What Tools Does the DevOps Toolchain Actually Include?

Posted on Views: Waline: Word count in article: 1.4k Reading time ≈ 5 mins.
This article outlines the modules and corresponding tools within the DevOps toolchain, covering 9 modules such as code management, CICD, and configuration management. It introduces tools like Git, Jenkins, and Ansible, and mentions that tool selection should consider factors such as compatibility with existing processes.

Introduction

Whether in job interviews, technical exchange meetings, or communication among colleagues, the term “DevOps toolchain” is frequently heard. Of course, I also use it myself. So, what tools does the DevOps toolchain actually include? And how are these tools used? I’ll try to sort out this information.

When it comes to the DevOps toolchain, many people may first think of a diagram similar to the following (the image is sourced from here):

![DevOps toolchain](../images/DevOps toolchain.png)

To be honest, I don’t like this kind of image. Often, the author tries to stuff in as many related icons as possible to make the picture look impressive, but in fact, it doesn’t explain the DevOps toolchain clearly.

In this article, I’ll attempt to categorize it into several modules based on my experience and provide some specific tool examples for each module.

What Modules Can the DevOps Toolchain Be Divided Into?

Based on my experience and past practices, the DevOps toolchain can be divided into the following modules:

  • Code Management (also known as Version Management)
  • CICD
  • Configuration Management
  • Containerization and Container Orchestration
  • Monitoring and Logging
  • Security Analysis
  • Artifact Management
  • Project Management, Collaboration, and Communication
  • Automated Testing

There are a total of 9 modules, and each module may contain multiple tools. Let’s introduce them one by one according to each module.

Code Management

The most popular code management tool nowadays is definitely Git. Regarding the server side, GitLab, Github, and Gitee are the more common ones. Each enterprise will choose a different platform based on various factors.

However, regardless of which platform is chosen, most enterprises currently opt for private deployment rather than using SaaS services. People are still concerned about code security.

Of course, some very large enterprises choose to develop a Git server according to their own needs. For example, Huawei developed its own.

There is also a relatively niche option. Currently, mainstream public cloud providers basically offer code management functions based on the Git protocol, but each is different. Without very special reasons, I highly do not recommend this option because the code management platform can greatly influence the choice of other tools in the DevOps toolchain. The code management functions of cloud providers are closed, which will limit the selection range of other tools and may likely have compatibility issues.

CICD

CICD refers to Continuous Integration and Continuous Deployment (Continuous Delivery).

The most well - known tool is probably Jenkins. It is powerful and has a large number of plugins, which can meet most requirements. If you don’t know which CICD tool to use, you can give it a try.

If your code management tool is Github, especially Github Enterprise Edition, then Github Action may be a better CICD tool for you. It is natively integrated with Github and has a simple configuration.

If most of your company/team’s applications are deployed based on k8s, then argoCD may be more suitable for you. argoCD can be said to be a CD tool specifically designed for k8s, following the gitops concept.

Configuration Management

Configuration management tools are mainly used to uniformly manage and configure server and application settings. Common ones include Ansible, Chef, Puppet, SaltStack.

In fact, there is another tool that I think should also be included in configuration management, and that is Terraform.
To be precise, Terraform is an IaC (Infrastructure as Code) tool, but it can be regarded as a configuration management tool in the era of public clouds.

Configuration management tools are often used not only by the DevOps team but also by the product team (project team). Because server resources are sometimes directly managed by the project team, or they may be managed by a dedicated operations team. Each company may be different.

Containerization and Container Orchestration

There is probably not much controversy in this part. Docker and k8s are the most popular.

There’s not much to say about Docker, but there are actually many details in k8s. For example, there are specialized tools for permission control, network isolation, logging, and monitoring in k8s. A good DevOps team should provide all of these.

Monitoring and Logging

The monitoring and logging part can be a bit complicated. Because there is an issue of boundaries.

In my opinion, as a DevOps team, basic monitoring and logging should be provided.

Specifically, monitoring should include computing resources such as CPU and memory, network, server status, and pod status. It may not include higher - level content such as SLO and SLI, because these should be done by SRE. (But in fact, many times one has to do both DevOps and SRE…)

Logging should include parsed application logs and system logs. However, the parsing is generally done by the project team themselves, as they are more familiar with their own logs.

Security Analysis

In the DevOps toolchain, security analysis can be further subdivided into the following categories according to different stages and objectives of security checks:

Static Application Security Testing (SAST)

  • Purpose: Analyze source code or binary code to find potential security vulnerabilities without running the program.
  • Example Tools:
    • SonarQube: It can not only detect code quality issues but also identify security vulnerabilities and code smells.
    • Checkmarx: Provides a comprehensive SAST solution, supporting multiple programming languages and frameworks.

Software Composition Analysis (SCA)

  • Purpose: Check whether the libraries and frameworks on which the project depends have known security vulnerabilities.
  • Example Tools:
    • Snyk: Focuses on discovering and fixing security vulnerabilities in dependent libraries.
    • WhiteSource: Offers extensive language and framework support, automatically detecting and fixing open - source security vulnerabilities.

Container Security and Scanning

  • Purpose: Ensure the security of container images in a containerized environment, including image scanning and runtime protection.
  • Example Tools:
    • Aqua Security: Provides a container security platform, including functions such as image scanning and runtime protection.
    • Clair, Trivy: Open - source projects focusing on vulnerability scanning of container images.

Code Style and Specification Checks

  • Purpose: Ensure that the code follows certain coding standards and styles, which helps reduce security risks. Such tools are generally just a plugin or a command - line tool, without a server - side, and do not require centralized maintenance.
  • Example Tools:
    • ESLint: A plug - in - based static code analysis tool for JavaScript, which can be used to check common code and style issues.
    • Pylint: A static code analysis tool for the Python programming language, used to find coding errors, enforce coding standards, and recommend coding styles.

Artifact Management

  • Purpose: Tools for managing, storing, and distributing software packages. These software packages may be third - party software libraries, frameworks, or programs you developed yourself.
  • Example Tools:
    • JFrog Artifactory: A universal artifact repository that supports multiple software package repository types, such as Maven, Docker, npm, etc. This is a commercial product.
    • Harbor: An open - source enterprise - level Docker image repository that supports image storage, scanning, and signing.
    • Sonatype Nexus: Provides artifact storage and distribution for multiple technologies, such as Maven, npm, Docker, etc.

Project Management, Collaboration, and Communication

I actually hesitated whether to include this part in the DevOps toolchain. Because generally, it is supposed to be provided by the IT department in the company. However, the choice of such tools can sometimes have a great impact on the DevOps process, and thus affect development efficiency. So, it is still included in the DevOps toolchain.

There are many such tools now. Domestic ones include DingTalk and Feishu, and there are also old - fashioned ones like Jira and Confluence wiki.

Automated Testing

As an essential part of DevOps, automated testing should be included in the scope of the DevOps toolchain. However, this part of the work is basically not done by the DevOps team, as testing is generally carried out by the project team or a dedicated testing team.

A good DevOps team will add some checks and controls in the CICD workflow. For example, if testing is not done during a certain CICD process, it is not allowed to proceed to the CD stage.

Conclusion

The above is my simple overview of the DevOps toolchain. In actual use, the selection of each tool actually needs to consider many factors, such as its compatibility with existing processes, the acceptance level of the project team for this tool, and whether the security strategy is compliant, etc.