In modern software development practices, it’s imperative to understand continuous integration (CI) and continuous delivery/deployment (CD). The goal of a CI/CD pipeline is to allow developers to apply new software changes to a production environment quickly and safely in order to generate business value. Understanding the difference between continuous integration, delivery, and deployment is important for both engineers and business leaders. Often, teams get stuck doing only continuous integration and disregard the delivery or deployment part.
In 2020, new concepts like GitOps and MLOps emerged, putting CI/CD in a prime spot to enable interdisciplinary alignment and new ways of working. These developments make it especially important to look at CI/CD holistically.
The major public cloud providers are also increasing their CI/CD capabilities, making it worthwhile to watch Microsoft Azure DevOps, AWS CodeStar, and Google Cloud Build. While they have room to grow as CI solutions, they might still change the CI/CD landscape.
In this article, we'll help you navigate seven of the most popular CI/CD tools currently available so you can make the best decision for your development teams.
A Closer Look Into The Most Popular CI/CD Technologies Available
With several CI/CD tools available in the market it can be challenging to select the one that best suits your organization's needs. Today, we'll look at the following tools:
Jenkins is undoubtedly the most well-known tool for building CI/CD pipelines. A robust, open-source project, Jenkins has a devoted community worldwide.
Jenkins architecture is divided into two parts: a controller server that orchestrates the necessary actions for each project CI/CD pipeline, and one or more agent nodes that execute those defined steps. While some companies offer Jenkins as a service, it is usually up to organizations to set up Jenkins on their own infrastructure.
The Jenkins agent can be installed in different types of machines (physical, virtual, containers) and runs on top of different OSs (Windows, Linux, OSX). This flexibility makes it possible for Jenkins to support any complex build and test environment needed for CI/CD. Alternatively, the entire CI/CD pipeline can be written as code using the Groovy language, and stored in a version control system like Git—making it easy to manage, audit, and replicate as needed.
Another great feature of Jenkins is the built-in, community-driven plugin system. With thousands of plugins at your fingertips, it’s easy to extend and add new functionalities.
However, this ability to fulfill virtually any type of use case with thousands of possible combinations also makes it overwhelming for less experienced engineers to set up an optimal CI/CD pipeline. Additionally, the lack of a de facto SaaS version implies that Jenkins often requires a higher operational management overhead than alternative solutions.
Created by a Berlin-based company under the same name, Travis CI is a popular SaaS solution.
One of the secrets behind Travis CI’s popularity was making it freely available to open-source projects. The CI/CD-as-a-service concept wasn’t well known, and for those who did know, it came at a cost. It filled a gap in open-source projects that usually lack corporate infrastructure and CI/CD capabilities.
While the company still has a free plan (now available to both private and open-source repositories) it is limited to a one-time allowance in the form of credits or can be unlimited on a case-by-case basis. The pricing is variable depending on the number of concurrent jobs and credits. In every plan, customers benefit from unlimited users as well as support for Windows, Linux, macOS, and FreeBSD.
Alternatively, the Travis CI Enterprise edition is a self-hosted approach with a separate pricing scheme. This is a good fit for enterprise customers that want full control over the management plane, as well as execution agents, deployment, and operations.
With ready-made GitHub and Bitbucket integrations, it’s easy to get started with Travis CI. The service includes features like built-in Docker support, build matrices for parallel testing, and add-ons for tasks like headless testing with browsers. Travis CI offers an intuitive UX, but it also supports pipeline-as-a-code using YAML.
In short, Travis is a developer-friendly solution for organizations that want modern features and a low operational overhead.
As a cloud-based solution, CircleCI offers a number of pricing options that range from a free plan (limited to single jobs and Windows and Linux builds) to a tailored plan with custom pricing. Note that features vary according to which plan you choose.
With more robust plans, customers can access GPU support, larger compute capacity, macOS builds, and the ability to use self-hosted execution agents (called “runners”). In addition to out-of-the-box support for popular programming languages, the Orbs feature enables snippets of code to be reused to automate repetitive tasks and build lightweight pipelines as code using YAML.
Circle CI also offers Server, a self-hosted edition which is priced per user and includes an enterprise support plan, thus enabling full control over deployment as well as both the management plane and execution agents.
Instead of adopting a separate technology for CI/CD, GitLab customers have the option to implement CI/CD pipelines in addition to using the platform to host code repositories.
When it comes to pricing, the CI/CD features are blended into the GitLab subscription, so you need to look at it as a holistic platform rather than just the CI/CD functionalities. The free plan includes 400 minutes of pipeline execution and the ability to bring self-hosted agents. From there, plans vary based on the number of CI/CD minutes and users needed.
While GitLab is essentially a SaaS product, there is also a self-managed option available to customers who want full control over the deployment and operational aspects.
With Gitlab, customers have end-to-end visibility and management of the software development lifecycle in a single platform. While CI/CD capabilities are not the sole focus of the product, there is built-in support for multiple platforms (Unix, Windows, and macOS) and languages (Java, PHP, Ruby, C, among others). Docker support, including your own private registry, is also available. Pipelines can also be developed as a code using YAML.
The tight integration between the different Gitlab features make it an attractive option for organizations looking for an all-in-one platform with a low operational overhead.
Bitbucket Pipelines & Bamboo
It comes as no surprise that Atlassian, the company behind popular software development tools such as Bitbucket, Jira, Trello, and Opsgenie, also provides CI/CD capabilities. Customers can choose between Bamboo—an on-premises standalone product—or Bitbucket Pipelines—a feature available in their code repository and version management product.
Because Bitbucket has both SaaS and on-premises versions, to some extent Bamboo and Bitbucket Pipelines compete for customers who want a self-hosted solution.
Both services offer modern capabilities, such as container and multi-platform support. And there is an advantage for customers who are already familiar with the Atlassian suite and can benefit from the built-in integrations between their products and third-party services using the Atlassian Marketplace.
While the entire Attlasian suite is freely available to open-source projects, the cost of using Bamboo can be steep for small organizations. Bamboo customers are charged based on the number of remote agents they need, while enjoying unlimited jobs and local agents. Conversely, Bitbucket Pipelines is included in the Bitbucket subscription, starting with a limited free plan. Further costs are based on the number of users, build time, and storage required.
The “Actions” feature from Microsoft-owned GitHub was initially announced in 2019 and became generally available in 2020. As a long-time home for many open-source and commercial projects, there was a natural interest in taking advantage of GitHub’s CI/CD capabilities.
One of the most interesting aspects of the service is that Actions works as a marketplace, where users can create their pipeline workflow based on snippets from popular third-party vendors, making it easy to get started. While GitHub Actions may not be as complete as its direct competitors yet, the fast pace of new releases suggests that they’ll catch up soon.
In terms of pricing, GitHub Actions is freely available for public repositories. Private repositories enjoy 2000 minutes of runner build time in the free plan and additional minutes are available in paid plans. If customers go over a plan’s allotted minutes, they’re billed per-minute based on the runner platform (Windows, Linux, and macOS). However, self-hosted runners are free and the CI/CD capabilities are also available in the GitHub Enterprise self-hosted version.
With seamless integration within the Github ecosystem, an intuitive developer experience, and abundant free minutes, GitHub Actions shook up the CI/CD toolkit market.
What to Consider When Drawing Comparisons
With GitHub Actions entering the market, CI/CD capabilities are now available in all major code repository management providers (GitHub, GitLab, and Bitbucket). But there are trade-offs to consider in this scenario: top-notch integration with increased vendor lock-in, versus separate tools and vendors for more flexibility but with less natural integration.
Any of the tools discussed here will enable you to implement a working CI/CD pipeline, but the effort and operational overhead varies. When drawing comparisons between them, it’s important to consider your current status quo. Ask yourself what your team’s strengths and blind spots are when it comes to developing pipelines as code and organization policies (e.g., SaaS versus self-hosted).
There is no “one size fits all” tool, so consider what best suits your project (and organization) needs. Ultimately, the real goal of any tool you implement is to empower your teams to deliver great quality software.
Independent of the tool you use for your CI/CD process, you’ll deal with build failures. Thundra Sidekick helps application teams debug these issues with non-intrusive debugging and distributed tracing (see video here). Get started with Thundra.