Travis CI vs CircleCI: Comparing The Best CI/CD Tools

Over the past few years, the Continuous Integration and Continuous Deployment space have been gaining prominence. CI/CD tools have now become an integral part of the DevOps testing activity. However, there are multiple choices when it comes to choosing the best CI/CD tools making it even more challenging to choose one.

The silver lining- a select few providing full Continuous Integration & Continuous Delivery systems, with CircleCI & Travis CI leading the pack. Here, we would be doing a comprehensive CircleCI vs Travis CI comparison to help you select the best CI/CD tools suitable for your project.

CI/CD Yardstick

If you want to pick the right CI/CD tools, you need to find answers to these questions.

Why should you opt for CircleCI or Travis CI, the purpose-driven tools for DevOps testing?

• Is it easy to get started with the CI/CD tool?
• Is the tool scalable and reliable in terms of deployment and support?
• Does the tool support a CI/CD pipeline to expedite the deployment and testing process?
• How easy (or difficult) is switching from an existing CI/CD tool (e.g., Jenkins) to a new one?
• Does the tool support only have an on-premise solution or a cloud-based SaaS offering or both?

Let’s get started with an in-depth analysis of CircleCI vs Travis CI, and by the end of this comparison, you would be in a much better position to choose the best-suited CI/CD tool for DevOps testing.

Introduction To CircleCI & Travis CI

The first step to start comparing CircleCI vs Travis CI is to understand each of them. Unless stated otherwise, we usually assumed that the best CI/CD tools have standard capabilities like:

• Support for a wide range of languages & runtimes.
• Support for CI/CD pipelines.
• Support for configuration-as-code (as popularly called in Jenkins) for specifying the build & deployment process in the popular YAML format (or code)
• Support for Docker to run tests

Our CircleCI vs Travis CI comparison is based on their unique capabilities and how these CI/CD tools look like in terms of usage, scalability, extensibility, and more.

Travis CI and CircleCI are extremely popular CI/CD tools that provide a continuous and automated delivery cycle. These factors are integral as far as DevOps testing is concerned.

Read: Best CI/CD Pipeline Practices To Speed Test Automation

A Top-Level View Of CircleCI

CircleCI is a cloud-based tool that is instrumental in automating the integration and deployment process. CircleCI server is the on-premise (or self-hosted) solution, and CircleCI Cloud is the hosted (or SaaS) solution from CircleCI.

CircleCI also supports any language that builds on Linux, Windows, or macOS. It also includes languages such as C++, NET, PHP, Python, JavaScript, and Ruby. CircleCI also supports containers and provides runtimes for Linux, Windows, and macOS.

The self-hosted (CircleCI Server) and cloud (CircleCI Cloud) offerings from CircleCI support Windows, Linux, Android, and Docker builds. Only CircleCI Cloud supports macOS builds.

Using CircleCI does not require any installation; hence, it is a complete solution that requires minimal configurations. It also supports CI/CD pipelines called ‘workflows.’ The workflows in CircleCI make it possible to write and build jobs with excellent visual representation.

CircleCI cloud also supports ‘Orbs,’ which are reusable snippets of code that help in automation of repetitive processes, speed-up the project setup. It makes it easy to integrate with third-party tools. Orbs for standard CircleCI configurations are available on the official website.

Salient Features Of CircleCI

I hope the details provided above have given you better clarification about the CircleCI vs Travis CI analysis. Before jumping off to a top-level view of Travis CI, let us take a look at some of the most pivotal features of CircleCI-

• Light-weight YAML file for configuration.
• Ease of setup.
• Offers CI/CD pipelines called ‘workflows.’
• Supports popular programming languages out-of-the-box.
• Provides on-premise and cloud-based solutions.
• Offers REST APIs.

This CircleCI Tutorial for beginners and professionals will help you learn how to use CircleCI, one of the most popular CI/CD tools used in DevOps.

A Top-Level View Of Travis CI

Travis CI is one of the early players in the CI/CD tools market. Initially created only for the open-source projects, it later migrated to closed source projects as well. Travis CI is written in Ruby and is maintained by the Travis CI community.

Initially available only for the projects hosted on GitHub, Travis CI now also supports projects that are hosted on BitBucket. Along with a hosted tool, Travis CI also offers Travis CI Enterprise – a self-hosted tool that is built for easy integration with GitHub. Travis CI Enterprise lets enterprises run the CI builds seamlessly on the private cloud (or a self-hosted platform).

Travis CI can be used across various programming languages like Java, C#, Python, Julia, PHP, Perl, Objective C, and many more. In all, it supports over 30 different languages.

It also supports container builds and provides runtimes on Linux, macOS, and Windows (early stage). For using Travis CI, no installation is required. You can signup and add a project using credentials of GitHub or BitBucket.
The ‘build matrix’ feature in Travis CI lets you run the build parallely on different combinations of environments, languages, package versions, and runtimes. It also supports integration with Email, Slack, HipChat, etc. so that notifications are sent to the team about builds’ status.

Salient Features Of Travis CI

• YAML file for configuration.
• Easy to set up using GitHub and Bitbucket.
• ‘Build Matrices’ feature for parallel testing.
• Supports 30+ programming languages out-of-the-box.
• Flexibility to run tests on Linux and macOS X at the same time.
• Support for Docker for running tests.
• Travis CI Enterprise lets organizations use Travis CI for DevOps testing on a private cloud.
• Offers REST APIs.

Wondering about Travis CI vs Jenkins?

CircleCI vs Travis CI – A Detailed Comparison

Now that you have the most useful information to compare CircleCI vs Travis CI, it is time to make an informed decision. Here are some of the features that can help you in picking between CircleCI vs. Travis CI:

Setup & Installation

To get started using CircleCI for your project, you just need to sign up with GitHub or Bitbucket to authorize CircleCI for accessing the code.

If you are using GitHub to use CircleCI, you also have the flexibility to prevent CircleCI access to private repositories. Once authorized, you can use the CircleCI Pipelines Dashboard and start adding the project. The CircleCI config.yml file has to be located in a folder called .circleci at the top of the project.

CircleCI has self-hosted and cloud-based offerings called CircleCI Server and CircleCI Cloud, respectively. There is an option to set up CircleCI on the cloud or on a private server where the onus for setup, security, and maintenance lies with the in-house team.

To get started with Travis CI with GitHub, you need to signup on travis-ci.com. Once signed up, enable the repository for testing from the Travis settings page and add travis.yml to the root folder. The signup on Travis CI can also be done using Bitbucket.

Both CircleCI and Travis CI can integrate with popular version control systems like GitHub, Bitbucket, and others. A number of steps are run whenever there are changes in the source code. Travis CI and CircleCI are good options for using GitHub (or Bitbucket), especially if you are using a cloud-hosted solution.

Integrations

CircleCI can be integrated with several tools (including Cloud service providers) from categories such as code analysis, deployment, reporting, monitoring, testing, and more. CircleCI integration with LambdaTest boosts the go-to-market delivery and lets you perform automated cross browser testing on a cloud-based Selenium Grid.

Integrations in CircleCI are available in the form of CircleCI Orbs. Orbs are reusable snippets of code that are instrumental in automating repeated processes and accelerating the project setup. To ease the integration process with third-party tools, you can also use the Orbs (or CircleCI configuration for popular integration options) from the Orbs registry. The orb for accelerating the process of cross browser testing with LambdaTest Tunnel and CircleCI pipelines is available in LambdaTest CircleCi orb.

Travis CI works by default with GitHub and BitBucket. Additionally, it also comprises a number of tools for websites, mobile applications, desktop and plugins & libraries that can interact with Travis CI. More information about the tools, plugins, and libraries for Travis CI is available in the official plugin’s documentation of Travis CI. Travis CI can also be used with third-party tools like LambdaTest, Coveralls, and more.

Both CI/CD tools score well on the plugins and third-party integrations front.

Pipeline Experience

The pipeline is one of the dominant factors as far as CircleCI vs Travis CI comparison is concerned. CircleCI and Travis CI offer different ways for the creation and maintenance of a CI/CD pipeline. Both the CI/CD tools provide a visual web-based wizard for creating a CI/CD pipeline. The wizard is primarily useful for someone who is just getting started with pipelines.

For developers who are well-versed with YAML files and pipelines, hand-coding would do a much better job creating and maintaining the CI/CD pipeline. When it comes to CircleCI, it provides you with a sample YAML file based on your responses. This means that the newly created YAML file (config.yml) meets the requirements.

On the other hand, you have to manually create a YAML file (.travis.yml) by following their documentation. The open-source generate-travis CLI is an excellent command-line tool which, if installed globally, can be used for overriding built-in templates of travis.yml.

As far as the pipeline setup experience is concerned, CircleCI inches ahead of Travis CI due to the provision of a YAML configuration generator that lets you create CI/CD pipelines with much ease. This especially helps if you are relatively new with the creation of CI/CD pipelines.

Extensibility & Customization

As the complexity of the application being developed increases, it is likely that the complexity of the CI/CD pipeline might also increase. Before the pipeline becomes unmanageable to maintain, it is important to introduce custom steps in the pipeline for increasing the longevity of the pipelines’.

This is where extensibility and customization play a major role in minimizing the e maintenance front’s efforts. They are also important to consider for CircleCI vs Travis CI comparison. Both CircleCI and Travis CI offer RESTful APIs for extensibility. This lets you access the pipeline programmatically. The latest release of Travis CI APIs is API v3 and the API version v2 is not yet deprecated, as many front-end applications of Travis CI are still using the API. The Ruby Library for Travis CI is based on API v2.

Like Travis CI, CircleCI also supports REST APIs that can be used for accessing the pipeline programmatically. The latest release of CircleCI APIs is API v2, and nine new endpoints were introduced. The new endpoints let you retrieve a pipeline’s workflows, the latest pipelines on a particular project, an individual workflow, and more. Unlike the APIs offered by Travis CI that let you read and modify parts of the pipeline, the CircleCI APIs only let you perform Retrieve (or Read/GET) on a project (or pipeline’s workflow).

Command-line interfaces (CLIs) are available for CircleCI while Travis CI lets you interact with the tool remotely. As the complexities of pipelines can grow over a period of time, Travis CI and CircleCI provide options for easing the complexities involved in the management of pipelines. Though complex pipelines can be broken up into smaller (and maintainable) pipelines (or YAML files), both CI/CD tools have different techniques for reducing the complexities involved in pipeline management.

CircleCI lets you create custom Orbs that can contain a certain set of steps or the complete job. These shareable packages of CircleCI configuration can then be used across builds (as they are in a reusable format). On the other hand, Travis CI lets you break down the complex scripts into simpler & easy-to-maintain scripts that can be further stored in the same repository.

Hosting (On-Premise or Cloud)

Travis CI and CircleCI offer on-premise and cloud-based options. CircleCI Server, the on-premise variant of CircleCI, can be deployed through popular cloud platforms like AWS, Azure, Google Cloud, Windows Cloud, and more. Kubernetes deployments can also be automated with CircleCI Server and Kubernetes CircleCI orbs can help to get started with Kubernetes and CircleCI.

There is no process from CircleCI for migrating between CircleCI Cloud and CircleCI Server setup. A fresh install of CircleCI 2.0 has to be performed using AWS and Terraform.

The open-source option of Travis CI is based on the cloud. For enterprises looking to use Travis CI on a private server setup, they can opt for the self-hosted option called ‘Travis CI Enterprise.’ The self-hosted and hosted tools let you build using different languages and runtimes on macOS, Linux, and Windows.

Travis CI for Enterprise (or on-premise environment) supports popular cloud environments such as AWS, Google Compute Engine, VMware, OpenStack, Azure, Heroku, Kubernetes (from version 3.0 onwards), and more.

In the CircleCI vs Travis CI comparison for ‘Enterprise’ options, CircleCI Server is ahead from Travis CI Enterprise by a small margin since CircleCI orbs makes it easy to integrate with popular third-party tools (including cloud platforms).

Parallel Testing

Travis CI and CircleCI both support parallel testing. This also applies to the free version of these CI/CD tools.

Parallelism in CircleCI lets you run tests in parallel by spreading the tests across different executors. The parallelism level has to be specified in config.yml, indicating how many executors are spun out for the test job. Further, CLI for CircleCI or environment variables should be used for splitting the test files or configuring each parallel machine on an individual basis.

CircleCI recommends splitting the tests using timing data to optimize the test suite across a set of parallel executors. Splitting by timing data ensures that the tests are split in the most ideal manner, reducing the overall time involved in test execution. The build time in CircleCI can be reduced by running jobs in parallel with workflows.

The ‘build matrix’ feature in Travis CI lets you split up the unit tests and integration tests into two different build jobs. These tests can then be run in parallel for optimal utilization of available build capacity supported by the account.

Test suites can be executed faster by parallelizing the builds across different VMs (Virtual Machines). Necessary changes have to be done in travis.yml for parallelizing the tests. Integrating with tools like Slack, HipChat, Email, etc. helps in sending out notifications when the build is not successful. Further details about parallel testing and speeding up the builds for Travis CI are available in the Speeding up section on Travis CI website.

Community Support

There is no denying that community plays a key role in shaping up an open-source project (or tool e.g., Jenkins). Though Travis CI and CircleCI have different offerings for open-source projects and enterprises, they do not have a vast stronghold for online communities.

The CircleCI community lets you submit topics for discussion, feature requests, and more. On the other hand, the Travis CI community has a number of sections like enterprise, deployment, languages, discussion & feedback, and more.

If you have migrated from an open-source tool like Jenkins to something like Travis CI or CircleCI, you may miss the vibrancy that is witnessed in open-source forums☺.

Scalability & User Management

Both Travis CI and CircleCI are good options for DevOps testing if your organization is using GitHub Enterprise for software development and version control. CircleCI Cloud, CircleCI Server, and Travis CI Enterprise offer integration with GitHub Enterprise and there is no necessity for creating new accounts if the users are already using GitHub (i.e., the repositories are on GitHub).

CircleCI when used with GitHub enterprise provides options for preventing access to private repositories. Travis CI Enterprise can also sync permission settings from the GitHub Enterprise.

Travis CI and CircleCI use a per-user pricing model. In terms of pricing, CircleCI has three plans – Free, Performance, and Custom (or Custom Pricing) for the CircleCI Cloud and the CircleCI Server plan is at $35 per user with an annual plan.

Travis CI has four plans – Bootstrap, Startup ($118/per month billed annually), Small Business ($228/month billed annually), and Premium. If you plan to use Travis CI for open-source projects, there is a free plan where you receive unlimited builds and five concurrent jobs for free.

For leveraging the advantages offered by the respective CI/CD tools, a scalable infrastructure is necessary so that builds as well as tests can be run in parallel. This is especially relevant when self-hosted variants of CircleCI and Travis CI (i.e. CircleCI Server and Travis CI Enterprise) are in use.

As per the official documentation of CircleCI, if you are on a container-based plan, there is a high probability that the jobs may get queued-up if enough containers are not there for accommodating the workload. On the other hand, the Performance Plan in CircleCI is usage-based and not container-based. In the CircleCI vs Travis CI battle, Travis CI has a slight upper edge over CircleCI as far as availability and load balancing is concerned.

Migration & Vendor Lock-in

Travis CI and CircleCI are free for use with open-source projects. However, organizations (especially growth-stage startups and large organizations) would prefer their enterprise variants (i.e. Travis CI Enterprise and CircleCI Server) as the CI/CD tools can be deployed on a self-hosted platform. This means better control over the hardware, improved security, and better control over the pipelines than the cloud-based counterparts.

CircleCI and Travis CI follow a proprietary syntax for the respective YAML files (i.e., config.yml and travis.yml). Still, the advantages offered by the tools can be leveraged by opting for the self-hosted (or Enterprise) variant.

Travis CI and CircleCI have detailed sections on migrating from open-source CI/CD options like Jenkins to their platform. CircleCI has also built a tool called Jenkinsfile Converter to enable Jenkins users to start building on CircleCI by converting a Jenkinsfile to CircleCI config.yml

CircleCI vs Travis CI Comparison Snapshot

In the above sections, we have observed all the pivotal factors that would help us choose the best CI/CD tools when it comes to CircleCI vs Travis CI. In the below section, we present a table view of the entire comparison for your clarity. Here is the snapshot of CircleCI vs Travis CI comparison:

CircleCI vs Travis CI: Which CI/CD Tool Should You Choose?

As seen from the CircleCI vs Travis CI comparison above, the choice depends on your project needs and how you are planning to leverage the capabilities of the tool. The enterprise (or self-hosted) versions of both the CI/CD tools can be used with popular cloud-platforms such as AWS, Azure, Google Cloud, and more. CircleCI Orbs makes it easy to configure CI/CD pipelines at an accelerated pace and integrate with third-party tools with just a few lines of code, especially when it comes to browser compatibility testing.

And now, we would like to hear from you: which CI/CD tool would you prefer for DevOps testing? Or maybe you have a question. Either way, go ahead and leave a comment below.

Happy testing!

100+ Free Online Tools From LambdaTest!

LambdaTest has come up with an index of 100+ free online tools for developers and testers. From HTML, XML, and JSON formatters to robust data generators, and hash calculators. LambdaTest’s free online tools are built to help engineering teams accelerate and be more productive with their daily activities.

Code Tidy

• JSON Prettify
• JSON Minify
• HTML Prettify
• HTML Minify
• JavaScript Minify
• CSS Minify
• CSS Prettify
• XML Minify
• XML Prettify

Data Format

• IDN Encode
• IDN Decode
• XML to JSON Converter
• JSON to XML Converter
• BCD to Decimal
• HEX to Decimal
• Decimal to BCD
• UTF8 Decode
• UTF8 Encode
• HEX to RGB Converter
• RGB to HEX Converter
• HTML to Markdown Converter
• Markdown to HTML Converter
• Decimal to Gray Code Converter
• Gray Code to Decimal
• URL Decode
• URL Encode
• Base64 Encode
• Base64 Decode
• Text to HTML Entities Converter
• HTML Entities to Text Converter

Random Data

• Random JSON Generator
• Random XML Generator
• Random CSV Generator
• Random YAML Generator
• Placeholder Image Generator
• Random Binary Generator
• Random Character Generator
• Random Color Generator
• Random Date Generator
• Random Decimal Fraction Generator
• Random Decimal Generator
• Random GUID Generator
• Random HEX Generator
• Random Octal Generator
• Random IP Generator
• Random MAC Generator
• Random Number Generator
• Random Paragraph Generator
• Random Password Generator
• Random Time Generator
• Random UUID Generator
• Random Sentence Generator
• Random String Generator
• Random Word Generator
• Random Data from RegEXP
• Test Data Generator
• Lorem Ipsum Generator
• Credit Card Number Generator
• QR Code Generator
• Random Byte Generator

Security Tools

• Hash Calculator
• Hash MAC Generator
• CRC32 Hash Calculator
• CRC32B Hash Calculator
• Ripe MD 128 Hash Calculator
• Ripe MD 160 Hash Calculator
• Ripe MD 256 Hash Calculator
• Ripe MD 320 Hash Calculator
• MD2 Hash Calculator
• MD4 Hash Calculator
• Adler32 Hash Calculator
• Gost Hash Calculator
• Whirlpool Hash Calculator
• MD5 Hash Calculator
• SHA1 Hash Calculator
• SHA256 Hash Calculator
• SHA384 Hash Calculator
• SHA512 Hash Calculator

Utils

• Find and Replace String
• HTML Escape
• HTML Unescape
• Difference Checker
• Shuffle Letters
• Shuffle Text Lines
• Sorting List
• Split Your String
• Text Lowercase
• Text Uppercase
• Text Repeater
• Text Rotator
• Character Count
• Word Count
• Lines Count
• Sentence Count
• URL Parse
• JSON Escape
• JSON Unescape
• Extract Text from HTML
• Extract Text from JSON
• Extract Text from XML
• Strip HTML
• JSON Validator

Himanshu Sheth

Himanshu Sheth is a seasoned technologist and blogger with more than 15+ years of diverse working experience. He currently works as the ‘Lead Developer Evangelist’ and ‘Senior Manager [Technical Content Marketing]’ at LambdaTest. He is very active with the startup community in Bengaluru (and down South) and loves interacting with passionate founders on his personal blog (which he has been maintaining since last 15+ years).

See author's profile