CI/CD Pipeline - What is it and tools Needed to Implement It
Introduction
The concept of DevOps has already gained momentum and more organizations are moving this way. The adoption of the DevOps principle has changed the way how products are shipped in the software industry. It has brought in a significant change in the speed of delivery without compromising the quality of products shipped. Continuous Integration (CI), Continuous Testing, Continuous Delivery (CD) and Continuous Deployment have emerged to be the key catalysts that trigger high-quality delivery at high speed. These terms have become buzzwords in the DevOps world. Traditionally developers, testers and Ops teams have been pushing code, running tests, generating PRs and deploying code manually. While the concepts of CI/CD and continuous testing brings in automation at these stages and makes the entire delivery pipeline automated. Many DevOps team now implement a CI/CD pipeline to deliver their code changes more frequently and reliably. In this article we will see, what is meant by a CI/CD pipeline and will be discussing the tools needed to set up a CI/CD pipeline.
Continuous Integration/Continuous Delivery vs Continuous Deployment/
Before we jump into CI/CD pipeline, let’s have a look at the different terms- Continuous Integration (CI), Continuous Delivery (CD) and Continuous Deployment (CD) and understand their meaning.
Continuous Integration
Continuous Integration is all about integrating the work of individual developers into a common repository. So, rather than building features in isolation and submitting their code at the end of the cycle, developers frequently merge their code into the main branch of a common repository several times, on any given day. The idea behind CI is to bring in more collaboration between developers and reduce overall integration costs by having developers do it sooner and frequently. CI also helps in revealing defects early in the cycle.
Continuous Delivery
Continuous delivery is an extension of continuous integration which aims to automate the software delivery process. It involves automating various steps for the build deployment - push the code changes through various environments, perform various tests, etc. With all these release processes automated, Continuous delivery ensures that you can have a safe code release at any point in time by clicking on a button.
Continuous delivery requires continuous testing as you need to ensure that the code delivered is quality code. It involves executing automated tests as part of the software delivery pipeline for assessing business risk coverage and provides instant insights on whether a release candidate is safe to process through the delivery pipeline. It includes end-to-end tests that assess the end-user experience across all associated technologies (front end and back end back end). To sum up, continuous tests involves executing a right set of tests at the right stage of the delivery pipeline and delivers actionable feedback appropriate for each stage- to avoid any bottlenecks.
Continuous Deployment
Coming to continuous deployment, it takes continuous delivery to the next level, with a higher degree of automation, in which a build or deployment occurs automatically whenever a major change is made to the code. There is no human intervention required for deployment. In Continuous deployment, the system will deploy whichever build components/packages that successfully exit the delivery pipeline and only a failed test will prevent a change to be deployed to production. Continuous deployment accelerates user feedback on new deployments.
The CI/CD Pipeline
We have now seen what are Continuous Integration, Continuous Delivery, and Continuous Deployment. So, a CI/CD pipeline could either mean a delivery pipeline with Continuous Integration/Continuous Delivery or Continuous Integration/Continuous Deployment. With continuous delivery, your software is always released ready, but the timing of when to push the software to production is in business hands and is done manually. Continuous deployment is just like continuous delivery, except that even the releases happen automatically for any updated working version of the software. A CI/CD pipeline is the complete software delivery pipeline with Continuous Integration and Continuous delivery/deployment in place and can deliver software continuously. A Continuous delivery/deployment pipeline depends on a chain of tools.
A CI/CD pipeline consists of the following steps:
- Commit code: The developers commit their source code to a central repository called the source code manager.
- Build: The code is pulled from the repository and checked out on a server, and the artifact is created for the application to be able to come up.
- Test: Unit tests, Security tests, Integration tests, acceptance tests, etc are done to ensure the application is working as expected.
- Deploy: The build version is deployed to production.
#CI/CD Tools
The right set of tools is the most important aspect for the execution of a continuous delivery/continuous deployment system. There are many tools available in the market today for facilitating CI/CD. For setting up a CI/CD pipeline, first, you need to create infrastructure. In this section, let’s see the different types of tools we need to implement a CI/CD pipeline and some of the most commonly used tools for each type.
For a CI/CD pipeline, the different categories of tools you need are:
- Source Code Repository
- Continuous Integration Software
- Code Quality tools
- Artifact Repository Management tools
- Configuration Management tools
- Communication Tools
Source Code Repository Tools
It is the heart of integration from where the source code is taken for processing, building, testing, deploying and so on. Commonly used tools are:
- GitHub – A web-based hosting service for version control using Git. It offers plans for private repositories and free accounts. GitHub provides access control and various other collaboration features like bug tracking, task management, etc.
Website: https://github.com/ - GitLab: Another web-based hosting service for version control using Git similar to GitHub. One of the biggest differences between GitLab and GitHub is the built-in CI/CD of GitLab. GitLab offers its own CI for free and there is no need to use an external CI.
Website: https://about.gitlab.com/ - BitBucket- Bitbucket is a web-based version control repository hosting service owned by Atlassian, for source code and development projects that use either Mercurial or Git revision control systems. It integrates very well with other Atlassian tools like JIRA, Confluence, Bamboo, etc.
Website: https://bitbucket.org/
Continuous Integration Tools
- Jenkins: The most popular CI server which can be used to automate all sorts of tasks related to building, testing and delivering or deploying software. It is open source and has many plugins available for almost everything and also has great integrations with IDEs like Eclipse. This is the most preferred CI tool currently.
Website: https://jenkins.io/ - Team City: A Java-based build management and CI server from JetBrains. It is a proprietary CI system that integrates with Git and Mercurial.
Website: https://www.jetbrains.com/teamcity/ - Bamboo: Bamboo is the CI tool from Atlassian. It is a proprietary tool and lets you build, test and deploy software. It integrates well with other tools such as JIRA, Bitbucket, Fisheye, etc.
Website: https://www.atlassian.com/software/bamboo - Travis: It is a hosted, distributed, continuous integration service used to build and test software projects hosted at GitHub.
Website: https://travis-ci.org/
Code Quality tools
To ensure the code quality, it is necessary to include a code quality tool in your CI/CD pipeline. These tools help in displaying the estimation of code quality - easy and understandable visualizations of different code quality aspects: code coverage, tests failures, static code analysis and so on.
- SonarQube: It is an open source tool which is used for continuous inspection of code quality to perform automatic reviews with static analysis of code to detect vulnerabilities and bugs. It can be used for more than 20 languages.
Website: https://www.sonarqube.org/ - PMD: An open source cross-language static code analyzer that reports issues found in code like inefficient coding, bad programming habits, etc. It includes built-in rule sets and supports the ability to write custom rules.
Website: https://pmd.github.io/ - FindBugs: An Open source static code analyzer which detects possible bugs in Java programs.
Website: http://findbugs.sourceforge.net/
Artifact Repository Manager tools
An artifact repository manager is to be included in your CI/CD pipeline to optimize the download and storage of binary files used and produced in software development. It centralizes the management of all binary artifacts generated and used and overcomes the complexity arising from different binary artifact types. So the CI/CD pipeline takes the source code from source code repository, build it, test it, create versioned binary artifacts and put them in Binary Repository Manager. The CI/CD pipeline can later take these binaries and deploy them to various environments (like test, dev, staging, or prod).
- Nexus Repository OSS: One of the most widely used repository manager. It is a free artifact repository with support for all popular formats.
Website: https://www.sonatype.com/nexus-repository-oss - jFrog Artifactory: It is a universal repository manager supporting all major packaging formats, build tools and CI servers.
Website: https://jfrog.com/artifactory/ - Archiva: It is an extensible repository management software which offers several capabilities like proxying, security management, build artifact storage, delivery, browsing, and much more.
Website: https://archiva.apache.org/index.cgi
Configuration Management tools
Configuration management tool helps you to keep environments consistent throughout the software development process and ensures that your infrastructure environment deploys automatically and reliably every time. Common tools used are:
- Puppet: One of the most commonly used CM tool written in Ruby. A Puppet system allows the user to define the state of an IT infrastructure, and Puppet automatically enforces that state.
Website: https://puppet.com/solutions/configuration-management - Chef: An open source CM tool which is a competitor for Puppet. This tool also continuously configures systems against your desired state.
Website: https://www.chef.io/chef/ - Ansible: An open source software that automates software provisioning, configuration management, and application deployment.
Website: https://www.ansible.com/
Communication Tools
You need to have a cloud-based communication tool in your CI/CD pipeline to streamline communication, give feedback and enable immediate action. For example, your CI/CD pipeline can send a notification to communication tool if something goes wrong, or when a new version of the software is deployed.
- Slack: With more than 6 million daily users, Slack is the de facto choice for many companies right now.
Website: https://slack.com/
So, that’s all right now about CI/CD pipeline. A CI/CD pipeline definitely helps in increasing customer satisfaction and your ROI. Of course, there is a huge cost involved in setting up and challenges too. But it is worth the efforts and can help increase the efficiency of your team. Hope the article helps you in getting started with using a CI/CD pipeline.
How much is a great User Experience worth to you?
Browsee helps you understand your user's behaviour on your site. It's the next best thing to talking to them.
