Testing in CI/CD

If you are part of testing fraternity you would have heard CI/CD numerous times by now. It's the talk of the software testing town. In this article, we will cover the basics of CI/CD, what actually it is and how it is implemented in software testing.

What is continuous integration?

In simple terms, development teams can integrate their code into a shared repository.

This helps in maintaining code quality and identifying potential issues with the local version of the code at an early stage.

What is continuous delivery?

Everything that is being continuously merged by the development team, is continuously deployed to the live environment. Continuous delivery is often coined as ‘Continuous Deployment’ as well.

Since most developers work in parallel, continuously integrating their code into one repository would mean that the master branch is continuously updated with new features. In order to ensure that there is no compromise in the code quality with so many changes happening rapidly, testing must happen at the same pace. It should be no surprise that manual testing at this moment would not be the best approach in order to achieve this. Automated testing is the key to successful testing in the CI/CD pipeline.

Stages of Continuous Delivery

Develop: The developer builds the code as per the new requirement or the feature requested.

Writing tests: Once the code is written, tests need to be written for the same. At this point, these tests as usually unit tests written by the developers.

Local Testing: This is then locally tested in order to check if all tests are passing and the code does not break. Often as a percentage is set as the pass rate which the tests running need to meet.

Rebase & Resolve conflict: In an actual development scenario, there will be multiple people merging their code. The developer needs to make sure that his branch is updated at all times. Updating the branch with the latest merged code is called ‘Rebasing’. Once it’s rebased there will likely be some conflicts that need to be resolved. After that, the tests are run again against the rebased code.

Commit: Once the tests have passed, the code is then ready to be committed with all the changes.

Build: The source code developed is then combined to build a deployment artifact that can be run on an instance i.e., the server. This code is now ready to be deployed to different testing environments.

UAT: The code is then deployed to a test server where testers start to test the feature. These tests can be automated as well as manual.

Merge: If they commit that’s under testing is approved by the testers, this is then merged into the master branch.

Production deployment: Once the code is merged, this merged code is then deployed to production.

The above process needs to be done with every build coded by the developers.

Where does automation testing fall in this CI/CD pipeline?

Automated testing ideally happens once the build stage has been completed and the code can be deployed. Unit tests, UI tests, Integration tests can all be run at this stage.

These tests help in ensuring that the code meets a standard of quality. This phase can last from a few minutes to a couple of hours depending on how the automation that needs to be run is architected.

Tests can be run in parallel in order to execute them more quickly. If a code fails during the test phase, the build can be rejected without further investing in any manual testing time.

Tools used for CI/CD

• Jenkins: Jenkins is an open-source tool that is used for continuous integration. It’s free to use and jobs can be configured both by the interface as well as scripts.

• Travis CI: This tool is free of cost for open source projects, it hosted by GitHub.

• Gitlab: Gitlab is a version control tool that has its own CI methodology which is cloud-based. It is supported on multiple platforms that have both free and paid versions.

• Bamboo: Bamboo is a CI tool by JIRA. If the organization uses JIRA, then it would be beneficial to check this tool out. It supports automated merging on the approval of tickets as well.

Best practices for CI/CD pipeline to make the best out of test automation

• Incremental changes: It is always advisable to follow a feature by feature approach. If the feature is really big, it is good to break it down into smaller and quicker to test features. This is important in terms of automation because if there is an issue it is easier to figure out the root cause. If your commit is too big, isolating the cause of an issue would be a tough task.

• Identify what can be automated: It is very common for teams to dive fast and say ‘let’s automate everything’ but this is the main mistake made. We must know the purpose of automation and identify the test cases which should be automated.

• Parallel Tests: Tests should be run in parallel in order to make it more efficient and timely. It can greatly reduce the time taken to run tests and thus give the results much quicker. But it’s not sufficient to just execute these tests in parallel, it is also important to scale the server size where the tests are running in order to actually make them faster.

Conclusion

Automating tests is an important part of the successful deployment of projects while maintaining a standard of quality. Ensuring tests are run at every stage gives good transparency on the quality of the code. Bugs can be discovered at an early stage and any delays which can be caused by it can be planned in a timely manner. Having a CI/CD pipeline with tests integrated helps in speeding up the testing and deployment process.