Testing in DevOps – What it means to QA/Tester?

Dhanya Arumughom -


Organizations in the software Industry have a highly competitive environment today and every organization has to adopt strategies to stay in the Industry. Organizations worry about competitors leapfrogging them with ever faster ‘Time to Market’ or ‘Quality’ products. These competitive dynamics has resulted in the organizations lending towards methodologies like Agile and DevOps which aims in faster and quality delivery. DevOps, in fact, can be considered to be an extension of Agile. While Agile focuses on delivering “working software” rapidly, DevOps strives to focus on the overall service or software fully delivered to the customer instead of simply “working software”. Agile emphasized collaboration of client, developers, and testers as a team for faster delivery. DevOps takes the agile model to the next step by bringing closer the release and deployment activities to those of development and testing. This is why DevOps is often referred to as “Agile on steroids”.
DevOps simply stands for ‘Developers+Operations’. It emphasizes breaking down the barriers between the development team and operations in order to enable continuous and fast-paced delivery. So what is the role of testing/QA in DevOps? There is quite some confusion around the role of testers and misconception on the QA’s role in DevOps and a fear of the role being not necessary and developers itself is responsible for testing is on the rise. Some folks doubt if a separate role for performing testing is justified in a DevOps environment. Yeah, QA or tester roles are very much needed in DevOps environment as well. In a DevOps chain, although the basic responsibilities of testers remain same, the requirement in terms of skill sets have increased. So, how the DevOps culture has really changed the QA life? In this article, we try to answer this question that you might have in mind if you are a tester and help you in building your testing career and gear up yourself in DevOps world. Read our article on “An Introduction to DevOps Approach” for a better view of the DevOps approach.

How DevOps has changed testing?

In a traditional model, the developers, testers and operations, all were separate and worked in isolation. But DevOps culture brings all these groups under the same umbrella and calls for a high level of coordination between the groups to enable release of the code faster and more often. DevOps is all about Continuous development, Continuous Testing, Continuous Integration, Continuous Deployment, and Continuous monitoring. As you can see, in DevOps culture, it is “continuous testing”. In traditional approaches QA or testing acted as a gatekeeper between development and the release of a code and testing was carried out at the end of the release cycle. But in DevOps, testing takes a shift left approach where testing is done early in the release cycle and requires testers to take responsibility for quality across the entire development lifecycle. In DevOps, the testing mantra is “Test early, more often and continuously”.
DevOps, as you know, focus on continuous delivery which is achieved as a result of the developers, QA, Operations all working as a single team and being held responsible for everything in the whole cycle. So, the call on QA in the DevOps world is greater agility, continuity, and collaboration with developers, operations, release engineers, system admins, etc. It can be said that testing in DevOps spans the entire software development and delivery lifecycle. In DevOps, testers are not only responsible for verifying the functional/ performance tests, but the responsibilities extend to production as well. So the testers need to work together with the development team in testing the code committed into the Continuous integration system and also work with operations team to collaborate on monitoring; perhaps to continuously run smoke tests in production, monitor and analyze production data and logs and so on.
And of course automation! There is a great emphasis on automation in DevOps and in fact, every action in the DevOps chain need to be automated. It is all about automated builds and automated delivery pipelines using continuous integration tools such as Jenkins. To achieve the speed and agility required for the DevOps environment, it is highly critical to automate all testing processes and also to configure them to run automatically once the deployment is completed in the QA environment. Right from unit tests to integrated build tests, smoke tests, acceptance tests, systems tests, regressions tests- automating the various tests helps the teams in each stage of the cycle to benefit in terms of time, cost as well as quality. Building a mature automation testing framework is also very essential to have an efficient test creation process in place which enables the creation of well structured and clean test cases quickly. In a DevOps model, the choice of tools and various technologies become ever so important. For example, shift left testing techniques for testing like TDD, ATDD, BDD, etc which gives more importance to automation need to be adopted for better efficiency in a DevOps environment.
Automation and more automation can be considered the essence of successful DevOps cycles. But at the same time, every tester should know that not everything can be automated or should be automated in testing. ‘Too much automation is not good for testing’ stands true in DevOps as well. Manual testing of certain scenarios and testing techniques like exploratory testing for better efficiency need to be incorporated wherever apt.

Skills Required for Testers/QA in DevOps world

We have so far seen how the responsibilities of a tester have increased in DevOps chain as compared to the traditional models where testing was seen as a phase where testers owned testing strategy and were largely manual. Hence, testers in the DevOps world need to equip themselves with more skills and should have knowledge of the DevOps tools to strive in a DevOps world. They need to be able to understand the application better, prioritize testing using predictive analytics wherever necessary, understand tools including virtualization tools better, use automation effectively and enable continuous delivery and work effectively to deliver quality beyond just functional testing. Testers have to be ready to wear hats of multiple roles like Core testers, Automation testers, Software Developer Engineer Test (SDET) who requires testers and developer skills for TDD, BDD, etc, Specialized testers (performance testing, security testing, etc), operations testers, business users testers and so on.
Here is a summary of skills testers are expected to have in a DevOps world.

  • Good communication skills.
  • Knowledge of Continuous Integration/Continuous Delivery tools.
  • Knowledge of work management tools such as JIRA, Confluence, git, etc.
  • Automated testing skills like Selenium, JUnit, Cucumber, etc.
  • Knowledge of testing approaches like TDD, ATDD, BDD, etc.
  • Knowledge of performance testing tools (JMeter, Gatling, etc.),
  • Resilience testing, Operations testing, etc.
  • Knowledge of IaaS/PaaS platforms such as AWS (Amazon web services), Docker, Kubernetes, etc.
  • Basic networking knowledge.
  • Basic Unix/Shell scripting.

This was just a list of skills I could think of. Most importantly you always need to keep yourself updated with the skills necessary to keep in pace with changes in the software development world. Waterfall gave way to V model, which now has been replaced by agile as the preferred choice and now, DevOps or a mix of DevOps and agile is in trend. DevOps/Agile is currently the future of Development processes as well for now. Whatever be the future, it definitely is going to demand even more agility and pace of delivery. Hence, testing processes and tools also need to evolve accordingly to be fast and dynamic as the development processes.