Quality engineering strategy

In the world of agile quality, there are many obstacles that an R&D group needs to overcome, and for each of them, there are multiple solutions.

From my experience in the quality realm, R&D groups often lack knowledge of what is needed, how to implement it, and why it’s important. They also lack experience in how quality works and how it can be a trusty companion to high-quality products when the right mindset, processes, and tools are adopted.

Let’s take, for example, an R&D group working according to the agile methodology but lacking automation. The first step is typically to embed manual QA testers into the agile team. Suddenly, these testers (often just one person) are responsible for the team’s quality. As the team continues to deploy new features, the task of covering a full regression becomes increasingly difficult, and the quality phase often becomes a bottleneck for releases. If not managed properly, frustration grows, and important tests are omitted due to lack of time, introducing new production defects and creating a vicious cycle.

To address this, R&D often hires a QA automation manager to take the necessary steps to create an automation infrastructure/framework. This framework will be used by R&D or a dedicated automation team to develop automated test cases. However, this approach often fails to account for new challenges that may arise due to lack of vision and planning:

1. Regression failures due to maintenance or feature changes in existing code, where automation maintenance wasn’t considered. This produces false negatives, requiring additional work that wasn’t included in the initial planning, leading to a growing backlog.
2. End-to-end (E2E) testing issues, where the focus is primarily on UI automation (mirroring manual QA processes), introducing new frustrations:
   1. Long testing cycles: E2E testing is time-consuming to build and execute, even when running tests in parallel. This time consumption grows as more features are introduced.
   2. Bug fixing: As E2E testing treats the system as a black box, test failures don’t indicate where in the system layers the bug appears, leading QA and developers to spend hours investigating.
   3. Test flakiness: Inconsistent test results due to network issues or configuration problems erode developers’ and quality personnel’s faith in the automation infrastructure.
3. There are additional issues that may arise from relying solely or mostly on E2E testing.

The next step is often to explore a multi-layered automation solution, which introduces new needs and challenges. However, it doesn’t have to be this way. Overcoming quality issues in patches leads to high costs, time consumption, and frustrations. Understanding the needs, pains, obstacles, and the mindset that needs to be adopted is crucial. Planning a strategy that suits a specific R&D group is essential in the path to achieving a high-quality product.