Current state-of-the-art in QA

Since the 90s, automated software testing has aimed to simplify the repeated workflows conducted by manual testing. This has opened up the position of QA engineering, a role that sits close to the development team to build and maintain automations. Today, this role plays a crucial part in the software development lifecycle.

As software has grown more complex, so too have the frameworks used to test it. With the creation of frameworks like Playwright and Cucumber, the testing industry has made browser automations accessible to technical and non-technical stakeholders. Most companies have found, however, that scaling QA operations still requires scaling a large QA team, despite these frameworks.

This is primarily because the process of creating and maintaining a test is largely manual or engineering-intensive.

AI Agents

AI agents have received a lot of attention in the past few months. They aim to bring human-level functionality to tasks via planning, reasoning, and a working memory. As foundational large-language and large-action models improve, the applications of these agents grow more complex.

Some notable applications of AI agents include:

1. Workflow automation

2. Data extraction

3. Personalized user experiences

Many industries will shift their processes to leverage AI as models and architecture improve. It’s easy to imagine that something similar could happen in the QA world.

Applying LLMs to QA

Currently, QA engineers need to brainstorm test cases and variations, build the initial automated script, and maintain the entire testing suite as the product evolves.

Which parts of this could benefit from AI?

Test maintenance is often a problem of finding the right locator. These locators are fragile, leading to flaky or inconsistent tests. Rather than immediately fail a test when a locator doesn’t resolve, large-language models can use the original intent behind the test (natural language), the locator error, and the current web page to come up with a new locator. In this way, tests can dynamically heal themselves.

Coming up different variations of an existing test involves numerous, time-consuming tweaks. A well-constructed agent could come up with test variations that would require a “yes” or “no” from a human-in-the-loop. This reduces the process of increasing coverage to a simple decision that virtually anyone, technical or not, can make.

Impact on QA Industry

AI is still just a tool to enhance existing workflows. While agents are particularly good at exploring potential avenues for testing and evolving to learn more about a testing suite, they still lack a crucial quality for effective testing — good judgement and intuition. Humans are still the best judge of “good” vs. “bad” outcomes on a product. This requires a nuanced understanding of intent, one that an agent can’t quite grasp yet.

AI will enable engineering teams to automate all the tedious components of QA and shift quality left. It will help democratize access to end-to-end testing, so every company, no matter how big or small, can build quality into their product from day one.

There are plenty of fragmented QA frameworks that assist with parts of the QA lifecycle. Contour brings all end-to-end testing under one AI-powered platform. If you’re interested in learning more, book a demo with us!