When testing gets cheap, what's left for me?

For years, the testing community has talked about shifting left, building quality in from the start and making quality a shared responsibility. On my projects, that's often how things begin. Over time, though, QA drifts into the testing column. I end up waiting for tickets, checking them against the acceptance criteria and deciding whether they can move on. That leaves little time for the work I enjoy most, and that I'd argue brings the most value, like exploratory testing, planning, getting involved early and staying involved throughout the project.

AI could change that. The mechanical, repetitive parts of the job are exactly what AI is getting really good at. I've used agents to run through apps, translate acceptance criteria into test cases, write bug reports and run smoke tests. Handing over these parts of the job to AI could free up time for the work that brings the most value.

After four years as a QA engineer at ustwo, across web, native and React Native projects, I don't think the answer is to defend the testing column.

QA's value is no longer in checking work. It's in shaping the why, what and how of quality, and building the systems that let a team deliver it, even when I'm not in the room.

Checking is getting cheap. Judgement isn't.

James Bach and Michael Bolton make a distinction I find useful. Checking is confirming that the product does what we expect, the kind of work a machine can do. Testing is the wider work of learning about the product by exploring and experimenting with it. That first part is what we can hand over to AI.

Agentic tools can now drive apps directly through browsers and simulators. Recently, when the team asked me for a quick check of a new feature, I gave Claude a three-line prompt to test an app's companion website, where customers buy subscriptions. It chose its own setup, with a headless browser at desktop and mobile screen sizes and automated accessibility scans. It found real bugs, like a price on the page that didn't match the price at checkout, and several accessibility issues. It covered more ground than I would have in the same time. We shared its report with the team, and once we'd triaged the issues, the agent helped me create Linear tickets for the ones we agreed to fix.

It wasn't flawless, though. It found that you couldn't choose a subscription plan with the keyboard, and concluded that keyboard users couldn't get through checkout at all. It rated that High and made claims about Safari, a browser it had never tested. In fact, the subscription plan tiles only selected an option, and a separate button handled navigation. Agents can be confidently wrong when they lack context.

AI-generated tests raise a related problem. On one recent project, test automation started arriving as part of developers' pull requests, with AI generating UI, integration and end-to-end tests alongside the code it was testing. Because the code itself was developed with AI, generating the tests too took almost no extra effort. That can of course be convenient, but if developers and AI decide what to test without a shared view of what matters, and without QA's lead, we lose sight of what's actually being tested.

When the code and its tests are generated from the same acceptance criteria, they share the same interpretation, so a misunderstanding in one is built into the other. An AI-generated test can verify a wrong interpretation just as efficiently as a correct one. High coverage doesn't automatically mean high quality.

This takes us back to a classic question. Who watches the watchmen? When AI writes the tests and runs the checks, someone has to check that the checks are right. One way is to make QA a code owner of the UI test folders, so that every pull request that changes them automatically asks for a QA review. Documented guidelines for how those tests are written and reviewed keep that review consistent.

Of course AI is a threat to the way QA has traditionally worked. If our value is counted in tests executed and tickets verified, we're competing with automation that improves every month. I'd rather stop competing, explore the possibilities AI brings and make it work for us.

What this looks like in practice

The companion website I had Claude test belongs to a native iOS app I work on. On that project, there have been periods without QA on the team. Rather than leaving a gap when I'm away, I've been building things the team can run without me.

One clear example is our pre-release smoke test. I've set up a template for it in Notion, together with an agent prompt that pulls the diff between the current and previous release tags on GitHub and updates the smoke test to match what has actually changed. Our product lead now uses it to run the smoke tests himself when I'm not there. I decided what to test and why, and the team runs the checks.

Not all of it is about tooling. Early on, we wrote an ADR, an architecture decision record, for accessibility. Product, design and development agreed to treat accessibility as a requirement rather than an extra, with WCAG 2.1 AA as the baseline and keyboard and screen reader support from the first release. We also agreed on what could wait until later. It took some work, but it meant we didn't have to have the same discussion again with every feature, in development or in testing.

Closer to my own day-to-day, I've also let Claude drive the iOS simulator to run its own smoke tests. It worked really well, reporting back with detailed screenshots and test reports. It's turned out to be a good complement to have running on the side while I do manual regression testing.

None of this took away the need for QA. Someone still had to decide what to test and why. Instead, it changed where my time went. I spent less time executing checks and more on deciding what should be checked, building the tools to do it and reviewing the results. It also freed up time for what we as QAs do best. Exploratory testing is a big part of that. It means exploring the product with curiosity, following hunches and noticing when something feels off, even when every acceptance criterion passes. And asking whether those criteria were good enough in the first place.

Stepping away comes with a cost

In agency work, projects vary enormously in scope, complexity, length and budget. In my experience, QA staffing often ends up on or off. A QA engineer is part of the team, then removed, then sometimes brought back later, depending on the project's needs and its budget.

On that same project, during one of those periods without QA, a significant new feature came in, and I was brought back at 50% for two weeks to help test it. That turned out to be harder than it sounds. It took time to get back up to speed. I had to relearn the system and find my way back into the team's ways of working and pace. Maybe most importantly, I had to catch up on the context I'd missed while I was away.

That context is exactly what I need to ask whether we're building and testing the right things. And it's the first thing that's lost when QA steps away.

What helped was AI. I pulled the epic from Linear and had an agent read all of it, including the comments and discussion on the tickets, and structure it into one testable suite. That helped me grasp the full picture quickly, which mattered more than the 81 test cases it also produced. It only had what was written down, though.

A lot of small decisions and clarifications happen in 1:1s, in meetings or in passing, and get acted on without ever being documented. Our product lead told me that on another project, the team recorded and transcribed every meeting. The transcripts still missed the conversations in between, so the AI never had the full picture.

An agent can help me get caught up, but not on conversations it was never part of.

QA, redefined

I think quality expertise should work more like technical leadership. A tech lead doesn't write every feature to shape a project's technical direction. A QA engineer shouldn't need to run every test to shape a product's quality.

I don't think this needs a new title. Consultant, coach, lead and partner all come with their own baggage. It's still QA, but with a clearer idea of what the role is for, and most of it is already in the examples above.

The accessibility ADR is what being involved from the start looks like. Coming back at 50% demonstrated the counterpoint, how much context is lost when QA is switched off and on again. Even with low daily involvement, staying in product discussions keeps that context alive, so that QA can step up when the risk goes up, for example with new automation, complex functionality or a change in scope.

The smoke test is the kind of system I think QA should build, something the team can run without me. Test strategies, instructions for AI agents and acceptance criteria linked to test cases belong in the same category. And the agent that rated a keyboard issue High and made claims about a browser it never tested is why someone still has to check the checks. QA should review what AI and automation produce and help the team understand how far that evidence can be trusted.

This only works if the whole team shares ownership of quality. The team owns the responsibility for delivering with quality and acting on shared decisions. QA brings the expertise. In this model, QA spends less time on hands-on checking day to day. The team runs the checks, often with the help of agents, and acts on what they find.

And quality is more than technical correctness. It's also whether we're solving the right problem, whether the experience meets expectations and whether the team understands the consequences of its decisions. In the teams I've worked in, QA sits in the tech team, alongside the developers. Yet when I was away from the project, what our product lead said he missed was having "someone who knows the full breadth of the product in a way that a front-end engineer, back-end engineer or (to a lesser degree) designer doesn't necessarily need to." I think quality expertise should work across product, design and engineering, rather than act as development's final checkpoint.

That's also where the savings are. A misunderstanding caught in refinement costs a conversation. The same misunderstanding caught after release can cost a hotfix, support time and the client's trust. AI can make the checking cheaper, but it doesn't make a late problem any cheaper to fix.

What's left for me

I don't believe AI will make us QA engineers obsolete, at least not yet. But it does change how we justify our place in software teams. Checking is easy to count, but the problems good QA prevents are hard to measure. When we do our job well, you hardly notice us. I would argue AI makes human QA more important than ever. The more a team trusts AI with the checking, the more it needs someone asking whether that trust is earned.

So what's left for me when testing gets cheap? Strictly speaking, testing isn't getting cheap. Checking is. What's left is the part of the job that matters most. That means exploring the product the way only a curious human can, helping the team understand what we're building, whether we're testing the right things and how far we can trust the answers. That's what the testing community has been talking about for years, and AI finally gives us the time to do it.

Note on AI

This article was developed through conversations with Claude and ChatGPT, which helped me challenge my assumptions, explore related ideas and structure the text. The experiences, opinions and central arguments are my own.