The headline number from this year's World Quality Report 2025-26 is not the adoption rate. It is the gap. Around 89% of organisations are piloting or deploying generative AI in quality engineering, but only 15% have scaled it across the enterprise. That gap is not a tooling problem. It is a quality problem, and manual testers are the people best placed to close it.
What the 2026 data actually says
The pilot-to-scale gap is consistent across the major industry reports.
Capgemini's World Quality Report finds the top barriers to scaling are secure test data (cited by 60% of teams) and the difficulty of adopting AI-powered tools (58%). Synthetic data has now become the top Gen AI use case in testing, rising from 14% adoption in 2024 to 25% in 2025.
Applause's State of Digital Quality 2026 adds a sharper signal. Of organisations that released live AI features, 44.1% have since deactivated at least one because operational cost outpaced user value. Forty per cent of consumer respondents reported encountering an AI hallucination in the same period. Teams are shipping AI, then quietly pulling it back.
This is what the gap looks like in practice. Pilots run. Features ship. Quality fails to hold. Something has to catch the gap, and the available evidence points squarely at human-led evaluation.
Where manual testers are the limiting reagent
In Applause's data, 60.8% of organisations rely on human-led evaluations for AI features. A further 46.5% use human sentiment and usability to decide whether a feature is production ready. Only 32.8% currently use AI-assisted functional testing for AI feature assessment. The dominant verification method for AI is still a person.
The reason is structural. AI systems are probabilistic, not deterministic. The same input can produce different outputs. Coverage-driven test suites cannot establish confidence on their own. What works is risk-based assurance, where a tester defines the failure modes that matter, then designs evaluations against them.
This is exactly the territory experienced manual testers already operate in. Boundary analysis. Exploratory testing. Edge cases that requirements documents never mention. Behavioural drift across multi-turn conversations. Bias detection in outputs. None of these are tasks AI agents handle reliably without supervision.
A practical methodology manual testers can adopt now
Three practices are emerging as the working pattern across teams that have moved past pilots.
- Editor-in-Chief reviews of AI-generated tests. AI tools generate test cases quickly. Their weakness is missing negative scenarios, boundary conditions, and domain-specific risks. A human reviewing and rewriting before tests enter the suite is the difference between a noisy backlog and a useful one.
- Structured evaluation frameworks. Rather than pass-fail assertions, AI features are scored against accuracy, relevance, completeness, clarity, and tone. This converts subjective judgement into a repeatable scoring system that any manual tester can run.
- Continuous adversarial probing. Red teaming, jailbreak attempts, and prompt manipulation tests cannot be a single pre-release event. Agent behaviour shifts as models update and tool sets change. Schedule it like regression testing.
If you are designing an evaluation framework for AI features and want a structured way to define risks, scoring rubrics, and human review steps, EvalSpec is built for exactly this.
Try EvalSpec free - evaluate your AI system across all 5 dimensions. No code required.
Start your free evaluation