Exploring how quality engineering works when AI is a first-class collaborator.
🇨🇳 中文 | 🇬🇧 English
AI-Native QA Lab is an open exploration space for the next generation of quality engineering. We study the ways people, AI agents, automation, and engineering tools can work together to build software that is more reliable, observable, secure, and useful.
This is not a checklist repository or a collection of AI-generated test cases. It is a place to investigate a complete way of working: how quality is understood, designed, tested, evidenced, and improved across the software lifecycle.
AI-native QA does not mean adding an AI tool to an existing testing process. It means redesigning quality work around three principles:
- Quality is continuous. Quality risks are explored from product discovery through operation, not postponed until a testing phase.
- AI is a collaborator, not an authority. AI can accelerate exploration, reasoning, generation, and analysis, while humans remain accountable for context, risk, and decisions.
- Evidence drives confidence. Claims about quality should be connected to observable evidence, reproducible checks, and explicit uncertainty.
The goal is not to remove people from QA. The goal is to help teams spend more time on judgment, risk discovery, and product learning—and less time on repetitive coordination and low-value execution.
Our working model connects five activities:
- Understand the product — clarify users, business outcomes, system boundaries, and high-risk assumptions.
- Model quality — turn risks and expectations into quality characteristics, questions, oracles, and acceptance signals.
- Explore with AI and tools — use agents, prompts, automation, and domain-specific tools to generate hypotheses and investigate behavior.
- Verify with evidence — combine human review, automated checks, production signals, and traceable artifacts.
- Learn and improve — feed findings back into requirements, design, workflows, tools, and team practices.
Product context → Quality model → AI-assisted exploration → Evidence → Learning loop
AI may contribute at every stage, but responsibility and approval boundaries must remain visible.
- Risk-based workflows from idea to production
- AI-assisted test design and exploratory testing
- Agentic workflows with clear human checkpoints
- Quality gates that account for freshness, coverage, and evidence
- Faster feedback without confusing activity with confidence
- Testing agents and reusable QA Skills
- Prompt patterns, context packaging, and evaluation methods
- Browser, API, contract, security, performance, and accessibility automation
- Test data, environments, observability, and failure triage
- Evidence bundles, decision records, and quality reports
- Evaluating AI features beyond traditional functional correctness
- Reliability, safety, privacy, security, and misuse resistance
- Non-determinism, model drift, prompt changes, and dependency risk
- Human-in-the-loop design and graceful failure
- Measuring usefulness, trustworthiness, and operational impact
- Start with risk, not test-count targets.
- Prefer small, inspectable workflows over opaque autonomy.
- Keep prompts, context, tools, outputs, and decisions traceable.
- Treat generated output as a hypothesis until it is verified.
- Design for failure, ambiguity, and change.
- Use automation to increase learning, not merely execution volume.
- Share reusable patterns while documenting their limits.
This lab is for people exploring the intersection of quality and AI, including:
- QA engineers, SDETs, and quality engineers
- Software engineers and platform engineers
- Product managers, designers, and engineering leaders
- AI, agent, and developer-tool builders
- Researchers and practitioners interested in reliable software delivery
You do not need to use a specific AI provider or testing framework. The ideas should remain useful across tools and technology stacks.
As the lab grows, it may contain:
- Workflows — repeatable ways to perform AI-native QA work
- Skills and prompts — reusable instructions and task-specific guidance
- Tools and prototypes — experiments that make the workflows practical
- Evaluations — ways to measure quality, usefulness, and failure modes
- Examples and case studies — concrete applications and lessons learned
- Notes and principles — the reasoning behind the practices
Early material may be experimental. Each contribution should make its scope, assumptions, evidence, and known limitations clear.
Good contributions do more than add another tool or prompt. They help answer questions such as:
- What quality problem does this solve?
- Where does it fit in the workflow?
- What context and permissions does it require?
- Which parts must remain human-controlled?
- How do we know it works?
- Where can it fail or create misleading confidence?
Start with a focused issue, experiment, workflow, or written observation. Prefer small reproducible examples over broad claims.
This repository is an evolving laboratory. The practices, workflows, and tools are expected to change as experiments produce better evidence. Stable guidance will be separated from exploratory work as the body of knowledge grows.
License and contribution details will be added as the repository develops.