This page is part of the ForgeSDLC knowledge base — an AI-assisted, human-directed methodology for taking product work from concept to production. For the core operating model and vocabulary, see Forge SDLC overview and What is ForgeSDLC?.

Test automation frameworks — making sense of the landscape

Purpose: Provide a clear mental model for anyone selecting test automation tools. This document explains what each framework is, where it fits in the test pyramid, and how to choose between alternatives. It is the tooling companion to APPROACHES.md (which covers testing strategies independent of tools).

Audience: Developers, QA engineers, architects, and PMs who hear "Playwright," "Selenium," "Espresso," "Cypress," and "Appium" in the same sentence and want to know how they connect.

1. The four-tier framework taxonomy

Every test automation framework fits into one of four tiers, aligned with the layers of the modern test pyramid in APPROACHES.md §3.

┌──────────────────────────────────────────────────────────────┐
│  4. AI-augmented testing                                      │
│     Tools that use AI to generate, maintain, or execute tests │
│     Playwright Agents · Mabl · testRigor · Katalon · Applitools│
├──────────────────────────────────────────────────────────────┤
│  3. E2E / UI frameworks                                       │
│     Automate browser or mobile UI for system-level testing    │
│     Playwright · Cypress · Selenium · Espresso · Appium       │
├──────────────────────────────────────────────────────────────┤
│  2. API / contract frameworks                                 │
│     Validate inter-service communication and API behavior     │
│     Pact · REST Assured · Postman/Newman · Karate · Supertest │
├──────────────────────────────────────────────────────────────┤
│  1. Unit / component frameworks                               │
│     Test isolated functions, classes, and components           │
│     JUnit · pytest · Jest · XCTest · go test · Vitest         │
└──────────────────────────────────────────────────────────────┘

BDD runners (Cucumber, SpecFlow) are not a separate tier — they are an execution harness that sits between specification (Gherkin) and any tier's test code.

2. E2E / UI frameworks (Tier 3)

2.1 Web / cross-browser

Performance benchmarks (2026): Playwright ~1,240 tests/hour; Cypress ~857 tests/hour; Selenium ~670 tests/hour.

2.2 Mobile

Android decision: If your project is Android-only (like many projects using this blueprint), Espresso is the natural first choice for UI testing — it shares the build system (Gradle), language (Kotlin/Java), and IDE (Android Studio). Add Appium if you later need iOS coverage or language-agnostic test authoring.

2.3 Desktop

Desktop testing frameworks are niche — include in your test plan only when your product has desktop components.

3. API / contract frameworks (Tier 2)

When to adopt contract testing: When your system has multiple services with independently deployable APIs. Contract tests catch breaking interface changes without spinning up the entire system — faster and more reliable than full integration environments.

4. BDD runners

BDD runners are not test frameworks themselves — they are execution harnesses that connect Gherkin specifications (.feature files) to step definitions implemented with any test framework.

Relationship to AURORA-IA: AURORA-IA uses Gherkin behavioral contracts as a core lifecycle artifact, not just a test format. The Test Agent generates .feature files; the Code Agent implements step definitions — see AI-NATIVE-METHODOLOGIES.md.

5. AI-augmented testing tools (Tier 4)

AI-augmented testing tools overlay on other tiers — they generate, maintain, or execute tests using AI capabilities. The industry has moved through three epochs: scripted (Selenium-era), low-code (2020–2024), and agentic (2025+).

Coding agents as test generators: IDE-integrated AI (Cursor, Claude Code, Copilot) can generate unit and integration tests from existing code or specifications. This is distinct from the purpose-built testing tools above — coding agents are general-purpose tools with test generation as one capability. See AI-TOOLS-AND-MODELS-LANDSCAPE.md.

6. Cloud infrastructure and device farms

Running tests at scale — especially E2E/UI and mobile tests — often requires cloud infrastructure for parallel execution and device coverage.

When to adopt cloud infrastructure: When local CI runners cannot provide sufficient browser/device coverage, when parallel execution is needed to keep pipeline times acceptable, or when physical device testing is required for mobile apps.

7. Selection decision matrix

Use this matrix to narrow down framework choices based on your project characteristics.

7.1 By platform

7.2 By team context

7.3 Key decision questions

8. Framework evolution timeline

Understanding the generational shift helps contextualize each framework's position.

Each era does not fully replace the previous one — Selenium (scripted era) is still the enterprise incumbent; Playwright (modern era) is the fastest-growing; AI-augmented tools are emerging as an overlay on both.

9. Authoritative sources & further reading

Hub: README.md · Strategies: APPROACHES.md · Lifecycle: SDLC.md §7 · AI tools: AI-TOOLS-AND-MODELS-LANDSCAPE.md