Deze documentatie wordt actief uitgebreid — kom regelmatig terug.

ADR-00015: End-to-End Browser Test Tooling — Playwright + Chromium, Failsafe-bound

Approval Status

Status flow: Proposed → Accepted → Reviewed → Approved (or Rejected), or Not Required. A superseded ADR keeps its file; only its status changes to Superseded by NNNNN.

Governance body	Status	Last update
CTN Project Team	Proposed	2026-06-29
CTN Technical Advisory Board	Pending	—
CTN Steering Committee	Pending	—
BDI Conformance Team	Pending	—
BDI Framework Team	Pending	—

RdN: This ADR is about E2E testing. Playwright, chromium is fine for me. I would like to understand how the overall testing strategy will look like. For instance in the previous prototypes I have used testcases for API testing, which I would like to re-use and submit into testsuite. I also have E2E tests, obviously they all need to be altered a bit, so they will work with this solution. My goal is that we (product owner, end-users etc.) can add testcases, which will be added to the testsuite and which will be tested every time we deploy. Yup, old fashioned Test Driven Development, you never have enough test cases, regression issues will surface rather sooner than later dadadadidadida.

Context and Problem Statement

The eHerkenning control step-up (ADR-00014) is a browser-driven flow: the asr-demo-frontend applicant-edit.html page runs OIDC code + PKCE through keycloak-js, drives a Keycloak Application-Initiated-Action idp_link round-trip to the simulated eHerkenning broker realm, then POSTs the brokered token to the complete-token endpoint and reflects the recorded control attestation.

The existing EHerkenningStepUpBrokerIT (issue #45) proves the broker + token chain at the HTTP/token layer only, via RestAssured and the Keycloak admin client — no browser. Nothing exercises the frontend’s OIDC-PKCE init, the AIA idp_link confirmation, or the page’s complete-token POST + reload. That gap let frontend/OIDC behaviour pass mvnw verify green-but-broken. Issue #67 closes it with one real-browser tracer-bullet test, which needs a browser-automation framework wired into the build.

Scope. This decision is about e2e test tooling only. It is explicitly not a resolution of the open frontend-stack question for the ASR portals (tracked as an Open Question in ctn-coding-standards.md). The asr-demo-frontend is a throwaway static demo (plain HTML + vanilla JS, no build step), not the portal stack.

Relationship to other test ADRs. This ADR covers only the tactical, in-repo Java e2e tier — one real-browser test bound to the backend’s Maven build. The strategic UI-automation framework for the real portal (Playwright + TypeScript, TAaaS-owned) is a separate decision: ADR-000017. Both tiers sit under the overall test strategy in ADR-00023, which explains why the Java (here) and TypeScript (00017) toolchains are a deliberate per-layer split rather than a duplication.

Considered Options

Option 1: Playwright for Java + Chromium, bound to Maven Failsafe

Description: Add com.microsoft.playwright:playwright (test scope, pinned) and drive headless Chromium from a @QuarkusTest *IT running under Failsafe.
Pros: Single-language build (Java/Maven, no Node toolchain); auto-installs its bundled Chromium on first launch; first-class request route-interception (used here to serve keycloak-js offline); integrates with the existing DevServices-Keycloak + Docker-gated test path.
Cons: Pulls a Chromium binary on the first Docker-host run; pins ports the static demo hardcodes.

Option 2: Cypress (or another Node/JS e2e framework)

Description: Run the e2e suite in a separate Node toolchain.
Pros: Popular in frontend ecosystems; rich DX for a real SPA.
Cons: Introduces a Node/npm build alongside Maven for a backend repo with no frontend build step; would not bind cleanly into the one Docker-gated ./mvnw verify path; heavier for a single tracer-bullet test.

Option 3: Selenium / WebDriver

Description: Classic WebDriver-based browser automation from Java.
Pros: Mature, Java-native.
Cons: Driver/browser version management is fiddlier; no built-in request interception (needed to serve keycloak-js offline); slower, flakier API than Playwright’s auto-waiting locators.

Decision Outcome

Chosen option: Option 1 — Playwright for Java + Chromium, bound to Maven Failsafe, gated exactly like the database integration tests.

Rationale

Keeps the build single-toolchain (Maven only) for a backend repo whose only frontend is a buildless static demo.
Reuses the existing two-realm DevServices Keycloak harness (EHerkenningStepUpBrokerProfile) and the Docker gate, so the e2e test runs in the same ./mvnw verify path as the @Tag("requires-database") suite and is skipped identically when Docker is absent.
Playwright’s request route-interception cleanly mitigates the one frontend dependency the test cannot otherwise satisfy offline (see Risks).

Consequences

Positive: Real-browser coverage of OIDC-PKCE + AIA idp_link that the HTTP/token-layer IT cannot reach; one toolchain; deterministic Docker gating.
Negative: A Chromium binary is fetched on the first enabled run; the test pins fixed ports (the static demo hardcodes Keycloak/API coordinates).
Neutral: A second e2e framework could still be chosen later for the real portal once the frontend-stack Open Question is resolved; this decision does not constrain that.

Build wiring and gating

com.microsoft.playwright:playwright is a test-scoped, pinned dependency (version in <properties>, not BOM-managed).
Each real-browser test class (EHerkenningStepUpBrowserE2EIT, EHerkenningAdminAndStepUpBrowserE2EIT, BulkImportAdminBrowserE2EIT) carries two JUnit5 tags: @Tag("requires-database") (it needs the DevServices Keycloak + Postgres stack) and the dedicated @Tag("browser-e2e") that marks it as a real-browser test. The browser-e2e tag — not a class-name glob — is the single discriminator every profile filters on, so adding or renaming a browser e2e class needs no build-config edit. They run via the Failsafe database-integration-tests execution. Because they bind Keycloak to a fixed port they must run exactly once; requires-database keeps them out of the default *IT.java execution (which would otherwise run every *IT a second time and collide on the pinned port).
Not run in CI (excluded from the pipeline). The real-browser tier is slow, fragile, and pins ports, so asr-service-ci.yml deliberately does not run it: the build job passes -Pci-skip-browser-e2e, which adds <excludedGroups>browser-e2e</excludedGroups> to the DB Failsafe execution, and there is no dedicated e2e job (an earlier one was removed). Run the suite locally on demand with ./mvnw -Pci-only-browser-e2e verify (scopes Failsafe to the browser-e2e group in a single execution — one Keycloak boot, no pinned-port collision; do not pass -Dit.test).
On a Docker host (-DskipITs=false) a plain local ./mvnw verify runs them alongside the DB ITs. A Docker-less run (-DskipITs=true) activates the no-docker profile, whose excludedGroups=requires-database excludes them — reported SKIPPED, not green, mirroring the DB/IT policy.

Harness shape

EHerkenningBrowserE2EProfile extends EHerkenningStepUpBrokerProfile, trusting the dev quarkus-realm RS256 key (the browser posts the realm-minted token, not the self-signed token-path key), and pins the e2e Keycloak/API ports with KC_HOSTNAME on the full Keycloak URL (split-horizon).
StaticFrontendTestResource serves asr-demo-frontend/ over a JDK HttpServer, rewriting the page’s hardcoded dev coordinates to the e2e ports at serve time (so a developer’s running quarkus:dev on the dev ports does not conflict) — no frontend source is edited.
The profile re-imports both realms from temp copies whose front-channel :8888 is rewritten to the e2e Keycloak port (back-channel :8080 untouched), so the browser broker redirect and the brokered token iss reach the test Keycloak.

Risk: keycloak-js CDN dependency (the one frontend-coupling risk)

applicant-edit.html imports keycloak-js from cdn.jsdelivr.net. A sandboxed/offline CI cannot fetch it, so keycloak.init would never run. The harness route-intercepts the CDN URL via Playwright and fulfils it from a vendored copy bundled as a test resource (e2e/keycloak-js-26.2.0.esm.js). This keeps the test offline-safe and edits no frontend file — the interception lives entirely in the harness. If the page’s pinned keycloak-js version changes, refresh the vendored copy to match.

Status History

Date	Status	Notes
2026-06-15	Proposed	Initial proposal (issue #67)
2026-06-29	Proposed	Added dedicated `@Tag("browser-e2e")`; real-browser tier excluded from CI (dedicated `e2e` job removed), now local/on-demand only