Chief Agentic Quality Architect

EVA is in the business of acquiring enterprise software companies and transforming them into AI-native, profitable, scalable businesses. By prioritizing Customer Success, we enable companies with strong product-market fit to emerge as market leaders through sustained product innovation and high-quality professional services delivered via our global team. We are a remote-first company and our team is spread across the globe. 

WHY JOIN EVA 

We offer: 

• High ownership, flexibility, and impact in a fast-moving environment. 

• Opportunity to work across multiple products, industries, and technologies 

• Growth pathways across engineering, product, operations, and leadership 

• Remote-first flexibility 

ROLE OVERVIEW 

As our Chief Agentic Quality Architect, you will orchestrate the transition from traditional scripted testing to an AI-augmented quality ecosystem. Your primary goal is to leverage agentic AI tools to generate, execute, and maintain high-fidelity test suites that keep pace with a development environment where AI agents are writing a significant portion of the production code. 

WHAT YOU'LL DO 

1. Assess & Benchmark Quality Coverage 

• Take ownership of a new product or suite of products as a hands-on quality leader. Conduct a comprehensive audit of the existing test estate across all layers — unit, integration, API, UI, sanity, and regression — evaluating depth, coverage, and reliability of each suite. 

• Evaluate the current QA tooling, frameworks, and automation maturity against EVA’s quality benchmarks and coverage expectations. Identify systemic gaps, test debt, and high-risk areas with no automated coverage. 

• Produce a Current State & Gap Coverage Report that maps existing tooling, highlights missing coverage domains, and recommends the adoption tooling needed to close the gaps — the baseline for every roadmap that follows. 

2. Define the Quality Engineering Roadmap 

• Define missing test cases and user journeys. Prioritise end-to-end automation across critical business flows using agentic automation patterns, with a deliberate emphasis on left-heavy coverage — maximising unit and integration depth first, and extending rightward through black-box UI and regression testing. 

• Architect and own the phased quality engineering roadmap, structured as three delivery horizons: 

• Two-Week Plan: Complete the current-state assessment; identify the highest-risk coverage gaps; stand up quick-win automation on the most critical user journeys; evaluate and select the tooling to be adopted. 

• One-Month Plan: Core regression coverage established across primary domains; CI/CD quality gates operational and integrated into the development pipeline; agentic test generation producing its first validated suites. 

• Three-Month Plan: End-to-end agentic automation live and self-maintaining; EVA benchmark coverage achieved; AI guardrails enforced across all automated agent output; quality metrics visible to engineering leadership. 

3. Agentic Test Generation & Automation 

• Prompt-Based Engineering: Utilise AI agents to automatically transform business requirements and manual test cases into executable Playwright or Cypress scripts. 

• Synthetic Test Creation: Implement tools that autonomously generate test data and edge-case scenarios that human testers might overlook. 

• Autonomous Maintenance: Deploy self-healing automation frameworks that use AI to detect UI changes and update test selectors without human intervention. 

4. Guardrails for AI-Generated Code 

• Agent Regression Strategy: Design and own a comprehensive regression suite specifically tuned to catch the non-deterministic “hallucinations” or logic errors common in AI-generated code. 

• Behavioral Locking: Implement characterisation testing patterns to “lock in” the expected behaviour of legacy systems during AI-assisted refactoring. 

• Autonomous Output Validation: Define quality gates to validate the outputs of autonomous agents, ensuring they meet functional, security, and performance boundaries. 

5. Build & Mentor an AI-Skilled QA Team 

• Hire QA engineers with deep, hands-on capability in AI-assisted testing and agentic automation frameworks — not generalists, but specialists who can design, implement, and operate AI-driven quality systems in production. 

• Mentor and upskill the team on EVA’s agentic workflows, guardrail frameworks, and quality benchmarks. Set the technical bar and hold the team to it through code reviews, pairing, and structured coaching. 

• Instil a quality-first engineering culture where QA is embedded throughout the development lifecycle — not a downstream gate — and every AI-generated change is treated as a quality risk until validated. 

WHAT WE'RE LOOKING FOR 

• Automation Frameworks: Expert-level proficiency in Playwright, Cypress, or Selenium. 

• AI Tooling: Hands-on experience using LLMs (Claude, GPT-4, etc.) and agentic frameworks to generate code or automate workflows. 

• CI/CD Mastery: Deep understanding of integrating quality gates into AWS-based pipelines or similar environments. 

• Architectural Mindset: Ability to design "Behavioral Snapshots" to safeguard critical business logic during rapid transformations. 

• 10+ years in QA automation engineering, SDET, or test architecture roles 

NICE TO HAVE 

• Experience in remote-first or globally distributed engineering teams 

• Background in SaaS scale-ups, legacy modernization projects, or PE-backed environments 

• Exposure to performance, load, or security testing tooling 

• Comfort operating in high-ambiguity, high-ownership environments with evolving requirements