Note: The job is a remote job and is open to candidates in USA. Trilon is building a supercharged, technology-enabled future for their people and partners. The QA Engineer plays a critical role in ensuring that every AI-powered tool meets high standards for quality, reliability, and usability, while also being responsible for the full quality lifecycle and collaborating closely with engineers and product managers.
Responsibilities
- Own the full quality lifecycle for your product pod, from test planning through production sign-off
- Define acceptance criteria in partnership with engineers and product managers
- Design comprehensive test strategies that cover functionality, performance, and reliability
- Establish and maintain quality standards for all deliverables
- Design, build, and maintain automated test suites for APIs, services, and user interfaces
- Develop regression tests to detect issues as systems evolve
- Integrate automated testing into CI/CD pipelines to ensure continuous validation
- Write clean, maintainable test code that scales with the product
- Design and implement testing approaches for AI-generated outputs, including prompt evaluation and RAG validation
- Define evaluation criteria and benchmarks for accuracy, consistency, and usefulness
- Monitor for quality drift as models, prompts, or data sources change
- Partner with engineers to improve output reliability and performance
- Identify, document, and track bugs and quality issues across the stack
- Work closely with engineers to troubleshoot and resolve defects early in the development cycle
- Prioritize issues based on impact and risk to end users
- Validate that all features meet acceptance criteria and quality standards before release
- Assess production readiness, including performance, reliability, and edge cases
- Serve as the final sign-off authority for releases within the pod
- Partner with Lead Engineers, Applied AI Engineers, and Product Managers to improve development and testing practices
- Engage with operating company users to understand real-world expectations and feedback
- Continuously improve testing methodologies, tools, and frameworks
Skills
- 4+ years of experience in QA engineering, software testing, or a related role
- Strong experience with test automation frameworks and tools for API, backend, and UI testing
- Proficiency in at least one programming language such as Python, JavaScript, or similar
- Experience building and maintaining automated test suites and integrating with CI/CD pipelines
- Experience testing APIs and working with REST or similar architectures
- Familiarity with modern software development practices, including version control and agile methodologies
- Experience testing AI-enabled applications, including LLM outputs, prompt behavior, or RAG pipelines
- Ability to define evaluation criteria for non-deterministic systems and assess output quality
- Strong analytical and problem-solving skills, with attention to detail
- Ability to clearly communicate quality risks, tradeoffs, and recommendations
- Experience collaborating closely with engineers, product managers, and cross-functional teams
- Comfort working in a fast-paced environment with evolving requirements
- Curiosity and interest in understanding engineering workflows within the AEC industry
Benefits
- Fully remote
- As a People First company, we are focused on growing the careers of our people faster within the Trilon group than our peers.
- We invest heavily in developing and elevating talent across our family of companies.
- Trilon Group offers a multitude of career paths, spanning technical, project management, business management, business development, and operations.
Company Overview
Company H1B Sponsorship