AI Machine Learning Engineer

About the role

Join the Research and Development team of a leading European tech company and work on cutting‑edge AI engineering projects. You will design and implement testing tools and pipelines that ensure AI agents and applications meet rigorous quality standards.

Key responsibilities

• Design and build testing harnesses to evaluate AI agent behavior, tool use, and output against defined quality criteria.
• Develop and maintain a central quality context layer, including shared datasets, golden traces, evaluation sets, and metadata.
• Implement quality gates in CI/CD pipelines for both agent and application code to block regressions.
• Define, track, and visualise KPIs such as accuracy, latency, hallucination rate, task success, drift, and reliability.
• Create EL(T) pipelines to collect, transform, and store test data, logs, and evaluation results.
• Collaborate with AI/ML engineers, developers, data engineers, and product stakeholders to define quality expectations and automate checks.
• Continuously improve test coverage, harness reliability, and evaluation methodology as capabilities evolve.
• Document testing frameworks, standards, and usage guidelines for the broader engineering team.

Required profile

• Minimum 2 years of experience in AI/ML engineering.
• Hands‑on experience building or using testing frameworks, harnesses, or evaluation pipelines for traditional or LLM/agent‑based systems.
• Familiarity with CI/CD and DevOps practices, including automated quality gates.
• Experience with at least one major cloud platform (AWS, Azure, or GCP).
• Understanding of large language models (LLMs) and AI agents.

Required skills

• AI/ML engineering
• Testing frameworks and harness development
• CI/CD pipeline integration
• DevOps practices
• Cloud platforms: AWS, Azure, GCP

What we offer

• Opportunity to work in an agile, innovation‑focused environment.
• Access to cutting‑edge AI projects and scalable challenges.
• Collaboration with multidisciplinary engineering and product teams.