Why us? 🤝

We are part of the ALTEN Group – a leading European provider of engineering and technology consulting servics.

Join •••••••••••• and let’s grow together!

🌎Location: Kraków- hybird work (6 x month)

👉In this role, you will:

We are seeking an AI Evaluation & Development Engineer who will play a key role in testing, evaluating, and improving AI agents and GenAI applications while contributing to the development of supporting services and integrations.

Tasks associated with this role include:

  • Design, build, and maintain evaluation frameworks for AI agents and Generative AI solutions.

  • Define testing methodologies, benchmarks, and quality metrics for agent performance.

  • Evaluate AI outputs for accuracy, relevance, consistency, safety, and adherence to business requirements.

  • Identify and analyze hallucinations, failure cases, and context-related issues.

  • Create automated testing pipelines and regression testing strategies for GenAI applications.

  • Validate Retrieval Augmented Generation (RAG) systems, memory stores, and context-engineering mechanisms.

  • Collaborate with AI Engineers, Data Engineers, and Product teams to improve agent behavior and user experience.

  • Develop Python-based services and tools supporting evaluation and monitoring processes.

  • Analyze prompts and agent orchestration workflows to optimize performance and quality.

  • Support production monitoring and continuous improvement of AI solutions.

👉Key Skills and Qualifications:

  • Strong experience with Python development.

  • Hands-on experience testing AI, Machine Learning, or Generative AI systems.

  • Understanding of Large Language Models (LLMs), AI agents, and RAG architectures.

  • Experience creating test scenarios, evaluation datasets, and automated validation frameworks.

  • Knowledge of API testing and automation tools.

  • Experience working with cloud environments such as Azure or Google Cloud Platform.

  • Strong analytical mindset and ability to investigate complex AI behaviors.

🎯 We offer

We offer a full-time employment contract

  • Stable and long-term cooperation

  • Well-defined career path at the European leader in engineering & IT consulting

  • Opportunity to relocate and work in different •••••••••••• branches

  • After completion of the project, opportunity to engage in a subsequent one within the company

  • Introduction and cooperation with dedicated Competences Manager

🎯Benefits:

  • Medicover medical care

  • Medicover dental care

  • Medicover Benefits platform

  • Medicover Sport card

  • Referral Program

  • Layette for a newborn employee’s child

  • Group life insurance

  • Pension scheme

🎯We also offer cooperation based on a B2B or Contract of mandate (UZ)

🎯We also guarantee:

  • Access to professional certifications with our technology partners (e.g. Google, ISTQB)

  • Access to relevant industry events

  • Access to the Knowledge Sharing Program

  • Access to Referral program

Do not hesitate and join our team!

NOTICE: We kindly inform you that we will contact the selected people.

Do not hesitate and join our team!