Social network you want to login/join with: AI Quality Assurance & GenAI Operations Engineer, pavia col-narrow-left Client: altermAInd Location: Job Category: Other - EU work permit required: Yes col-narrow-right Job Reference: 830012664429569638433713 Job Views: 2 Posted: 28.04.2025 Expiry Date: 12.06.2025 col-wide Job Description: Do you want to be part of a technology company with an AI-first approach, shaping the invisible by blending AI innovation with human experience? altermAInd is a company that offers solutions enabling organizations to overcome limits, uncover hidden opportunities, and drive transformative growth. Location: Flexible / Remote About the Role We are looking for a hands-on engineer (mid-level or senior, depending on experience) who will own both Quality Assurance for our AI initiatives and the day-to-day running of our cloud-native Generative-AI applications. You will design and automate test strategies for large-language-model (LLM) workflows, keep production GenAI services healthy, and contribute improvements using Python, Docker, and modern open-source frameworks such as LlamaIndex and AutoGen. What will you do? AI Quality Assurance Create, automate, and execute test plans for LLM pipelines, data ingestion, and RAG (Retrieval- Augmented Generation) solutions. Monitor model accuracy, bias, and drift; investigate anomalies and coordinate fixes with data scientists. Integrate automated tests into our CI/CD pipelines for continuous validation. GenAI Operations & Maintenance Deploy, scale, and troubleshoot GenAI microservices packaged in Docker on our cloud-native platform (Kubernetes or managed container services). Maintain prompt libraries, embeddings stores, and vector indexes built with LlamaIndex, AutoGen, or similar frameworks. Implement observability (logging, metrics, alerting) and incident-response runbooks to keep SLAs on track. Partner with ML engineers, product managers, and DevOps teams to ship reliable AI features. Document test cases, operational playbooks, and post-incident analyses to build organizational knowledge. Who are you? 2 years in Software QA, SDET, DevOps, or a similar engineering role (experience with AI/ML systems is a plus). Practical knowledge of Python for scripting tests, data checks, or automation tasks. Hands-on experience with Docker and at least one major cloud provider (AWS, Azure, or GCP). Familiarity with CI/CD tooling (GitHub Actions, GitLab CI, or similar). Strong understanding of software testing principles and comfort working in Linux-based, cloud-native environments. Clear communication skills in Italian, both written and spoken. Good knowledge of English. Preferred (Nice-to-Have) Direct experience with LLM orchestration frameworks (LlamaIndex, AutoGen, LangChain). Kubernetes (EKS, AKS, GKE, or vanilla K8s) for container orchestration. Observability stacks such as Prometheus/Grafana, OpenTelemetry, or Datadog. Knowledge of vector databases (e.g., Pinecone, Weaviate) and embedding techniques. ISTQB, AWS DevOps, or other relevant certifications. What do we offer? Competitive salary with performance-based bonus. Remote-first culture and flexible working hours. Annual budget for learning, certifications, and conferences in AI/ML and DevOps. Modern equipment plus a stipend for your home office. The chance to shape production GenAI systems from the ground up in a fast-growing company. J-18808-Ljbffr