Observability & Evaluation Engineer

Tasks

• Help build, monitor, evaluate, and improve production-grade Agentic AI systems.
• Help improve the quality, reliability, transparency, and operational readiness of production Agentic AI systems.

Requirements

• 7+ years of engineering experience in observability, monitoring, automation, platform operations, or AI/ML systems.
• 5+ years of hands-on Python experience.
• Strong experience with dashboards, metrics, logs, tracing, alerting, SLOs, and production monitoring.
• Understanding of LLM evaluation, prompt evaluation, RAG evaluation, and AI quality assessment.
• Experience with OpenTelemetry concepts, Agile delivery, and production support.
• Preferred: GenAI observability and AI evaluation frameworks.
• Preferred: Kubernetes, cloud platforms, and CI/CD.
• Preferred: Experience in regulated environments and release governance.

Schedule

• Long-Term Contract
• Onsite

How to apply

• Share your updated resume by email.

Location

• Charlotte, NC