GenAI / AgentOps Engineer

• AgentOps & LangGraph: Design and implement agent workflows using LangGraph/LangChain, with strong focus on orchestration, observability, debugging, and evaluation.
• Evaluation Frameworks: Develop automated LLM evaluation pipelines (factuality, robustness, hallucination detection, guardrails) to ensure production readiness and reliability.
• Embeddings & Retrieval: Optimize embedding models, vector store integrations, and RAG pipelines for domain-specific unstructured data.
• Prompt & Workflow Optimization: Maintain prompt pipelines and fine-tune prompts to balance cost, performance, and accuracy in production.
• Data Systems: Architect data ingestion and vectorization pipelines across SQL/NoSQL databases and large-scale unstructured content sources.
• Production Deployment: Lead end-to-end LLMOps/MLOps workflows, from experimentation to deployment, monitoring, and continuous improvement (preferably in Azure: ML Studio, Azure OpenAI, Azure AI Foundry).
• Research & Innovation: Stay up-to-date with GenAI/AgentOps advances (LangGraph, LangSmith, eval frameworks, retrieval strategies) and bring best practices into production.
• Collaboration & Mentorship: Partner with stakeholders to translate requirements into agentic pipelines, and mentor junior engineers on LangGraph workflows, AgentOps best practices, and evaluation design.

• Hands-on expertise with LangGraph, LangChain, and LangSmith (or similar agent frameworks).
• Strong experience with AgentOps: orchestration, monitoring, debugging, and evaluation of LLM-based workflows.
• Proficiency in embedding models, vector databases (Pinecone, Weaviate, FAISS, Mongo Atlas Vector Search, etc.), and retrieval optimization.
• Proven track record deploying agentic GenAI systems into production with robust evaluation/monitoring pipelines.
• Solid engineering background with Python and libraries such as Transformers, FastAPI, spaCy.
• Experience with SQL and NoSQL databases for data modeling and retrieval.
• Strong understanding of LLMOps/MLOps practices (CI/CD, monitoring, scaling, cost optimization).
• Excellent communication skills, ability to clearly explain evaluation results and system behaviors to technical and non-technical stakeholders

• Experience with LLM fine-tuning/adaptation (LoRA, QLoRA, RLHF, DPO) as a complement to embeddings/prompting.
• Experience with graph databases (Neo4j, ArangoDB, TigerGraph) for hybrid RAG pipelines.
• Familiarity with domain-specific ontologies or knowledge graphs for structured + unstructured reasoning.
• Knowledge of continuous evaluation & A/B testing frameworks for agents in production.
• Strong knowledge of data streaming frameworks (Kafka, Flink, Spark) for real-time retrieval & evaluation.
• Experience with AI safety, interpretability, explainability or responsible AI practices.

• Flexible working arrangements (hybrid)
• Opportunity to work in a high impact role at the intersection on AI, SaaS and Compliance/ Regulatory intelligence
• Continuous learning and professional development

RegASK