AI and Agent Engineer - Senior Associate

Position Summary

The Senior AI and Agent Engineer will develop, integrate, test, deploy, and monitor AI-enabled applications and agents across the Microsoft ecosystem. The role is responsible for the intelligence, orchestration, actions, integrations, and behavioral quality of AI solutions.

This is an AI application engineering role, not a prompt-writing position. The successful candidate must be able to build production software, integrate enterprise systems, create evaluation frameworks, troubleshoot complex behavior, and operate deployed solutions.

Primary Responsibilities

• AI and agent development
• Build and configure Copilot Studio agents, topics, actions, triggers, workflows, and generative orchestration.
• Develop AI applications using Azure AI Foundry, Azure OpenAI, Azure AI Search, and supporting Azure services.
• Implement retrieval-augmented generation, structured outputs, tool use, multistep orchestration, and human approval patterns.
• Develop custom APIs, Azure Functions, connectors, and supporting application services.
• Integrate AI solutions with Microsoft Graph, SharePoint, Teams, Dataverse, Power Platform, and approved enterprise systems.
• Prompt, context, and behavior engineering
• Develop system instructions, prompts, grounding logic, response structures, fallback behavior, and escalation rules.
• Implement model selection and routing based on accuracy, risk, latency, and consumption requirements.
• Design agent behavior that distinguishes between answering, recommending, requesting clarification, escalating, and taking action.
• Maintain version control for prompts, instructions, tools, actions, and model configurations.
• Testing and evaluation
• Develop evaluation datasets with practice SMEs.
• Automate tests for groundedness, relevance, completeness, citation quality, task completion, response consistency, and escalation behavior.
• Perform adversarial testing involving prompt injection, data leakage, unsupported instructions, excessive agency, and unauthorized actions.
• Build regression tests to detect changes caused by models, prompts, source content, integrations, or application releases.
• Document known limitations and expected failure behavior.
• Operations and improvement
• Instrument solutions for usage, latency, failures, cost, task completion, user feedback, and model behavior.
• Review production telemetry and interaction patterns.
• Investigate incidents and implement corrective changes.
• Optimize token consumption, retrieval performance, application latency, and integration reliability.
• Develop reusable agent templates, integration components, evaluation harnesses, and error-handling patterns.

Required Technical Skills

• Strong Python development skills.
• TypeScript or JavaScript development experience.
• REST APIs, JSON, OAuth, OpenAPI, webhooks, and event-driven integration.
• Copilot Studio, Power Automate, Azure AI Foundry, Azure OpenAI, and Azure AI Search.
• Microsoft Graph, Dataverse, SharePoint, and Teams integration.
• Retrieval-augmented generation, embeddings, vector search, semantic search, hybrid search, reranking, and structured generation.
• Agent orchestration, state management, tool invocation, and human-in-the-loop controls.
• Automated testing, Git, pull requests, CI/CD, and production troubleshooting.
• Application Insights, Azure Monitor, logging, alerting, and performance diagnostics.
• Understanding of prompt injection, data exfiltration, overreliance, hallucination, unsafe actions, and AI-specific failure modes.

Preferred Experience

• 5 or more years of software, integration, automation, or data application development.
• Demonstrated production experience with generative AI or conversational AI applications.
• Experience building applications across Copilot Studio, Power Platform, and Azure.
• Familiarity with Semantic Kernel or comparable orchestration frameworks.
• Basic C# experience within the Microsoft ecosystem.
• Experience in regulated, confidential, or professional-services environments.

Success Measures

• Evaluation performance against approved acceptance thresholds.
• Task-completion and appropriate-escalation rates.
• Production defect and regression rates.
• Retrieval and grounding quality.
• Response latency and cost per transaction.
• Integration reliability.
• Reuse of standard components.
• Completeness of technical documentation and automated testing.
• Speed and effectiveness of incident remediation.

Role Boundaries

This role engineers the behavior of AI applications. Practice SMEs remain accountable for defining and validating authoritative professional content, business rules, acceptable conclusions, and circumstances requiring human review.

#WeaverIndia #LI-LS1