Ignite your passion for product innovation by leading customer-centric development, inspiring solutions, and shaping the future with your strategic vision and influence. Step into a role where you define how the firm builds and runs the next generation of AI applications, turning a powerful runtime platform into a true point of leverage for the teams you serve. You will remove friction, accelerate the path from idea to production, and set the standard for how secure, resilient infrastructure is delivered at scale.
As a Product Director in the Model & Serving team, you lead innovation through the development of products and features that delight customers. As a leader on the team, you leverage your advanced capabilities to challenge traditional approaches, remove barriers to success, and foster a culture of continuous innovation that helps inspire cross-functional teams create groundbreaking solutions that address customer needs.
You shape how enterprise teams access models through a runtime layer with routing, guardrails, observability, telemetry, evaluation, and cost management built in by default. Partnering across Engineering, Architecture, Cybersecurity, Risk, Finance, and the business, you deliver a secure, resilient platform that dramatically shortens the time from idea to production.
This role calls for a senior product leader with deep expertise in AI infrastructure, distributed systems, and model serving who can set a clear vision and lead execution at enterprise scale. You balance speed and simplicity for application teams with enterprise requirements for security, compliance, multi-tenancy, and cost control, ensuring the platform scales reliably as demand grows.
Job responsibilities
Oversees the product roadmap, vision, development, execution, risk management, and business growth targets
Leads the entire product life cycle through planning, execution, and future development by continuously adapting, developing new products and methodologies, managing risks, and achieving business targets like cost, features, reusability, and reliability to support growth
Coaches and mentors the product team on best practices, such as solution generation, market research, storyboarding, mind-mapping, prototyping methods, product adoption strategies, and product delivery, enabling them to effectively deliver on objectives
Owns product performance and is accountable for investing in enhancements to achieve business objectives
Monitors market trends, conducts competitive analysis, and identifies opportunities for product differentiation
Serves as the accountable product leader for the generative AI Gateway across both federated and managed deployment models, driving adoption and platform outcomes
Defines the target-state gateway architecture with Engineering and Architecture partners, including data, multi-cloud, and multi-region deployment strategies and consistent capabilities across federated and managed models
Leads token management and intelligent model routing across policy, cost, and latency to support multi-model and agentic workloads
Drives traffic management and resiliency capabilities so the platform scales reliably with demand, partnering with Engineering to optimize inference performance and cost
Makes security and governance default capabilities applied to every model call, and owns guardrails, observability, and telemetry that give teams full visibility into production behavior
Communicates strategy, delivery progress, risks, dependencies, and recommendations to senior leadership and governance forums
Required qualifications, capabilities, and skills
8+ years of experience or equivalent expertise delivering products, projects, or technology applications
Extensive knowledge of the product development life cycle, technical design, and data analytics
Proven ability to influence the adoption of key product life cycle activities including discovery, ideation, strategic development, requirements definition, and value management
Experience driving change within organizations and managing stakeholders across multiple functions
Experience building or scaling AI or cloud platform infrastructure, such as generative AI or large language model gateways, API gateways and API management, model serving platforms, cloud networking and traffic management, or enterprise observability and telemetry platforms
Experience building AI and machine learning platform products for enterprise customers on one or more major cloud providers (Amazon Web Services, Microsoft Azure, or Google Cloud), including compute, networking, storage, and security for AI workloads
Strong knowledge of generative AI architectures, large language models, and how AI applications consume models in production
Solid understanding of AI infrastructure and model serving, including inference performance, capacity planning, scalability, and reliability engineering
Experience with cost management for cloud or AI workloads, including usage metering, cost attribution, and optimization
Ability to translate complex technical concepts into clear product strategy, prioritized roadmaps, and executive communications
Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience
Preferred qualifications, capabilities, and skills
- Recognized thought leader within a related field
- Experience building or leading a generative AI gateway, large language model proxy, model routing, or machine learning serving product at enterprise scale
- Full-stack technical fluency across the AI platform, from GPU and accelerator infrastructure and low-latency serving stacks to APIs, SDKs, and the applications built on them
- Experience operating both federated and centrally managed platform models, including shared standards, policy enforcement, and multi-tenant design
- Familiarity with open-source and commercial gateway and model serving technologies, as well as techniques for optimizing inference performance and cost
- Familiarity with AI observability, evaluation, and guardrail tooling, and with agent orchestration
- Experience partnering closely with Legal, Compliance, and Risk teams, with working knowledge of the regulatory landscape for AI, data privacy, and third-party risk in financial services
- This position is subject to Section 19 of the Federal Deposit Insurance Act. As such, an employment offer for this position is contingent on JPMorganChase’s review of criminal conviction history, including pretrial diversions or program entries.
Free tool / Resume tailor
Tailor your resume for this job
Paste your resume and get 3 to 8 of your own lines rewritten in the words this posting uses, with the reason for each change. The job is already filled in, and you don't need an account.
Job description provided by JPMorgan Chase.
View the original posting