Research and implement novel techniques for accelerated evaluation creation, including automated task synthesis, programmatic verifiers, and model-assisted rubric induction. Architect pipelines to mine execution logs, agent traces, and runtime artifacts into high-fidelity, future-proof evals and training data that continuously self-improve. Design and build robust benchmarks evaluating complex, long-horizon agentic reasoning across code, math, and specialized domains; optimize for climbability, discriminability, and zero contamination.
Build and scale platform software that automates quality gates—such as AutoRaters and automated loss-pattern root-cause analyzers. Provide technical direction and architecture across AI Data and partners such as GDM and Cloud AI. Align distributed engineering teams and establish flexible approaches reusable across all types of benchmarks.
Job description provided by Google.
View the original posting
