Research, experiment and prototype novel approaches for analyzing model behavior and failure patterns. Conduct targeted empirical investigations into model outputs, agent trajectories, and error patterns to inform benchmark design and improve evaluation trustworthiness. Implement, test, and refine benchmark suites evaluating complex agentic reasoning across code, math, and specialized domains, ensuring high discriminability and zero data contamination.
Develop and maintain pipelines that transform benchmark/sandbox execution logs, agent traces, and runtime artifacts into high-fidelity evaluation datasets and training data. Partner closely with research scientists, senior engineers, and cross-functional teams to translate research insights into practical evaluation infrastructure.
Job description provided by Google.
View the original posting
