Direct the end-to-end technical strategy for BigQuery's storage formats, and determine the roadmap for our existing proprietary formats (Capacitor) versus adopting or contributing to emerging open standards (Vortex, Lance, BtrBlocks). Design storage layer innovations that unlock zero-copy column updates/additions, extremely efficient random access, and seamless integration with deep learning training pipelines. Partner with Dremel, BigLake, and GCS teams to ensure our execution engine seamlessly adopts next-generation formats without regressing SQL concurrency.
Position Google as a thought leader in the open data lakehouse ecosystem, steering the direction of the formats that will define the next decade of data infrastructure. Lead technical and Return on Investment (ROI) evaluations of open-source formats, and assess the engineering costs to integrate Arrow-native execution with BigQuery pipelines for wide tables and multimodal AI.
Job description provided by Google.
View the original posting
