Lead Cloud Engineer

• This job involves creating data pipelines using a configurable framework (developed using Databricks / pyspark), which
• are used to load data from multiple sources to Google Cloud. It also involves
• extensive collaboration with other team members (India & USA) to follow a
• plan, execute workflows, ensure data load accuracy and debug in case of issues.

Essential Responsibilities

• Understand the overall requirement and

create & execute resilient and high performant data pipeline jobs,

developed in Databricks / PySpark

• Understand data and its relation /

significance from a business perspective

• Create custom scripts in python / pyspark
• Minor development and enhancement

activities as needed to handle new / custom requirements

• Validate data between source and target

& highlight issues and rerun jobs as needed

• Lead other developers, review their code

and ensure quality & optimal code is being delivered

• Provide regular updates on the tasks,

status and risks to project manager

Requirements

• Bachelor’s degree from a reputed

university

• 7 to 12 years overall experience
• Excellent SQL / data knowledge (6+ years)

in creating complex SQL queries, Stored Procedures etc.

• Experience in developing enterprise grade

ETL applications (any tool)

• Experience in Google BigQuery or any

cloud database

• Working knowledge in Pyspark,

Dataproc/Apache Spark, Airflow, Iceberg

• Good communication and positive attitude
• Attitude to take ownership and drive

tasks without follow-ups

Good to have

• Knowledge in Starburst, DBT
• Knowledge in US Healthcare domain
• Knowledge in Agile