Data Engineer – Databricks (IBU Consumer)

• Build and maintain scalable data pipelines on Databricks for consumer-facing IBU data products.
• Develop ETL/ELT pipelines using PySpark, Spark SQL and Delta Lake
• Implement medallion architecture (bronze/silver/gold), Unity Catalog and Workflows
• Optimize cluster performance and cost; monitor and troubleshoot jobs in production

Requirements

Must have: 4+ yrs data engineering, Databricks, PySpark, SQL, Git/CI-CD.

Nice to have: pharma/consumer data, Databricks certification