UnitedHealth Group Healthcare/BioTech

Data Engineering Lead-Azure Data bricks, SQl,Python,Spark,Airflow

Bengaluru, India In person Full-time

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibilities

Design, develop, test, and deploy scalable data pipelines using Azure Databricks, Spark SQL, Python, and Airflow

Build and maintain data ingestion, transformation, and data curation processes across multiple enterprise data sources

Develop complex SQL queries and optimize data processing workflows for performance and scalability

Create, maintain, and support Airflow DAGs for orchestration, scheduling, monitoring, and error handling

Collaborate with Product Owners, Business Analysts, and stakeholders during requirements gathering, backlog refinement, and sprint planning activities

Implement data quality validations, reconciliation checks, and exception handling mechanisms

Support the development and maintenance of Delta Lake and Lakehouse architectures within Databricks

Participate in data platform modernization and cloud migration initiatives

Develop reusable frameworks, utilities, and automation solutions to improve engineering efficiency

Troubleshoot pipeline failures, performance bottlenecks, and complex data issues

Follow coding standards, security requirements, and engineering best practices

Contribute to CI/CD implementation and automated deployment processes

Work closely with cross-functional teams to support analytics, reporting, AI, and business operational needs

Prepare and maintain technical documentation, design specifications, and operational runbooks

Support production deployments, incident resolution, root cause analysis, and operational excellence initiatives

Utilize AI-assisted development tools and automation capabilities to improve delivery efficiency and code quality

Provide technical recommendations and participate in design reviews and architecture discussions

Demonstrate ownership, accountability, and commitment to project deliverables

Stay current with emerging technologies, cloud services, and modern data engineering practices

Comply with company policies, security standards, and organizational guidelines

Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Required Qualifications

Bachelor's Degree in Engineering, Computer Science, Information Technology, MCA, or equivalent experience

4+ years of relevant hands-on experience in Azure Databricks and Modern Data Engineering

Experience working in Agile/Scrum delivery environments

Technical Skills

Data Engineering Expertise

  • Data ingestion frameworks
  • Data transformation and enrichment
  • Batch processing pipelines
  • Data quality implementation
  • Query optimization
  • Performance tuning
  • Metadata and lineage concepts
  • Cloud Technologies knowledge

Microsoft Azure

Azure Data Factory (ADF)

Azure Key Vault

Azure DevOps or GitHub

CI/CD Pipelines

Proven core skills

Azure Databricks

Apache Spark

Spark SQL

SQL Development

Python Programming

Apache Airflow

DAG Development and Workflow Orchestration

Delta Lake

Azure Data Lake Storage (ADLS)

ETL / ELT Development

Data Warehousing Concepts

Data Modeling Fundamentals

Preferred Qualifications

  • Azure Databricks certification or related Azure certifications
  • Experience working with Healthcare, Claims, Member, Provider, Payment Integrity, or Affordability data domains
  • Experience with Snowflake and modern cloud data platforms
  • Experience using GitHub Actions, DevOps automation, and Infrastructure as Code
  • Healthcare payer industry experience
  • Experience supporting enterprise data platforms and analytics ecosystems
  • Experience with Payment Integrity, Risk Adjustment, Claims Analytics, or Provider Analytics
  • Exposure to AI/ML data engineering workflows and AI-enabled analytics solutions
  • Exposure to Generative AI, Azure OpenAI, or AI-assisted engineering tools
  • Understanding of Data Lakehouse, Data Mesh, or Data Fabric concepts
  • Exposure to AI-driven development and engineering productivity tools

Key Competencies

Solid analytical and problem-solving skills

Ability to independently deliver medium to complex technical solutions

Solid debugging and troubleshooting capabilities

Excellent written and verbal communication skills

Effective stakeholder collaboration and teamwork

Solid learning agility and growth mindset

Ability to work in a fast-paced and dynamic environment

Self-motivated with solid ownership and accountability

At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission.

Job description provided by UnitedHealth Group.
View the original posting