Radcube Technology/Software

Senior Data Engineer

Hyderabad, India In person Full-time

AWS Glue / Databricks

LOCATION: Hyderabad, India

EXPERIENCE: 6–10 years (preferred)

ROLE OVERVIEW

Build and operationalize robust data engineering pipelines using AWS Glue, PySpark and Delta Lake to create governed,

high-quality enterprise datasets.

KEY RESPONSIBILITIES

  • Develop AWS Glue / PySpark transformations from Raw to Refined datasets in Amazon S3.
  • Implement schema enforcement, type conversion, PII tokenization, deduplication and merges.
  • Develop Delta Lake pipelines and, when required, Gold datasets and Redshift loads.
  • Orchestrate jobs using Airflow or AWS Step Functions with completeness and failure checks.
  • Register datasets in Unity Catalog and support governance, metadata and consumer access.
  • Perform reconciliation, performance tuning, unit testing and production troubleshooting.
  • Prepare CI/CD artifacts, deployment configuration, runbooks and technical documentation.

Requirements

REQUIRED SKILLS

  • Python, PySpark, Spark SQL and strong SQL
  • AWS Glue, Amazon S3, Delta Lake and Databricks
  • ETL/ELT, data modelling and data-quality validation
  • Airflow or AWS Step Functions and failure recovery
  • Git, CI/CD, data reconciliation and production support

PREFERRED EXPERIENCE

Unity Catalog, Amazon Redshift, Immuta, ServiceNow integration and regulated-data experience.

Free tool / Resume tailor

Tailor your resume for this job

Paste your resume and get 3 to 8 of your own lines rewritten in the words this posting uses, with the reason for each change. The job is already filled in, and you don't need an account.

Job description provided by Radcube.
View the original posting