AWS Glue / Databricks
LOCATION: Hyderabad, India
EXPERIENCE: 6–10 years (preferred)
ROLE OVERVIEW
Build and operationalize robust data engineering pipelines using AWS Glue, PySpark and Delta Lake to create governed,
high-quality enterprise datasets.
KEY RESPONSIBILITIES
- Develop AWS Glue / PySpark transformations from Raw to Refined datasets in Amazon S3.
- Implement schema enforcement, type conversion, PII tokenization, deduplication and merges.
- Develop Delta Lake pipelines and, when required, Gold datasets and Redshift loads.
- Orchestrate jobs using Airflow or AWS Step Functions with completeness and failure checks.
- Register datasets in Unity Catalog and support governance, metadata and consumer access.
- Perform reconciliation, performance tuning, unit testing and production troubleshooting.
- Prepare CI/CD artifacts, deployment configuration, runbooks and technical documentation.
Requirements
REQUIRED SKILLS
- Python, PySpark, Spark SQL and strong SQL
- AWS Glue, Amazon S3, Delta Lake and Databricks
- ETL/ELT, data modelling and data-quality validation
- Airflow or AWS Step Functions and failure recovery
- Git, CI/CD, data reconciliation and production support
PREFERRED EXPERIENCE
Unity Catalog, Amazon Redshift, Immuta, ServiceNow integration and regulated-data experience.
Free tool / Resume tailor
Tailor your resume for this job
Paste your resume and get 3 to 8 of your own lines rewritten in the words this posting uses, with the reason for each change. The job is already filled in, and you don't need an account.
Job description provided by Radcube.
View the original posting