We are seeking a highly motivated Data Engineer to design, build, and maintain scalable data platforms and pipelines that enable analytics, reporting, machine learning, and AI-driven business solutions. The ideal candidate will have expertise in data integration, cloud technologies, data warehousing, and modern data engineering practices, with a strong focus on data quality, performance, and reliability.
The Data Engineer will work closely with business stakeholders, data architects, analysts, data scientists, and application teams to deliver trusted and scalable data solutions.
Data Pipeline Development
- Design, develop, and maintain scalable ETL/ELT data pipelines.
- Build and optimize batch and real-time data ingestion frameworks.
- Integrate data from multiple sources, including databases, APIs, SaaS applications, and third-party systems.
- Ensure data consistency, quality, and reliability across platforms.
Data Platform Engineering
- Develop and support enterprise Data Lake, Data Warehouse, and Lakehouse architectures.
- Implement scalable data solutions on cloud platforms.
- Design data models to support reporting, analytics, and AI/ML use cases.
- Optimize data storage, processing, and query performance.
Data Quality & Governance
- Establish data validation, cleansing, and monitoring processes.
- Implement data governance, lineage, cataloging, and metadata management practices.
- Ensure compliance with security, privacy, and regulatory requirements.
- Support audit and data quality initiatives.
Performance Optimization
- Monitor and tune data pipelines and databases for performance and scalability.
- Identify and resolve data bottlenecks and processing issues.
- Optimize SQL queries and large-scale data transformations.
Collaboration & Stakeholder Engagement
- Collaborate with Data Architects to implement enterprise data solutions.
- Work closely with BI developers, analysts, and data scientists to support analytical needs.
- Participate in requirements gathering, solution design, and technical reviews.
- Provide production support and troubleshooting for critical data platforms.
DevOps & Automation
- Implement CI/CD pipelines for data engineering solutions.
- Automate deployment, monitoring, and operational processes.
- Support infrastructure-as-code and cloud automation initiatives.
Education
Bachelor's degree in Computer Science, Information Technology, Engineering, Data Science, or related field.
Master's degree is a plus.
Experience
- 12+ years of experience in Data Engineering, Data Warehousing, or Big Data environments.
- Experience building and supporting large-scale enterprise data platforms.
- Experience working in Agile development environments.
Technical Skills
Programming Languages
Python
SQL
Java (Preferred)
Data Engineering Tools
Apache Spark
Databricks
dbt
Cloud Platforms
Microsoft Azure
AWS
Google Cloud Platform (GCP)
Data Platforms
Snowflake
Azure Synapse Analytics
Amazon Redshift
BigQuery
Teradata
Databases
SQL Server
Oracle
PostgreSQL
MySQL
MongoDB
Business Intelligence Tools
Tableau
Power BI
Looker
Qlik Sense
DevOps & Version Control
Git
Azure DevOps
Jenkins
Terraform
Docker
Kubernetes
Preferred Skills
Data Lakehouse Architecture
Real-Time Streaming Architecture
Data Mesh Concepts
Master Data Management (MDM)
Data Governance Frameworks
Machine Learning Data Pipelines
API Development & Integration
Event-Driven Architecture
Soft Skills
- Strong analytical and problem-solving abilities.
- Excellent communication and stakeholder management skills.
- Ability to work independently and collaboratively within cross-functional teams.
- Strong documentation and presentation skills.
- Attention to detail and commitment to delivering high-quality solutions.
Preferred Certifications
Cloud Certifications
Microsoft Azure Data Engineer Associate (DP-203)
AWS Certified Data Analytics Specialty
Google Professional Data Engineer
Platform Certifications
Snowflake SnowPro Certification
Databricks Certified Data Engineer Associate
Tableau Certified Professional (Preferred)
Job description provided by Zensar.
View the original posting