Senior Data Engineer – AWS Cloud Data Platform

We are seeking a highly skilled, hands-on Senior Data Engineer to architect, build, modernise, and manage enterprise-scale data platforms on AWS.

The ideal candidate must have extensive experience in cloud data engineering, Data Lakehouse architecture, enterprise data warehousing, cloud migration, data modelling, distributed data processing, and performance optimisation. This role requires an engineer who can independently translate complex business requirements into secure, scalable, resilient, and cost-effective data solutions.

The candidate will be responsible for establishing engineering standards, designing reusable data frameworks, implementing governance and data quality controls, and enabling analytics, reporting, AI, and business intelligence use cases across the organisation.

This is not a purely development or support-oriented position. The successful candidate is expected to demonstrate strong architectural thinking, advanced hands-on engineering capabilities, production troubleshooting skills, and the ability to provide technical leadership across complex enterprise data initiatives.

Experience with Informatica PowerCenter or Informatica Intelligent Cloud Services (IICS) is a strong advantage but is not mandatory.

Key Responsibilities

• Enterprise Data Architecture and Strategy
• Design and implement enterprise-scale data architectures aligned with business, analytics, and technology objectives.
• Architect scalable Data Lake, Data Lakehouse, and Enterprise Data Warehouse solutions on AWS.
• Define architecture principles, engineering standards, reusable patterns, and development best practices.
• Design multi-layer data architectures such as Raw, Standardised, Curated, and Consumption layers.
• Evaluate technical requirements and select appropriate AWS services, data formats, processing engines, and orchestration patterns.
• Translate complex business requirements into scalable, maintainable, and production-ready technical solutions.
• Participate in architecture reviews and provide technical direction for data platform initiatives.
• Evaluate emerging data technologies and recommend platform improvements based on business value, scalability, security, and cost.
• AWS Data Platform Design and Engineering
• Architect and develop end-to-end data solutions using AWS-native services.
• Build scalable frameworks for batch, near-real-time, event-driven, and streaming data processing.
• Design secure, highly available, fault-tolerant, and cost-optimised cloud data platforms.
• Build metadata-driven and configuration-driven ingestion and transformation frameworks.
• Design solutions for loading data from databases, APIs, files, message queues, SaaS applications, and enterprise platforms.
• Implement orchestration, dependency management, retries, restartability, error handling, and operational recovery capabilities.
• Establish monitoring, alerting, logging, auditability, and observability standards for production pipelines.
• Optimise storage, processing, and query performance across AWS data services.
• Data Lakehouse and Data Warehouse Engineering
• Design and implement modern Data Lakehouse architectures using technologies such as:

Apache Parquet

Define appropriate partitioning, compaction, data retention, schema evolution, and file optimisation strategies.

Design enterprise and dimensional data models using

Fact and dimension modelling

Star and snowflake schemas

Slowly Changing Dimensions

• Data Vault modelling
• Create scalable ETL and ELT frameworks supporting enterprise reporting, analytics, self-service BI, data science, and AI workloads.
• Design data models at the correct business grain and establish reliable relationships across multiple source systems.
• Implement history management, incremental processing, change data capture, late-arriving data handling, reconciliation, and audit controls.
• Enable both batch and near-real-time data consumption while maintaining data consistency and traceability.

Advanced Data Engineering

• Develop complex production-grade data pipelines using Python, PySpark, SQL, AWS Glue, and related AWS services.
• Process high-volume structured, semi-structured, and unstructured datasets.
• Design efficient transformation logic capable of handling large-scale distributed workloads.
• Develop reusable components for ingestion, transformation, validation, reconciliation, exception handling, and operational monitoring.
• Review code and ensure compliance with engineering, security, performance, and maintainability standards.
• Implement unit testing, integration testing, regression testing, and automated data validation.
• Diagnose and resolve complex data processing, performance, stability, and production issues.
• PySpark and Distributed Processing
• Develop and optimise complex PySpark workloads for large-scale data processing.
• Analyse Spark execution plans and identify performance bottlenecks.

Apply appropriate optimisation techniques, including

• Partitioning and repartitioning
• Broadcast joins
• Join strategy optimisation
• Data-skew handling
• Predicate pushdown
• Partition pruning
• Caching and persistence
• Memory and executor optimisation
• Small-file management
• Troubleshoot failed or slow-running Spark jobs and implement sustainable fixes.
• Select suitable processing patterns based on data size, workload behaviour, latency, and cost requirements.
• Amazon Redshift Engineering and Optimisation
• Design and maintain scalable analytical data models in Amazon Redshift.

Demonstrate advanced knowledge of

Distribution styles and distribution keys

Sort keys

Workload Management

Query Monitoring Rules

Materialised views

Redshift Spectrum

• Data sharing
• Query execution plans
• Table maintenance and statistics
• Optimise SQL queries and data models for performance, concurrency, and workload efficiency.
• Diagnose query failures caused by data skew, excessive joins, intermediate row growth, memory pressure, or disk spilling.
• Design appropriate loading, incremental processing, deduplication, and reconciliation patterns.
• Monitor Redshift workload behaviour and recommend improvements to queries, physical design, and processing patterns.

Enterprise Data Integration

• Design and oversee integrations across enterprise platforms, including:
• ERP systems
• CRM systems
• Supply chain and planning applications
• Logistics and manufacturing systems
• Finance platforms
• eCommerce applications
• Third-party and SaaS solutions
• Design cross-system mapping and reference-data solutions where different platforms use different identifiers.
• Define data contracts, interface specifications, schema standards, and integration controls.
• Establish mechanisms for duplicate detection, referential validation, lineage tracking, and source-to-target reconciliation.
• Work with business and technical stakeholders to validate data grain, business rules, transformation logic, and expected outcomes.

Cloud Data Migration

• Lead migration of legacy and on-premises data platforms to AWS.
• Assess existing databases, ETL workflows, dependencies, data volumes, and operational requirements.
• Define migration strategies covering re-platforming, refactoring, parallel runs, validation, cutover, and rollback.
• Re-engineer legacy pipelines into scalable cloud-native solutions where appropriate.
• Implement reconciliation and validation controls to confirm completeness and accuracy between source and target platforms.
• Identify migration risks, technical constraints, and performance issues and recommend mitigation measures.
• Data Governance, Quality, and Security
• Implement engineering controls for data governance, metadata management, lineage, cataloguing, and data quality.

Establish automated data quality validations covering

Completeness

Uniqueness

Accuracy

Consistency

Referential integrity

Timeliness

• Design audit frameworks to track source-to-target movement, processing status, exceptions, and reconciliation results.
• Implement role-based access controls, encryption, secrets management, and least-privilege access.
• Apply data protection controls for sensitive, confidential, and personally identifiable information.
• Partner with data governance, architecture, infrastructure, and cybersecurity teams to maintain organisational standards.
• Support data ownership, stewardship, glossary, metadata, and lineage initiatives.
• DevOps, Automation, and Infrastructure as Code
• Implement CI/CD practices for data pipelines and cloud infrastructure.
• Automate build, testing, deployment, configuration, and rollback activities.
• Use infrastructure-as-code tools such as AWS CloudFormation or Terraform.
• Implement source control, branching, peer review, and release management standards.
• Separate code, configuration, credentials, and environment-specific parameters.
• Build deployment processes that support development, test, staging, and production environments.
• Improve engineering productivity through reusable templates, libraries, and automation.
• Production Support and Operational Excellence
• Take ownership of complex production incidents and drive them through resolution.
• Perform root-cause analysis and implement permanent corrective and preventive actions.
• Define operational metrics, pipeline service-level expectations, recovery procedures, and support documentation.
• Improve platform reliability through monitoring, proactive alerting, automation, and failure recovery.
• Support production releases and ensure appropriate technical validation before deployment.
• Identify recurring operational issues and eliminate them through engineering improvements.
• Technical Leadership and Collaboration
• Provide technical leadership and guidance to data engineers.
• Mentor engineers on AWS, SQL, PySpark, data modelling, testing, and engineering best practices.
• Lead solution design discussions, architecture reviews, code reviews, and technical problem-solving sessions.
• Work closely with business stakeholders, architects, analysts, product owners, BI teams, and data governance teams.
• Challenge incomplete or inefficient designs and recommend scalable alternatives.
• Communicate technical risks, dependencies, trade-offs, and recommendations clearly to both technical and non-technical stakeholders.
• Independently own technical delivery from requirement analysis through production implementation.
• Bachelor’s or Master’s degree in Computer Science, Engineering, Information Technology, or a related discipline.
• 8–10 years of experience in data engineering, data warehousing, data integration, or enterprise data platforms.
• At least 5 years of hands-on experience designing and implementing AWS-based data solutions.
• Proven experience delivering enterprise-scale data platforms and complex data engineering programmes.
• Demonstrated ability to independently own solution design, development, deployment, performance optimisation, and production support.
• Strong experience working with large-volume and complex datasets from multiple source systems.
• Strong understanding of cloud-native architecture, distributed processing, data security, and operational resilience.
• Experience providing technical direction, conducting design reviews, and mentoring other engineers.

Mandatory Technical Skills

AWS Data Services

The candidate must have strong hands-on experience with most of the following:

Amazon S3

AWS Glue

Amazon Redshift

Amazon Athena

Amazon EMR

AWS Lambda

AWS Step Functions

Amazon MWAA / Apache Airflow

AWS Lake Formation

Amazon EventBridge

Amazon SQS

Amazon SNS

AWS Identity and Access Management

AWS Secrets Manager

Amazon CloudWatch

AWS CloudFormation or Terraform

Programming and Data Processing

Advanced SQL

Python

PySpark

Apache Spark

• Shell scripting
• Distributed data processing
• Performance tuning and troubleshooting
• Data structure and algorithm fundamentals relevant to data processing
• Data Architecture and Modelling
• Data Lakehouse architecture
• Enterprise Data Warehouse design
• Dimensional modelling
• Data Vault modelling
• Fact and dimension design
• Star and snowflake schemas

Slowly Changing Dimensions

• Data partitioning and lifecycle management
• Metadata-driven processing
• Master and reference-data concepts
• Database and Query Optimisation
• Advanced Amazon Redshift development and tuning
• Query execution-plan analysis
• Distribution and sort-key design
• Workload Management and Query Monitoring Rules
• High-volume join optimisation
• Incremental loading and deduplication
• PostgreSQL, Oracle, SQL Server, or comparable relational databases
• Engineering and DevOps
• Git-based source control
• CI/CD pipeline implementation
• Infrastructure as Code
• Automated testing
• Code review and branching standards
• Release and deployment automation
• Monitoring, logging, and operational observability

Good-to-Have Skills

Informatica

Experience with one or more of the following will be an advantage:

Informatica PowerCenter

Informatica Intelligent Cloud Services

IICS Cloud Data Integration

Informatica Application Integration

Informatica Data Quality

Informatica Metadata Management

Informatica Master Data Management

Migration from Informatica or other legacy ETL platforms to AWS-native solutions

Additional Technologies

Databricks

Snowflake

Apache Kafka

dbt

Apache Iceberg

Delta Lake

• Change Data Capture technologies
• Data cataloguing and lineage tools
• API and event-driven integration
• Containerisation and Kubernetes

Preferred Domain Experience

Experience in one or more of the following domains is preferred:

Supply chain

Planning

Logistics

Procurement

Manufacturing

ERP

Finance

Commercial analytics

Enterprise master data

Rewards designed for you

Our Total Rewards is our way of saying: We see you and we value you. It’s more than just pay and benefits—it’s a meaningful investment in you. It is designed to help you perform, grow, feel safe, and elevate your potential. The package helps you care for yourself and your family, plan your future, grow your skills and career, collaborate in an inclusive workplace, and contribute to your community. At Schneider Electric, we’re here for what matters most to you. Discover more at our Career Page.

• Country-specific programs and initiatives may be available.

Looking to make an IMPACT with your career?

When you are thinking about joining a new team, culture matters. At Schneider Electric, our values and behaviors are the foundation for creating a great culture to support business success. We believe that our IMPACT values – Inclusion, Mastery, Purpose, Action, Curiosity, Teamwork – starts with us.

IMPACT is also your invitation to join Schneider Electric where you can contribute to turning sustainability ambition into actions, no matter what role you play. It is a call to connect your career with the ambition of achieving a more resilient, efficient, and sustainable world.

We are looking for IMPACT Makers; exceptional people who turn sustainability ambitions into actions at the intersection of automation, electrification, and digitization. We celebrate IMPACT Makers and believe everyone has the potential to be one.

Become an IMPACT Maker with Schneider Electric – apply today!

€40 billion global revenue

+9% organic growth

150 000+ employees in 100+ countries

You must submit an online application to be considered for any position with us. This position will be posted until filled.

Schneider Electric aspires to be the most inclusive and caring company in the world, by providing equitable opportunities to everyone, everywhere, and ensuring all employees feel uniquely valued and safe to contribute their best. We mirror the diversity of the communities in which we operate, and ‘inclusion’ is one of our core values. We believe our differences make us stronger as a company and as individuals and we are committed to championing inclusivity in everything we do.

At Schneider Electric, we uphold the highest standards of ethics and compliance, and we believe that trust is a foundational value. Our Trust Charter is our Code of Conduct and demonstrates our commitment to ethics, safety, sustainability, quality and cybersecurity, underpinning every aspect of our business and our willingness to behave and respond respectfully and in good faith to all our stakeholders. You can find out more about our Trust Charter here

Schneider Electric is an Equal Opportunity Employer. It is our policy to provide equal employment and advancement opportunities in the areas of recruiting, hiring, training, transferring, and promoting all qualified individuals regardless of race, religion, color, gender, disability, national origin, ancestry, age, military status, sexual orientation, marital status, or any other legally protected characteristic or conduct.