**Greetings of the day!!**
I am Arumugam Veera, reaching out to you regarding an exciting career opportunity with **Naveera Technology LLC.** I would be happy to connect and discuss the opportunity further. You can also connect with me on **_LinkedIn:https://www.linkedin.com/in/arumugamv/_**
**Specialties**
Data Engineering & Modern Data Stack, Generative AI Solutions & Model Deployment, Application Development (Web, Mobile & Enterprise), Artificial Intelligence (Predictive, Conversational, Computer Vision), IT Infrastructure Services (Cloud & On-Prem), Security, DR, Cloud Transformation & Microservices, DevOps, API & Systems Integration, Extended Technology Teams & Dedicated Delivery, Real-Time Streaming & Analytics, and BI & Data Warehousing
**Job Title: Data Engineering Manager – GCP Cloud**
**Experience: 15+ Years**
**Location: Remote (USA)**
**Primary Focus: GCP Data Engineering |AWS → GCP Migration | Data Engineering | Data Architecture | Engineering Leadership**
**Position Overview**
We are looking for an experienced Engineering Manager with strong hands-on expertise in AWS and GCP Data Engineering to lead a large-scale AWS-to-GCP data platform migration.
The ideal candidate will have strong experience designing enterprise data platforms on AWS and migrating them to Google Cloud Platform (GCP). The role requires a combination of technical architecture, hands-on engineering, migration leadership, team management and stakeholder management.
The candidate should have strong hands-on experience with AWS services such as S3, Glue, Redshift, Athena, Step Functions and AWS DMS, along with strong GCP expertise across BigQuery, Dataflow, Pub/Sub, Cloud Storage and Cloud Composer.
The Engineering Manager will work closely with US-based stakeholders, architects, engineers, DevOps teams, Data Science and BI teams to define the migration strategy and ensure successful execution.
**Key Responsibilities**
**1\. GCP Data Platform Architecture**
- Architect and implement scalable enterprise data platforms on GCP.
- Design Data Lake and Lakehouse architectures using GCS and BigQuery.
- Define Bronze, Silver and Gold/Atomic data layers.
- Design scalable data ingestion, transformation and consumption frameworks.
- Establish standards for data modeling, partitioning, clustering and storage.
- Design multi-tenant and multi-location data architectures.
- Define schema-on-read and schema-on-write strategies.
- **2\. AWS to GCP Migration Leadership**
- Lead the end-to-end migration of enterprise data platforms from AWS to GCP.
- Assess existing AWS architecture, data pipelines, workloads, dependencies and operational processes.
- Define the target-state GCP architecture and migration roadmap.
Develop migration strategies for
Amazon S3 → Google Cloud Storage
Amazon Redshift → BigQuery
- AWS Glue → Dataflow / Dataproc / BigQuery
- AWS Step Functions → Cloud Composer / Workflows
- AWS DMS → GCP-native CDC solutions
Amazon Athena → BigQuery
- Identify opportunities to modernize AWS workloads rather than performing a simple lift-and-shift migration.
- Define migration phases, technical dependencies, risks and rollback strategies.
- Lead architecture reviews and technical design discussions.
**3\. AWS Data Platform Expertise**
Analyze and optimize existing AWS data platforms before migration.
AWS Lake Formation
- Understand existing AWS ETL/ELT pipelines, data models, workloads and dependencies.
- Identify equivalent or improved GCP services for each AWS workload.
- Prepare technical mapping and migration plans between AWS and GCP services.
- **4\. GCP Streaming & Real-Time Data Engineering**
Architect real-time data pipelines using
Google Pub/Sub
Cloud Storage
- Design high-volume event ingestion, enrichment and transformation pipelines.
- Implement event-driven architectures and appropriate delivery guarantees.
- Optimize streaming pipelines for latency, throughput and scalability.
- Design BigQuery streaming ingestion patterns.
- Implement monitoring, logging and alerting for real-time workloads.
- **5\. ETL / ELT & Data Processing**
- Design and implement scalable batch and real-time ETL/ELT pipelines.
- Migrate AWS Glue-based pipelines to appropriate GCP services.
Develop transformation frameworks using
Python
PySpark
SQL
Dataflow / Apache Beam
BigQuery
- dbt
- Design CDC pipelines and real-time ingestion patterns.
- Build orchestration workflows using Cloud Composer / Airflow.
- Optimize data processing jobs and query performance.
**6\. Data Modeling & BigQuery**
- Design enterprise data models for analytics and reporting.
- Define dimensional, normalized and denormalized data models.
- Develop multi-tenant data structures.
- Design BigQuery partitioning and clustering strategies.
- Optimize BigQuery SQL and query execution.
- Design data models supporting both real-time and batch workloads.
- Work closely with BI and Analytics teams to create scalable consumption models.
**7\. Data Governance, Security & Quality**
- Establish data governance and data quality standards across the GCP platform.
- Implement automated data quality checks and validation frameworks.
- Establish data lineage, metadata and ownership standards.
- Ensure appropriate security controls across all GCP data layers.
Implement
IAM
Least-privilege access
Encryption
- Service accounts
- Network security
- Data access policies
- Work with governance and security teams to ensure compliance requirements are met.
- Experience with Dataplex, Data Catalog and data lineage is preferred.
**8\. DevOps, Infrastructure & Automation**
- Lead infrastructure automation using Terraform.
- Build repeatable and secure GCP infrastructure deployments.
- Implement CI/CD pipelines for data engineering workloads.
Work with
Terraform
Git
GitHub
Cloud Build
- CI/CD pipelines
- Automate data pipeline deployment, testing and infrastructure provisioning.
- Establish Dev, QA, UAT and Production deployment standards.
**9\. Performance & Cost Optimization**
Lead performance optimization initiatives across GCP data workloads.
Optimize
- BigQuery query performance
- Partitioning and clustering
- Dataflow pipelines
- Spark workloads
Cloud Storage
- Streaming workloads
- Analyze AWS workloads and determine the most cost-effective GCP architecture.
- Develop cloud FinOps and cost optimization strategies.
- Establish performance benchmarks and SLAs for critical workloads.
**10\. Engineering Management & Team Leadership**
- Lead and mentor a team of Data Engineers, Senior Data Engineers and Technical Leads.
- Provide technical direction and establish engineering standards.
- Conduct architecture and code reviews.
- Define technical roadmaps and engineering priorities.
- Break complex migration requirements into actionable deliverables.
- Track engineering progress, risks, dependencies and delivery milestones.
- Promote best practices around coding, testing, CI/CD, security and documentation.
- Mentor engineers on GCP, data architecture and modern data engineering practices.
**11\. Stakeholder & Client Management**
- Act as the primary technical point of contact for US-based stakeholders.
- Work closely with Business, Product, Data Science, BI and DevOps teams.
- Translate business requirements into scalable technical solutions.
- Present architecture decisions, migration strategies and technical roadmaps.
- Communicate technical risks, dependencies, timelines and trade-offs.
- Collaborate with business teams to define operational and analytical KPIs.
## Tasks
- **15+ years of experience in Data Engineering, Data Architecture or Cloud Engineering.**
- **5+ years of hands-on GCP Data Engineering experience.**
- **3+ Yrs Proven experience working on AWS-to-GCP migration projects.**
- Strong experience designing enterprise Data Lake / Lakehouse platforms.
- Experience migrating AWS data workloads to GCP.
- Strong knowledge of AWS and GCP service mapping and cloud migration patterns.
- Expert-level SQL and strong Python/PySpark skills.
- Strong data modeling and data warehousing experience.
- Experience with Terraform and CI/CD.
- Experience managing and mentoring data engineering teams.
- Strong communication skills with experience working with US-based stakeholders
## Requirements
**Required Technical Skills & Cloud Platforms**
**AWS – Strong Existing Platform Experience**
Amazon S3
AWS Glue
AWS Glue Data Quality
Amazon Redshift / Redshift Serverless
Amazon Athena
AWS Step Functions
AWS DMS
AWS Lake Formation
IAM
**GCP – Target Platform**
BigQuery
Google Cloud Storage
Pub/Sub
Dataflow / Apache Beam
Cloud Composer / Airflow
Dataproc / Spark
Cloud Monitoring
Cloud Logging
Dataplex / Data Catalog
**Data Engineering**
Advanced Python
Expert-level SQL
Strong PySpark / Apache Spark
ETL/ELT
CDC
- Batch and streaming data processing
- Event-driven architecture
- Data pipeline optimization
## Benefits
- Lead large-scale AWS-to-GCP cloud transformation initiatives.
- Work on enterprise Data Lakehouse and analytics modernization projects.
- Flexible remote work environment.
- Exposure to global enterprise customers.
- Collaborative, innovation-driven engineering culture.
- Continuous learning and certification opportunities
Job description provided by Naveera Technology.
View the original posting