Data Engineer III

Position Title

Pay Rate

$83,000 - $121,000

Multiple factors, including Individual experience, skills and abilities will determine where an employee is ultimately placed in the pay range.

Category/Shift

Salaried Full-Time

Physical Location

75 5th Street NW

Atlanta, GA 30308

United States

The Job You Will Perform

The Data Engineer is an important member of the Cost Innovation Team, which is a strategic, innovative, and cross-functional team that is focused on creating value for the company through the advanced analytics of manufacturing data. This position offers a flexible working arrangement provided that the home office is within a typical commuting distance to the A2C office.

Core Responsibilities

Analytics & Data Science Enablement

• Partner with analysts, data scientists, engineers, and business stakeholders to translate functional requirements into scalable technical data solutions.
• Deliver self-service, analytics-ready datasets that empower downstream teams to consume data without deep knowledge of underlying source systems.
• Enable industrial analytics use cases by providing trusted, contextualized time-series, asset, event, and operational data for analytics platforms and applications.
• Eliminate redundant data-preparation and contextualization efforts across disparate analytics and application teams.
• Help establish standard data definitions, reusable data products, and consistent business context across enterprise and industrial analytics environments.
• Support downstream analytics workflows, including advanced analytics, data science, visualization, and industrial process analytics.

Data Ingestion & Integration

• Design, build, and maintain robust pipelines to ingest data from relational databases, operational systems, industrial historians, APIs, cloud platforms, and flat files.
• Establish reliable and secure connections between source systems, cloud data platforms, and industrial data platforms.
• Write optimized SQL queries and extraction logic for high-performance data retrieval.
• Implement flexible ingestion patterns including batch, incremental, streaming, and replication-based approaches based on business latency requirements.
• Integrate high-volume time-series data and industrial metadata while maintaining relationships between assets, measurements, events, and other operational entities.
• Support ingestion and integration patterns used by platforms such as Cognite Data Fusion (CDF), Seeq, and Microsoft Fabric.

Data Storage & Lakehouse Engineering

• Design and manage scalable storage structures across raw (bronze), intermediate (silver), and curated (gold) layers.
• Leverage Spark-native formats and engines such as Delta Lake to support scalable analytics workloads.
• Design curated data structures that support both enterprise analytics and industrial data consumption.
• Enforce data retention, archiving, and purging policies to manage data lifecycles.
• Monitor and optimize storage, compute costs, and cloud resource utilization.

Data Transformation

• Develop Python and SQL-based transformations to clean, standardize, enrich, and unify data.
• Develop, tune, and proactively optimize scalable PySpark applications to efficiently process high-volume datasets.
• Build modular, reusable transformation components rather than siloed, application-specific logic.
• Resolve complex data-quality anomalies including schema drift, missing values, duplicates, timestamp alignment, and inconsistent identifiers.
• Engineer efficient incremental processing patterns to handle high-volume, frequently changing datasets.
• Transform raw operational and historian data into analytics-ready time series, calculations, conditions, events, and contextualized datasets for downstream consumption.
• Enable reusable data transformations that can support platforms such as Seeq, Cognite, Fabric, and custom analytics applications.

Semantic Modeling & Industrial Data Contextualization

• Translate complex source schemas into intuitive, business-aligned semantic entities, asset hierarchies, and industrial data models.
• Build and maintain relationships between assets, equipment, time series, events, files, and other operational data.
• Standardize naming conventions, identifiers, metadata, and relationship mappings across source systems.
• Perform data contextualization to connect operational data with relevant equipment, process, and business context.
• Support asset-centric and knowledge-graph-style data models commonly used in industrial data platforms such as Cognite.

Pipeline Orchestration & Operations

• Architect end-to-end workflows that seamlessly coordinate ingestion, transformation, contextualization, validation, and publication.
• Orchestrate data movement between source systems, lakehouse environments, industrial data platforms, and analytics applications.
• Implement resilient error handling including retries, automated recovery, checkpointing, and alerting.
• Monitor pipeline health, SLA compliance, data freshness, and data drift using centralized observability tools.
• Design pipelines capable of supporting both traditional enterprise data and high-frequency industrial time-series workloads.

Data Governance, Security & DevOps

• Enforce data security and privacy standards, including row- and column-level security, RBAC (Role-Based Access Control), and masking of sensitive data.
• Apply consistent access-control and governance patterns across enterprise data platforms and industrial analytics environments.
• Embed automated validation checks as needed to enforce data quality.
• Maintain Git-based development workflows using branching, code reviews, and pull requests.
• Implement automated CI/CD pipelines for code testing, data validation, configuration management, and repeatable environment deployments.
• Support deployment and lifecycle management of data pipelines, transformation logic, semantic models, and analytics configurations across development, test, and production environments.
• Maintain comprehensive documentation of data architectures, pipeline schemas, source-to-target mappings, contextual relationships, and lineage maps.

Preferred Experience

Bachelor’s Degree in Computer Science, Software Engineering, Data Science, Engineering, or related field + 2 years of relevant experience; equivalent Master’s degree accepted; non-relevant bachelor’s degree requires 5+ years of equivalent hands-on experience.

Experience working with

SQL and Python for data engineering and transformation.

PySpark and Microsoft Fabric for lakehouse transformations.