Bristlecone Technology/Software

Data Engineer

Bengaluru, India In person Full-time
  • Develop API-driven systems to govern, manage, and monitor large-scale batch Big Data applications.
  • Build scalable backend services and data engineering solutions that support data processing and operational workflows.
  • Develop and maintain data transformation processes using Spark, SQL, Hive, Python, Scala, and related technologies.
  • Work with AWS cloud services such as S3, EC2, EMR, Lambda, DynamoDB, and API Gateway to build cloud-native data and backend solutions.
  • Build and enhance workflow orchestration using Apache Airflow, including advanced DAG design, dependency management, scheduling, monitoring, and failure handling.
  • Define technical scope, objectives, and implementation approaches by participating in requirements gathering, technical research, and process definition.
  • Participate in architecture reviews, code reviews, performance tuning, and operational readiness activities.
  • Contribute to test planning and validation for application integrations, functional areas, and project deliverables.
  • Partner with product owners, data engineers, backend engineers, QA, DevOps, and other cross-functional teams to deliver high-quality solutions in an Agile/Scrum environment.
  • Troubleshoot production issues, improve system reliability, and drive automation across data and backend workflows.
  • 5+ years of hands-on experience developing enterprise-scale applications, data platforms, or distributed systems.
  • 5+ years of experience developing, and operating Big Data platforms and cloud-based infrastructure services, preferably using AWS EMR and the Hadoop ecosystem.
  • Hands-on experience with AWS, Spark, Python and/or Scala, Airflow, SQL, Hive, and related data processing technologies.
  • Proficiency in either Python or Scala, with the ability to build production-grade data pipelines and backend services.
  • Experience with AWS cloud services, including S3, EC2, EMR, Lambda, DynamoDB, and API Gateway.
  • Strong experience with Apache Airflow orchestration, including complex workflows, scheduling, dependency handling, monitoring, and operational support.
  • Strong understanding of data engineering concepts, including data pipelines, data transformation, batch processing, real-time processing, data quality, and performance optimization.
  • Excellent analytical and problem-solving skills.
  • Strong communication and presentation skills, with the ability to explain technical concepts clearly to both technical and non-technical audiences.
  • Experience with version control and CI/CD tools, including Git and Jenkins, for source code management, build automation, testing, and deployment workflows.
  • Ability to work effectively in cross-functional Agile/Scrum teams.
  • Experience with LLMs, Generative AI, Agentic AI, or AI-assisted engineering workflows.
  • Experience with API design, microservices, event-driven architecture, or serverless backend development.
  • Experience with CI/CD pipelines, infrastructure as code, automated testing, and production deployment practices.
  • Experience with data governance, metadata management, lineage, data observability, or operational controls for enterprise data platforms

Job description provided by Bristlecone.
View the original posting