Senior Data Engineer

Description

• Equal Experts is an innovative consultancy specialising in the delivery of custom software solutions for
• blue-chip enterprise and public sector clients across a range of industry sectors. We work on important
• projects for our clients, deliver market-leading propositions across the digital, online and mobile
• channels, and are recognised for our leadership in the application of agile and lean delivery methods to
• assure delivery. We have offices in Australia, India, South Africa, the UK, the EU and the USA.
• We’re looking for Data Engineers with proven senior-level experience to join the Equal Experts
• network.
• Our Data Engineers are highly skilled in creating data pipelines for ingest, transformation and storage of
• data, usually at scale, and are familiar with standing-up analytics clusters.
• This role involves using modern agile technical practices - including Continuous Integration and
• Deployment, TDD, and XP - to deliver timely and pragmatic solutions, as well as helping others to do
• their jobs in a more effective way.
• This role involves working in teams that use modern agile technical practices - including continuous
• integration and deployment and fast feedback loops - to deliver timely and pragmatic solutions, as well
• as helping others to do their jobs in a more effective way.

Responsibilities

• Working with the client and the team to determine data processing and access needs and how

best to deliver them.

• Creation and support of highly available data pipelines and storage solutions (e.g. data

warehouses).

• Delivery of software using agile (pair programming, TDD, CI/CD, etc) practices.
• Automation of data infrastructure and deployments.
• Advocating agile practices to the client organisation.
• Mentoring client team members.
• Helping improve the team and clients’ data capabilities.

Required Skills

• Significant experience working on data pipelines, platforms and projects.
• Experience in developing solutions in one or more of the main Cloud providers (AWS, GCP,

Azure).

• Strong background in at least one programming language, Python or Scala preferable.
• Working with a variety of databases, data warehouses and distributed file systems in production

environments.

• Strong experience in SQL and data modeling.
• Exposure with data processing at scale.
• Knowledge about GDPR and data security. Working with source control and infrastructure as code (Git/Github, Terraform, ansible etc.) and CI/CD tools.
• Understanding and being able to support data scientists or business intelligence teams - in data

prep, mentoring and support functions.

• Strong collaboration and communication skills, including the ability to communicate effectively

while working remotely with globally distributed teams.