Senior Data Engineer (Big Data Processing)

Role

As a senior data engineer in a team that is responsible for mission-critical data analytics systems, you will contribute to the development and maintenance of high-performance data pipelines and systems related to national security. You will be involved in complex technical projects to collect, parse and process large datasets with the aim to turn them into useful information and insights. Join us to seize the opportunity to use cutting-edge technologies to protect the nation.

Responsibilities

Design and build data pipelines that are scalable, robust and maintainable to ingest crucial data using cutting-edge platforms and technologies

Work with product managers, engineering managers and key stakeholders to deliver impactful data products that meet our business needs

Optimise performance of large-scale data pipelines for low-latency, high-throughput and reliability

Ensure data integrity, quality and governance through robust design and operational monitoring

Troubleshoot and resolve complex issues in critical data pipelines

Implement best practices in data engineer, including testing, orchestration and monitoring of data pipelines

Lead technical discussions with the team and key stakeholders to shape the future of our data products

Mentor junior data engineers and contribute to building a strong data engineering culture within the team

Requirements

Background in computer science or equivalent

5+ years of experience in data engineering, with a strong focus on building large-scale data pipelines

Proven experience in writing optimised ETL code to handle large datasets

Proficient in programming languages such as Scala or Java

Familiarity with the following technologies: Spark, Kafka, Hadoop, Elastic stack, Kudu

Strong understanding of modern data architectures and data modelling paradigms

Able to communicate technical details effectively and clearly

Good interpersonal, analytical and problem-solving sklils to get things done in a dynamic environment

Why join us?

The work is purposeful and meaningful

You will get to work with talented and driven people

We work with modern technologies and tech stacks

We have excellent engineering culture and work-life balance

We aspire for engineering and operational excellence

We empower to innovate