Where

Lead Data Engineer (S)

Recruitment Hive
Sydney Full-day Full-time

Description:

  • Design, build, test and maintain data pipelines to move data between systems
  • NV1 clearance required - 12 + 12 month contract
  • Surry Hills, NSW - Flexible work arrangements

Our client, a large Federal government agency based in Surry Hills, NSW, requires two lead data engineers for 12 + 12 month contracts.

Duties

  • Design: Creating scalable data pipelines that meet project requirements.
  • Maintain: Optimising pipelines for performance and scalability.
  • Monitor: Troubleshooting issues and documenting processes.
  • Collaborate: Working with analysts and data scientists to understand data needs.
  • Automate: Writing scripts to automate repetitive tasks.

Candidates will have:

  • Technical expertise in scripting languages (e.g. AWS Glue, HashiCorp Terraform).
  • Solid understanding of cloud-based architecture relating to data storage and pipelines.
  • Understanding of data processing of streams (e.g. AWS Kinesis).
  • Understanding and being able to implement security and access controls around data pipelines.
  • Experience in data pipeline ETL tools (e.g. AWS Glue, Informatica, DBT, Talend etc).
  • Very good SQL skills.
  • Tertiary qualifications in an ICT related field or applicable industry certifications.

Selection Criteria

Essential criteria

  • Five years' experience working as a Data Pipeline Engineer in a Cloud Computing environment.
  • Demonstrated AWS as it relates to Data Storage, Extraction, Transformation, Data processing frameworks.
  • A proficient understanding of security concepts around the above areas.
  • Experience with Agile development methodologies.

Desirable criteria

  • Experience with GoAnywhere MFT.
  • Experience with Ctrl-M orchestration.
  • Experience with CI/CD automation / GITHub / Bitbucket.

To apply for these lead Data Engineer contract opportunities, please click the Quick Apply button to forward your CV to Andy at the Recruitment Hive.

Please note our internal Job ID is AE 10708 and this RFQ closes on the 16th of April 2025.

2 Apr 2025;   from: uworkin.com

Similar jobs

  • Talenza
  • Sydney
Description: Lead Data Engineer Sydney, Australia $230,000 (including ... institution to lead regulatory initiatives and drive innovation in data engineering, utilizing ... . Provide leadership and mentorship to data engineers, fostering a culture of innovation ...
15 days ago
  • Simple Machines
  • Sydney
Description: Simple Machines ANZ - Job Ad - Principal / Senior Data Engineer Position: Senior Data Engineer Location: Darlinghurst, Sydney Simple Machines. Data Engineered to Life™ Simple Machines is a leading independent boutique technology firm with a ...
18 days ago
  • Optiver
  • Sydney
... Optiver Sydney is seeking a seasoned Data Engineer to support and advance the ... are looking for a highly skilled Data Engineer with: 3+ years of experience in ... Databricks highly desirable.Ability to lead projects autonomously, prioritise tasks, and ...
26 days ago
Description: Looking for a Lead AI Engineer / Data Scientist to join a cutting edge ... , they’re seeking a talented Lead AI Engineer / Data Scientist to join their fast ... business objectives. Design and implement data science and engineering solutions, including ...
23 days ago