Description:
Lead Data Engineer
Sydney, Australia
$230,000 (including Super) + Bonus
About the Role:
Join a high-performing Data Team at a leading financial services institution to lead regulatory initiatives and drive innovation in data engineering, utilizing AI and automation to enhance platform efficiency.
Key Responsibilities:
- Lead the development and transformation of the regulatory reporting platform.
- Collaborate across business and technical teams to deliver complex data solutions.
- Implement and optimize data integration pipelines and data warehouse solutions.
- Oversee the design and architecture of large-scale data solutions using AWS services.
- Drive platform efficiency through automation and AI enhancements.
- Ensure data governance and compliance with internal and external regulations.
- Provide leadership and mentorship to data engineers, fostering a culture of innovation and excellence.
Required Skills and Experience:
- Proficient in Spark, Hive, Presto, and big data storage formats (Parquet, ORC, Avro).
- Experience with data pipeline orchestration tools such as Airflow.
- Hands-on experience with data integration tools (Talend, DataStage, or Informatica).
- Strong AWS experience, including EC2, S3, RDS, and EMR.
- Expertise in data modeling techniques and data architecture best practices.
- Knowledge of data lineage, technical metadata, data quality, and reconciliation processes.
- Advanced skills in Spark SQL and Python for data automation.
- Ability to tune and optimize data queries for performance.
- Strong communication and negotiation skills.
- Proven experience in leading cross-platform data solutions and complex projects.
- Ability to mentor and lead technical teams.
If you feel you possess the relevant skills to the above, please respond with most updated CV!
20 Mar 2025;
from:
uworkin.com