Engineer - Data Platforms

Westpac

To become number one for customer service

Westpac is Australia’s oldest bank and company, one of four major banking organisations in Australia and one of the largest banks in New Zealand. We provide a broad range of banking and financial services in these markets, including consumer, business and institutional banking and wealth management services.

The** Data Platforms (DP) **function comprises of a team of data professionals dedicated to building game-changing data capabilities for Westpac Group and shaping the future of banking. We understand the big picture and focus on and deliver initiatives that will significantly and positively impact the Westpac Group. In this hands-on role, you will be a strong technical expert, responsible for building real-time and batch data solutions. You will handling multiple programs concurrently and be a go-to SME for the development of engineering solutions.

Responsibilities include:

• Enhancement, maintenance, and support of the technical deliverables including hands-on coding, ensuring the quality of team outputs, representing the team in various forums, and aligning team contributions with the overall strategic roadmap.
• Design and implement scalable, reliable, and secure solutions, and collaborating with cross-functional teams to gather requirements and ensure successful delivery.
• Collaborate with fellow Engineers across the domain to define technical direction for service offerings, and work with Service Owners and key business stakeholders to shape the delivery roadmap and coordinate with support teams to ensure smooth delivery.

What do I need?

• Solid engineering experience developing large-scale applications using Scala and Python, plus handling large datasets using PySpark in numerous production workloads.
• Proven expertise in real-time streaming jobs, time data processing and pipelines in production environments.
• Extensive experience in optimising Spark code using Directed Acyclic Graphs (DAGs) involving several strategies to enhance performance and efficiency.
• Experienced working with HDFS, pig, hive, HBase, Phoenix etc.
• Proven experience and understanding of cloud platforms, particularly within Azure, having worked with ALDS, EventHub, Stream Analytics, HDInsight, Synapse Analytics, Data Factory, Cosmos DB, Kubernetes. Similar experience with AWS/GCP is also fine.
• Strong functional programming and ability with complex data structures, with understanding and experience in data warehousing concepts.
• Expertise in DevOps functions, contributing to CI/CD pipelines, and experience in implementing MLOps pipelines in production is advantageous.
• Proficient in using source control tools, preferably Bitbucket, with a good understanding of branching and merging strategies.

This role is part in the UNITE program which will be Westpac’s biggest transformation project ever. It is a business-led, tech-enabled transformation - delivering simplification and benefits for our customers, employees and shareholders.

A major priority for the Group over the next 3 to 4 years, UNITE is about delivering ‘one best way’ to make our processes, systems and tech simpler so we can deliver a better experience for customers, make things easier for our people, and reduce risk for generations to come. 

• 🔀

  Data Pipelines

  Develop and maintain processes for the extraction, flow, and transformation of data

• 🔲

  SQL and Databases

  Use SQL to query databases to extract and process data

• 🤖

  Scripting and Automation

  Develop scripts and automated workflows to improve the efficiency and accuracy of repeated tasks

• 👥

  Collaboration

  Works with others by being open, clear in communication and listening to achieve goals

• 🧮

  Numerical problem solving

  Works with numerical information and performs mathematical calculations to solve problems

• 🔍

  Attention to detail

  Accurately identifies and rectifies discrepancies or errors that exists in information and deliverables