Senior Data Engineer - PySpark & Data Ingestion

Westpac

Westpac invites a Senior Data Engineer to join their Information Security Group, striving to enhance their Security Insights platform operating on Palantir. The role requires collaboration with the Service Operations Manager and technical teams to ensure strategic implementation and compliance.

In this role you can expect to have the responsibilities:

• Develop and maintain a roadmap for updates and deployment requirements.
• Establish technical dependencies across the technology stack.
• Act as the primary point of contact with service managers for roadmap and patching activities.
• Ensure compliance with technical controls and standards.
• Collect and verify necessary artefacts.
• Serve as the technical subject matter expert on currency and security compliance.
• Collaborate with teams to ensure timely execution of the roadmap.

This role comes with the following benefits:

• Opportunities for career development and flexible working arrangements.
• Comprehensive parental leave policies for various family arrangements.
• Continuous professional growth supported by a collaborative team.

This role requires you to have:

• Expertise in transforming datasets using PySpark/Python, with consideration for Java/Scala.
• Experience with data ingestion via REST API, SFTP, Bash, and SQL.
• Manage ITSM data services within an Agile Scrum framework, improving CI/CD processes.
• Build efficient data pipelines in distributed environments.

You would benefit from having:

• Knowledge of data governance for onboarding and quality control.
• Experience with multi-tenancy data lake platforms like Databricks, Snowflake, or Cloudera.

Westpac values diversity and inclusion, ensuring a supportive environment for all employees. They offer personalised career development and uphold equal opportunity practices.