Big Data Engineer (Apache Spark)

Innosphere SDG Ltd

Innosphere SDG Ltd, established in 1997, specializes in providing technology staffing across North America and has embraced remote work for over 15 years. They offer a collaborative environment, supporting both remote and intermittent office work. The role aims to bolster their data engineering capabilities with advanced big data solutions.

In this role you can expect to have the responsibilities:

• Design and develop scalable data processing solutions using Apache Spark.
• Build and maintain data pipelines supporting advanced analytics.
• Collaborate with engineers to understand and meet data requirements.
• Optimize Spark applications for enhanced performance.
• Implement data quality checks and ensure data integrity.
• Resolve performance bottlenecks in Spark applications.
• Ensure compliance with data security and governance standards.

This role comes with the following benefits:

• Generous time off and flexible work schedule.
• Comprehensive health benefits including dental and vision.
• RRSP with company match.
• Professional development budget and fitness benefits.
• Home office benefit program and annual in-person celebrations.

This role requires you to have:

• Proficiency with Apache Spark for large-scale data processing.
• Strong SQL skills.
• Experience in programming with Python, Scala, or Java.
• Experience with cloud platforms such as AWS, GCP, or Azure.

You would benefit from having:

• Experience with Hadoop, Hive, Kafka, or similar technologies.
• Familiarity with CI/CD pipelines and version control systems.
• Knowledge of containerization and orchestration tools like Docker and Kubernetes.

Innosphere is committed to inclusivity and provides accommodations throughout the selection process. Applicants must reside in Canada and be eligible to work legally. A background check is mandatory.