Data Engineer

Epsilon Solutions Ltd

Epsilon Solutions Ltd is offering an exciting opportunity for an Advanced Data Engineer to lead innovative data solutions in a full-time remote position. This role plays a crucial part in enabling the effective use of data as a strategic asset within the organization, requiring collaboration with cross-functional teams. The engineer will utilize advanced technologies, focusing on scalable and secure data engineering practices.

In this role you can expect to have the responsibilities:

• Offer technical leadership and facilitate collaboration to solve complex data engineering challenges.
• Design, build, and maintain scalable data pipelines for data ingestion, transformation, and integration.
• Leverage innovative technologies to modernize data platforms.
• Develop feature engineering pipelines for machine learning workflows.
• Implement automated testing frameworks for ensuring data quality.
• Optimize data workflows for performance and cost efficiency.
• Mentor team members in data principles and practices.
• Create and review documentation for data solutions.
• Provide cost/benefit analysis to guide architectural decisions.

This role requires you to have:

• 4+ years of Data Development
• 4+ years of SQL and NoSQL technologies
• 3+ years building data pipelines
• 5+ years developing with Java
• 2+ years developing with Python
• 3+ years with Kafka solutions
• 2+ years in feature engineering
• Experience with GCP services
• CI/CD pipelines
• Automated testing
• Version control with Git
• ETL and Data Warehousing concepts
• Understanding of Agile principles

You would benefit from having:

• Knowledge of Structured Streaming
• Experience with GitHub Actions
• Understanding Databricks concepts
• Experience with PySpark