Senior Data Engineer

Enformion

Enformion is a leading data and analytics firm that strengthens digital marketplaces through AI-driven solutions for fraud prevention and risk management. Their innovation aids in fostering trust between shoppers and merchants by using vast data intelligence and advanced behavioral analysis. They are seeking a Senior Data Engineer to join their fast-paced, startup-minded team to push the limits of big data technology.

In this role you can expect to have the responsibilities:

• Implement and maintain the big data platform and infrastructure.
• Optimize and develop MySQL stored procedures, scripts, and indexes.
• Create Hive schemas, Spark jobs using PySpark and Scala, and UDFs in Java.
• Design and maintain efficient ETL processes for large-scale datasets.
• Develop pipelines using Airflow for data monitoring and analysis.
• Troubleshoot and optimize Hadoop cluster issues.

This role comes with the following benefits:

• Flexible PTO.
• 6% 401K match.
• Full coverage of employee medical, dental, and vision costs.
• Coverage for dependents' medical, dental, and vision at 75%.

This role requires you to have:

• Proficiency in Java, Scala, PySpark, Perl, Shell Scripting, and Python.
• Extensive knowledge of Hadoop ecosystem applications.
• Experience with multi-terabyte data warehouses and performance issues.
• Strong SQL and No-SQL skills.
• Expertise in data modeling and architectural database design.
• Experience using Git for version control.

You would benefit from having:

• Cluster configuration and Hadoop administration.
• Distributed computing principles and performance tuning.
• Machine Learning.