Data Engineer

Shamrock Trading Corporation

Shamrock Trading Corporation, a leading provider in the transportation, finance, and technology sectors, is frequently celebrated as a premier workplace. Based in Overland Park, KS, and with offices nationwide, the company continues to grow at a remarkable pace, driven by an exceptional team and strategic evolution in the market. They are seeking a Data Engineer to join their Data Services team, aiming to enhance data capabilities and support business intelligence initiatives.

In this role you can expect to have the responsibilities:

• Develop and maintain scalable ETL/ELT data pipelines using Databricks, Apache Spark, and Delta Lake.
• Optimize Spark-based workloads through performance tuning techniques.
• Design and implement cloud data architectures using AWS services such as S3, Glue, and Redshift.
• Build streaming data solutions with Kafka and Databricks Structured Streaming.
• Implement data validation, observability, and governance practices.
• Collaborate with analysts and data scientists to ensure data aligns with business needs.
• Apply CI/CD best practices and infrastructure-as-code for data pipelines using Terraform, dbt, and GitHub Actions.

This role comes with the following benefits:

• Fully paid healthcare, dental, and vision for employees and dependents.
• Competitive PTO and paid leave policies.
• Generous 401(k) contributions and employee stock ownership.
• Onsite gym and discounted fitness center memberships.

This role requires you to have:

• Data engineering expertise, particularly with cloud platforms.
• Hands-on experience with Databricks, Spark, and Delta Lake.
• Strong SQL skills for data transformation and performance optimization.
• Knowledge of AWS services including Lambda, ECS, and MSK.
• Experience with Python, Apache Spark, and Kafka ecosystems.
• Understanding of data security and governance.

You would benefit from having:

• Familiarity with relational and NoSQL databases like Postgres and MongoDB.
• Experience with version control using Git.
• Knowledge of data visualization techniques in PowerBI and Excel.

Shamrock Trading Corporation provides an environment focused on career growth with ample training opportunities, aiming to be a long-term career choice for employees.