Lead Data Engineer

BOI (Board of Innovation)

BOI is seeking a skilled Lead Data Engineer to design and manage scalable data pipelines that fuel their AI-driven applications. This role is pivotal for ensuring that data from various sources is harmonized and ready for integration into advanced models. The position is ideal for someone with consulting experience who excels in project-based environments.

In this role you can expect to have the responsibilities:

• Develop ETL pipelines for data extraction, transformation, and loading.
• Design and maintain databases and data storage solutions.
• Implement workflows for data harmonization using tools like Apache Airflow.
• Collaborate with AI engineers for data preparation.
• Ensure data integrity and security across pipelines.
• Optimize data pipelines for performance.
• Support development of adaptive agentic tools.
• Stay updated on best practices in data engineering.

This role requires you to have:

• Bachelor’s or Master’s degree in a relevant field.
• Six years' experience in data engineering and ETL pipeline management.
• Consulting or agency experience in project-focused work.
• Proficiency in Python, SQL, and data transformation libraries.
• Strong cloud data platform experience.
• Experience with data pipeline orchestration tools.
• Solid database design and optimization skills.
• Familiarity with API integration for data processing.
• Advanced English communication skills.

You would benefit from having:

• Experience with vector database systems.
• Knowledge of RAG workflows.
• Understanding of data governance and security.