Principal Data Engineer (Azure)

Tiger Analytics

Tiger Analytics, a leading AI and analytics consulting firm headquartered in Silicon Valley, is at the forefront of data-driven innovations. The company operates global delivery centers and boasts a substantial remote workforce. Employees are a critical part of the AI revolution, working in teams that push technological limits. Recognized as a Great Place to Work-Certified™ organization, Tiger Analytics values expertise and collaboration.

In this role you can expect to have the responsibilities:

• Design and develop scalable, metadata-driven data ingestion pipelines for both batch and streaming datasets.
• Execute and conceptualize high-performance data processing and harmonization for structured and unstructured data.
• Schedule, orchestrate, and validate data pipelines.
• Implement exception handling and log monitoring for debugging purposes.
• Collaborate across teams to guide decisions on tech stack and tools.
• Work with consulting, data science, and app development teams to deliver analytical solutions.

This role comes with the following benefits:

• Career growth opportunities in a vibrant entrepreneurial environment with significant responsibility.

This role requires you to have:

• Experience with Azure Data Factory, PySpark, Databricks, and Azure SQL Database.
• Strong programming, unit testing, and debugging skills in SQL, Python, Scala, or Java.
• Experience with big data technologies such as Hadoop, Spark, Airflow, NiFi, Kafka, Hive, Neo4J, and Elasticsearch.

You would benefit from having:

• Knowledge of Azure Synapse Analytics, Event Hub & Streaming Analytics, Cosmos DB, and Purview.
• Understanding of file formats like Delta Lake, Avro, Parquet, JSON, and CSV.
• Experience with REST APIs, data lake, and lakehouse projects.
• Certifications such as Data Engineering on Microsoft Azure (DP-203) or Databricks Certified Developer.

Compensation is one of the best in the industry and aligned with expertise and experience.