Lead Data Engineer for Databricks Solutions

Shape the Databricks Lakehouse framework on Azure as a Lead Data Engineer. This position requires hands-on experience in building scalable data solutions and governing best practices. In this role, you will design and ensure the integrity of a secure Databricks architecture that meets enterprise standards. You will lead the migration from legacy ETL processes while focusing on optimizing job performance and configurations. Collaboration with key stakeholders is essential to bridge requirements and effective delivery. Key Responsibilities:

• Design and own secure Databricks architectures aligned with standards
• Develop and review PySpark/Spark SQL data workflows
• Oversee the migration of legacy ETL workloads to Databricks
• Enhance performance and implement right-sizing across clusters
• Spearhead data governance initiatives using Unity Catalog

• Minimum 12 years in data platforms or engineering roles
• 4+ years delivering solutions using Databricks
• Expertise in CI/CD tools and practices
• Solid background in data governance and lineage techniques
• Familiarity with analytics tools like Power BI and Azure ML