Data Engineer

ShyftLabs is a fast‑growing data and technology company partnering with Fortune 500 organizations to deliver innovative, data‑driven solutions. We specialize in building scalable, enterprise‑grade platforms that help businesses unlock the value of their data through modern cloud technologies, advanced analytics, and AI‑driven capabilities. We are seeking an experienced Data Architect to work directly with enterprise clients, providing architectural leadership and hands‑on technical expertise across modern data engineering initiatives. This is a client‑facing role that combines solution architecture, technical consulting, and delivery oversight, requiring the ability to translate complex business requirements into scalable, secure, and reliable data solutions. You will collaborate closely with client stakeholders, engineering teams, and technology partners to define architectural direction, establish engineering standards, and guide data platform implementations from design through production. You will also review partner‑developed architectures, code, data models, and pipelines to ensure solutions meet enterprise requirements and align with established best practices. The ideal candidate is a technically strong, hands‑on architect who can confidently engage with senior client stakeholders, communicate complex technical concepts clearly, and influence architectural decisions while supporting distributed engineering teams. Lead Client‑Facing Architecture Discussions: Work directly with client stakeholders to understand business objectives, technical requirements, existing data environments, and architectural challenges. Translate these needs into practical, scalable data architecture solutions. Define Enterprise Data Architecture: Design and recommend modern data platform architectures using Azure Databricks, Delta Lake, and cloud‑based data engineering technologies, ensuring alignment with client objectives and enterprise standards. Drive Technical Discovery and Solution Design: Facilitate technical workshops, discovery sessions, and architecture reviews to assess current‑state environments, identify gaps, evaluate solution options, and define target‑state architectures. Provide Hands‑On Technical Leadership: Develop, validate, and review data engineering solutions using SQL, Python/PySpark, Apache Airflow, and dbt Core, balancing architectural oversight with practical implementation expertise. Guide Data Pipeline Development: Establish architecture and implementation standards for ingestion pipelines that integrate APIs, databases, files, and cloud data sources using Databricks‑native capabilities such as Lakeflow Connect and Auto Loader. Establish Data Architecture Standards: Define and promote reusable architectural patterns, implementation guidelines, and engineering best practices across client projects and distributed delivery teams. Design and guide the implementation of Bronze, Silver, and Gold data layers to support reliable ingestion, standardized transformations, curated data products, and downstream analytics. Drive Data Governance and Modeling: Establish appropriate practices for Unity Catalog, data access controls, lineage, semantic modeling, data discoverability, and enterprise data governance. Oversee Technical Delivery and Quality: Review partner and engineering team deliverables, including architecture designs, code, data models, and pipelines, ensuring alignment with requirements, performance expectations, security standards, and long‑term maintainability. Champion Data Quality and Reliability: Define standards for data validation, automated testing, pipeline monitoring, error handling, and operational readiness to ensure reliable production data products. Guide implementation of Git‑based workflows, automated testing, CI/CD pipelines, and controlled deployment practices across development, testing, and production environments. Advise on Performance and Cost Optimization: Evaluate platform performance, processing efficiency, scalability, and cloud consumption, recommending improvements that balance technical requirements with business value. Drive AI‑Assisted and Spec‑Driven Development: Encourage the effective use of AI‑assisted development tools and specification‑driven engineering practices while maintaining architectural integrity, code quality, and delivery standards. Coordinate Distributed Engineering Teams: Provide technical direction, resolve architectural questions, and facilitate collaboration between client stakeholders, internal engineers, and external technology partners. Communicate Progress and Technical Risks: Clearly communicate architectural decisions, implementation trade‑offs, dependencies, risks, and recommendations to both technical and non‑technical stakeholders. Maintain architectural ownership throughout the project lifecycle, from discovery and solution design through implementation, deployment, and ongoing opti