Databricks ETL Developer Position

4 weeks ago

Toronto, Ontario, Canada Paralucent Full time

Paralucent Databricks ETL Developer Job Description

We are seeking a skilled Databricks ETL Developer to support our data warehousing, data lakes, and analytics initiatives. In this role, you will be responsible for developing, maintaining, and optimizing ETL processes within Databricks to efficiently transform and move data.

Key Responsibilities:

• Design, develop, and implement ETL pipelines using Databricks and related tools to ingest, transform, and store large-scale datasets.

• Utilize Databricks, Delta Lake, Delta Live Tables, and Spark to process both structured and unstructured data.

• Collaborate with business teams to design scalable data solutions that enhance data accessibility and analytical capabilities.

Required Skills and Experience:

• Proven experience in developing ETL pipelines using Databricks.

• Strong knowledge of Delta Lake for data management and optimization.

• Hands-on experience with CDC tools (e.g., GoldenGate) for managing real-time data.

• Familiarity with Databricks Workflows for scheduling and orchestrating tasks.

Technical Knowledge and Expertise:

• Proven experience in developing and managing ETL pipelines, jobs, and workflows in Databricks.

• In-depth understanding of Delta Lake for building data lakes and managing ACID transactions, schema evolution, and data versioning.

• Experience in automating ETL pipelines using Delta Live Tables, including handling Change Data Capture (CDC) for incremental data loads.

• Proficiency in structuring data pipelines with the Medallion Architecture to scale data pipelines and ensure data quality.

• Hands-on experience in developing streaming tables in Databricks using Structured Streaming and readStream to handle real-time data.

• Expertise in integrating CDC tools like GoldenGate or Debezium for processing incremental updates and managing real-time data ingestion.

• Experience with Unity Catalog for managing data governance, access control, and ensuring compliance.

• Skilled in managing clusters, jobs, autoscaling, monitoring, and performance optimization in Databricks environments.

• Knowledge of using Databricks Autoloader for efficient batch and real-time data ingestion.

• Experience with data governance best practices, including implementing security policies, access control, and auditing with Unity Catalog.

• Proficient in creating and managing Databricks Workflows to orchestrate job dependencies and schedule tasks.

• Strong knowledge of Python, PySpark, and SQL for data manipulation and transformation.

• Experience in integrating Databricks with cloud storage solutions such as Azure Blob Storage, AWS S3, or Google Cloud Storage.

• Familiarity with external orchestration tools like Azure Data Factory.

Certifications:

• Certified in one or more of the following:

o Databricks Certified Data Engineer Associate

o Databricks Certified Professional Data Engineer

o Microsoft Certified: Azure Data Engineer Associate

o AWS Certified Data Analytics - Specialty

o Google Cloud Professional Data Engineer

Communication and Leadership Skills:

• Ability to collaborate effectively with cross-functional teams and articulate complex technical concepts to non-technical stakeholders.

• Strong problem-solving skills with experience working in Agile or Scrum environments.

• Provide technical guidance and support to team members on Databricks best practices.

  • Databricks ETL Developer

    4 weeks ago

    Toronto, Ontario, Canada Paralucent Full time

    Job Title: Databricks ETL DeveloperJob Summary:We are seeking a skilled Databricks ETL Developer to support our data warehousing, data lakes, and analytics initiatives. As a key member of our team, you will be responsible for designing, developing, and maintaining ETL processes within Databricks to ensure efficient data transformation and movement.Key...

  • Databricks ETL Developer

    4 weeks ago

    Toronto, Ontario, Canada Paralucent Full time

    Job Title: Databricks ETL DeveloperWe are seeking a skilled Databricks ETL Developer to support our data warehousing, data lakes, and analytics initiatives. In this role, you will be responsible for developing, maintaining, and optimizing ETL processes within Databricks to ensure efficient data transformation and movement.Key Responsibilities: Design,...

  • Databricks ETL Developer

    2 weeks ago

    Toronto, Ontario, Canada Paralucent Full time

    We are seeking a highly skilled Databricks ETL Developer to join our team at Paralucent in Toronto. This is a 5-month contract position with the possibility of yearly extension.As a Databricks ETL Developer, you will be responsible for designing, developing, and implementing scalable data pipelines using Databricks and related tools. Your primary focus will...

  • Databricks ETL Engineering Specialist

    1 week ago

    Toronto, Ontario, Canada Paralucent Full time

    We are seeking an experienced Databricks ETL engineer to support our data warehousing, data lakes, and analytics initiatives. As a key member of our team, you will be responsible for designing, developing, and maintaining efficient ETL pipelines using Databricks and related tools.Key ResponsibilitiesDesign and implement scalable ETL processes to ingest,...

  • Data Engineer with Expertise in ETL and Databricks

    6 days ago

    Toronto, Ontario, Canada Maarut Inc Full time

    About the RoleMaarut Inc is seeking a highly skilled Data Engineer to join our team. The ideal candidate will have extensive experience in designing, developing, and implementing ETL pipelines using Databricks and related tools.The successful candidate will be responsible for creating high-quality ETL mappings/scripts/notebooks, developing data pipelines...

  • ETL Developer

    2 weeks ago

    Toronto, Ontario, Ontario, Canada TEEMA Full time

    ETL Developer4+ months contract3 days onsite per week is required/Toronto LocationMust Haves:10 years using ETL tools such as Microsoft SSIS, stored procedures, T-SQL 2+ Delta Lake, Databricks and Azure Databricks pipelinesStrong knowledge of Delta Lake for data management and optimization.Familiarity with Databricks Workflows for scheduling and...

  • Data Engineer ETL Specialist

    7 days ago

    Toronto, Ontario, Canada Wirehead Full time

    Wirehead is seeking an experienced Data Engineer ETL Specialist to join our team on a 4-month contract with a potential extension of up to 1 year. The successful candidate will be based in Toronto and work in a hybrid environment, commencing within 2-3 weeks.Key Responsibilities:The ideal candidate will have extensive experience working with ETL tools,...

  • Business Intelligence ETL Developer

    4 weeks ago

    Toronto, Ontario, Canada Foilcon Full time

    Job Title: Business Intelligence ETL DeveloperJob Summary:Foilcon is seeking a highly skilled Business Intelligence ETL Developer to join our team. As a key member of our data analytics team, you will be responsible for designing, developing, and implementing data ingestion and transformation solutions using Azure Data Factory and Databricks.Key...

  • Business Intelligence ETL Developer

    4 weeks ago

    Toronto, Ontario, Canada Foilcon Full time

    Job DescriptionFoilon is seeking a highly skilled Business Intelligence ETL Developer to join our team. As a key member of our data team, you will be responsible for designing, developing, and implementing data ingestion and transformation frameworks using Azure Data Factory and Databricks.Key Responsibilities:Design and develop data ingestion frameworks...

  • Azure Databricks Solutions Architect

    2 weeks ago

    Toronto, Ontario, Canada Astra North Infoteck Inc. Full time

    Azure Databricks Role OverviewAstra North Infoteck Inc. is seeking an experienced Azure Databricks Architect to help us design and implement data processing pipelines. The successful candidate will have extensive experience with Azure Data Factory, Databricks, and Pyspark based data processing.Key ResponsibilitiesDesign and implement data processing...

  • Senior Data Engineer

    2 weeks ago

    Toronto, Ontario, Canada Rubicon Path Full time

    As a Senior Data Engineer at Rubicon Path, you will be responsible for designing, developing, and maintaining large-scale data pipelines using Databricks and Python. Your expertise in ETL tools, such as Microsoft SSIS, and strong knowledge of Delta Lake for data management and optimization, will be crucial in ensuring the efficient transformation and...

  • Business Intelligence ETL Developer

    4 weeks ago

    Toronto, Ontario, Canada Foilcon Full time

    Job DescriptionFoilocn is seeking a highly skilled Business Intelligence ETL Developer to join our team. The ideal candidate will have experience in designing, developing, and implementing data ingestion frameworks from Oracle sources to Azure Data Lake.Key Responsibilities:Design and develop data ingestion frameworks using Oracle Golden Gate and Azure Data...

  • Databricks Developer

    2 weeks ago

    Toronto, Ontario, Ontario, Canada Atlantis IT Group Full time

    Role name: Developer Role Description: Databricks Developer Competencies: Digital : Databricks Experience (Years): 6-8 Essential Skills: Databricks Developer Desirable Skills: Databricks Developer Country: Canada Branch | City | Location: TCCI - Canada TORONTO BRAMPTON, ONTARIO

  • Business Intelligence ETL Specialist

    3 weeks ago

    Toronto, Ontario, Canada Foilcon Full time

    Job DescriptionFoIicone is seeking an experienced Business Intelligence ETL Specialist to join our team.Key Responsibilities:Design and Develop ETL Framework: Design and implement an ETL framework from Oracle source to Azure Data Lake for initial load and incremental ETL.Tools and Technologies: Utilize Oracle Golden Gate for data ingestion and change data...

  • Data Engineer

    1 month ago

    Toronto, Ontario, Canada Foilcon Full time

    Job DescriptionFoIalcon is seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and implementing data ingestion frameworks from Oracle sources to Azure Data Lake, including initial load and incremental ETL. You will work with various tools such as Oracle Golden Gate, Azure Data...

  • Senior ETL Developer Position

    3 weeks ago

    Toronto, Ontario, Canada Futran Tech Solutions Pvt. Ltd. Full time

    Role: Senior ETL Developer Location: Toronto, Canada (Candidate will be visiting the office 4 days per week)Job Overview: We are seeking an ETL Developer with BA experience to join our stimulating and dynamic team.About the Position: The ideal candidate will have 6+ years of capital markets experience and strong skills in data integration and ETL...

  • Senior ETL Developer Position

    1 month ago

    Toronto, Ontario, Canada Futran Tech Solutions Pvt. Ltd. Full time

    Senior ETL Developer OpportunityWe are seeking a highly skilled Senior ETL Developer to join our dynamic team at Futran Tech Solutions Pvt. Ltd. in Toronto, Canada.About the RoleThe ideal candidate will have 6+ years of experience in capital markets and a strong background in ETL development. The successful candidate will be responsible for designing,...

  • Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Lyons Consulting Group Full time

    Job Title: Senior Software EngineerAbout the Role:We are seeking a highly skilled Senior Software Engineer to join our team at Lyons Consulting Group. As a Senior Software Engineer, you will be responsible for designing and implementing efficient ETL or ELT pipelines to move data between systems, ensuring data quality and consistency through cleansing,...

  • Senior Data Engineer

    1 week ago

    Toronto, Ontario, Canada Foilcon Full time

    Job Summary:FoIICON is seeking a highly skilled Senior Data Engineer to join our team. As an experienced professional in designing and implementing data ingestion frameworks, you will play a key role in shaping the future of our organization.About the Role:We are looking for a talented individual with expertise in Azure-based data engineering solutions. The...

  • Highly Skilled ETL Data Specialist for Large-Scale Data Projects

    1 week ago

    Toronto, Ontario, Canada Paralucent Full time

    Job OverviewWe are seeking a seasoned ETL (Extract, Transform, Load) specialist to join our team at Paralucent. This is an exciting opportunity to work on large-scale data projects and contribute to the development of cutting-edge data solutions.Job Responsibilities:- Design and develop efficient ETL pipelines using Databricks and related tools to handle...