Expert Data Engineer for Large-Scale Databricks Solutions
4 days ago
Rubicon Path is seeking an experienced Data Engineer to join our team. As a key member of our engineering group, you will design, develop, and implement large-scale data pipelines using Databricks.
Job Description
In this role, you will be responsible for developing and managing ETL pipelines, jobs, and workflows in Databricks. You will work closely with cross-functional teams to ensure the efficient transformation and movement of data to meet business needs. Your expertise in Delta Lake, Databricks Workflows, and Python/PySpark will enable you to automate ETL pipelines and handle Change Data Capture (CDC) for incremental data loads.
Key Responsibilities
- Develop and manage ETL pipelines, jobs, and workflows in Databricks.
- Collaborate with cross-functional teams to design, develop, and implement large-scale data pipelines.
- Automate ETL pipelines using Delta Live Tables and handle Change Data Capture (CDC) for incremental data loads.
- Ensure the efficient transformation and movement of data to meet business needs.
- Work closely with data architects and business teams to optimize data quality and performance.
Requirements
To be successful in this role, you will need:
- 7+ years of experience using ETL tools such as Microsoft SSIS, stored procedures, T-SQL.
- 2+ years of experience working with Delta Lake, Databricks, and Azure Databricks pipelines.
- Strong knowledge of Delta Lake for data management and optimization.
- Familiarity with Databricks Workflows for scheduling and orchestrating tasks.
- 2+ years of experience working with Python and PySpark.
- Solid understanding of the Medallion Architecture (Bronze, Silver, Gold) and experience implementing it in production environments.
- Hands-on experience with CDC tools (e.g., GoldenGate) for managing real-time data.
- SQL Server, Oracle experience.
Salary Range
$140,000 - $170,000 per year based on experience and qualifications.
-
Technical Director
5 days ago
Toronto, Ontario, Canada Databricks Full timeAbout DatabricksDatabricks is a leading cloud-based data and AI platform that enables businesses to unlock the full potential of their data. Our team of experts activates and accelerates the value customers get from their data and AI, securely delivering AI's potential at scale.Salary Range: $180,000 - $220,000 per annumJob Description: As a dynamic Solution...
-
Data Engineering Professional
4 weeks ago
Toronto, Ontario, Canada Fluent, LLC Full timeAbout FluentAt Fluent, we're revolutionizing the advertising industry with cutting-edge technology and innovative solutions. Our goal is to build a seamless and relevant experience for online shoppers, leveraging machine learning and artificial intelligence to drive business growth.Job OverviewWe're seeking an experienced Data Engineer to join our team,...
-
Big Data Engineering Expert
10 hours ago
Toronto, Ontario, Canada Quantum World Technologies Inc. Full timeQuantum World Technologies Inc. is looking for a skilled Big Data Engineering Expert to join our team in Toronto, ON.We offer a competitive salary of $120,000 per year, plus benefits.About the Role:The successful candidate will have 6-12 years of experience in big data engineering and will be responsible for designing, implementing, and maintaining...
-
Databricks Engineer
2 weeks ago
Toronto, Ontario, C6A, Ontario, Canada Smart IT Frame LLC Full timeHi There,We have a contract role with one of our clients. Please find the below details.Role: Databricks EngineerLocation: Toronto, Ontario (Hybrid)Type: ContractRequired Skills:* Proficiency in Databricks and Apache Spark.* Experience with one or more cloud platforms (Azure Data Lake, AWS S3, GCP BigQuery).* Strong programming skills in Python, Scala, or...
-
Data Engineer
4 weeks ago
Toronto, Ontario, Canada Rubicon Path Full timeAbout the RoleRubicon Path is seeking an experienced Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining large-scale data pipelines using Databricks, Delta Lake, and Azure.Key ResponsibilitiesDesign and implement ETL processes using Databricks, Delta Lake, and Azure Data...
-
Data Engineer
2 days ago
Toronto, Ontario, Canada Capgemini Engineering Full timeAbout Capgemini EngineeringCapgemini Engineering is a global leader in engineering and technology consulting. We help organizations accelerate their digital transformation by providing innovative solutions that drive business value.About the JobWe are currently seeking a highly skilled Data Engineer - Large Scale Big Data Infrastructure to join our team. As...
-
Senior Data Engineer
1 week ago
Toronto, Ontario, Canada Rubicon Path Full time**Job Summary:**We are seeking a highly skilled Senior Data Engineer with expertise in designing, developing, and implementing large-scale data pipelines using Databricks and related tools. As an ETL Specialist at Rubicon Path, you will play a critical role in ingesting, transforming, and storing large-scale datasets for our business.Key...
-
Data Engineering Specialist
2 weeks ago
Toronto, Ontario, Canada Rubicon Path Full timeRubicon Path Job DescriptionJob Title: Data Engineering Specialist - Cloud Data ArchitectureEstimated Salary Range: $140,000 - $200,000 per yearJob SummaryRubicon Path is seeking an experienced Data Engineering Specialist to join our team. As a key member of our cloud data architecture team, you will be responsible for designing, developing, and maintaining...
-
Databricks Developer
4 weeks ago
Toronto, Ontario, Ontario, Canada Tata Consultancy Services Full timeInclusion without Exception:Tata Consultancy Services (TCS) is an equal opportunity employer, and embraces diversity in race, nationality, ethnicity, gender, age, physical ability, neurodiversity, and sexual orientation, to create a workforce that reflects the societies we operate in. Our continued commitment to Culture and Diversity is reflected in our...
-
Data Engineer
2 days ago
Toronto, Ontario, Canada Strategic Solutions Technology Staffing Inc. Full timeAbout Our ClientStrategic Solutions Technology Staffing Inc. is a multi-strategy alternative asset manager with global expertise in private equity, venture capital, credit, and more.We are seeking an experienced Data Engineer to join our data-driven team. The ideal candidate will have a strong background in building and managing large-scale data pipelines...
-
Toronto, Ontario, Canada Rubicon Path Full timeJob Summary: We are seeking an experienced Enterprise Architecture Expert to join our team at Rubicon Path. As a key member of our architecture team, you will be responsible for designing and implementing large-scale solutions that meet the complex needs of our clients.About Us: Rubicon Path is a leading provider of innovative technology solutions. Our team...
-
Machine Learning Engineer
2 weeks ago
Toronto, Ontario, Canada NearSource Full timeAbout NearSource TechnologiesWe're a forward-thinking company driving innovation in data-driven decision-making across our digital platform.Job SummaryThis is a unique opportunity to contribute to impactful machine learning initiatives as part of a collaborative team.Compensation and Benefits$120,000 - $160,000 per year, depending on experience. We offer a...
-
Data Engineer Specialist
4 weeks ago
Toronto, Ontario, Canada Q1 Technologies, Inc. Full timeKey ResponsibilitiesWe are seeking an experienced Data Engineer to join our team at Q1 Technologies, Inc. The ideal candidate will have a strong background in cloud-based data engineering and experience with AWS technologies.Key SkillsAWS Glue, Python, AWS Redshift, Airflow, and AWS CloudWatchGood experience in cloud data engineeringThe successful candidate...
-
Toronto, Ontario, Canada S.i. Systems Full timeS.i. Systems Seeks Experienced Salesforce ExpertAt S.i. Systems, we are committed to delivering exceptional solutions that meet our clients' needs. We are currently seeking a highly skilled Sr. Salesforce Solution Architect with experience in Service Cloud and Experience Cloud to support a large-scale Salesforce implementation.Key Responsibilities:Develop...
-
Toronto, Ontario, Canada Fluent, LLC Full timeJob Description:A Senior Data Engineer opportunity has arisen at Fluent, LLC, a high-flying performance marketing company. As part of our team, you will play a crucial role in building data products to power our business lines using your expertise in Databricks pipeline development.The successful candidate will be responsible for developing, deploying, and...
-
Data Engineer for Large Language Models
4 weeks ago
Toronto, Ontario, Canada Bitstrapped Full timeBitstrapped is a fast-growing data engineering and cloud consulting company.We help companies invest in forward-thinking data, machine learning solutions, and cloud-native infrastructure.Job SummaryWe are seeking an experienced Data Engineer to work with our team on developing and deploying large language models.The ideal candidate will have strong expertise...
-
Cloud Data Engineer
2 days ago
Toronto, Ontario, Canada Maarut Inc Full timeJob SummaryMaarut Inc is seeking a highly skilled Cloud Data Engineer - Senior to join our team. The successful candidate will be responsible for designing, developing, and implementing large-scale cloud-based data systems, with a focus on scalability, performance, and reliability.About UsMaarut Inc is a leading provider of innovative solutions in the field...
-
Cloud Data Engineer
4 weeks ago
Toronto, Ontario, Canada Tata Consultancy Services Full timeAbout the RoleWe are seeking a highly skilled Cloud Data Engineer to join our team in Canada. In this role, you will be responsible for designing, implementing, and managing large-scale data platforms on Azure.Key ResponsibilitiesDesign and develop scalable data pipelines using Azure Data Factory and DatabricksWork with cross-functional teams to implement...
-
Data Scientist and Innovator
2 weeks ago
Toronto, Ontario, Canada Disability Solutions Full timeJob OverviewThe position of Data Scientist and Innovator is a key role within our organization, focused on leveraging data science methodologies to drive business growth and improvement. As a member of our team, you will collaborate with cross-functional groups to identify opportunities for innovation and develop strategies for extracting insights from...
-
Toronto, Ontario, Canada Royal Bank of Canada Full timeAbout the Role:This is a unique opportunity to lead a team of developers in delivering high-performing cloud-based big data solutions. As a Senior Data Architecture Lead, you will be responsible for designing and implementing data pipelines using big data technologies such as Spark and Databricks on medium to large scale datasets.Key Responsibilities:-...