See more Collapse

Senior Data Engineer(Databricks)

3 months ago

Old Toronto, Canada Prodigy Labs Full time

Data Engineer with Databricks & Spark Experience

We are looking for a data engineer who has strong experience in building scalable and reliable data pipelines using Databricks and Spark. You will be working with various data sources and formats, and transforming them into valuable insights for our business.

Responsibilities

  • Design, develop, and maintain data pipelines using Databricks and Spark, and other cloud technologies as needed
  • Optimize data pipelines for performance, scalability, and reliability
  • Ensure data quality and integrity throughout the data lifecycle
  • Collaborate with data scientists, analysts, and other stakeholders to understand and meet their data needs
  • Troubleshoot and resolve data-related issues, and provide root cause analysis and recommendations
  • Document data pipeline specifications, requirements, and enhancements, and communicate them effectively to the team and management
  • Create new data validation methods and data analysis tools, and share best practices and learnings with the data engineering community
  • Implement ETL processes and data warehouse solutions, and ensure compliance with data governance and security policies

Please Note: This is a hybrid position and will require at least 2 days in the office per week. Successful candidates will need to complete a background check.

Qualifications

  • Bachelor’s degree in Computer Science, Engineering, or related field, or equivalent work experience
  • 5+ years of experience in data engineering with Databricks and Spark
  • Proficient in SQL and Python, and familiar with Java or Scala
  • Experience with cloud platforms, such as Azure or AWS
  • Experience with big data technologies, such as Kafka, Hadoop, Hive, etc.
  • Experience with data warehouse and data lake concepts and architectures
  • Experience with data integration and ETL tools, such as Azure Data Factory or Talend
  • Experience with data visualization and reporting tools, such as Power BI or Tableau
  • Strong analytical and problem-solving skills
  • Excellent communication and teamwork skills

Benefits

  • We are a collective group of people and collaboration is key to our process
  • We don’t work for our clients, we work with them
  • A Flexible Hybrid Working Environment
  • Easily accessible downtown location
  • Competitive compensation commensurate with experience
  • Everyone brings something valuable to the table in our supportive, challenging, and collaborative, diverse work environment

All Employees Can Participate in:

  • Company paid health benefits: 100% medical, dental and vision coverage
  • Corporate-discounted Gym Membership through GoodLife
  • Company discount program including Travel, Shopping, Attractions, Wellness, & Sporting events, just to name a few
  • Access to an Employee & Family Assistance program (EAP)
  • Employee Referral Program
  • Employee Opportunity Program
  • Professional Development Program
  • Philanthropic Events
  • Social Events



Inclusion & Diversity

We foster an inclusive and diverse workforce, believing our strength stems from our individual differences. Our employees, partners, and clients continuously benefit from the innovation and creativity grounded in these values. We strive to be a company that attracts a diverse group of highly skilled people who know that their contributions will be valued and that they will be heard. We are committed to building a corporate culture with people who are excited to join our team, do their best work, and grow with us

#J-18808-Ljbffr

We have other current jobs related to this field that you can find below

  • Senior Data Engineer

    2 weeks ago

    Toronto, Canada Pacific Data Integrators Full time

    Position:Senior Data EngineerHybrid Role ROLE AND RESPONSIBILITIES1. Work closely with the Enterprise Data Engineering team to design and construct data architectures, databases, and processing systems that support big data transformation and structuring.2. Leverage deep understanding of SQL, data warehousing, ETL processes, and Python to develop large scale...

  • Senior Data Engineer

    2 weeks ago

    Toronto, Canada Pacific Data Integrators Full time

    Position:Senior Data EngineerHybrid Role ROLE AND RESPONSIBILITIES1. Work closely with the Enterprise Data Engineering team to design and construct data architectures, databases, and processing systems that support big data transformation and structuring.2. Leverage deep understanding of SQL, data warehousing, ETL processes, and Python to develop large scale...

  • Senior Data Engineer

    2 weeks ago

    Toronto, Canada Pacific Data Integrators Full time

    Position: Senior Data Engineer Hybrid Role ROLE AND RESPONSIBILITIES 1. Work closely with the Enterprise Data Engineering team to design and construct data architectures, databases, and processing systems that support big data transformation and structuring. 2. Leverage deep understanding of SQL, data warehousing, ETL processes, and Python to develop...

  • Senior Data Engineer

    2 weeks ago

    Toronto, Canada Pacific Data Integrators Full time

    Position:Senior Data EngineerHybrid Role ROLE AND RESPONSIBILITIES1. Work closely with the Enterprise Data Engineering team to design and construct data architectures, databases, and processing systems that support big data transformation and structuring.2. Leverage deep understanding of SQL, data warehousing, ETL processes, and Python to develop large scale...

  • Senior Data Engineer

    2 weeks ago

    Toronto, ON, Canada Pacific Data Integrators Full time

    Position:Senior Data EngineerHybrid Role ROLE AND RESPONSIBILITIES1. Work closely with the Enterprise Data Engineering team to design and construct data architectures, databases, and processing systems that support big data transformation and structuring.2. Leverage deep understanding of SQL, data warehousing, ETL processes, and Python to develop large scale...

  • Solutions Architect Toronto, Ontario, Canada

    1 month ago

    Old Toronto, Canada Databricks Inc. Full time

    While candidates in the listed locations are encouraged for this role, we are open to remote candidates in other locations in eastern CanadaAs a Solutions Architect at Databricks within the Field Engineering org you will partner with our customers to design scalable data architectures using Databricks technology and services. You have technical depth and...

  • Solutions Architect

    1 month ago

    Toronto, Canada Databricks Full time

    P-225While candidates in the listed locations are encouraged for this role, we are open to remote candidates in other locations in eastern CanadaAs a Solutions Architect at Databricks within the Field Engineering org you will partner with our customers to design scalable data architectures using Databricks technology and services. You have technical depth...

  • Solutions Architect Montréal, Canada

    2 weeks ago

    Old Toronto, Canada Databricks Inc. Full time

    While candidates in the listed locations are encouraged for this role, we are open to remote candidates in other locations in eastern Canada.As a Solutions Architect at Databricks within the Field Engineering org, you will partner with our customers to design scalable data architectures using Databricks technology and services. You have technical depth and...

  • Senior Enterprise Sales Executive

    2 days ago

    Toronto, Ontario, Canada Databricks Full time

    Position Overview:As a Senior Enterprise Sales Executive at Databricks, you will leverage your extensive experience in engaging with large enterprise clients. Your expertise in driving innovation and facilitating change will be key in advancing customer visions and accelerating decision-making processes. You possess a deep understanding of our products and...

  • Senior Azure Data Engineering

    2 weeks ago

    Old Toronto, Canada Avanade Full time

    Introduction:Join our Data Engineering team and create cutting-edge solutions within Avanade’s Business Intelligence practice. We’re seeking ambitious technologists passionate about Microsoft cloud technology. Early access to the newest Microsoft technologies ensures you stay ahead of the curve. Come join us! As a member of the Data Engineering team,...

  • Senior Azure Data Engineering

    2 weeks ago

    Old Toronto, Canada Avanade Full time

    Introduction:Join our Data Engineering team and create cutting-edge solutions within Avanade’s Business Intelligence practice. We’re seeking ambitious technologists passionate about Microsoft cloud technology. Early access to the newest Microsoft technologies ensures you stay ahead of the curve. Come join us! As a member of the Data Engineering team,...

  • Senior Azure Data Engineering

    2 weeks ago

    Old Toronto, Canada Avanade Full time

    ```html Introduction: Join our Data Engineering team and create cutting-edge solutions within Avanade’s Business Intelligence practice. We’re seeking ambitious technologists passionate about Microsoft cloud technology. Early access to the newest Microsoft technologies ensures you stay ahead of the curve. Come join us! As a member of the Data Engineering...

  • Senior Azure Data Engineering

    2 weeks ago

    Old Toronto, Canada Avanade Full time

    ```html Introduction: Join our Data Engineering team and create cutting-edge solutions within Avanade’s Business Intelligence practice. We’re seeking ambitious technologists passionate about Microsoft cloud technology. Early access to the newest Microsoft technologies ensures you stay ahead of the curve. Come join us! As a member of the Data Engineering...

  • Databricks Platform Engineer

    2 months ago

    Toronto, Canada Guidepoint Full time

    Overview:  Guidepoint’s Technology team thrives on problem-solving and creating happier users. As Guidepoint works to achieve its mission of making individuals, businesses, and the world smarter through personalized knowledge-sharing solutions, the engineering team is taking on challenges to improve our internal application architecture and create new...

  • Data Engineering

    3 weeks ago

    Old Toronto, Canada Avanade Full time

    Introduction: Join our Data Engineering team and create cutting-edge solutions within Avanade’s Business Intelligence practice. We’re seeking ambitious technologists passionate about Microsoft cloud technology. Early access to the newest Microsoft technologies ensures you stay ahead of the curve. Come join us! As a member of the Data Engineering team,...

  • Data Engineering

    3 weeks ago

    Old Toronto, Canada Avanade Full time

    Introduction: Join our Data Engineering team and create cutting-edge solutions within Avanade’s Business Intelligence practice. We’re seeking ambitious technologists passionate about Microsoft cloud technology. Early access to the newest Microsoft technologies ensures you stay ahead of the curve. Come join us! As a member of the Data Engineering team,...

  • Data Engineering

    3 weeks ago

    Old Toronto, Canada Avanade Full time

    Introduction: Join our Data Engineering team and create cutting-edge solutions within Avanade’s Business Intelligence practice. We’re seeking ambitious technologists passionate about Microsoft cloud technology. Early access to the newest Microsoft technologies ensures you stay ahead of the curve. Come join us! As a member of the Data Engineering team,...

  • Senior Azure Data Engineering

    2 weeks ago

    Old Toronto, Ontario, CA Avanade Full time

    Introduction:Join our Data Engineering team and create cutting-edge solutions within Avanade’s Business Intelligence practice. We’re seeking ambitious technologists passionate about Microsoft cloud technology. Early access to the newest Microsoft technologies ensures you stay ahead of the curve. Come join us! As a member of the Data Engineering team,...

  • Senior Data Engineer

    3 weeks ago

    Old Toronto, Canada Nielseniq Full time

    At NIQ, we deliver the most complete and clear understanding of consumer buying behavior that reveals new pathways to growth. We are looking to add a Senior Data Engineer to our Activate team.The NIQ Activate team serves dozens of retailers and brands worldwide using cutting-edge Big Data, real-time analytics, and data science automation. Our solution is...

  • Senior Data Engineer

    14 hours ago

    Old Toronto, Canada Nielseniq Full time

    At NIQ, we deliver the most complete and clear understanding of consumer buying behavior that reveals new pathways to growth. We are looking to add a Senior Data Engineer to our Activate team.The NIQ Activate team serves dozens of retailers and brands worldwide using cutting-edge Big Data, real-time analytics, and data science automation. Our solution is...