Senior Data Engineer, Big Data

3 weeks ago

Toronto, Ontario, Canada Autodesk Full time

Job Overview

Autodesk is seeking a Senior Data Engineer, Big Data to join our Enterprise Data Engineering & Visualisation (EDEV) team. The EDEV Team is responsible for building data pipelines and solutions that enable our users to gain valuable business insights. These solutions serve as the foundation for all data engineers, data scientists, and data analysts within the organisation. Additionally, this team establishes and promotes data engineering best practices across Autodesk, driving efforts to make data an integral part of decision-making at Autodesk.

Key Responsibilities

  • Automate data transformations against data sourced from various systems, including desktop, web, and mobile product usage logs and business systems data.
  • Develop solutions applying CI/CD processes to orchestrate automated batch and NRT (Near Real Time) pipelines running PySpark and dbt data transformations using Apache Airflow.
  • Utilise programming languages used for data manipulation, including SQL, Python, Spark, PySpark, Spark SQL, Java, and Jinja.
  • Design, develop, execute, and document software solutions to address complex data collection, processing, transformation, and testing.
  • Collaborate with peer organisations, DevOps, and support teams on technical issues and help troubleshoot code-level problems and performance issues.
  • Recommend architectural standards, best practices, and quality assurance processes for data-related systems and applications.
  • Build data quality and durability tracking mechanisms to provide visibility into and address changes in data ingestion, processing, and storage.
  • Lead and/or help design data schemas to ensure common protocols and storage mechanisms are used across all data services.
  • Partner with data and software architects to design data models, APIs, or other architectural elements.

Requirements

  • 5 years of experience in cloud ETL, ELT, and near-real-time data collection, transport, and processing technologies.
  • 3+ years of experience with data modeling, including the design of schemas optimised for data retrieval.
  • 5+ years of experience in programming with SQL, Python, Spark, PySpark, Spark SQL, Java, Jinja, and related languages.
  • 3+ years of experience working with a big data environment, including Hadoop, Hive, Spark, and Presto.
  • Experience creating detailed technical documentation addressing business requirements.
  • Experience architecting and implementing data testing solutions.
  • Experience working with a big data environment, including Hadoop, Hive, Spark, and Presto.
  • Proficiency in building data transformation and analysis using PySpark and dbt.
  • Experience working with workflow management tools, like Airflow and Azkaban.
  • Experience working with GitHub, Jira, and Jenkins.
  • Familiarity with data governance frameworks, SDLC (Software Development Life Cycle), and Agile methodology.
  • Strong technical and problem-solving skills and continuous interest in learning best-of-class technologies around data warehousing, data wrangling, data quality, data governance, and data ethics.

About Autodesk

Welcome to Autodesk. Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.

We take great pride in our culture here at Autodesk – our Culture Code is at the core of everything we do. Our values and ways of working help our people thrive and realise their potential, which leads to even better outcomes for our customers.

When you're an Autodesker, you can be your whole, authentic self and do meaningful work that helps build a better future for all. Ready to shape the world and your future? Join us.

Salary Transparency

Salary is one part of Autodesk's competitive compensation package. Offers are based on the candidate's experience and geographic location. In addition to base salaries, we also have a significant emphasis on discretionary annual cash bonuses, commissions for sales roles, stock or long-term incentive cash grants, and a comprehensive benefits package.

Diversity & Belonging

We take pride in cultivating a culture of belonging and an equitable workplace where everyone can thrive. Learn more here:

  • Senior Data Engineer

    4 weeks ago

    Old Toronto, Ontario, Canada Data Engineer Jobs Full time

    About This RoleWe are seeking a highly skilled Senior Data Engineer to join our Analytics Engineering team. As a key member of this team, you will be responsible for designing and building scalable data models and ETL pipelines to support business decisions.Key Responsibilities:Collaborate with data scientists to design data models and answer questions.Work...

  • Senior Big Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Apollo Full time

    About the RoleWe are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team at Apollo. The ideal candidate will have a strong background in developing batch processing systems, with extensive experience in the Apache Hadoop ecosystem.Key ResponsibilitiesDesign and implement scalable batch processing systems using...

  • Senior Data Engineer, Big Data

    4 weeks ago

    Toronto, Ontario, Canada Autodesk Full time

    Job OverviewAutodesk is seeking a Senior Data Engineer, Big Data to join our Enterprise Data Engineering & Visualisation (EDEV) team. The EDEV Team is responsible for building data pipelines and solutions that enable our users to gain valuable business insights. These solutions serve as the foundation for all data engineers, data scientists, and data...

  • Senior Data Engineer, Big Data

    4 weeks ago

    Toronto, Ontario, Canada Autodesk Full time

    Job OverviewAutodesk is seeking a Senior Software Developer, Big Data to join our Enterprise Data Engineering & Visualisation (EDEV) team. The EDEV Team is responsible for building data pipelines and solutions that enable our users to gain valuable business insights. These solutions serve as the foundation for all data engineers, data scientists, and data...

  • Data Analyst

    4 weeks ago

    Toronto, Ontario, Canada Data Privacy Full time

    Unlock the Power of Data in Wholesale FinanceRBC's Wholesale Finance Data Strategy team is seeking a skilled Data Analyst to join their ranks. As a key member of the team, you will play a vital role in exploring and operationalizing big data sources to drive business insights and growth.Key Responsibilities:Develop predictive data models and quantitative...

  • Senior Big Data Engineer

    3 weeks ago

    Toronto, Ontario, Canada Astra North Infoteck Inc. Full time

    Astra North Infoteck Inc. is seeking a highly skilled Solution Architect to lead the design and implementation of our cloud-based data solutions. The ideal candidate will have extensive experience with Java, Spark, Hadoop, and Big Data technologies, as well as a strong understanding of case management systems.Key Responsibilities:Design and implement...

  • Data Engineer

    3 weeks ago

    Toronto, Ontario, Canada Royal Bank of Canada Full time

    Job SummaryWe are seeking a highly skilled Data Engineer to join our Business Insights and Analytics team at Royal Bank of Canada. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and architectures.Key ResponsibilitiesDesign, develop, implement, and support emerging big data analytics...

  • Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Royal Bank of Canada Full time

    Job SummaryWe are seeking a highly skilled Data Engineer to join our Business Insights and Analytics team at Royal Bank of Canada. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and architectures.Key ResponsibilitiesDesign, develop, implement, and support emerging big data analytics...

  • Senior Data Engineer

    3 weeks ago

    Toronto, Ontario, Canada Global Pharma Tek Full time

    Job Title: Senior Data EngineerAbout the Role:The Senior Data Engineer will be responsible for designing, developing, and maintaining large-scale data systems, including data warehouses, data lakes, and data pipelines. The ideal candidate will have expertise in data engineering, cloud platforms, and big data technologies such as Azure Databricks, Delta Lake,...

  • Big Data Engineer

    4 weeks ago

    Old Toronto, Ontario, Canada Electronic Arts Inc. Full time

    Job Title: Big Data Engineer Temps PleinAs a key member of the Adrenaline team at Electronic Arts Inc., you will play a pivotal role in developing and implementing cutting-edge data architectures that support the central technology team. This position requires collaboration with a diverse range of team members, including game studios, tech teams, technology...

  • Data Engineering Leader for Big Data Solutions

    5 days ago

    Toronto, Ontario, Canada Autodesk Full time

    Overview of the PositionAutodesk is seeking a Senior Software Developer, Big Data to join our Enterprise Data Engineering & Visualization (EDEV) team. The EDEV Team's mission is to build data pipelines and solutions that enable users to gain valuable business insights.Estimated Salary: $150,000 - $180,000 per yearThe successful candidate will be responsible...

  • Big Data Architect

    3 days ago

    Toronto, Ontario, Canada Astra North Infoteck Inc. Full time

    Job Description for Big Data LeadOverviewAstra North Infoteck Inc. is seeking an experienced Big Data Lead to lead our team in designing and implementing cutting-edge big data solutions. The ideal candidate will have a strong background in data quality, architecture, and engineering, with experience in leading teams and driving results.Key...

  • Senior Data Engineer

    3 weeks ago

    Toronto, Ontario, Canada Crescens Inc. Full time

    Job Title: Senior Data EngineerJob Summary:Crescens Inc. is seeking a highly skilled Senior Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data pipelines and architectures. Key Responsibilities:Design and implement data pipelines using Java and big...

  • Big Data Engineer: Building Data Solutions

    3 days ago

    Toronto, Ontario, Canada Autodesk Full time

    Job OverviewAutodesk is seeking a highly skilled Senior Software Developer, Big Data to join our Enterprise Data Engineering & Visualization (EDEV) team. The EDEV Team is responsible for designing and implementing data pipelines and solutions that enable our users to gain valuable business insights. These solutions serve as the foundation for all data...

  • Lead Data Architect in Big Data Engineering

    1 week ago

    Toronto, Ontario, Canada Royal Bank of Canada Full time

    Job OverviewRoyal Bank of Canada (RBC) is a leading Canadian bank seeking an experienced Manager, Data Engineering to join its Canadian Banking Operations Data and Analytics Center of Excellence. This role offers the opportunity to lead the development and maintenance of automated data pipelines and infrastructure.About the RoleWe are looking for a skilled...

  • Big Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada TSX Inc. Full time

    Unlock the Power of Data with Our TeamAt TSX Inc., we're on a mission to harness the potential of data to drive innovation and growth. As a seasoned Big Data Engineer, you'll play a critical role in shaping our data strategy and building cutting-edge solutions that transform the way we operate.Key Responsibilities:Design and implement efficient data pipeline...

  • Senior Data Engineer

    3 days ago

    Toronto, Ontario, Canada Scotiabank Full time

    About This RoleScotiabank, a leading financial institution, seeks an exceptional Senior Data Engineer to join its Client Central technology team. As a key member of this team, you will be responsible for designing, developing, and implementing large-scale data processing solutions using big data technologies and Python.Job DescriptionThe ideal candidate will...

  • Data Engineer with Expertise in Cloud and Big Data

    3 days ago

    Toronto, Ontario, Canada Astra North Infoteck Inc. Full time

    Job Title: Data Engineer with Expertise in Cloud and Big DataAstra North Infoteck Inc. is seeking an experienced Data Engineer to lead the development of our big data solutions.We are offering a competitive salary of $140,000 - $180,000 per year, depending on experience and qualifications.About the Role:We are looking for a skilled engineer who can design,...

  • Senior Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Capgemini Full time

    Job Title: Senior Data EngineerJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team at Capgemini. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining large-scale data processing systems. You will work closely with our team of experienced data engineers to develop innovative solutions for...

  • Senior Data Engineering Analyst

    4 weeks ago

    Toronto, Ontario, Canada CB Canada Full time

    Job DescriptionAs a Senior Data Engineering Analyst at CB Canada, you will play a key role in designing and developing state-of-the-art data analytics solutions for business problems. Your expertise in big data analytics will help us drive business insights and inform strategic decisions.Key Responsibilities:Architect and implement big data analytics...