Senior Data Engineer, Big Data

2 weeks ago

Toronto, Ontario, Canada Autodesk Full time

Job Overview

Autodesk is seeking a Senior Data Engineer, Big Data to join our Enterprise Data Engineering & Visualisation (EDEV) team. The EDEV Team is responsible for building data pipelines and solutions that enable our users to gain valuable business insights. These solutions serve as the foundation for all data engineers, data scientists, and data analysts within the organisation. Additionally, this team establishes and promotes data engineering best practices across Autodesk, driving efforts to make data an integral part of decision-making at Autodesk.

Key Responsibilities

  • Automate data transformations against data sourced from various systems, including desktop, web, and mobile product usage logs and business systems data.
  • Develop solutions applying CI/CD processes to orchestrate automated batch and NRT (Near Real Time) pipelines running PySpark and dbt data transformations using Apache Airflow.
  • Utilise programming languages used for data manipulation, including SQL, Python, Spark, PySpark, Spark SQL, Java, and Jinja.
  • Design, develop, execute, and document software solutions to address complex data collection, processing, transformation, and testing.
  • Collaborate with peer organisations, DevOps, and support teams on technical issues and help troubleshoot code-level problems and performance issues.
  • Recommend architectural standards, best practices, and quality assurance processes for data-related systems and applications.
  • Build data quality and durability tracking mechanisms to provide visibility into and address changes in data ingestion, processing, and storage.
  • Lead and/or help design data schemas to ensure common protocols and storage mechanisms are used across all data services.
  • Partner with data and software architects to design data models, APIs, or other architectural elements.

Requirements

  • 5 years of experience in cloud ETL, ELT, and near-real-time data collection, transport, and processing technologies.
  • 3+ years of experience with data modeling, including the design of schemas optimised for data retrieval.
  • 5+ years of experience in programming with SQL, Python, Spark, PySpark, Spark SQL, Java, Jinja, and related languages.
  • 3+ years of experience working with a big data environment, including Hadoop, Hive, Spark, and Presto.
  • Experience creating detailed technical documentation addressing business requirements.
  • Experience architecting and implementing data testing solutions.
  • Experience working with a big data environment, including Hadoop, Hive, Spark, and Presto.
  • Proficiency in building data transformation and analysis using PySpark and dbt.
  • Experience working with workflow management tools, like Airflow and Azkaban.
  • Experience working with GitHub, Jira, and Jenkins.
  • Familiarity with data governance frameworks, SDLC (Software Development Life Cycle), and Agile methodology.
  • Strong technical and problem-solving skills and continuous interest in learning best-of-class technologies around data warehousing, data wrangling, data quality, data governance, and data ethics.

About Autodesk

Welcome to Autodesk. Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.

We take great pride in our culture here at Autodesk – our Culture Code is at the core of everything we do. Our values and ways of working help our people thrive and realise their potential, which leads to even better outcomes for our customers.

When you're an Autodesker, you can be your whole, authentic self and do meaningful work that helps build a better future for all. Ready to shape the world and your future? Join us.

Salary Transparency

Salary is one part of Autodesk's competitive compensation package. Offers are based on the candidate's experience and geographic location. In addition to base salaries, we also have a significant emphasis on discretionary annual cash bonuses, commissions for sales roles, stock or long-term incentive cash grants, and a comprehensive benefits package.

Diversity & Belonging

We take pride in cultivating a culture of belonging and an equitable workplace where everyone can thrive. Learn more here:

  • Senior Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Data Engineer Jobs Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems that support our business operations.Key ResponsibilitiesDesign and implement data models, data warehouses, and data pipelines to support business...

  • Senior Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Data Engineer Jobs Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems that support our business operations.Key ResponsibilitiesDesign and implement data models, data warehouses, and data pipelines to support business...

  • Senior Data Engineer

    3 weeks ago

    Old Toronto, Ontario, Canada Data Engineer Jobs Full time

    About This RoleWe are seeking a highly skilled Senior Data Engineer to join our Analytics Engineering team. As a key member of this team, you will be responsible for designing and building scalable data models and ETL pipelines to support business decisions.Key Responsibilities:Collaborate with data scientists to design data models and answer questions.Work...

  • Senior Big Data Engineer

    3 weeks ago

    Toronto, Ontario, Canada Apollo Full time

    About the RoleWe are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team at Apollo. The ideal candidate will have a strong background in developing batch processing systems, with extensive experience in the Apache Hadoop ecosystem.Key ResponsibilitiesDesign and implement scalable batch processing systems using...

  • Big Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada CB Canada Full time

    Job Title: Big Data EngineerWe are seeking a highly skilled Big Data Engineer to join our team at CB Canada. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Job Summary:The successful candidate will have a strong background in cloud computing, big data, and data analytics. You...

  • Big Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada CB Canada Full time

    Job Title: Big Data EngineerWe are seeking a highly skilled Big Data Engineer to join our team at CB Canada. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Job Summary:The successful candidate will have a strong background in cloud computing, big data, and data analytics. You...

  • Senior Data Engineer, Big Data

    3 weeks ago

    Toronto, Ontario, Canada Autodesk Full time

    Job OverviewAutodesk is seeking a Senior Data Engineer, Big Data to join our Enterprise Data Engineering & Visualisation (EDEV) team. The EDEV Team is responsible for building data pipelines and solutions that enable our users to gain valuable business insights. These solutions serve as the foundation for all data engineers, data scientists, and data...

  • Senior Data Engineer, Big Data

    2 weeks ago

    Toronto, Ontario, Canada Autodesk Full time

    Job OverviewAutodesk is seeking a Senior Software Developer, Big Data to join our Enterprise Data Engineering & Visualisation (EDEV) team. The EDEV Team is responsible for building data pipelines and solutions that enable our users to gain valuable business insights. These solutions serve as the foundation for all data engineers, data scientists, and data...

  • Senior Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Sage Lake Senior Living Full time $276

    Unlock Your Potential as a Senior Data EngineerAre you ready to take your data engineering career to the next level? We're seeking a highly skilled and experienced Senior Data Engineer to join our team at Enable Global Inc.About the RoleAs a Senior Data Engineer, you will play a key role in designing, developing, and deploying our SaaS products. You will...

  • Senior Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Sage Lake Senior Living Full time $276

    Unlock Your Potential as a Senior Data EngineerAre you ready to take your data engineering career to the next level? We're seeking a highly skilled and experienced Senior Data Engineer to join our team at Enable Global Inc.About the RoleAs a Senior Data Engineer, you will play a key role in designing, developing, and deploying our SaaS products. You will...

  • Data Analyst

    4 weeks ago

    Toronto, Ontario, Canada Data Privacy Full time

    Finance Data Analyst RoleRBC's Wholesale Finance Data Strategy team is seeking a skilled Data Analyst to join their team. As a key member of the team, you will play a crucial role in exploring and operationalizing big data sources to provide valuable insights and contribute to the team's objectives.Key Responsibilities:Develop predictive data models and...

  • Data Analyst

    4 weeks ago

    Toronto, Ontario, Canada Data Privacy Full time

    Finance Data Analyst RoleRBC's Wholesale Finance Data Strategy team is seeking a skilled Data Analyst to join their team. As a key member of the team, you will play a crucial role in exploring and operationalizing big data sources to provide valuable insights and contribute to the team's objectives.Key Responsibilities:Develop predictive data models and...

  • Data Analyst

    3 weeks ago

    Toronto, Ontario, Canada Data Privacy Full time

    Unlock the Power of Data in Wholesale FinanceRBC's Wholesale Finance Data Strategy team is seeking a skilled Data Analyst to join their ranks. As a key member of the team, you will play a vital role in exploring and operationalizing big data sources to drive business insights and growth.Key Responsibilities:Develop predictive data models and quantitative...

  • Senior Big Data Engineer

    1 month ago

    Toronto, Ontario, Canada Rackspace Full time

    About the Role:We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team at Rackspace. The ideal candidate will have a strong background in developing batch processing systems, with extensive experience in the Apache Hadoop ecosystem (Map Reduce, Oozie, Hive, Pig, HBase, Storm). This role involves working in Java and...

  • Senior Big Data Engineer

    1 month ago

    Toronto, Ontario, Canada Rackspace Full time

    About the Role:We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team at Rackspace. The ideal candidate will have a strong background in developing batch processing systems, with extensive experience in the Apache Hadoop ecosystem (Map Reduce, Oozie, Hive, Pig, HBase, Storm). This role involves working in Java and...

  • Senior Big Data Engineer

    2 weeks ago

    Toronto, Ontario, Canada Astra North Infoteck Inc. Full time

    Astra North Infoteck Inc. is seeking a highly skilled Solution Architect to lead the design and implementation of our cloud-based data solutions. The ideal candidate will have extensive experience with Java, Spark, Hadoop, and Big Data technologies, as well as a strong understanding of case management systems.Key Responsibilities:Design and implement...

  • Big Data Engineer

    3 weeks ago

    Toronto, Ontario, Canada TSX Inc. Full time

    Unlock Your Potential as a Big Data EngineerAt TSX Inc., we're on a mission to revolutionize the way we think about data. As a seasoned Big Data Engineer, you'll play a critical role in shaping our data-driven future.Key Responsibilities:Design and implement efficient data pipeline architectures to drive business growth.Assemble complex data sets that meet...

  • Big Data Engineer

    3 weeks ago

    Toronto, Ontario, Canada TSX Inc. Full time

    Unlock Your Potential as a Big Data EngineerAt TSX Inc., we're on a mission to revolutionize the way we think about data. As a seasoned Big Data Engineer, you'll play a critical role in shaping our data-driven future.Key Responsibilities:Design and implement efficient data pipeline architectures to drive business growth.Assemble complex data sets that meet...

  • Data Engineer

    2 weeks ago

    Toronto, Ontario, Canada Royal Bank of Canada Full time

    Job SummaryWe are seeking a highly skilled Data Engineer to join our Business Insights and Analytics team at Royal Bank of Canada. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and architectures.Key ResponsibilitiesDesign, develop, implement, and support emerging big data analytics...

  • Data Engineer

    3 weeks ago

    Toronto, Ontario, Canada Royal Bank of Canada Full time

    Job SummaryWe are seeking a highly skilled Data Engineer to join our Business Insights and Analytics team at Royal Bank of Canada. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and architectures.Key ResponsibilitiesDesign, develop, implement, and support emerging big data analytics...