Big Data Engineer: Building Data Solutions

7 days ago

Toronto, Ontario, Canada Autodesk Full time
Job Overview

Autodesk is seeking a highly skilled Senior Software Developer, Big Data to join our Enterprise Data Engineering & Visualization (EDEV) team. The EDEV Team is responsible for designing and implementing data pipelines and solutions that enable our users to gain valuable business insights. These solutions serve as the foundation for all data engineers, data scientists, and data analysts within the organization.

About the Role

This role involves building complex data transformation pipelines using PySpark and dbt data transformations with Apache Airflow orchestration. You will work closely with peer organizations, DevOps, and support teams to resolve technical issues and troubleshoot code-level problems and performance issues. Additionally, you will collaborate with data and software architects to design data models, APIs, or other architectural elements.

Key Responsibilities
  • Automate data transformations against various systems, including desktop, web, and mobile product usage logs, and business systems data.
  • Develop CI/CD processes to orchestrate automated batch and NRT (Near Real-Time) pipelines.
  • Use programming languages such as SQL, Python, Spark, PySpark, Spark SQL, Java, and Jinja for data manipulation.
  • Design, develop, execute, and document software solutions to address complex data collection, processing, transformation, and testing.
  • Collaborate with peer organizations, DevOps, and support teams on technical issues and help troubleshoot code-level problems and performance issues.
Requirements
  • 5+ years of experience in cloud ETL, ELT, and near-real-time data collection, transport, and processing technologies.
  • 3+ years of experience with data modeling, including schema design optimized for data retrieval.
  • 5+ years of experience in programming with SQL, Python, Spark, PySpark, Spark SQL, Java, Jinja, and related languages.
  • Experience working with big data environments, including Hadoop, Hive, Spark, and Presto.
  • Proficiency in building data transformation and analysis using PySpark and dbt.
  • Experience working with workflow management tools, such as Airflow and Azkaban.
  • Familiarity with data governance frameworks, SDLC (Software Development Life Cycle), and Agile methodology.
Estimated Salary Range

$120,000 - $180,000 per year, depending on experience and location.

About Autodesk

At Autodesk, we believe in fostering a culture of belonging and an equitable workplace where everyone can thrive. We take pride in our Culture Code, which guides everything we do. Join us in shaping the future and creating amazing things

  • Senior Data Engineer

    1 month ago

    Old Toronto, Ontario, Canada Data Engineer Jobs Full time

    About This RoleWe are seeking a highly skilled Senior Data Engineer to join our Analytics Engineering team. As a key member of this team, you will be responsible for designing and building scalable data models and ETL pipelines to support business decisions.Key Responsibilities:Collaborate with data scientists to design data models and answer questions.Work...

  • Data Engineering Leader for Big Data Solutions

    1 week ago

    Toronto, Ontario, Canada Autodesk Full time

    Overview of the PositionAutodesk is seeking a Senior Software Developer, Big Data to join our Enterprise Data Engineering & Visualization (EDEV) team. The EDEV Team's mission is to build data pipelines and solutions that enable users to gain valuable business insights.Estimated Salary: $150,000 - $180,000 per yearThe successful candidate will be responsible...

  • Senior Data Engineer, Big Data

    4 weeks ago

    Toronto, Ontario, Canada Autodesk Full time

    Job OverviewAutodesk is seeking a Senior Data Engineer, Big Data to join our Enterprise Data Engineering & Visualisation (EDEV) team. The EDEV Team is responsible for building data pipelines and solutions that enable our users to gain valuable business insights. These solutions serve as the foundation for all data engineers, data scientists, and data...

  • Senior Data Engineer, Big Data

    4 weeks ago

    Toronto, Ontario, Canada Autodesk Full time

    Job OverviewAutodesk is seeking a Senior Software Developer, Big Data to join our Enterprise Data Engineering & Visualisation (EDEV) team. The EDEV Team is responsible for building data pipelines and solutions that enable our users to gain valuable business insights. These solutions serve as the foundation for all data engineers, data scientists, and data...

  • Data Engineer with Expertise in Cloud and Big Data

    7 days ago

    Toronto, Ontario, Canada Astra North Infoteck Inc. Full time

    Job Title: Data Engineer with Expertise in Cloud and Big DataAstra North Infoteck Inc. is seeking an experienced Data Engineer to lead the development of our big data solutions.We are offering a competitive salary of $140,000 - $180,000 per year, depending on experience and qualifications.About the Role:We are looking for a skilled engineer who can design,...

  • Big Data Engineer

    1 month ago

    Toronto, Ontario, Canada TSX Inc. Full time

    Unlock the Power of Data with Our TeamAt TSX Inc., we're on a mission to harness the potential of data to drive innovation and growth. As a seasoned Big Data Engineer, you'll play a critical role in shaping our data strategy and building cutting-edge solutions that transform the way we operate.Key Responsibilities:Design and implement efficient data pipeline...

  • Big Data Solutions Architect

    2 weeks ago

    Toronto, Ontario, Canada Amazon Development Centre Canada ULC Full time

    Job DescriptionAs a Software Development Manager at Amazon Development Centre Canada ULC, you will be responsible for leading the development of our big data processing platform. You will define and drive the multi-year vision for this platform, build and manage a team of innovative engineers, and deliver robust and scalable solutions towards your vision.Key...

  • Big Data Architect

    7 days ago

    Toronto, Ontario, Canada Astra North Infoteck Inc. Full time

    Job Description for Big Data LeadOverviewAstra North Infoteck Inc. is seeking an experienced Big Data Lead to lead our team in designing and implementing cutting-edge big data solutions. The ideal candidate will have a strong background in data quality, architecture, and engineering, with experience in leading teams and driving results.Key...

  • Xsenior Big Data Engineer

    3 weeks ago

    Toronto, Ontario, Canada Royal Bank of Canada Full time

    Opportunity SummaryRoyal Bank of Canada is seeking a seasoned Data Engineer with hands-on expertise in building solutions leveraging Azure Big Data technologies, Azure Data Factory, Azure platform management, and Azure DevOps.Key Responsibilities- Design, develop, and implement end-to-end data integration solutions using Azure Data Factory (ADF)- Stay...

  • Data Engineer

    1 month ago

    Toronto, Ontario, Canada Royal Bank of Canada Full time

    Job SummaryWe are seeking a highly skilled Data Engineer to join our Business Insights and Analytics team at Royal Bank of Canada. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and architectures.Key ResponsibilitiesDesign, develop, implement, and support emerging big data analytics...

  • Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Royal Bank of Canada Full time

    Job SummaryWe are seeking a highly skilled Data Engineer to join our Business Insights and Analytics team at Royal Bank of Canada. The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines and architectures.Key ResponsibilitiesDesign, develop, implement, and support emerging big data analytics...

  • Senior Big Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Astra North Infoteck Inc. Full time

    Astra North Infoteck Inc. is seeking a highly skilled Solution Architect to lead the design and implementation of our cloud-based data solutions. The ideal candidate will have extensive experience with Java, Spark, Hadoop, and Big Data technologies, as well as a strong understanding of case management systems.Key Responsibilities:Design and implement...

  • Staff Data Engineer

    1 month ago

    Old Toronto, Ontario, Canada NTT DATA, Inc. Full time

    Job Title: Staff Data EngineerWe are seeking a highly skilled Staff Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing and implementing tailored data solutions to meet customer needs and use cases.Key Responsibilities:Provide thought leadership and recommend the most appropriate technologies...

  • Data Analyst

    1 month ago

    Toronto, Ontario, Canada Data Privacy Full time

    Unlock the Power of Data in Wholesale FinanceRBC's Wholesale Finance Data Strategy team is seeking a skilled Data Analyst to join their ranks. As a key member of the team, you will play a vital role in exploring and operationalizing big data sources to drive business insights and growth.Key Responsibilities:Develop predictive data models and quantitative...

  • Data Engineer

    1 month ago

    Toronto, Ontario, Canada NTT DATA Services Full time

    Job Summary:We are seeking a highly skilled Staff Data Engineer with expertise in Google Cloud Platform to join our Data Intelligence team. The ideal candidate will be a growth-minded, adaptable, and collaborative individual with a strong background in designing and implementing tailored data solutions.Key Responsibilities:Provide thought leadership in...

  • Big Data Lead

    4 weeks ago

    Toronto, Ontario, Canada Astra North Infoteck Inc. Full time

    About the Role:Astra North Infoteck Inc. is seeking a highly skilled Big Data Lead - Data Quality Expert to join our team. As a key member of our data engineering team, you will be responsible for leading the development and implementation of big data solutions, ensuring high-quality data processing and analysis.Key Responsibilities:Lead the design,...

  • Big Data Solutions Architect

    7 days ago

    Toronto, Ontario, Canada Jay Analytix Full time

    Job OverviewJay Analytix is seeking a highly skilled Big Data Solutions Architect to join our team. As a key member of our technology group, you will be responsible for designing and implementing scalable data engineering solutions.About the RoleThis is a full-time position that involves working closely with portfolio managers, traders, operations staff, and...

  • Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada SADA Full time

    Unlock Your Potential as a Cloud Data EngineerSADA is seeking a highly skilled Cloud Data Engineer to join our accomplished team. As a Cloud Data Engineer, you will play a key role in designing, building, and maintaining advanced data infrastructures on Google Cloud Platform (GCP). You will work closely with top-tier architects and engineers to recommend,...

  • Big Data Engineer with Java

    2 months ago

    Toronto, Ontario, Ontario, Canada VLink Inc Full time

    Big Data Engineer with JavaToronto, ON, JR1017573FulltimeJob Description:We are seeking a highly skilled and experienced Big Data Engineer to join our dynamic team. The ideal candidate will have a solid background in application development with specific expertise in Hadoop, Spark, and cloud technologies. This role involves developing and optimizing big data...

  • Big Data Developer

    3 weeks ago

    Toronto, Ontario, Ontario, Canada First Derivative Full time

    Are you a technologist who has a hunger to dig deep and get involved in providing first-class solutions to real-world problems?First Derivative is a people-first company. We are powered by our ever-growing teams of specialist problem-solvers and process-evolvers. Our extensive capabilities offer agile solutions and exciting career opportunities – whether...