Senior Software Developer, Big Data

4 weeks ago

Toronto, Canada Autodesk Full time

Position Overview

Autodesk is looking for a Senior Software Developer, Big Data with experience building data pipelines to join our Enterprise Data Engineering & Visualisation (EDEV) team. The EDEV Team is chartered with building data pipelines and solutions that allow our users to obtain important business insights. These solutions serve as the building blocks for all data engineers, data scientists and data analysts throughout the organisation. Also, this team establishes and promotes data engineering best practices across Autodesk, leading efforts to make data an integral part of decision-making at Autodesk. You will build data solutions for complex data transformation pipelines, and lending technical expertise within the group.

You will report to Senior Software Development Manager based in the US. This role is based out of Toronto, in a hybrid mode. Autodesk supports FLEX working model.

Responsibilities

Automate data transformations against data sourced from a variety of systems including desktop, web, and mobile product usage logs and business systems data

Develop solutions applying CI/CD processes to orchestrate automated batch and NRT (Near Real Time) pipelines running PySpark and dbt data transformations using Apache Airflow

Use programming languages used for data manipulation, including SQL, Python, Spark, PySpark, Spark SQL, Java, Jinja

Design, develop, execute, and document software solutions to address complex data collection, processing, transformation, and testing

Collaborate with peer organisations, DevOps, support organisations on technical issues and help troubleshoot code level problems and performance issues

Recommend architectural standards, best practices and quality assurance processes for data related systems and applications

Build data quality and durability tracking mechanisms to provide visibility into and address changes in data ingestion, processing, and storage

Lead and/or help design data schemas to ensure common protocols and storage mechanisms are used across all data services

Partner with data and software architects to design data models, APIs or other architectural elements

Minimum Qualifications

5 years of experience in cloud ETL, ELT and near-real time data collection, transport, and processing technologies

3+ years of experience with data modeling, including the design of schemas optimized for data retrieval

5+ year of experience in programming with SQL, Python, Spark, PySpark, Spark SQL, Java, Jinja and related languages

3+ years of experience working with a big data environment including Hadoop, Hive, Spark and Presto

Experience creating detailed technical documentation which addresses business requirements

Experience architecting and implementing data testing solutions

Experience working with a big data environment including Hadoop, Hive, Spark and Presto

Proficiency in building data transformation and analysis using PySpark and dbt

Experience working with workflow management tools, like Airflow and Azkaban

Experience working with GitHub, Jira, and Jenkins

Familiarity with data governance frameworks, SDLC (Software Development Life Cycle), and Agile methodology

Strong technical and problem-solving skills and continuous interest in learning best of class technologies around data warehousing, data wrangling, data quality, data governance and data ethics

Learn More

About Autodesk
Welcome to Autodesk Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.

We take great pride in our culture here at Autodesk – our Culture Code is at the core of everything we do. Our values and ways of working help our people thrive and realize their potential, which leads to even better outcomes for our customers.

When you’re an Autodesker, you can be your whole, authentic self and do meaningful work that helps build a better future for all. Ready to shape the world and your future? Join us

Salary transparency

Salary is one part of Autodesk’s competitive compensation package. Offers are based on the candidate’s experience and geographic location. In addition to base salaries, we also have a significant emphasis on discretionary annual cash bonuses, commissions for sales roles, stock or long-term incentive cash grants, and a comprehensive benefits package.

Diversity & Belonging
We take pride in cultivating a culture of belonging and an equitable workplace where everyone can thrive. Learn more here:

Are you an existing contractor or consultant with Autodesk?

Please search for open jobs and apply internally (not on this external site).

  • Big Data Engineer

    1 day ago

    Old Toronto, Canada Big Island Motors Full time

    Job Description: Design, develop, and maintain data pipelines using Hadoop, Apache Spark, and PySpark. Integrate data from various sources and formats into the big data platform. Troubleshoot and resolve performance issues related to data processing. Work closely with data scientists, analysts, and other stakeholders to understand data requirements and...

  • Big Data Engineer

    1 day ago

    Old Toronto, Canada Big Island Motors Full time

    Job Description: Design, develop, and maintain data pipelines using Hadoop, Apache Spark, and PySpark. Integrate data from various sources and formats into the big data platform. Troubleshoot and resolve performance issues related to data processing. Work closely with data scientists, analysts, and other stakeholders to understand data requirements and...

  • Senior Java/Big Data Developer

    3 weeks ago

    Toronto, Canada First Derivative Full time

    Who are we?First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting).Senior Java/Big Data Developer:Responsibilities: Create...

  • Senior Java/Big Data Developer

    3 weeks ago

    Toronto, Canada First Derivative Full time

    Who are we? First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting). Senior Java/Big Data...

  • Senior Java/Big Data Developer

    1 week ago

    Toronto, ON, Canada First Derivative Full time

    Who are we?First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting).Senior Java/Big Data Developer:Responsibilities: Create...

  • Senior Java/Big Data Developer

    3 weeks ago

    Toronto, ON, Canada First Derivative Full time

    Who are we?First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting).Senior Java/Big Data Developer:Responsibilities: Create...

  • Senior Java/Big Data Developer

    3 weeks ago

    Toronto, ON, Canada First Derivative Full time

    Who are we?First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting).Senior Java/Big Data Developer:Responsibilities: Create...

  • Senior Java/Big Data Developer

    2 weeks ago

    Toronto, ON, Canada First Derivative Full time

    Who are we?First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting).Senior Java/Big Data Developer:Responsibilities: Create...

  • Senior Java/Big Data Developer

    2 weeks ago

    Toronto, ON, Canada First Derivative Full time

    Who are we?First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting).Senior Java/Big Data Developer:Responsibilities: Create...

  • Senior Java/Big Data Developer

    3 weeks ago

    toronto, Canada First Derivative Full time

    Who are we?First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting).Senior Java/Big Data Developer:Responsibilities: Create...

  • Senior Java/Big Data Developer

    3 weeks ago

    toronto, Canada First Derivative Full time

    Who are we?First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting).Senior Java/Big Data Developer:Responsibilities: Create...

  • Senior Java/Big Data Developer

    3 weeks ago

    Toronto, Canada First Derivative Full time

    Who are we?First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting).Senior Java/Big Data Developer:Responsibilities: Create...

  • Senior Java/Big Data Developer

    3 weeks ago

    Toronto, Canada First Derivative Full time

    Who are we? First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting). Senior Java/Big Data Developer: Responsibilities: ...

  • Senior Java/Big Data Developer

    2 weeks ago

    Toronto, Canada First Derivative Full time

    Who are we? First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting). Senior Java/Big Data Developer: Responsibilities: ...

  • Senior Java/Big Data Developer

    2 weeks ago

    Toronto, Canada First Derivative Full time

    Who are we?First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting).Senior Java/Big Data Developer:Responsibilities: Create...

  • Senior Java/Big Data Developer

    2 weeks ago

    Toronto, Canada First Derivative Full time

    Who are we?First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting).Senior Java/Big Data Developer:Responsibilities: Create...

  • Software Development Manager

    1 week ago

    Toronto, Canada Amazon Development Centre Canada ULC Full time

    Passionate about Big Data and Distributed Systems? Interested in building new state-of-the-art products at Petabyte scale? Be part of a team of industry leading experts that operates one of the largest big data analytics systems at Amazon. We own the campaign traffic reporting product for all of advertising and are responsible for end to end processing of...

  • Senior Java/Big Data Developer

    3 weeks ago

    Toronto, ON, Canada First Derivative Full time

    Who are we? First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting). Senior Java/Big Data Developer: Responsibilities: ...

  • Tech - Senior Big Data Java Developer

    3 weeks ago

    Toronto, ON, Canada First Derivative Full time

    First Derivatives (FD) is a leading software and services company, with world-leading intellectual property in ultra-high-performance analytics (KX) across industries, and extensive domain expertise and capabilities in capital markets systems and technology (managed services and consulting). Senior Java/Spark Developer – FX Trading Our client within...

  • Software Development Manager

    7 days ago

    Old Toronto, Canada Amazon Full time

    Software Development Manager - Big Data, MADS Traffic ReportingPassionate about Big Data and Distributed Systems? Interested in building new state-of-the-art products at Petabyte scale? Be part of a team of industry leading experts that operates one of the largest big data analytics systems at Amazon. We own the campaign traffic reporting product for all of...