Senior Big Data Engineer

2 months ago

Old Toronto, Ontario, Canada Rackspace Full time
About the Role

We are seeking a highly skilled and experienced Senior Big Data Engineer to lead our team's efforts in developing and maintaining large-scale batch processing systems. The ideal candidate will have a strong background in the Apache Hadoop ecosystem, with extensive experience in Map Reduce, Oozie, Hive, Pig, HBase, and Storm.

Key Responsibilities:

  • Design and develop scalable and robust code for batch processing systems using Hadoop, Oozie, Pig, Hive, Map Reduce, Spark, and Java.
  • Develop, manage, and maintain batch pipelines supporting Machine Learning workloads.
  • Leverage GCP for scalable big data processing and storage solutions.
  • Implement automation and DevOps best practices for CI/CD, IaC, and other infrastructure.

Requirements:

  • Proficiency in the Hadoop ecosystem with Map Reduce, Oozie, Hive, Pig, HBase, and Storm.
  • Strong programming skills with Java, Python, and Spark.
  • Knowledge in public cloud services, particularly in GCP.
  • Experience in infrastructure and applied DevOps principles.
  • Ability to tackle complex challenges and devise effective solutions.
  • Strong written and verbal communication skills for remote collaboration.
  • Proven experience in engineering batch processing systems at scale.
  • Hands-on experience in public cloud platforms, particularly GCP.

Must-Have Qualifications:

  • Experience with batch pipelines supporting Machine Learning workloads.
  • Strong experience in programming languages such as Java.
  • Strong experience in the Apache Hadoop ecosystem.
  • 10+ years of experience in customer-facing software or technology.
  • 5+ years of experience with on-premises to cloud migrations or IT transformations.
  • Technical degree in Computer Science, Software Engineering, or related field.

Good to Have:

  • Familiarity with Terraform.
  • Familiarity with Python.
  • 5+ years of experience building and operating solutions on GCP.

  • Senior Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Data Engineer Jobs Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems that support our business operations.Key ResponsibilitiesDesign and implement data models, data warehouses, and data pipelines to support business...

  • Senior Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Data Engineer Jobs Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems that support our business operations.Key ResponsibilitiesDesign and implement data models, data warehouses, and data pipelines to support business...

  • Senior Data Engineer

    3 weeks ago

    Old Toronto, Ontario, Canada Data Engineer Jobs Full time

    About This RoleWe are seeking a highly skilled Senior Data Engineer to join our Analytics Engineering team. As a key member of this team, you will be responsible for designing and building scalable data models and ETL pipelines to support business decisions.Key Responsibilities:Collaborate with data scientists to design data models and answer questions.Work...

  • Senior Big Data Engineer

    3 weeks ago

    Toronto, Ontario, Canada Apollo Full time

    About the RoleWe are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team at Apollo. The ideal candidate will have a strong background in developing batch processing systems, with extensive experience in the Apache Hadoop ecosystem.Key ResponsibilitiesDesign and implement scalable batch processing systems using...

  • Big Data Engineer

    1 month ago

    Toronto, Ontario, Canada CB Canada Full time

    Job Title: Big Data EngineerWe are seeking a highly skilled Big Data Engineer to join our team at CB Canada. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Job Summary:The successful candidate will have a strong background in cloud computing, big data, and data analytics. You...

  • Big Data Engineer

    1 month ago

    Toronto, Ontario, Canada CB Canada Full time

    Job Title: Big Data EngineerWe are seeking a highly skilled Big Data Engineer to join our team at CB Canada. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Job Summary:The successful candidate will have a strong background in cloud computing, big data, and data analytics. You...

  • Big Data Engineer

    4 weeks ago

    Old Toronto, Ontario, Canada Electronic Arts Inc. Full time

    Job Title: Big Data Engineer Temps PleinAs a key member of the Adrenaline team at Electronic Arts Inc., you will play a crucial role in developing data architectures that support the central technology team. Your expertise will be sought after by a wide range of team members, including game studios, tech teams, technology partners, and others.Key...

  • Big Data Engineer

    4 weeks ago

    Old Toronto, Ontario, Canada Electronic Arts Inc. Full time

    Job Title: Big Data Engineer Temps PleinAs a key member of the Adrenaline team at Electronic Arts Inc., you will play a crucial role in developing data architectures that support the central technology team. Your expertise will be sought after by a wide range of team members, including game studios, tech teams, technology partners, and others.Key...

  • Big Data Engineer

    4 weeks ago

    Old Toronto, Ontario, Canada Electronic Arts Inc. Full time

    Job Title: Big Data Engineer Temps PleinAs a key member of our Adrenaline team, you will play a crucial role in developing data architectures that support our central technology team. You will collaborate with a wide range of team members, including game studios, tech teams, technology partners, and other teams, to bring high-level subject matter expertise...

  • Big Data Engineer

    4 weeks ago

    Old Toronto, Ontario, Canada Electronic Arts Inc. Full time

    Job Title: Big Data Engineer Temps PleinAs a key member of our Adrenaline team, you will play a crucial role in developing data architectures that support our central technology team. You will collaborate with a wide range of team members, including game studios, tech teams, technology partners, and other teams, to bring high-level subject matter expertise...

  • Big Data Engineer

    4 weeks ago

    Old Toronto, Ontario, Canada Electronic Arts Full time

    Job Title: Big Data Engineer Temps PleinAs a key member of the Adrenaline team, you will play a crucial role in developing data architectures that support the central technology team. You will collaborate with a diverse range of team members, including game studios, tech teams, technology partners, and other teams. Your expertise will be instrumental in...

  • Big Data Engineer

    4 weeks ago

    Old Toronto, Ontario, Canada Electronic Arts Full time

    Job Title: Big Data Engineer Temps PleinAs a key member of the Adrenaline team, you will play a crucial role in developing data architectures that support the central technology team. You will collaborate with a diverse range of team members, including game studios, tech teams, technology partners, and other teams. Your expertise will be instrumental in...

  • Big Data Engineer

    3 weeks ago

    Old Toronto, Ontario, Canada Electronic Arts Inc. Full time

    Job Title: Big Data Engineer Temps PleinAs a key member of the Adrenaline team at Electronic Arts Inc., you will play a pivotal role in developing and implementing cutting-edge data architectures that support the central technology team. This position requires collaboration with a diverse range of team members, including game studios, tech teams, technology...

  • Big Data Engineer

    4 weeks ago

    Old Toronto, Ontario, Canada Electronic Arts Inc. Full time

    Job Title: Big Data Engineer Temps PleinAs a key member of our Adrenaline team, you will play a crucial role in developing data architectures that support our central technology team. You will collaborate with a wide range of team members, including game studios, tech teams, technology partners, and other teams, to drive the development of our data strategy...

  • Big Data Engineer

    4 weeks ago

    Old Toronto, Ontario, Canada Electronic Arts Inc. Full time

    Job Title: Big Data Engineer Temps PleinAs a key member of our Adrenaline team, you will play a crucial role in developing data architectures that support our central technology team. You will collaborate with a wide range of team members, including game studios, tech teams, technology partners, and other teams, to drive the development of our data strategy...

  • Senior Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Sage Lake Senior Living Full time $276

    Unlock Your Potential as a Senior Data EngineerAre you ready to take your data engineering career to the next level? We're seeking a highly skilled and experienced Senior Data Engineer to join our team at Enable Global Inc.About the RoleAs a Senior Data Engineer, you will play a key role in designing, developing, and deploying our SaaS products. You will...

  • Senior Data Engineer

    4 weeks ago

    Toronto, Ontario, Canada Sage Lake Senior Living Full time $276

    Unlock Your Potential as a Senior Data EngineerAre you ready to take your data engineering career to the next level? We're seeking a highly skilled and experienced Senior Data Engineer to join our team at Enable Global Inc.About the RoleAs a Senior Data Engineer, you will play a key role in designing, developing, and deploying our SaaS products. You will...

  • Senior Big Data Engineer

    1 month ago

    Toronto, Ontario, Canada Rackspace Full time

    About the Role:We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team at Rackspace. The ideal candidate will have a strong background in developing batch processing systems, with extensive experience in the Apache Hadoop ecosystem (Map Reduce, Oozie, Hive, Pig, HBase, Storm). This role involves working in Java and...

  • Senior Big Data Engineer

    1 month ago

    Toronto, Ontario, Canada Rackspace Full time

    About the Role:We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team at Rackspace. The ideal candidate will have a strong background in developing batch processing systems, with extensive experience in the Apache Hadoop ecosystem (Map Reduce, Oozie, Hive, Pig, HBase, Storm). This role involves working in Java and...

  • Senior Data Engineer, Big Data

    3 weeks ago

    Toronto, Ontario, Canada Autodesk Full time

    Job OverviewAutodesk is seeking a Senior Data Engineer, Big Data to join our Enterprise Data Engineering & Visualisation (EDEV) team. The EDEV Team is responsible for building data pipelines and solutions that enable our users to gain valuable business insights. These solutions serve as the foundation for all data engineers, data scientists, and data...