Data Pipeline Engineer

2 days ago

Mississauga, Ontario, Canada CG Consulting Group Full time
Job Title: Data Pipeline Developer

We are seeking a highly skilled Data Pipeline Developer to join our team at CG Consulting Group. As a Data Pipeline Developer, you will be responsible for designing, developing, and implementing high-performance, scalable, and reliable data pipelines using Java and Apache Spark.

Key Responsibilities:
  • Design and develop Java applications leveraging Apache Spark for distributed data processing and analytics.
  • Collaborate with data engineers and data scientists to implement data pipelines, ETL processes, and machine learning workflows using Spark.
  • Optimize and tune Spark jobs to ensure efficient utilization of computing resources and high throughput for data processing tasks.
  • Integrate Java applications with Spark clusters, leveraging Spark's APIs and libraries for data manipulation, transformation, and analysis.
  • Develop and deploy real-time and batch processing applications using Spark Streaming and Spark SQL for data ingestion and analysis.
  • Implement data caching, partitioning, and parallel processing techniques to optimize Spark job performance and resource utilization.
Requirements:
  • Bachelor's degree in Computer Science, Engineering, or a related field.
  • Proven experience as a Java Developer with hands-on experience in Apache Spark for data processing and analytics.
  • Strong proficiency in Java programming language with a focus on scalable and distributed systems.
  • Experience with Apache Spark, Spark Streaming, Spark SQL, and related Spark ecosystem tools and libraries.
  • Knowledge of big data technologies such as Hadoop, HDFS, and distributed computing frameworks for large-scale data processing.
  • Strong understanding of data structures, algorithms, and database technologies for data manipulation and analytics.
What We Offer:

CG Consulting Group offers a dynamic and challenging work environment, competitive salary and benefits, and opportunities for professional growth and development.

We are an equal opportunities employer and welcome applications from diverse candidates.

  • Senior Data Engineer

    5 days ago

    Mississauga, Ontario, Canada Data Engineer Jobs Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at AutoCapital Canada. As a Data Engineer, you will play a critical role in designing, implementing, and maintaining robust data pipelines and ETL processes for efficient extraction, transformation, and loading of large-scale datasets.Key ResponsibilitiesDesign and implement data...

  • Senior Data Engineer

    5 days ago

    Mississauga, Ontario, Canada Data Engineer Jobs Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at AutoCapital Canada. As a Data Engineer, you will play a critical role in designing, implementing, and maintaining robust data pipelines and ETL processes for efficient extraction, transformation, and loading of large-scale datasets.Key ResponsibilitiesDesign and implement data...

  • Senior Data Engineer

    2 weeks ago

    Mississauga, Ontario, Canada Data Engineer Jobs Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at AutoCapital Canada. As a key member of our Engineering team, you will be responsible for designing, implementing, and maintaining robust data pipelines and ETL processes to extract, transform, and load large-scale datasets.Key ResponsibilitiesDesign and Implement Data Pipelines:...

  • Senior Data Engineer

    2 weeks ago

    Mississauga, Ontario, Canada Data Engineer Jobs Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team at AutoCapital Canada. As a key member of our Engineering team, you will be responsible for designing, implementing, and maintaining robust data pipelines and ETL processes to extract, transform, and load large-scale datasets.Key ResponsibilitiesDesign and Implement Data Pipelines:...

  • Data Pipeline Engineer

    2 weeks ago

    Mississauga, Ontario, Canada Aviya Aerospace Systems Full time

    About the RoleAviya Aerospace Systems, a leading provider of engineering services and solutions for critical Aerospace and Defense applications, is seeking a skilled Software Development Professional to join its engineering team in Ontario.Key ResponsibilitiesDesign and implement efficient ETL pipelines, databases, and APIs for seamless data integration with...

  • Data Pipeline Engineer

    3 weeks ago

    Mississauga, Ontario, Canada Aviya Aerospace Systems Full time

    About the RoleAviya Aerospace Systems, a leading provider of engineering services and solutions for critical Aerospace and Defense applications, is seeking a skilled Software Development Professional to join its engineering team in Ontario.Key ResponsibilitiesDesign and implement efficient ETL pipelines, databases, and APIs for seamless data integration with...

  • Senior Data Pipeline Engineer

    2 weeks ago

    Mississauga, Ontario, Canada Egen Full time

    About EgenEgen is a rapidly expanding and entrepreneurial organization with a data-driven approach.We bring together top engineering talent working with cutting-edge technology platforms, including Google Cloud and Salesforce, to help clients drive action and impact through data and insights.We are committed to being a workplace where the best people choose...

  • Senior Data Pipeline Engineer

    2 weeks ago

    Mississauga, Ontario, Canada Egen Full time

    About EgenEgen is a rapidly expanding and entrepreneurial organization with a data-driven approach.We bring together top engineering talent working with cutting-edge technology platforms, including Google Cloud and Salesforce, to help clients drive action and impact through data and insights.We are committed to being a workplace where the best people choose...

  • Software Engineer for Data Pipeline Development

    3 days ago

    Mississauga, Ontario, Canada Aviyatech Full time

    Job Title: DPHM Software EngineerAviya Aerospace Systems is a leading provider of engineering services and solutions for mission-critical aerospace and defense applications. We are seeking a highly skilled DPHM Software Engineer to join our engineering team in Mississauga, Ontario.Job SummaryThe successful candidate will be responsible for designing,...

  • Software Engineer for Data Pipeline Development

    3 days ago

    Mississauga, Ontario, Canada Aviyatech Full time

    Job Title: DPHM Software EngineerAviya Aerospace Systems is a leading provider of engineering services and solutions for mission-critical aerospace and defense applications. We are seeking a highly skilled DPHM Software Engineer to join our engineering team in Mississauga, Ontario.Job SummaryThe successful candidate will be responsible for designing,...

  • Data Pipeline Architect

    2 weeks ago

    Mississauga, Ontario, Canada Egen Full time

    About EgenWe are a fast-growing and entrepreneurial company with a data-first mindset. Our mission is to bring together the best engineering talent to work with the most advanced technology platforms, including Google Cloud and Salesforce, to help clients drive action and impact through data and insights.We are committed to being a place where the best...

  • Data Pipeline Architect

    2 weeks ago

    Mississauga, Ontario, Canada Egen Full time

    About EgenWe are a fast-growing and entrepreneurial company with a data-first mindset. Our mission is to bring together the best engineering talent to work with the most advanced technology platforms, including Google Cloud and Salesforce, to help clients drive action and impact through data and insights.We are committed to being a place where the best...

  • Senior Data Engineer

    1 week ago

    Mississauga, Ontario, Canada Roche Full time

    The OpportunityWe are seeking a highly skilled Senior Data Engineer to join our Agile team in developing and maintaining robust data pipelines and data management systems. As a key member of our team, you will partner closely with data science and business intelligence teams to understand business needs and co-create solutions that drive business value.Key...

  • Senior Data Engineer

    1 week ago

    Mississauga, Ontario, Canada Roche Full time

    The OpportunityWe are seeking a highly skilled Senior Data Engineer to join our Agile team in developing and maintaining robust data pipelines and data management systems. As a key member of our team, you will partner closely with data science and business intelligence teams to understand business needs and co-create solutions that drive business value.Key...

  • Senior Data Engineer

    2 weeks ago

    Mississauga, Ontario, Canada Roche Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our Agile team in developing and maintaining robust data pipelines and data management systems. As a key member of our team, you will work closely with data science and business intelligence teams to understand business needs and co-create solutions that drive business value.Key...

  • Senior Data Engineer

    2 weeks ago

    Mississauga, Ontario, Canada Roche Full time

    About the RoleWe are seeking a highly skilled Senior Data Engineer to join our Agile team in developing and maintaining robust data pipelines and data management systems. As a key member of our team, you will work closely with data science and business intelligence teams to understand business needs and co-create solutions that drive business value.Key...

  • DPHM Software Engineer

    2 weeks ago

    Mississauga, Ontario, Canada Aviya Aerospace Systems Full time

    Aviya Aerospace Systems is a leading provider of engineering services and solutions for mission-critical aerospace and defense applications. Our company offers expertise in program management, systems, software, mechanical, and hardware engineering to top-tier aerospace and defense contractors.Job SummaryWe are seeking a highly skilled DPHM Software Engineer...

  • DPHM Software Engineer

    2 weeks ago

    Mississauga, Ontario, Canada Aviya Aerospace Systems Full time

    Aviya Aerospace Systems is a leading provider of engineering services and solutions for mission-critical aerospace and defense applications. Our company offers expertise in program management, systems, software, mechanical, and hardware engineering to top-tier aerospace and defense contractors.Job SummaryWe are seeking a highly skilled DPHM Software Engineer...

  • Data Engineer

    2 weeks ago

    Mississauga, Ontario, Canada EllisDon Full time

    About EllisDonEllisDon is a leading construction company that values innovation and excellence in all aspects of its operations. We are committed to providing our clients with exceptional service and delivering projects on time, within budget, and to the highest quality standards.About the RoleWe are seeking a highly skilled Data Engineer to join our team....

  • Data Engineer

    2 weeks ago

    Mississauga, Ontario, Canada EllisDon Full time

    About EllisDonEllisDon is a leading construction company that values innovation and excellence in all aspects of its operations. We are committed to providing our clients with exceptional service and delivering projects on time, within budget, and to the highest quality standards.About the RoleWe are seeking a highly skilled Data Engineer to join our team....