Data pipeline developer

3 months ago


Mississauga, Canada CG Consulting Group Full time
div>Sr Java Spark Developer

This is with a large IT services firm for a major US based bank.
Candidate must have legal work status for Canada.
Client Location is Mississauga, Ontario
Hybrid - In-Office 3 days per week.
Perm/FT role, salary and benefits OR Contract (6 months to start)
Market rate

Profile
We are looking for a talented and experienced Java Spark Developer to join our dynamic team. As a Java Spark Developer, you will be responsible for designing, developing, and implementing high-performance, scalable, and reliable applications using Java and Apache Spark. You will work on data-intensive projects, leveraging the power of Spark for large-scale data processing and analytics.
Keys: Java, Apache Spark, data streaming

Responsibilities:
Design, develop, and maintain Java applications leveraging Apache Spark for distributed data processing and analytics.
Collaborate with data engineers and data scientists to implement data pipelines, ETL processes, and machine learning workflows using Spark.
Optimize and tune Spark jobs to ensure efficient utilization of computing resources and high throughput for data processing tasks.
Integrate Java applications with Spark clusters, leveraging Spark's APIs and libraries for data manipulation, transformation, and analysis.
Develop and deploy real-time and batch processing applications using Spark Streaming and Spark SQL for data ingestion and analysis.
Implement data caching, partitioning, and parallel processing techniques to optimize Spark job performance and resource utilization.
Work with cloud-based platforms and big data technologies to deploy and manage Spark-based applications in distributed environments.
Collaborate with cross-functional teams to understand business requirements, data models, and analytics use cases, and implement relevant solutions using Spark.

Requirements:
Bachelor's degree in Computer Science, Engineering, or a related field.
Proven experience as a Java Developer with hands-on experience in Apache Spark for data processing and analytics.
Strong proficiency in Java programming language with a focus on scalable and distributed systems.
Experience with Apache Spark, Spark Streaming, Spark SQL, and related Spark ecosystem tools and libraries.
Knowledge of big data technologies such as Hadoop, HDFS, and distributed computing frameworks for large-scale data processing.
Strong understanding of data structures, algorithms, and database technologies for data manipulation and analytics.
Experience with Agile development methodologies and tools (e.g., br/>Certification in Apache Spark or related big data technologies is advantageous.

  • Mississauga, Ontario, Canada Data Engineer Jobs Full time

    AutoCapital Canada Inc. is a leading Canadian financial services company with diverse portfolios across automotive, consumer, and merchant lending.About the RoleThis position offers an exciting opportunity to leverage big data and cloud computing technologies in writing impactful data pipelines that drive business growth.Design, implement, and maintain...

  • Senior Data Engineer

    1 month ago


    Mississauga, Ontario, Canada F. Hoffmann-La Roche Gruppe Full time

    Rochefosters diversity, equity, and inclusion, representing the communities we serve. In the field of healthcare, diversity is key to success. We believe that inclusion unlocks understanding of people's varied healthcare needs. Together, we celebrate individuality and share a passion for exceptional care.The RoleWe are always innovating, bringing together...

  • Data Engineer

    1 month ago


    Mississauga, Ontario, Canada Capgemini Full time

    Job Title: Data Engineer - Scalable Pipeline DeveloperAbout the RoleCapgemini is seeking a Data Engineer - Scalable Pipeline Developer to join its team of innovative professionals. The ideal candidate will have a strong background in data engineering, with expertise in scalable pipeline development using Java or Python.This is a unique opportunity to work...

  • Senior Data Engineer

    1 month ago


    Mississauga, Ontario, Canada Data Engineer Jobs Full time

    About the RoleWe are seeking an experienced Data Engineer to join our team at AutoCapital Canada. As a Data Engineer, you will play a crucial role in designing, implementing, and maintaining robust data pipelines and ETL processes for efficient extraction, transformation, and loading of large-scale datasets.Key ResponsibilitiesData Pipeline Design and...


  • Mississauga, Ontario, Canada CB Canada Full time

    CB Canada is seeking a Senior Data Pipeline Specialist to support our Retail Sector client.The ideal candidate will have expertise in building data pipelines in Hadoop/GCP and SQL analysis.Key responsibilities include:Designing and implementing scalable and resilient data systemsCollaborating with cross-functional teams to meet business objectivesStaying...

  • Senior Data Architect

    4 weeks ago


    Mississauga, Ontario, Canada Data Engineer Jobs Full time

    Unlock career opportunities as a Senior Data Architect with AutoCapital Canada Inc., a leading national financial services company. Our team focuses on innovative data solutions and cutting-edge technology to drive business growth.We are looking for an experienced Senior Data Architect to join our Engineering team in Toronto, Ontario. As a key member of the...

  • Data Engineer, Finance

    2 months ago


    Mississauga, Ontario, Canada Data Engineer Jobs Full time

    Job Title: Data Engineer, FinanceAutoCapital Canada Inc. is a leading financial services company with a strong presence in the Canadian market. We are seeking a highly skilled Data Engineer to join our team and contribute to the development of our data-driven solutions.Key Responsibilities:Design, implement, and maintain robust data pipelines and ETL...


  • Mississauga, Ontario, Canada Data Engineer Jobs Full time

    Established financial services company, AutoCapital Canada Inc., is a leader in automotive, consumer, and merchant lending portfolios. Our vertically integrated business model spans the entire lifecycle of finance receivables, from credit adjudication to post-charge-off recoveries.This dynamic work environment revolves around data: creation, extraction,...


  • Mississauga, Ontario, Canada Data Engineer Jobs Full time

    Job Title: Senior Data Processing EngineerCompany Overview:AutoCapital Canada Inc. is a Canadian national financial services company with a diverse portfolio of automotive, consumer, and merchant lending.We specialize in managing the entire lifecycle of finance receivables, from credit adjudication to contract administration and customer service.We work...


  • Mississauga, Ontario, Canada Astra North Infoteck Inc. Full time

    Job SummaryAstra North Infoteck Inc. is seeking a skilled Senior Data Systems Developer to join our team.About UsAstra North Infoteck Inc. is a leading provider of data engineering solutions.SalaryThe estimated salary for this position is $120,000 - $180,000 per year.About the RoleWe are looking for a highly experienced Data Engineer to lead our data...


  • Mississauga, Ontario, Canada Synechron Full time

    We are seeking a skilled Python PySpark Developer to join our team at Synechron. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions.About the RoleThe ideal candidate will have strong programming expertise in Python and experience with Apache Spark, particularly in data processing and...


  • Mississauga, Canada Data Engineer Jobs Full time

    AutoCapital Canada Inc. is part of privately-owned Canadian national financial services company with multiple verticals across automotive, consumer, and merchant lending portfolios. We manage the entire lifecycle of the finance receivable from credit adjudication through to contract administration, customer service, default management and post charge-off...


  • Mississauga, Canada Aviyatech Full time

    p>Aviya Aerospace Systems is a leader in engineering services and solutions for mission critical Aerospace and Defense applications. Aviya provides expertise in program management, systems, software, mechanical, and hardware engineering to many of the top-tier Aerospace and Defense Contractors. Aviya has an opening for a DPHM Software Engineer to join our...


  • Mississauga, Ontario, Canada Astra North Infoteck Inc. Full time

    About Astra North Infoteck Inc.We are a leading company in the industry, dedicated to delivering high-quality solutions to our clients. We are seeking an experienced Senior Java Developer and Data Architect to join our team.Job OverviewThe successful candidate will be responsible for designing and developing scalable data systems using Java, as well as...


  • Mississauga, Ontario, Canada Virtusa Full time

    About the RoleVirtusa is seeking an experienced Data Scientist to join our team. As a Data Scientist, you will be responsible for developing and implementing data-driven solutions using Python and Big Data technologies.Key ResponsibilitiesDesign, develop, and deploy data pipelines using Python and Big Data tools such as PySpark.Analyze and process large...

  • Office Administrator

    6 months ago


    Mississauga, Canada PipeLine Ontario Full time

    **About us** PipeLine Ontario is a small business in Mississauga, ON L4W 2M4. We are professional, and our goal is to provide Toronto and the GTA with plumbing and mechanical service. Our work environment includes: - Modern office setting **Responsibilities**: - Confirm on-site jobs and appointments. - Manage work orders. - Order material through...

  • Office Administrator

    6 months ago


    Mississauga, Canada PipeLine Ontario Full time

    **About us** PipeLine Ontario is a small business in Mississauga, ON L4W 2M4. We are professional, and our goal is to provide the GTA and southwestern Ontario with plumbing and mechanical service. Our work environment includes: - Modern office setting **Responsibilities**: - Confirm on-site jobs and appointments. - Manage work orders. - Order material...

  • Big Data Developer

    1 month ago


    Mississauga, Ontario, Peel region, Canada Hays Full time

    Software Engineer - Big dataSalary: 100-110 CADJob DescriptionYour new company Our client, a leading global chemistry company across US, Canada, Asia and Europe, is going through a Digital Transformation. They are building a Digital Hub for the global business in Toronto city, therefore we are hiring multiple technology talents with ambition to join the...


  • Mississauga, Canada Data Privacy Full time

    Chartwell Retirement Residences is looking for a Data Visualization Analyst to convert complex data into actionable insights aiding strategic planning, finance, and operations. Collaborating with the Data & Analytics team, the role aims to create impactful dashboards that drive decision-making across the organization.In this role you can expect to have the...

  • Big Data Developer

    2 weeks ago


    Mississauga, Canada Dexian Full time

    Job Title: Big Data Developer Location: Mississauga, ON (Hybrid) Contract Duration: 12 months (Potential Extension) Key Responsibilities: Strong background in building and optimizing large-scale data processing applications. Design, develop, and optimize large-scale data processing applications using PySpark. Collaborate with cross-functional teams to...