Lead Big Data Developer

1 week ago

Toronto, Ontario, Canada Vybog Full time $120,000 - $180,000 per year

Overview

Are you interested in building the next-generation platform that handles
petabytes of data
and powers
data insight creation
using cutting-edge
data science technologies
?

Are you excited about leveraging
Big Data
,
AI / Machine Learning
, and
Data Analytics
to deliver actionable insights?

We are a
dynamic team of top-notch engineers
dedicated to making a dent in the
FinTech universe
.

We're rapidly expanding in
Vancouver, beautiful British Columbia
, and if you wish to be part of this
Priceless experience
, read below.

The
Marketing Services (MS) Technology team
is looking for a
Senior Data Engineer
who is passionate about data and analytics, highly motivated, and capable of taking technical ownership of data solutions with an emphasis on
engineering efficiency
and
on-time delivery
.

The ideal candidate will be familiar with the fast-paced world of Big Data technologies and have experience building data solutions using
emerging tools
while ensuring
platform stability
. You'll work with diverse datasets and be at the forefront of transforming the way
Mastercard captures, processes, stores, and visualizes data
.

Role & Responsibilities

In this role, you will:

  • Design and develop batch and streaming solutions
    using various Big Data technologies.
  • Work closely with architects
    to design secure, scalable, and efficient data platforms.
  • Take ownership of
    assessing technologies and approaches
    for data ingestion, transformation, and storage.
  • Develop software
    utilizing open-source technologies to interface with distributed and relational data sources.
  • Collaborate with
    cross-functional Mastercard teams
    to identify functional and system requirements.

All About You

The ideal candidate for this position should have:

  • Hands-on experience
    with Big Data and analytics technologies, including
    Hadoop
    ,
    Spark
    , and related frameworks.
  • Programming expertise
    in
    Java
    ,
    Scala
    , or
    Python
    , with a focus on developing data processing and transformation logic.
  • Real-time and batch data processing
    experience, building robust data pipelines for real-time or near-real-time ingestion.
  • Proficiency in
    data modeling
    and
    schema design
    for both relational and non-relational databases, supporting high data volume and optimized query performance.
  • Exposure to
    cloud technologies
    with hands-on experience in
    Databricks
    and
    Cloudera
    , including knowledge of
    cloud architecture
    ,
    data security
    , and
    scalability best practices
    .
  • Experience in
    optimizing and managing ETL workflows
    using scheduling, monitoring, and debugging tools.
  • Strong
    problem-solving abilities
    with a mindset for
    automation and efficiency improvements
    .
  • Excellent
    communication and collaboration skills
    to work effectively across technical and business teams, promoting knowledge sharing and alignment on data engineering best practices.
  • Proven track record of
    delivering software projects
    and a willingness to roll up your sleeves to get the job done.
  • Excellent oral and written communication skills.
  • Familiarity with
    Agile / Scrum methodologies.
  • Ability to
    manage multiple priorities simultaneously.
  • A
    BS/BA degree
    in
    Computer Science, Information Systems
    , or a related field.

  • Big Data Developer

    7 days ago

    Toronto, Ontario, Canada fc71e8c5-deda-48e9-b38c-23c0611c543b Full time $70,000 - $120,000 per year

    Job OverviewWe are seeking a dynamic and innovative Big Data Developer to join our data engineering team. In this role, you will harness the power of large-scale data processing and analytics to drive strategic insights and support business growth. Your expertise in big data technologies, cloud platforms, and data integration will be crucial in designing,...

  • Big Data Developer

    4 days ago

    Toronto, Ontario, Canada Triton Digital Full time $120,000 - $180,000 per year

    We are open to candidates in Ontario or Quebec.Are you passionate about new technologies and looking to be part of a software development team that is constantly striving for innovation in a warm, relaxed, and stimulating environment?We are looking for aBig Data Developer / Machine Learning Engineerto join our "Audience" team.Our success depends on our...

  • Big Data Developer

    2 weeks ago

    Toronto, Ontario, Canada Tekshapers Full time $80,000 - $120,000 per year

    Job Title: Big data DeveloperLocation: Toronto, ONFull TimeTotal Experience: 8-10 yearsRole Description:• We are looking for a big data developer with hands on experience on developing APIs.• The candidate should also have knowledge of Kubernetes and worked as a Java developer.• The candidate should be able to exhibit strong coding skills in above tech...

  • Big Data Developer

    7 days ago

    Toronto, Ontario, Canada Accuro Full time $90,000 - $120,000 per year

    Job Tite: Big Data DeveloperLocation: Toronto, ON (Hybrid)Job Type: FTEJob Description:Primary Skill:• Min 5+ years in developing and optimizing Big Data applications using Java/Scala and Spark on Cloudera/HDP.• Experience in developing/designing micro-service architecture.• Proven hands-on experience in containerization – Docker, Kubernetes,...

  • Big Data Developer

    7 days ago

    Toronto, Ontario, Canada ehub global Full time $72,000 - $96,000 per year

    Role-Big-Data developerLocation- Toronto, ON (Onsite from day 1)JD:Experience: 8+ yearsBig Data: Apache Spark, Hadoop ecosystem(HDFS, Hive, Yarn).Programming : Expert in Scala and JavaBackend :Spring Boot, RESTful APIsDatabase : Microsoft SQL Server(MSSQL)Nice to have : Experience in migrating on-premise applications to cloud (Azure/AWS), familiarity with...

  • Big Data Developer

    2 weeks ago

    Toronto, Ontario, Canada Info Way Solutions Full time $80,000 - $120,000 per year

    Big Data Developer: (No of position – 1)JD:• Experience: 8+ years• Big Data: Apache Spark, Hadoop ecosystem(HDFS, Hive, Yarn).• Programming : Expert in Scala and Java• Backend :Spring Boot, RESTful APIs• Database : Microsoft SQL Server(MSSQL)• Nice to have : Experience in migrating on-premise applications to cloud (Azure/AWS), familiarity with...

  • Big Data Developer

    2 weeks ago

    Toronto, Ontario, Canada Smart IT Frame LLC Full time $80,000 - $120,000 per year

    Role :Bigdata(Spark,Scala), Java,Sql,Unix,OCPLocation :TorontoJD:6 - 0 years of experience in Big Data technologies, especially Spark and Scala.Strong programming skills in Java and Scala.Proficient in writing complex SQL queries and working with relational databases.Hands-on experience with Unix/Linux environments and shell scripting.Oracle Certified...

  • Consultant, Big Data Development

    7 days ago

    Toronto, Ontario, Canada CIBC Full time $80,000 - $120,000 per year

    We're building a relationship-oriented bank for the modern world. We need talented, passionate professionals who are dedicated to doing what's right for our clients.At CIBC, we embrace your strengths and your ambitions, so you are empowered at work. Our team members have what they need to make a meaningful impact and are truly valued for who they are and...

  • Big Data Platform Engineer

    2 weeks ago

    Toronto, Ontario, Canada CISPL Full time $110,000 - $135,000 per year

    Big Data Platform EngineerPosition OverviewWe are looking for someone armed with a strong tool-kit to develop and maintain technical solutions that adhere to engineering and architectural design principles while meeting business requirements. You'll be a technology owner providing technical expertise with a focus on efficiency, reliability, scalability, and...

  • Senior Data Engineer – Big Data

    7 days ago

    Toronto, Ontario, Canada AstraNorth Full time $120,000 - $180,000 per year

    • Exp Required: 10EngineerSkills Required: Agile Way of Working Digital : Python Digital : Bigdata and Hadoop Ecosystem - MapR PL/SQLJob Description: Required Technical Skills-10 years of experience in designing and implementing the applications.8 year of experience in Spark, Scala, and Java8 years of experience in SQL Server RDBMS.Knowledge of...