Sr. Data Engineer

3 months ago

Old Toronto, Canada Paytm Full time

About Paytm Labs:

At Paytm Labs, we’re on a mission to provide useful technological solutions that enrich and empower millions of people in their daily lives. We apply big data, artificial intelligence, and machine learning to bring the next generation of financial products and services to global markets.

We recently soft-launched Pi, our full end-to-end fraud management system that can make real-time decisions. It works as a dynamic rule engine to make day-to-day analyses and procedures. Check out more about Pi here: pi.paytm.com.

Job Description:

If working with billions of events, petabytes of data, and optimizing for the last millisecond is something that excites you then read on We are looking for a Senior Data Engineer who has seen their fair share of messy data sets and has been able to structure them for further fraud detection and prevention; anomaly detection and other AI products.

You will be working on writing frameworks for real-time and batch pipelines to ingest and transform events from 100’s of applications every day. These events will be consumed by both machines and people. Our ML and Software engineers consume these events to build new and optimize existing models to detect and fight new fraud patterns. You will also help optimize the feature pipelines for fast execution and work with software engineers to build event-driven microservices.

You will get to put cutting-edge tech in production and the freedom to experiment with new frameworks, try new ways to optimize, and resources to build the next big thing in fintech using data

This position will operate on a full onsite model and be based in our Toronto office, mandatory in-office every week Monday to Friday.

Responsibilities:

  • Work directly with the Platform Engineering Team to create reusable experimental and production data pipelines and centralize the data store.
  • Unbox, deep-dive, understand, tune, and master the frameworks and technologies used day-to-day.
  • Adopt problem-solving as a way of life – always go to the root cause.
  • Keep the data whole, safe, and flowing with expertise on high-volume data ingest and streaming platforms (like Spark Streaming, Kafka, etc).
  • Make the data available for online and offline consumption by machines and humans.
  • Maintain and optimize underlying storage systems to perform according to the set SLAs.
  • Sheppard and shape the data by developing efficient structures and schema for the data in storage and transit.
  • Explore new technology options for data processing, storage, and share them with the team.
  • Develop tools and contribute to open source wherever possible.

Qualifications:

  • Degree in Computer Science, Engineering, or a related field.
  • Proficient in Spark/Scala/Python/Java.
  • You are passionate about producing clean, maintainable, and testable code as part of a real-time data pipeline.
  • You have worked with Spark and Kafka before and have experimented with or heard about Spark Streaming/Kafka Streams/Flink and understand when to use one over the other.
  • You have experience implementing offline and online data processing flows and understand how to choose and optimize underlying storage technologies. You have worked or experimented with NoSQL databases such as Cassandra.
  • You can connect different services and processes together even if you have not worked with them before and follow the flow of data through various pipelines to debug data issues.
  • You have previously worked on building serious data pipelines ingesting and transforming > 10 ^6 events per minute and terabytes of data per day.
  • On a bad day, maintaining and bringing up a cluster doesn’t bother you.
  • You may not be a computer network expert but you understand issues with ingesting data from applications in multiple data centers across geographies, on-premises and cloud, and will find a way to solve them.

Why join Paytm Labs:

  • For the fifth year in a row, we are proud to announce that we have been certified as a Great Place to Work.
  • We are an open work environment that fosters collaboration, ownership, creativity, and urgency.
  • We ensure flexible hours outside of our core working hours.
  • Enrolment in the group health benefits plan right from day 1, no waiting period.
  • Team building events.
  • We support continued learning and self-improvement.
  • Fuel for the day: Weekly delivery of groceries, and all types of snacks.
  • Catered desserts every month.

“We pooled our knowledge of the space and our world-class engineering talent to produce Pi. It’s everything we wanted in an FRM, saving us hundreds of millions of dollars.” - Harinder Takhar, CEO

Go Big or Go Home

Paytm Labs believes in diversity and equal opportunity and we will not tolerate any forms of discrimination or harassment. Our people are critical to our success and we know the more inclusive we are, the better our work will be.

We thank all applicants, however, only those selected for an interview will be contacted.

Paytm Labs is committed to meeting the accessibility needs of all individuals in accordance with the Accessibility for Ontarians with Disabilities Act (AODA) and the Ontario Human Rights Code (OHRC). Should you require accommodations during the recruitment and selection process, please let us know.

#J-18808-Ljbffr

  • Senior Data Engineer: Driving Business Growth with Data Insights

    3 weeks ago

    Old Toronto, Canada Data Engineer Jobs Full time

    As a Senior Data Engineer at Mozilla, you will play a pivotal role in shaping the company's data strategy and driving business growth through informed decision-making.About the RoleWe are seeking an experienced data engineer to join our Analytics Engineering team. In this role, you will work closely with data scientists to design and implement scalable data...

  • M Mozilla Senior Data Engineer

    2 months ago

    Old Toronto, Canada Data Engineer Jobs Full time

    To learn the Hiring Ranges for this position, please select your location from the Apply Now dropdown menu.The Mozilla Corporation is wholly owned by the non-profit 501(c) Mozilla Foundation. This means we aren't beholden to any shareholders --- only to our mission. Along with thousands of volunteer contributors and collaborators all over the world,...

  • Sr. Data Engineer

    1 week ago

    Toronto, Ontario, Ontario, Canada Lorven Technologies Inc. Full time

    Job Title: Sr. Data EngineerLocation: Toronto (Hybrid)Job Type: ContractSkills Must haveConcepts: Data Mesh, Data Product - BasicData Pipelines: PySpark, Hudi, Iceberg, Airflow, EMR- ExpertData Operations: Operational Excellence, monitoring, troubleshooting -ExpertData Quality: Great Expectations, AWS Glue DQ, Deequ- ExpertEntitlements: AWS Lakeformation...

  • Data & AI Engineering - Data Engineer

    4 weeks ago

    Old Toronto, Canada Lyons Consulting Group Full time

    Job Description: As Sr Data Engineer, you will work on one of the world's largest social media platforms which deals with a few petabytes of data coming to the system daily. You will chip in as part of an R&D self-organized team working in a complicated, innovative environment for our client. Investigate, build, and implement the solutions for many technical...

  • Sr. Consultant, Data Mesh Enablement

    1 month ago

    Old Toronto, Canada Apex Systems Full time

    Sr. Consultant, Data Mesh EnablementApex Systems, a global IT services provider, is seeking a Sr. Consultant with extensive expertise in analytics tools & Data Mesh experience to place at a leading Top Five Bank.Client: Big 5 BankLocation: Hybrid; 1-2 days a week in TorontoTerms: 12 Month Contract, strong possibility of extensionHours: 37.5 hours/week...

  • Sr. Data Engineer

    3 weeks ago

    Greater Toronto Area, Canada OnX Full time

    Sr Data Engineer6mo contract (high probability of extension)4d per week onsite (downtown) is a non-negotiableRate: fair market rateMust-Haves:Strong Data Modelling & Data Warehouse ExperienceStrong with Informatica IICSStrong Snowflake Strong SQLStrong PythonStrong cloud experience, preferably AWS but open to either Azure and GCPStrong CommunicationStrong...

  • Sr. Software Engineer in Test

    1 month ago

    Old Toronto, Canada AutoTrader.ca Full time

    TRADER Corporation is a trusted Canadian leader in online media, dealer and lender services. The company is comprised of AutoTrader.ca, AutoSync and Dealertrack Canada. AutoTrader.ca (AutoHebdo.net in Quebec) offers the largest inventory of new cars and used cars in Canada, receiving over 25 million monthly visits to its marketplace. With over 3,500...

  • Sr. Data Engineer

    3 weeks ago

    Greater Toronto Area, Canada OnX Full time

    Sr Data Engineer6mo contract (high probability of extension)4d per week onsite (downtown) is a non-negotiableRate: fair market rateMust-Haves:Strong Data Modelling & Data Warehouse ExperienceStrong with Informatica IICSStrong Snowflake Strong SQLStrong PythonStrong cloud experience, preferably AWS but open to either Azure and GCPStrong CommunicationStrong...

  • Sr. Data Engineer

    3 weeks ago

    Greater Toronto Area, Canada OnX Full time

    Sr Data Engineer6mo contract (high probability of extension)4d per week onsite (downtown) is a non-negotiableRate: fair market rateMust-Haves:Strong Data Modelling & Data Warehouse ExperienceStrong with Informatica IICSStrong Snowflake Strong SQLStrong PythonStrong cloud experience, preferably AWS but open to either Azure and GCPStrong CommunicationStrong...

  • Sr. Data Engineer

    2 weeks ago

    Greater Toronto Area, Canada OnX Full time

    Sr Data Engineer 6mo contract (high probability of extension) 4d per week onsite (downtown) is a non-negotiable Rate: fair market rate Must-Haves: Strong Data Modelling & Data Warehouse Experience Strong with Informatica IICS Strong Snowflake Strong SQL Strong Python Strong cloud experience, preferably AWS but open to either Azure and GCP Strong...

  • Sr. Data Engineer

    4 weeks ago

    Greater Toronto Area, Canada OnX Full time

    Sr Data Engineer6mo contract (high probability of extension)4d per week onsite (downtown) is a non-negotiableRate: fair market rateMust-Haves:Strong Data Modelling & Data Warehouse ExperienceStrong with Informatica IICSStrong Snowflake Strong SQLStrong PythonStrong cloud experience, preferably AWS but open to either Azure and GCPStrong CommunicationStrong...

  • Sr. Cybersecurity Platform Engineer

    2 months ago

    Old Toronto, Canada Canada Goose Full time

    Sr. Cybersecurity Platform EngineerLocation: TorontoAddress: 100 Queens Quay East, Toronto, Ontario M5E 1V3, CanadaCanada Goose isn't like anything else. We've built something great, something special - an iconic lifestyle brand with an inspirational and authentic story. At the heart of it is our promise to inspire and enable all people to thrive in the...

  • Data Engineer for Cloud Data Warehouse

    4 weeks ago

    Toronto, Ontario, Canada Lyons Consulting Group Full time

    Job OverviewSr. Data Engineer - Cloud Data WarehouseAbout the RoleWe are seeking an experienced Sr. Data Engineer to join our team at Lyons Consulting Group. As a key member of our cloud data warehouse team, you will be responsible for designing, building, and maintaining scalable data pipelines to support our clients' business needs.The ideal candidate will...

  • Sr. Data Engineer

    1 month ago

    Greater Toronto Area, Canada, Ontario OnX Full time

    Sr Data Engineer6mo contract (high probability of extension)4d per week onsite (downtown) is a non-negotiableRate: fair market rateMust-Haves:Strong Data Modelling & Data Warehouse ExperienceStrong with Informatica IICSStrong Snowflake Strong SQLStrong PythonStrong cloud experience, preferably AWS but open to either Azure and GCPStrong CommunicationStrong...

  • Data Engineer

    5 days ago

    Toronto, Canada Compunnel Inc. Full time

    Hi,Good MorningHope you are doing well.Please let me know if you are interested in this position.Title: Sr. Data Engineer LeadLocation: Toronto, Ontario – Day 1 Onsite Job Description :8+ years of experience in Data Engineer with AWS, Glue, Lambda, SQL, Python, Dovps, Redshift.Must have working knowledge in designing and implementing data pipelines on any...

  • Sr. Cloud Engineer

    1 week ago

    Toronto, Canada Thermo Fisher Scientific Inc. Full time

    Work Schedule Standard (Mon-Fri) Environmental Conditions Office Job Description Job Title: Sr. Cloud Engineer (IT/OT/IIoT) Are you a Cloud Engineering expert ready to take on an ambitious challenge? Thermo Fisher Scientific Inc. is looking for a Sr. Cloud Engineer to join our remote global innovative Innovation & IIoT / AI Hub team. This is a unique...

  • Data Engineer

    3 weeks ago

    Toronto, Canada NTT DATA Full time

    Job Description Req ID: 299022 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to remotely join our team in or near Tornoto, Ontario (CA-ON), Canada (CA).We are...

  • Sr. Cloud Engineer

    1 week ago

    Toronto, Canada Thermo Fisher Scientific Inc. Full time

    Work Schedule Standard (Mon-Fri) Environmental Conditions Office Job DescriptionJob Title: Sr. Cloud Engineer (IT/OT/IIoT)Are you a Cloud Engineering expert ready to take on an ambitious challenge? Thermo Fisher Scientific Inc. is looking for a Sr. Cloud Engineer to join our remote global innovative Innovation & IIoT / AI Hub team. This is a unique...

  • Sr. Manager, Data Science

    6 months ago

    Toronto, Canada Cineplex Full time

    Work location:Home Office 1303 Yonge StCity:TorontoWhat you will do:Sr. Manager, Data ScienceHybrid work environment 3 days per week from the Cineplex corporate headquarters at 1303 Yonge Street, Toronto (Yonge and St. Clair).The Sr. Manager, Data Science will join the Data Insights & Monetization leadership team to enable the organization with...

  • Sr. Data Engineer

    3 weeks ago

    Old Toronto, Canada Fluent Full time

    Fluent is building the next generation advertising network, Partner Monetize & Advertiser Acquisition. Our vision is to build an ML/AI first network of advertisers and publishers to achieve a common objective, elevating relevancy in E-commerce for everyday shoppers. As a Data Engineer you will bring your Databricks pipeline expertise to execute on building...