Senior Data Engineer

4 days ago

Toronto Ontario CA, Ontario Xoriant Full time

Ideal Candidate:

  • An undergraduate or Master’s degree in Computer Science or equivalent engineering experience
  • 6+ years of professional software engineering and programming experience (Java, Python) with a focus on designing and developing complex data-intensive applications
  • 3+ years of architecture and design (patterns, reliability, scalability, quality) of complex systems
  • Advanced coding skills and practices (concurrency, distributed systems, functional principles, performance optimization)
  • Professional experience working in an agile environment
  • Strong analytical and problem-solving ability
  • Strong written and verbal communication skills
  • Experience in operating and maintaining production-grade software
  • Comfortable with tackling very loosely defined problems and thrive when working on a team which has autonomy in their day to day decisions


Preferred Skills

  • In-depth knowledge of software and data engineering best practices
  • Experience in mentoring and leading junior engineers
  • Experience in serving as the technical lead for complex software development projects
  • Experience with large-scale distributed data technologies and tools
  • Strong experience with multiple database models ( relational, document, in-memory, search, etc )
  • Strong experience with Data Streaming Architecture ( Kafka, Spark, Airflow, SQL, NoSQL, CDC, etc )
  • Strong knowledge of cloud data platforms and technologies such as GCS, BigQuery, Cloud Composer, Pub/Sub, Dataflow, Dataproc, Looker, and other cloud-native offerings
  • Strong Knowledge of Infrastructure as Code (IaC) and associated tools (Terraform, ansible etc)
  • Experience pulling data from a variety of data source types including Mainframe (EBCDIC), Fixed Length and delimited files, databases (SQL, NoSQL, Time-series)
  • Strong coding skills for analytics and data engineering (Java, Python, and Scala)
  • Experience performing analysis with large datasets in a cloud-based environment, preferably with an understanding of Google’s Cloud Platform (GCP)
  • Understands how to translate business requirements to technical architectures and designs
  • Comfortable communicating with various stakeholders (technical and non-technical)
  • Experience with Airflow and Spark:
  • Airflow: Proven experience in using Apache Airflow for orchestrating and scheduling workflows. Ability to design, implement, and manage complex data pipelines. Understanding of DAGs (also how to dynamically create them), task dependencies, and error handling within Airflow.
  • Spark: Hands-on experience with Apache Spark for large-scale data processing and analytics. Proficiency in writing Spark jobs in Java (PySpak also fine as we're moving in that direction). Also, contains the ability to optimizie performance, and handling data transformations and aggregations at scale.
  • Familiarity with GCP Services:
  • BigQuery: Experience with Google BigQuery for running SQL queries on large datasets, optimizing queries for performance, and in general managing data warehousing solutions.
  • Composer: Knowledge of Google Cloud Composer for managing and orchestrating workflows.
  • Dataproc: Experience with Dataproc for managing and scaling Spark clusters, including configuring clusters, running jobs, and integrating with other GCP services.
  • Proficiency in Python, Java, and SQL:
  • Python: Strong foundation in Python, with experience in writing clean, efficient code and utilizing libraries such as Pandas and NumPy for data manipulation. Proficient in debugging, testing, and using Python for API interactions and external service integration.
  • Java: Proficiency in Java, especially for integrating with data processing frameworks. Experience with Java-based libraries and tools relevant to data engineering is a plus.
  • SQL: Experience in writing and optimizing complex SQL queries for data extraction, transformation, and analysis.
  • Knowledge of Terraform (Optional but Preferred):
  • Terraform: Familiarity with Terraform to automate the provisioning and management of cloud resources. Ability to write and maintain Terraform scripts to define and deploy GCP resources, ensuring infrastructure consistency and scalability.


Nice to have Skills (though not required):

  • Exposure to data-science or machine-learning packages (Pandas, Pytorch, Keras, TensorFlow, etc)
  • Contributions to open-source software (code, docs, or mailing list posts)
  • GCP Professional Data Engineer Certification

  • Data Engineering Senior Manager

    2 weeks ago

    Toronto, Ontario, C6A, Ontario, Canada OANDA Full time

    Everyone at OANDA is focused on our vision to transform how our customers can meet all their currency needs. We are revolutionising the world of currency trading by providing innovative trading experiences, currency data and analytics solutions. Dare to be open, bold, focused - own it and apply! The future is now!Join us and:Be the team lead for a group of...

  • Senior Data Engineer

    4 days ago

    Toronto, Ontario, C6A, Ontario, Canada YS Accounting & Develop Services Full time

    About the Role:We are seeking a talented and experienced Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines, data warehouses, and data lakes. You will play a key role in transforming raw data into valuable insights.Responsibilities:Data Ingestion: Develop robust data ingestion...

  • Senior Data Engineer

    1 month ago

    Toronto, Ontario, Ontario, Canada Finance Professionals Inc. Full time

    Senior Data Engineer Job Description Location: Hybrid, 3 days onsite – Downtown, Toronto Duration: 1 year (Extension - Yes depending on performance (looking for a long-term person)What you will do:• Work with business stakeholders and cross-functional teams to understand data requirements and deliver scalable data solutions. • Design, develop, and...

  • Data Engineer

    2 days ago

    Toronto, Ontario, C6A, Ontario, Canada Patch Tech Staffing Full time

    We are seeking a highly motivated and skilled Data Engineer to join our growing data team. In this role, you will play a crucial part in designing, building, and maintaining our data infrastructure and pipelines. You will be responsible for ingesting, transforming, and loading data from various sources into our data warehouse, ensuring its quality,...

  • Data Engineer

    2 weeks ago

    Toronto, Ontario, C6A, Ontario, Canada Quarry Consulting Full time

    Title: Data EngineerLocation: Old Toronto, ON - 2/3 times a week on-siteDuration: Permanent role FTKey Responsibilities:Design, develop, and maintain data pipelines for handling large volumes of data streams using Apache Kafka.Implement real-time data processing solutions using Apache Flink or Apache Spark.Build and maintain RESTful APIs using Spring Boot to...

  • Data Engineer

    10 hours ago

    Toronto, Ontario, C6A, Ontario, Canada Insight Global Full time

    6 month contract + possible extensions Hybrid 2x per week in Toronto Pay: $50-60/hr incorporated Must haves:3-5 years of experience as a Data Engineer.Strong experience with ETL development, including creating new ETLs to move information.3-5 years of Python experience Strong experience with large data sets Proven expertise in building out data pipelines...

  • Data Engineer

    4 days ago

    Toronto, Ontario, C6A, Ontario, Canada Themesoft Inc. Full time

    Position: Data EngineerLocation: Toronto -HybridRequired Skills: 1. Azure Data Engineer 2. Strong In Database skills 3. DatabricksJob Summary: The ideal candidate will have extensive knowledge and hands-on experience with Azure Synapse, Azure Data Factory ETL, Azure Data Lake, Azure Data Lake Store, and Azure Databricks.Responsibilities:Lead the design and...

  • Data Engineer

    4 days ago

    Toronto, Ontario, C6A, Ontario, Canada Strategic Solutions Technology Staffing Inc. Full time

    Our client is a multi-strategy alternative asset manager with global expertise in private equity, venture capital, credit, and more. They are committed to delivering innovative solutions that drive value for investors and stakeholders. As part of their team, you’ll work in a dynamic and collaborative environment that emphasizes data-driven decision-making...

  • GCP Data Engineer

    4 days ago

    Toronto, Ontario, C6A, Ontario, Canada Quantum World Technologies Inc. Full time

    GCP Data Engineer :Toronto,ONThe focus is on GCP Data Engineer, who has hand-on experience on data pipelines, data proc,composer, hive,bigquery,sql,python, airflow.10+ years of experience with Data Warehouse / Data Platforms5+ years of experience creating ELT data pipelines from scratch, working with structured, semi-structured, and unstructured data and...

  • Data Engineer

    2 weeks ago

    Toronto, Ontario, C6A, Ontario, Canada Tata Consultancy Services Full time

    Inclusion without ExceptionTCS is an equal opportunity employer, and embraces diversity in race, nationality, ethnicity, gender, age, physical ability, neurodiversity, and sexual orientation, to create a workforce that reflects the societies we operate in. Our continued commitment to Culture and Diversity and is reflected in our people stories across our...

  • Senior Data Engineering DevOps

    2 weeks ago

    Toronto, Ontario, C6A, Ontario, Canada Luxoft Full time

    Project Description:Applying to the position, you may join one of the most respected and fastest growing institutional investors in the world. Our client is a professional investment management organization that globally invests the funds to ensure long-term sustainability. The company invests in all major asset classes, including public equity, private...

  • Data Engineer

    10 hours ago

    Toronto, Ontario, C6A, Ontario, Canada Compunnel Inc. Full time

    We are hiring for our client in the financial services industry.Duration: 12+ monthsJob Type: Hybrid (3 days in office)Responsibilities:Support an Advanced Analytics Team in data enablement efforts.Transform and organize data in a cloud-based environment for effective use by analytics teams.Design and implement upstream processes to modularize data and...

  • Senior Data Engineer

    3 hours ago

    Toronto, Ontario, Canada Wirehead Full time

    Job TitleSenior Data EngineerAbout UsAt Wirehead, we are committed to delivering innovative solutions that drive business success. Our team of experts is dedicated to helping our clients achieve their goals through data-driven insights.Salary RangeThe estimated annual salary for this role is $120,000-$180,000, depending on experience and qualifications.Job...

  • Senior DevOps Engineer with Data Engineering tools

    4 days ago

    Toronto, Ontario, C6A, Ontario, Canada Luxoft Full time

    Project descriptionApplying to the position, you may join one of the most respected and fastest growing institutional investors in the world. Our client is a professional investment management organization that globally invests the funds to ensure long-term sustainability. The company invests in all major asset classes, including public equity, private...

  • Senior Data Engineer

    1 month ago

    Toronto, Ontario, Ontario, Canada Quantum World Technologies Inc. Full time

    Data Engineer Hybrid Onsite from Toronto ON CanadaJob Description :Minimum 8+ years of experience in Data Engineer with AWS, Glue, Lambda, SQL, Python, Devops, Redshift.Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred). Must be able to work with large volumes of data coming from various...

  • Senior Data Engineer

    4 weeks ago

    Toronto, Ontario, Ontario, Canada Compunnel Inc. Full time

    Title: Sr. Data Engineer LeadLocation: Toronto, Ontario – Day 1 Onsite Job Description:8+ years of experience in Data Engineer with AWS, Glue, Lambda, SQL, Python, Dovps, Redshift.Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred). Must be able to work with large volumes of data coming...

  • Sr GCP Data Engineer

    4 days ago

    Toronto, Ontario, C6A, Ontario, Canada E-Solutions Full time

    Role: Sr GCP Data EngineerLocation: Toronto Canada (Fully onsite)Ideal Candidate:• An undergraduate or Master’s degree in Computer Science or equivalent engineering experience• 10+ years of professional software engineering and programming experience (Java, Python) with a focus on designing and developing complex data-intensive applications• 3+ years...

  • Chief Developer Engineering Data

    2 months ago

    Toronto, Ontario, C6A, Ontario, Canada National Bank Full time

    A career as a Chief Data Engineering Developer on the Data Solutions team at National Bank Independent Network (NBIN) means being an expert in building and maintaining our data warehouse and data pipelines. In this role, you will be responsible for designing, developing, and maintaining ETL processes, reporting, and data models to support data-driven...

  • Principal Data Engineer

    4 days ago

    Toronto, Ontario, C6A, Ontario, Canada Premium Stairs & Railings Inc. Full time

    About the Role:We are seeking a highly skilled and experienced Principal Data Engineer to lead our data engineering team. As a Principal Data Engineer, you will be responsible for designing, developing, and implementing robust data pipelines, data warehouses, and data lakes. You will also play a key role in driving data-driven decision-making across the...

  • Senior Data Analyst

    2 weeks ago

    Toronto, Ontario, C6A, Ontario, Canada Finance Professionals Inc. Full time

    Senior Data Governance AnalystLocation: Hybrid – 2 days a week Downtown TorontoContract Duration: 7 months + possibility of extension Typical Day in Role:• Support the Senior Manager on strategic data and innovation initiatives, presentations, and analysis with executive level visibility for the Treasury Management team.• Experience working in data...