Senior Data Engineer

Select how often (in days) to receive an alert: Requisition ID: 234564 Join a purpose driven winning team, committed to results, in an inclusive and high‑performing culture. The Wealth Data engineering team within the Global Wealth Engineering (GWE) is the key team in meeting the operational data needs of the various stakeholders within Wealth Management. The Lead Data Engineer will play a key role in designing and implementing data solutions using Big Data, Hadoop (Cloudera) and Google Cloud, working closely with the enterprise data team, data architects, solution architects, business systems analysts and data engineers. Is this role right for you? In this role you will: Help development efforts in ingesting and transforming data from various sources. In the Big Data/Hadoop environment, be hands‑on with writing code, building scripts, writing specifications and responsible for end‑to‑end delivery of data in the Enterprise Data Lake environment. Build distributed, reliable and scalable data pipelines to ingest and process data from multiple data sources. Design, build and operationalise the data platform using Google Cloud Platform (GCP) data services such as DataProc, Dataflow, CloudSQL, BigQuery, CloudSpanner in combination with third parties such as Spark, Apache Beam, DBT, Cloud PubSub, Confluent Kafka, Cloud storage, Cloud Functions and GitHub. Design and implement data ingestion patterns that will support batch, streaming and API interfaces on both ingress and egress. Guide a team of data engineers and work hands‑on in developing frameworks and custom code using best practices that meet demanding performance requirements. Lead the design and build of production data pipelines from ingestion to consumption using GCP services, Java, Python, Scala, BigQuery, DBT, SQL, etc. Experience using Cloud Dataflow with Java/Python for deploying streaming jobs in GCP as well as batch jobs using text/JSON files and writing them to BigQuery. Build and manage data pipelines with a deep understanding of workflow orchestration, task scheduling and dependency management. Provide end‑to‑end technical guidance and expertise on how to effectively use Google Cloud to build solutions; creatively apply cloud infrastructure and platform services to solve business problems and communicate these approaches to different business users. Provide guidance on implementing application logging, notifications, job monitoring and performance monitoring. Do you have the skills that will enable you to succeed in this role? We’d love to work with you if you have: 2+ years of experience in data engineering and performance optimisation for large OLTP applications. Knowledge of Hadoop HDFS, Hive, Pig, Flume and Sqoop. Knowledge of the primary managed data services within GCP, including DataProc, Dataflow, BigQuery/DBT, Cloud Spanner, Cloud SQL, Cloud Pub/Sub, etc. Knowledge of Google Cloud Platform databases (SQL, Spanner, PostgreSQL). Working experience in HQL. Good knowledge of Hadoop concepts. Knowledge of relational/NoSQL databases. Knowledge of data streaming technologies such as Kafka and Spark‑streaming. Knowledge of Infrastructure as Code (IaC) practices and frameworks like Terraform. Knowledge of Java microservices and Spring Boot. Strong architecture knowledge with experience in providing technical solutions for cloud infrastructure. Working knowledge of developing and scaling Java REST services, using frameworks such as Spring. Good communication and problem‑solving skills; ability to effectively convey ideas to business and technical teams. Nice‑To‑Have Skills: Understanding of the Wealth business line and the various data domains required for building end‑to‑end solutions. What's in it for you? Diversity, Equity, Inclusion & Allyship – creating an inclusive culture where every employee is empowered to reach full potential, respected for who they are and embraced through bias‑free practices across Scotiabank. Accessibility and Workplace Accommodations – commitment to an inclusive and accessible environment, eliminating barriers for all employees. Upskilling through online courses, cross‑functional development opportunities, and tuition assistance. Competitive rewards program including bonus, flexible vacation, personal and sick days, with benefits commencing day one. Dynamic ecosystem – free tea & coffee, universal washrooms, and ample space for team collaboration. Location(s): Canada : Ontario : Toronto #J-18808-Ljbffr