Sr Gcp Data Engineer

2 days ago

Toronto ON, Canada E-Solutions Full time

Role:
Sr GCP Data Engineer

Location:
Toronto Canada (Fully onsite)

Ideal Candidate:

- An undergraduate or Master’s degree in Computer Science or equivalent engineering experience

- 10+ years of professional software engineering and programming experience (Java, Python) with a focus on designing and developing complex data-intensive applications

- 3+ years of architecture and design (patterns, reliability, scalability, quality) of complex systems

- Advanced coding skills and practices (concurrency, distributed systems, functional principles, performance optimization)

- Professional experience working in an agile environment

- Strong analytical and problem-solving ability

- Strong written and verbal communication skills

- Experience in operating and maintaining production-grade software

- Comfortable with tackling very loosely defined problems and thrive when working on a team which has autonomy in their day to day decisions

Preferred Skills

- In-depth knowledge of software and data engineering best practices

- Experience in mentoring and leading junior engineers

- Experience in serving as the technical lead for complex software development projects

- Experience with large-scale distributed data technologies and tools

- Strong experience with multiple database models ( relational, document, in-memory, search, etc )

- Strong experience with Data Streaming Architecture ( Kafka, Spark, Airflow, SQL, NoSQL, CDC, etc )

- Strong knowledge of cloud data platforms and technologies such as GCS, BigQuery, Cloud Composer, Pub/Sub, Dataflow, Dataproc, Looker, and other cloud-native offerings

- Strong Knowledge of Infrastructure as Code (IaC) and associated tools (Terraform, ansible etc)

- Experience pulling data from a variety of data source types including Mainframe (EBCDIC), Fixed Length and delimited files, databases (SQL, NoSQL, Time-series)

- Strong coding skills for analytics and data engineering (Java, Python, and Scala)

- Experience performing analysis with large datasets in a cloud-based environment, preferably with an understanding of Google’s Cloud Platform (GCP)

- Understands how to translate business requirements to technical architectures and designs

- Comfortable communicating with various stakeholders (technical and non-technical)

- Experience with Airflow and Spark:

- Airflow:
Proven experience in using Apache Airflow for orchestrating and scheduling workflows. Ability to design, implement, and manage complex data pipelines. Understanding of DAGs (also how to dynamically create them), task dependencies, and error handling within Airflow.

- Spark:
Hands-on experience with Apache Spark for large-scale data processing and analytics. Proficiency in writing Spark jobs in Java (PySpak also fine as we're moving in that direction). Also, contains the ability to optimizie performance, and handling data transformations and aggregations at scale.

- Familiarity with GCP Services:

- BigQuery:
Experience with Google BigQuery for running SQL queries on large datasets, optimizing queries for performance, and in general managing data warehousing solutions.

- Composer:
Knowledge of Google Cloud Composer for managing and orchestrating workflows.

- Dataproc:
Experience with Dataproc for managing and scaling Spark clusters, including configuring clusters, running jobs, and integrating with other GCP services.

- Proficiency in Python, Java, and SQL:

- Python:
Strong foundation in Python, with experience in writing clean, efficient code and utilizing libraries such as Pandas and NumPy for data manipulation. Proficient in debugging, testing, and using Python for API interactions and external service integration.

- Java:
Proficiency in Java, especially for integrating with data processing frameworks. Experience with Java-based libraries and tools relevant to data engineering is a plus.

- SQL:
Experience in writing and optimizing complex SQL queries for data extraction, transformation, and analysis.

- Knowledge of Terraform (Optional but Preferred):

- Terraform:
Familiarity with Terraform to automate the provisioning and management of cloud resources. Ability to write and maintain Terraform scripts to define and deploy GCP resources, ensuring infrastructure consistency and scalability.

Nice to have Skills (though not required):

- Exposure to data-science or machine-learning packages (Pandas, Pytorch, Keras, TensorFlow, etc...)

- Contributions to open-source software (code, docs, or mailing list posts)

- GCP Professional Data Engineer Certification

  • Sr Gcp Data Engineer

    2 days ago

    Toronto, ON, Canada E-Solutions Full time

    Role: Sr GCP Data Engineer Location: Toronto Canada (Fully onsite) Ideal Candidate: - An undergraduate or Master’s degree in Computer Science or equivalent engineering experience - 10+ years of professional software engineering and programming experience (Java, Python) with a focus on designing and developing complex data-intensive applications - 3+...

  • Sr GCP Data Engineer

    3 days ago

    Toronto, Canada E-Solutions Full time

    Role: Sr GCP Data EngineerLocation: Toronto Canada (Fully onsite)Ideal Candidate:• An undergraduate or Master’s degree in Computer Science or equivalent engineering experience• 10+ years of professional software engineering and programming experience (Java, Python) with a focus on designing and developing complex data-intensive applications• 3+ years...

  • Sr GCP Data Engineer

    4 days ago

    Toronto, Canada E-Solutions Full time

    Role: Sr GCP Data EngineerLocation: Toronto Canada (Fully onsite)Ideal Candidate:• An undergraduate or Master’s degree in Computer Science or equivalent engineering experience• 10+ years of professional software engineering and programming experience (Java, Python) with a focus on designing and developing complex data-intensive applications• 3+ years...

  • Sr GCP Data Engineer

    3 days ago

    Toronto, Canada E-Solutions Full time

    Role: Sr GCP Data Engineer Location: Toronto Canada (Fully onsite) Ideal Candidate: • An undergraduate or Master’s degree in Computer Science or equivalent engineering experience • 10+ years of professional software engineering and programming experience (Java, Python) with a focus on designing and developing complex data-intensive applications • 3+...

  • Sr GCP Data Engineer

    3 days ago

    Toronto, Canada E-Solutions Full time

    Role: Sr GCP Data EngineerLocation: Toronto Canada (Fully onsite)Ideal Candidate:• An undergraduate or Master’s degree in Computer Science or equivalent engineering experience• 10+ years of professional software engineering and programming experience (Java, Python) with a focus on designing and developing complex data-intensive applications• 3+ years...

  • Sr GCP Data Engineer

    5 days ago

    Toronto, Ontario, C6A, Ontario, Canada E-Solutions Full time

    Role: Sr GCP Data EngineerLocation: Toronto Canada (Fully onsite)Ideal Candidate:• An undergraduate or Master’s degree in Computer Science or equivalent engineering experience• 10+ years of professional software engineering and programming experience (Java, Python) with a focus on designing and developing complex data-intensive applications• 3+ years...

  • GCP Data Engineer F/H

    2 days ago

    Toronto, ON, Canada E-Solutions Full time

    Role: Sr GCP Data Engineer Location: Toronto Canada (Fully onsite) Ideal Candidate: - An undergraduate or Master’s degree in Computer Science or equivalent engineering experience - 10+ years of professional software engineering and programming experience (Java, Python) with a focus on designing and developing complex data-intensive applications...

  • GCP Data Engineer

    3 days ago

    Toronto, Canada Quantum World Technologies Inc. Full time

    GCP Data Engineer :Toronto,ONThe focus is on GCP Data Engineer, who has hand-on experience on data pipelines, data proc,composer, hive,bigquery,sql,python, airflow.10+ years of experience with Data Warehouse / Data Platforms5+ years of experience creating ELT data pipelines from scratch, working with structured, semi-structured, and unstructured data and...

  • GCP Data Engineer

    1 day ago

    Toronto, Canada Quantum World Technologies Inc. Full time

    GCP Data Engineer :Toronto,ONThe focus is on GCP Data Engineer, who has hand-on experience on data pipelines, data proc,composer, hive,bigquery,sql,python, airflow.10+ years of experience with Data Warehouse / Data Platforms5+ years of experience creating ELT data pipelines from scratch, working with structured, semi-structured, and unstructured data and...

  • GCP Data Engineer

    3 days ago

    Toronto, Canada Quantum World Technologies Inc. Full time

    GCP Data Engineer : Toronto,ON The focus is on GCP Data Engineer, who has hand-on experience on data pipelines, data proc,composer, hive,bigquery,sql,python, airflow . 10+ years of experience with Data Warehouse / Data Platforms 5+ years of experience creating ELT data pipelines from scratch, working with structured, semi-structured, and unstructured data...

  • GCP Data Engineer

    3 days ago

    Toronto, Canada Quantum World Technologies Inc. Full time

    GCP Data Engineer :Toronto,ONThe focus is on GCP Data Engineer, who has hand-on experience on data pipelines, data proc,composer, hive,bigquery,sql,python, airflow.10+ years of experience with Data Warehouse / Data Platforms5+ years of experience creating ELT data pipelines from scratch, working with structured, semi-structured, and unstructured data and...

  • GCP Data Engineer

    4 days ago

    Toronto, Canada Quantum World Technologies Inc. Full time

    GCP Data Engineer :Toronto,ONThe focus is on GCP Data Engineer, who has hand-on experience on data pipelines, data proc,composer, hive,bigquery,sql,python, airflow.10+ years of experience with Data Warehouse / Data Platforms5+ years of experience creating ELT data pipelines from scratch, working with structured, semi-structured, and unstructured data and...

  • GCP Data Engineer

    5 days ago

    Toronto, Ontario, C6A, Ontario, Canada Quantum World Technologies Inc. Full time

    GCP Data Engineer :Toronto,ONThe focus is on GCP Data Engineer, who has hand-on experience on data pipelines, data proc,composer, hive,bigquery,sql,python, airflow.10+ years of experience with Data Warehouse / Data Platforms5+ years of experience creating ELT data pipelines from scratch, working with structured, semi-structured, and unstructured data and...

  • Data Engineer

    4 weeks ago

    Toronto, ON, Canada CorGTA Full time

    Role: Data Engineer (Fortune 1000 Client) Structure: Contract (6 months initially then extension options) Location: Toronto, ON (Hybrid 2-3 days in office per week)- The role: We currently have an opportunity for a contract for a hybrid remote Data Engineer working for one of our Toronto based Fortune 1000 clients. - The required skills for this role will...

  • GCP Data Architect

    4 weeks ago

    Toronto, Canada Avanciers Full time

    We are seeking a highly skilled GCP Data Architect to join our team at Avanciers in Toronto- Hybrid (FTE/Contract) environment.Estimated salary for this position is around $150,000 - $200,000 per year, depending on the candidate's experience and qualifications.About the RoleThe successful candidate will have 10+ years of experience with data warehouse and...

  • GCP Data Engineer Role with Advanced Cloud Services

    4 weeks ago

    Toronto, Canada Avanciers Full time

    Are you a skilled data professional looking to leverage your expertise in Google Cloud Platform (GCP) to drive business growth? Avanciers is seeking an experienced GCP Data Engineer to join our team in Toronto, working in a hybrid environment.Salary: $170,000 - $200,000 per annum, commensurate with experience.Job Description:We are on the hunt for a seasoned...

  • GCP Data Architect Specialist

    5 days ago

    Toronto, Canada Quantum World Technologies Inc. Full time

    At Quantum World Technologies Inc., we are seeking a highly skilled GCP Data Architect Specialist to join our team.Salary: $180,000 - $220,000 per year.Job DescriptionWe are looking for an experienced professional with expertise in designing and implementing scalable data architectures on Google Cloud Platform (GCP).The ideal candidate will have a strong...

  • Sr. Data Engineer

    2 days ago

    Toronto, ON, Canada GalaxE.Solutions Full time

    What you will do: Implement methods to improve data reliability and quality. Deploy programmable, easily customizable infrastructure that embraces change and automates the deployment process to reduce risk Analyze all platform level changes, monitor application impact, and provide appropriate technical solutions to resolve all issues quickly and...

  • Sr. Data Engineer

    2 days ago

    Toronto, ON, Canada GalaxE.Solutions Full time

    What you will do: Implement methods to improve data reliability and quality. Deploy programmable, easily customizable infrastructure that embraces change and automates the deployment process to reduce risk Analyze all platform level changes, monitor application impact, and provide appropriate technical solutions to resolve all issues quickly and...

  • Data Engineering Leader for Cloud-Based Data Platforms

    2 days ago

    Toronto, Ontario, Canada E-Solutions Full time

    E-Solutions is seeking a highly skilled Sr GCP Data Engineer to lead the design and development of our cloud-based data platforms.As a key member of our team, you will be responsible for architecting and implementing scalable data solutions using Google Cloud Platform (GCP) technologies such as BigQuery, Cloud Composer, Dataproc, and Pub/Sub.We are looking...