Big Data/Hadoop/Splunk Tester

3 weeks ago

Halifax, Canada Compunnel Inc. Full time

Job Summary

We are seeking a Data QA / Validation Engineer (Contractor) to ensure data quality across Databricks/Spark/Delta Lake pipelines.

The role focuses on validating clean, accurate datasets that power downstream analytics, machine learning, and real-time decision-making systems.

You will design and implement automated data quality tests for ingestion and transformation layers (Bronze ? Silver ? Gold), enforce schema and data contracts, detect drift, validate feature pipelines, and monitor data anomalies.

Key Responsibilities

- Develop and execute automated data quality tests for ingestion and transformation layers.
- Enforce schema and data contracts across pipelines.
- Detect and address data drift and anomalies.
- Validate feature pipelines to ensure accuracy and reliability.
- Monitor data quality and maintain issue logs for critical pipelines.
- Deliver production-grade validation suites and provide QA sign-off for key data processes.

Required Qualifications

- 8+ years of experience in SQL.
- 5+ years of experience with Databricks and Spark.
- 3+ years of experience in data quality frameworks (e.g., Great Expectations, Deequ, Soda, dbt tests).
- 3+ years of experience using GenAI tools (Cursor, Windsurf, GitHub Copilot, Databricks Assistant) for test generation and validation workflows.
- 5+ years of experience delivering production-grade validation suites and QA sign-off for critical pipelines.

  • Big Data/Hadoop/Splunk Tester

    3 weeks ago

    Halifax, Canada Compunnel Inc. Full time

    Job Summary We are seeking a Data QA / Validation Engineer (Contractor) to ensure data quality across Databricks/Spark/Delta Lake pipelines. The role focuses on validating clean, accurate datasets that power downstream analytics, machine learning, and real-time decision-making systems. You will design and implement automated data quality tests for ingestion...

  • Big Data/Hadoop/Splunk Tester

    3 weeks ago

    Halifax, NS, Canada Compunnel Inc. Full time

    Job Summary We are seeking a Data QA / Validation Engineer (Contractor) to ensure data quality across Databricks/Spark/Delta Lake pipelines. The role focuses on validating clean, accurate datasets that power downstream analytics, machine learning, and real-time decision-making systems. You will design and implement automated data quality tests for ingestion...

  • Big Data/Hadoop/Splunk Tester

    3 weeks ago

    Halifax, NS, Canada Compunnel Inc. Full time

    Job Summary We are seeking a Data QA / Validation Engineer (Contractor) to ensure data quality across Databricks/Spark/Delta Lake pipelines. The role focuses on validating clean, accurate datasets that power downstream analytics, machine learning, and real-time decision-making systems. You will design and implement automated data quality tests for ingestion...

  • Big Data/Hadoop/Splunk Tester

    3 weeks ago

    Halifax, NS, Canada Compunnel Inc. Full time

    Job Summary We are seeking a Data QA / Validation Engineer (Contractor) to ensure data quality across Databricks/Spark/Delta Lake pipelines. The role focuses on validating clean, accurate datasets that power downstream analytics, machine learning, and real-time decision-making systems. You will design and implement automated data quality tests for ingestion...

  • Automation Tester

    4 days ago

    Halifax, Canada Compunnel, Inc. Full time

    The Automation Tester is responsible for isolating, reproducing, tracking defects, and verifying fixes within software products. This role involves developing test data and procedures to ensure that software products meet organizational standards and end-user requirements. The Automation Tester is highly familiar with the software development lifecycle...

  • Automation Tester

    4 days ago

    Halifax, Canada Compunnel, Inc. Full time

    The Automation Tester is responsible for isolating, reproducing, tracking defects, and verifying fixes within software products. This role involves developing test data and procedures to ensure that software products meet organizational standards and end-user requirements. The Automation Tester is highly familiar with the software development lifecycle...

  • Big Data Automation Lead

    2 weeks ago

    Halifax, Canada Royal Bank of Canada Full time

    **Job Summary** **WHAT IS THE OPPORTUNITY?** **WHAT WILL YOU DO?** - Lead the design, development, and implementation of automation solutions for big data pipelines and workflows. - Develop and maintain robust automation scripts using Python, Pandas, and PySpark. - Implement and manage test automation using Robot Framework and BDD frameworks. - Design and...

  • Etl Tester

    1 week ago

    Halifax, Canada NTT DATA Full time

    Job Description **Company Overview**: **Req ID**: 250152 We are currently seeking a ETL Tester to join our team in REMOTE, Nova Scotia (CA-NS), Canada (CA). The Tester isolates, reproduces, and tracks defects and verifies fixes. Develops test data and procedures for ensuring the software products meet organization standards and end-user requirements. Very...

  • Senior Big Data Developer

    4 weeks ago

    Halifax, Canada RBC Full time

    Join to apply for the Senior Big Data Developer role at RBC 1 day ago Be among the first 25 applicants Join to apply for the Senior Big Data Developer role at RBC What is the opportunity? Compliance is a team within Global Functions Technology that deals with trading and employee surveillance, privacy, and risk governance. This position is within the...

  • Senior Big Data Developer

    4 weeks ago

    Halifax, Canada RBC Full time

    Join to apply for the Senior Big Data Developer role at RBC 1 day ago Be among the first 25 applicants Join to apply for the Senior Big Data Developer role at RBC What is the opportunity? Compliance is a team within Global Functions Technology that deals with trading and employee surveillance, privacy, and risk governance. This position is within the...