ETL QA Engineer Databricks PySpark
4 days ago
Winnipeg, Manitoba, Canada
KUMARAN SYSTEMS
Full-time
Free with email or Google
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
Free with email or Google
By continuing, you agree to our Terms & Privacy Policy.
Validate enterprise data pipelines with Databricks and PySpark as an ETL QA Engineer at Kumaran Systems. Apply AI-assisted testing expertise.
This ETL QA Engineer validates large-scale data pipelines and ensures data quality. The work centers on Databricks, PySpark, ETL/ELT concepts, and data warehousing. You will use GitHub, GitHub Copilot, and GitHub Actions/Runners for CI/CD pipelines. The position also includes prompt engineering, AI agents, and AI-powered knowledge bases.
Key Responsibilities:
Validate large-scale data pipelines across enterprise data environments Perform data quality checks and reconciliation techniques Use Databricks notebooks, jobs, clusters, Delta Lake, and SQL Improve testing efficiency and coverage with AI-assisted tools Build and validate AI-powered knowledge bases Requirements:
Strong hands-on Databricks experience across core platform capabilities Expert PySpark skills in transformations, actions, windows, and optimization Understanding of ETL/ELT, warehousing, and dimensional data modeling Experience with GitHub, Copilot, and Actions/Runners Experience crafting prompts for coding, testing, and documentation Bring Databricks, PySpark, data validation, and AI-assisted QA skills to Kumaran Systems’ transformation engineering practice.
Validate large-scale data pipelines across enterprise data environments Perform data quality checks and reconciliation techniques Use Databricks notebooks, jobs, clusters, Delta Lake, and SQL Improve testing efficiency and coverage with AI-assisted tools Build and validate AI-powered knowledge bases Requirements:
Strong hands-on Databricks experience across core platform capabilities Expert PySpark skills in transformations, actions, windows, and optimization Understanding of ETL/ELT, warehousing, and dimensional data modeling Experience with GitHub, Copilot, and Actions/Runners Experience crafting prompts for coding, testing, and documentation Bring Databricks, PySpark, data validation, and AI-assisted QA skills to Kumaran Systems’ transformation engineering practice.