Data Engineer

4 weeks ago

New Canada NS Avanciers Inc. Full time

Avanciers is seeking a highly skilled Data Engineer - TDM, IDMC to join our team for an exciting Fulltime role with one of our esteemed Fortune 500 clients, based in Canada. This is a REMOTE role. Position: Data Engineer - TDM, IDMC Location: [CANADA (REMOTE) Job Description: The Test Data Engineer is a critical role responsible for establishing, implementing, and governing the Test Data Management (TDM) strategy. This position ensures that high-quality, secure, and compliant test data is reliably available to support rigorous software testing across all non-production environments. The role requires a strong blend of data engineering, security (masking), and automation expertise, particularly within the Azure cloud ecosystem. Core Responsibilities TDM Strategy & Governance: Define, implement, and lead the organization's TDM strategy, methodology, and governance, establishing the TDM function as a core service. Test Data Supply Chain: Design and maintain the end-to-end data flow (supply chain) from production to non-production, guaranteeing data quality, volume, and referential integrity. Data Security & Masking: Mandate the use of and execute data masking/protection techniques (using TDM tools) to secure sensitive customer information (PII) for compliance in lower environments. Data Sourcing & Preparation: Utilize complex SQL queries to perform data identification, analysis, and extraction across relational (SQL Server, Oracle, DB2) and NoSQL sources for specific testing scenarios. TDM Tooling & Automation: Implement TDM functions using tools like Informatica IDMC, CA, or Delphix for sub-setting, synthetic data generation, and golden copy creation. Automate test data refresh and provisioning, integrating TDM services into CI/CD pipelines and Test Automation Frameworks. Cloud Data Engineering: Design and build data pipelines using Azure Data Factory (ADF), PySpark, and Databricks Notebooks to manage and transform test data in the Azure cloud environment. Design & Consultation: Analyze business requirements, conduct feasibility studies (POCs), and create high-level design documents for optimal test data solutions. Minimum Positions Qualifications/Education Requirements: 8+ years of software product development, testing and delivery experience. 5+ years of TDM/SQL experience. 3+ years of Informatica Power Center, IDQ, IICS/IDMC experience. 2+ years of experience in some of the Azure Cloud tools: ADF, Pyspark, databricks notebooks 3+ Experience with Python and libraries like Panads, Numpy & Matplotlib. Familiar with Latest Cloud Technologies. Mandate Key Skills: Expertise in TDM Strategy and Implementation: including defining and driving TDM strategy, governance, and delivery model as a core function. Strong proficiency in SQL for data identification, extraction, transformation, and analysis: SQL, NonSQL, Relational Database. Experience with TDM Tools & Automation: Hands-on experience with TDM tools like Informatica IDMC, CA, Delphix Experience with Azure Cloud tools: Azure Data Factory, PySpark, Databricks Notebooks. Experience with test management tools like: HP ALM, Jira, QMetry

  • (Cloud) Data Engineer

    4 weeks ago

    New Canada, NS Realist AI Full time

    Role: Cloud Data Engineer SUBCONTRACTOR Realist AI, an industry leader in data and cloud solutions, is seeking data engineers to join our growing network of subcontractors. We are looking for professionals with a passion for building scalable, reliable data pipelines to support cutting-edge Cloud applications and to support the many data offerings Realist AI...

  • Data Engineer

    1 week ago

    New Canada, NS Mastech Digital Full time

    Role Details: This job is for someone who makes sure data is flowing smoothly, accurately, and without problems across the company. You will: Watch and monitor data systems to catch issues early Investigate and fix data problems when something goes wrong Work closely with the Lead Engineer and partners to solve technical challenges Make sure the company...

  • Data Engineer

    4 weeks ago

    New Canada, NS TechDoQuest Full time

    Assessment & Responsibilities: Experience in Data Connection, extraction from different systems including both cloud based and On Premise systems (e.g. Azure, Oracle, SAP, BMC, SNOW, Salesforce etc.) and transformation & loading into the process mining tool. Support validation of data (counts, values between source systems and process mining tool)....

  • Senior Data Engineer

    4 weeks ago

    New Canada, NS Quarry Consulting Full time

    Role: Senior Data Engineer Location: Remote Contract: 1 year Requirements: Programming experience in Spark using modern languages such as Python, Scala Experience working with modern data architectures like Azure Data Lake Storage, Azure Databricks, Azure Synapse (formerly SQL Data Warehouse) and Delta Lakes Experience leading Data Engineering principles...

  • Data Engineer

    4 weeks ago

    New Canada, NS Open Systems Technologies Full time

    We provide: • A robust career development path offering numerous opportunities for growth, learning and advancement. • A supportive, learning-oriented environment in collaboration with development within fast-feedback agile delivery squads • Collaborative work within cross-functional squads, following agile practices and utilizing both cloud and...

  • Data Quality Engineer

    4 weeks ago

    New Canada, NS Open Systems Technologies Full time

    Job Responsibilities • Collaborate with cross divisional project managers, technology product owners, and firm data office engineering teams to ensure alignment on goals and deliverables for the global regulatory program delivering data quality reporting on critical reporting • Conducts a communication style that effectively "bridges the gaps" between...

  • Azure Data Engineer

    3 weeks ago

    New Canada, NS TALENDICA Full time

    Position: Senior/Lead Data Engineer Location: New Brunswick, CA (Remote) Duration: 6+ month Responsibilities Design, develop, and maintain data pipelines and orchestration workflows using Azure Synapse and Microsoft Fabric. Build, enhance, and support ETL processes for both on-premise data stores and enterprise data warehouses. Integrate, transform, and...

  • Microsoft Azure Data Engineer

    3 weeks ago

    New Canada, NS TALENDICA Full time

    Position: Senior/Lead Data Engineer Location: New Brunswick, CA (Remote) Duration: 6+ month Responsibilities Design, develop, and maintain data pipelines and orchestration workflows using Azure Synapse and Microsoft Fabric. Build, enhance, and support ETL processes for both on-premise data stores and enterprise data warehouses. Integrate,...

  • Data Centre network Design Engineer

    3 weeks ago

    New Canada, NS Atlantis IT Group Full time

    Required Skills: - Senior in Network design engineering. More than 12-15 years' experience - Certifications: CCIE or equivalent -. Ethernet technologies: SPT, 802.1Q, VPC, Multilayer Switching; Leaf-Spine IP Fabric - IP Routing: RIP, OSPF, BGP, MPBGP, TCP/IP - Multicast routing: PIM, IGMP, MSDP. VxLAN - Platform knowledge on Cisco Switching & Routing...

  • Data Engineer

    1 week ago

    New Canada Mastech Digital Full time

    Role Details:This job is for someone who makes sure data is flowing smoothly, accurately, and without problems across the company.You will:Watch and monitor data systems to catch issues earlyInvestigate and fix data problems when something goes wrongWork closely with the Lead Engineer and partners to solve technical challengesMake sure the company meets...