ETL Developer

ETL Developer (ETL Azure Data Factory, Databricks, Logic App and Function App, IBM Cognos and Microsoft PowerBI, SQL Server Stored Procedure, Oracle PL/SQL)

1 year contract (with possibility of renewal)

Toronto (hybrid)

Deliverables section:

Scope of Services and Deliverables - The Services and Deliverables to be provided by the Vendor will include the following:

· Participate in the review of project, work request and maintenance release artefacts - Use Case, System Requirement Specification, etc.

· Deliver detailed system design documents and any other supporting documentation conforming to Ministry standards.

· Design logical and physical data models using Sybase Power Designer.

· Produce transformation mapping document.

· Implement data pipelines using Azure Data Factory (ADF), stored procedure, and Informatica PowerCenter performing extraction, transformation, and loading activities.

· Implement solutions using Logic App and Function App.

· Generate structured JSON file using ADF/ SQL.

· Develop Azure CI/CD pipeline to automate ADF release.

· Develop complex Oracle PL/SQL program to fulfill project requirements.

· Implement complex data conversion, e.g., binary to character, EBCDIC to UTF.

· Implement complex data transformation(s) for derived and calculated values.

· Integrate data sets from diverse source systems.

· Implement ADF for initial data load and incremental load.

· Promote ADF and Informatica ETL/ELT through all Ministry environments including Development, Integration Testing, QA, UAT, Production.

· Resolve and troubleshoot ADF pipeline, stored procedure, and Informatica workflow technical problems.

· Monitor and analyze incident, provide timely resolution.

· Collaborate with IT Professionals throughout the Software Development Life Cycle.

· Document information from diverse business area stakeholders and subject matter experts.

· Optimize performance of ADF pipeline, Azure SQL and Synapse databases, and Informatica workflow.

· Monitor application functionality and performance on daily basis.

· Provide effective knowledge transfer to Ministry staff at all stages of this assignment.

SkillsExperience and Skill Set Requirements

Evaluation Table

ETL Development-Data Integration %

· Extensive experience in gathering requirements and business process knowledge in order to design correct and high-quality data transformation.

· Extensive experience in design, development, and implementation with Azure Data Factory, Databricks, including:

o Self-Hosted Integration Runtime

o Data movement, ETL pipelines

o Performance tuning on large data volumes

o Complex data transformations

o ADF Debugger to validate data transformation.

o Azure Roles

· Experience with Informatica, including:

o Workflow Manager

o Repository Manager

o Designer

o Workflow Monitor

· Experience with developing solutions using Logic App and Function App.

· Experience in rapid application development (RAD) methodologies.

· Experience in maintaining and improving existing ETL processes.

· Experience in investigating data to identify potential issues within ETL pipelines.

· Experience with Business Intelligence tools IBM Cognos and Microsoft PowerBI.

Azure Technologies %

· Experience with CI/CD (DevOps) pipelines and concepts, including:

o Azure Resource Management

o GitHub Repo

o Source code version control

o Branching, pull requests, build, release, multiple parallel development repositories merge.

· Experience with Azure DevOps, including:

o Azure Boards

o Azure Test Plans

o Azure Pipelines

o Azure Repos

o Azure Artefacts

Oracle PL/SQL Development %

· Extensive experience in designing and developing SQL Server Stored Procedure, Oracle PL/SQL programs.

Database Technologies %

· Experience with Oracle, Microsoft SQL Server database and tools.

· Experience with relational and hierarchical database technologies.

· Extensive knowledge of Azure SQL DB and Synapse platforms, including:

o Performance tuning on large data volume