Current jobs related to ETL Data Engineer with Pyspark - Halifax - VLink Inc

  • Etl Tester

    5 months ago

    Halifax, Canada NTT DATA Full time

    Job Description **Company Overview**: **Req ID**: 250152 We are currently seeking a ETL Tester to join our team in REMOTE, Nova Scotia (CA-NS), Canada (CA). The Tester isolates, reproduces, and tracks defects and verifies fixes. Develops test data and procedures for ensuring the software products meet organization standards and end-user requirements. Very...

  • Senior Data Engineer

    1 month ago

    Halifax, Nova Scotia, Canada Smart TechLink Solutions Inc. Full time

    Job SummaryWe are seeking a highly skilled Data Engineer to join our team at Smart TechLink Solutions Inc. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Key ResponsibilitiesDevelop and implement data processing solutions using PySpark and Python, with a strong...

  • Senior Data Engineer

    1 month ago

    Halifax, Nova Scotia, Canada Smart TechLink Solutions Inc. Full time

    Job SummaryWe are seeking a highly skilled Data Engineer to join our team at Smart TechLink Solutions Inc. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Key ResponsibilitiesDevelop and implement data processing solutions using PySpark and Python, with a strong...

  • Data Engineer

    4 weeks ago

    Halifax, Nova Scotia, Canada Myticas Consulting Full time

    Job Title: Programmeur Pipeline DataWe are seeking experienced professionals to join our team at Myticas Consulting in the financial sector.Key Responsibilities:Utilize R and Python for programming tasks.Perform data engineering using ETL, SQL, and NoSQL, with Apache Hadoop.Work within Agile environments (Kanban and Scrum).Develop user interfaces with...

  • Senior Data Engineer

    4 weeks ago

    Halifax, Nova Scotia, Canada Smart TechLink Solutions Inc. Full time

    Key ResponsibilitiesDevelop and deliver complex software solutions using PySpark and Python, leveraging 8+ years of experience in engineering solutions.Design and implement robust SQL development strategies to drive business growth.Collaborate with cross-functional teams to create cohesive technology solutions using microservices architecture.Drive...

  • Senior Data Architect

    5 days ago

    Halifax, Nova Scotia, Canada Smart TechLink Solutions Inc. Full time

    As a key member of our team at Smart TechLink Solutions Inc., the Senior Data Engineer will be responsible for developing and implementing cloud-based data solutions using PySpark and Python. With a strong background in SQL development and experience working in cloud environments, this individual will be able to design and deliver cohesive technology...

  • Data Pipeline Engineer

    4 weeks ago

    Halifax, Nova Scotia, Canada Myticas Consulting Full time

    Job Title: Programmeur Pipeline DataWe are seeking two experienced professionals to join our team at Myticas Consulting in the financial sector.Key Responsibilities:Utilize R and Python for programming tasks, including data engineering and ETL processes.Work within Agile environments, such as Kanban and Scrum, to deliver high-quality results.Develop user...

  • Data Pipeline Engineer

    4 weeks ago

    Halifax, Nova Scotia, Canada Myticas Consulting Full time

    Job Title: Programmeur Pipeline DataWe are seeking two experienced professionals to join our team at Myticas Consulting in the financial sector.Key Responsibilities:Utilize R and Python for programming tasks, including data engineering and ETL processes.Work within Agile environments, such as Kanban and Scrum, to deliver high-quality results.Develop user...

  • Data Engineering

    4 months ago

    Halifax, Canada Avanade Full time

    Introduction: Join our Data Engineering team and create cutting-edge solutions within Avanade’s Business Intelligence practice. We’re seeking ambitious technologists passionate about Microsoft cloud technology. Early access to the newest Microsoft technologies ensures you stay ahead of the curve.Come join us As a member of the Data Engineering team,...

  • Data Engineer

    4 weeks ago

    Halifax, Nova Scotia, Canada Randstad Full time

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Randstad. As a Data Engineer, you will play a crucial role in designing, building, and optimizing our data integration platform using Microsoft Azure and Databricks.Key Responsibilities:Design and implement data pipelines using Azure Data Factory and DatabricksDevelop...

  • BI System Administrator

    5 months ago

    Halifax, Canada NTT DATA Full time

    **Job Description**: We are currently seeking a **BI System Administrator** to join our team Hybrid in Halifax, Nova Scotia (CA-NS), Canada (CA). - Administer all requests, analyze issues, and provide efficient resolutions. - Develop technical documentation for installation and configuration. - Install and manage ETL platforms like Informatica, Matillion,...

  • Senior Data Acquisition Engineer

    4 weeks ago

    Halifax, Nova Scotia, Canada LifeRaft Full time

    Senior Data Acquisition EngineerWe are seeking a highly skilled Senior Data Acquisition Engineer to join our team at LifeRaft. As a key member of our engineering team, you will play a critical role in building and enhancing our data integration capabilities to support our threat intelligence platform, Navigator.Key ResponsibilitiesDesign and develop new data...

  • Senior Data Acquisition Engineer

    3 weeks ago

    Halifax, Nova Scotia, Canada LifeRaft Full time

    We are seeking a highly skilled Senior Data Acquisition Engineer to join our team at LifeRaft. As a key member of our engineering leadership, you will play a critical role in building on our sophisticated open source data collection capabilities.The ideal candidate will have at least 5 years of experience working with Python or similar programming languages,...

  • Senior Azure Data Engineering Specialist

    4 weeks ago

    Halifax, Nova Scotia, Canada Avanade Full time

    Unlock Your Potential in Data EngineeringAre you a tech-savvy individual passionate about Microsoft cloud technology? Do you want to create cutting-edge solutions that drive business growth? We're seeking ambitious technologists to join our Data Engineering team at Avanade, a global leader in digital innovation.About the RoleAs a Senior Azure Data...

  • Senior Azure Data Engineering

    4 months ago

    Halifax, Canada Avanade Full time

    Introduction: Join our Data Engineering team and create cutting-edge solutions within Avanade’s Business Intelligence practice. We’re seeking ambitious technologists passionate about Microsoft cloud technology. Early access to the newest Microsoft technologies ensures you stay ahead of the curve.Come join us As a member of the Data Engineering team,...

  • Senior Software Architect

    2 days ago

    Halifax, Nova Scotia, Canada Smart TechLink Solutions Inc. Full time

    About Smart TechLink Solutions Inc.We are a forward-thinking company that provides innovative solutions to complex problems. Our mission is to deliver cutting-edge technology that transforms the way businesses operate.Job SummaryThis Senior Data Engineer position plays a critical role in driving our technological advancements. The ideal candidate will have...

  • Lead Data Engineer, GFT

    1 month ago

    Halifax, Nova Scotia, Canada Royal Bank of Canada Full time

    Job DescriptionWe are seeking a highly skilled Lead Data Engineer to join our team at Royal Bank of Canada. As a key member of our Technology and Operations division, you will play a crucial role in designing and developing a cutting-edge data platform that supports our climate goals.What is the Opportunity?As a Lead Data Engineer, you will be responsible...

  • Lead Data Engineer, GFT

    1 month ago

    Halifax, Nova Scotia, Canada Royal Bank of Canada Full time

    Job DescriptionWe are seeking a highly skilled Lead Data Engineer to join our team at Royal Bank of Canada. As a key member of our Technology and Operations division, you will play a crucial role in designing and developing a cutting-edge data platform that supports our climate goals.What is the Opportunity?As a Lead Data Engineer, you will be responsible...

  • Senior Data Engineer

    5 months ago

    Halifax Regional Municipality, Canada Smart TechLink Solutions Inc. Full time

    Responsibilities8+ Years in engineering solutions using PySpark and PythonStrong SQL development ExperienceExperienced in working in a Cloud environmentAble to create cohesive technology solutions using microservicesour system and implementing new featuresCollaborating with other team members in a Feature Team perspectiveDeveloping and delivering complex...

  • DevOps Engineer

    4 months ago

    Halifax, Canada NTT DATA Full time

    Job Description NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a DevOps Engineer (Groovy Scripting on Jenkins) (Onsite) to join our team in Halifax, Nova Scotia (CA-NS),...

ETL Data Engineer with Pyspark

3 months ago

Halifax, Canada VLink Inc Full time

Job Title: Data Engineer

Location: Halifax , NS

Employment Type: (FTE )

Duration: FTE


About VLink: Started in 2006 and headquartered in Connecticut, VLink is one of the fastest growing digital technology services and consulting companies. Since its inception, our innovative team members have been solving the most complex business, and IT challenges of our global clients.


What is the role:

• Lead a group of developers building data pipelines using big data technologies (Spark, Flink, Databricks) on medium to large scale datasets for batch as well as real time streaming patterns

• Maintain existing data pipelines to support on premise clients and build the next generation ETL processes leveraging Databricks in Cloud

• Work in a creative & collaborative environment driven by agile methodologies with focus on CI/CD, Application Resiliency Standards

• Experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the team

• Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in around data ingestion & transformation onprem and in cloud

• Collaborate with product owners, business stakeholders to understand requirements, prioritize tasks and work with the scrum master/development team to meet program timelines

• Mentor team on technical aspects, participate in design/code reviews to ensure code is designed well, elegantly coded, and effectively tuned for performance.


Qualifications

• Bachelor's Degree

• 8+ years of hands-on experience in solution, design and development of ETL applications using Spark (Pyspark preferred ) within Big Data space.

• 5+ years' experience managing a team of developers focused on data pipelines, providing oversight on implementation, performing code reviews and managing Production rollouts

• 2+ years of experience in streaming applications using Spark Streaming / Flink .

• 2+ year's experience building and working with rest api's using Java or python

• Knoledge of best practices , architectural patterns, limitations within Big Data ecosystem

• 2+ years of experience working with Agile Engineering practices.

• Knowledge and experience in non-functional requirements like Security, Performance, Vulnerabilities, etc.

Nice to Have:

• Good to have some experience / knowledge on cloud-based solutions on Azure, Databricks