Senior Big Data Engineer

4 days ago

Mississauga Ontario CW, Peel region, Canada Droisys Full time

Droisys is an innovation technology company focused on helping companies accelerate their digital initiatives from strategy and planning through execution. We leverage deep technical expertise, Agile methodologies, and data-driven intelligence to modernize systems of engagement and simplify human/tech interaction.


Amazing things happen when we work in environments where everyone feels a true sense of belonging and when candidates have the requisite skills and opportunities to succeed. At Droisys, we invest in our talent and support career growth, and we are always on the lookout for amazing talent who can contribute to our growth by delivering top results for our clients. Join us to challenge yourself and accomplish work that matters.


Droisys is seeking Senior Big Data Engineer job offering Onsite Work for a long-term job opportunity in Mississauga, ON area of Canada.


Here are the job details


Job Title Senior Big Data Engineer

Job Location Mississauga, ON [Onsite Work]

Duration Full Time Employment

Salary: $130 CAD per annum + benefits.

Client: CITI Bank through Altimetrik

Mandatory Skills: Python, Cloud experience, Kafka, Openshift ECS

Responsibilities

• Integral team member of our Data Engineering team responsible for design and development of Big data solutions Partner with domain experts, product managers, analyst, and data scientists to develop Big Data pipelines in Hadoop or Snowflake Responsible for delivering data as a service framework

• Responsible for moving all legacy workloads to cloud platform

• Work with data scientist to build Client pipelines using heterogeneous sources and provide engineering services for data science applications

• Ensure automation through CI/CD across platforms both in cloud and on-premises

• Ability to research and assess open source technologies and components to recommend and integrate into the design and implementation

• Be the technical expert and mentor other team members on Big Data and Cloud Tech stacks

• Define needs around maintainability, testability, performance, security, quality and usability for data platform

• Drive implementation, consistent patterns, reusable components, and coding standards for data engineering processes

• Convert SAS based pipelines into languages like PySpark, Scala to execute on Hadoop and non-Hadoop ecosystems

• Tune Big data applications on Hadoop and non-Hadoop platforms for optimal performance

• Evaluate new IT developments and evolving business requirements and recommend appropriate systems alternatives and/or enhancements to current systems by analyzing business processes, systems and industry standards.

• Supervise day-to-day staff management issues, including resource management, work allocation, mentoring/coaching and other duties and functions as assigned


Qualifications

• 8+ years of experience in hadoop/big data technologies.

• 3+ years of experience in spark.

• 2+ years experience in Snowflake

• 2+ year of experience working on Google or AWS cloud developing data solutions. Certifications preferred.

• Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required;

• Experience with containerization and related technologies (e.g. Docker, Kubernetes)

• Experience with all aspects of DevOps (source control, continuous integration, deployments, etc.)

• 1 year Hadoop administration experience preferred

• 1+ year of SAS experience preferred.

• Comprehensive knowledge of the principles of software engineering and data analytics

• Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)

• Knowledge of agile (scrum) development methodology is a plus

• Strong development/automation skills

• Proficient in programming in Java or Python with prior Apache Beam/Spark experience a plus.

• System level understanding - Data structures, algorithms, distributed storage & compute

• Can-do attitude on solving complex business problems, good interpersonal and teamwork skills


Education

• Bachelor’s degree/University degree or equivalent experience


Droisys is an equal opportunity employer. We do not discriminate based on race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law. Droisys believes in diversity, inclusion, and belonging, and we are committed to fostering a diverse work environment.

  • Senior Data Engineer

    4 days ago

    Mississauga, Ontario, C4W, Peel region, Canada Themesoft Inc. Full time

    Job Title: Sr. Data EngineerLocation- Mississauga, ON ( Locals Only)Salary - CAD 130K/annum+benefits Ideally 9+ years of experience Python is Mandatory.Mandatory Skills: Python, Cloud experience, kafka , Openshift ECSResponsibilities:⦁ Integral team member of our Data Engineering team responsible for design and development of Big data solutions Partner...

  • Big Data Developer

    4 days ago

    Mississauga, Ontario, C4W, Peel region, Canada Open Systems Technologies Full time

    Open Systems Tech is looking for a Big Data Developer for one of our financial services clients. This is a time sensitive opportunity, please apply ASAP.Job Title: Big Data Developer Location: Mississauga - 3 days - HybridTerm: 12 months contract plus extensionJob Description: • 10+ years of hands-on experience in big data development using Python and...

  • Data Engineer

    11 hours ago

    Mississauga, Ontario, C4W, Peel region, Canada Galactic Minds INC Full time

    Mandatory Skills: Python, Cloud experience, kafka , Openshift , ECS.Qualifications:8+ years of experience in hadoop/big data technologies.3+ years of experience in spark.2+ years’ experience in Snowflake2+ year of experience working on Google or AWS cloud developing data solutions. Certifications preferred.Hands-on experience with Python/Pyspark/Scala and...

  • Data Engineer

    11 hours ago

    Mississauga, Ontario, C4W, Peel region, Canada Smart IT Frame LLC Full time

    Must Have:SnowflakeSQLPythonADFPySparkData Warehouse Concepts Job Description :This position is for a Cloud Data engineer with a background in Python, Pyspark, SQL and data warehousing for enterprise level systems. Major Responsibilities:Build and optimize data pipelines for efficient data ingestion, transformation and loading from various sources while...

  • Data Engineer

    2 weeks ago

    Mississauga, Ontario, C4W, Peel region, Canada TechMatrix Inc Full time

    Job Title: Senior Data EngineerJob Description:We are seeking an experienced Senior Data Engineer with strong programming skills and a deep understanding of cloud technologies. The ideal candidate will have 9+ years of experience in designing, implementing, and optimizing data systems at scale.Required Skills:Strong programming expertise in Python.Experience...

  • Big Data Developer

    1 month ago

    Mississauga, Ontario, Peel region, Canada IT Connex Full time

    About the JobLooking for a Big Data developer for financial client with extensive experience with Pyspark to build/optimize large scale data processing systemQualifications6-8 years of experience in big data development using PySpark and Spark EcosystemExtensive experience with PySpark/PythonSolid understanding of Hadoop, Hive and HDFSProficiency in SQL...

  • Data Engineer

    4 days ago

    Mississauga, Ontario, C4W, Peel region, Canada Centrilogic Full time

    Data Engineer Purpose: Over 15 years, we have become a premier global provider of multi-cloud management, cloud-native application development solutions, and strategic end-to-end digital transformation services. Headquartered in Canada and with regional headquarters in the U.S. and the United Kingdom, Centrilogic delivers smart, streamlined solutions to...

  • Lead Data Engineer

    4 days ago

    Mississauga, Ontario, C4W, Peel region, Canada NexGen Tech Solutions Full time

    Position: Lead Data EngineerJob Type: Full Time Contract Location: Mississauga, ON (Hybrid)Must Have: Azure DevOps , ADF, SSIS, Databricks, SQL Must HaveAs our new Data Engineer, you will play a role in designing, developing, and maintaining our data architecture on the Azure platform. You can expect to collaborate with cross-functional teams, leverageyour...

  • Data Engineer

    2 weeks ago

    Mississauga, Ontario, C4W, Peel region, Canada Presto Recruiting Solutions Full time

    Job Description:We are seeking an experienced Data Engineer with a strong background in AWS (EMR & Athena), Python, PySpark and Apache Airflow to join our technology team. The ideal candidate will be responsible for developing, optimizing, and maintaining our Snowflake data warehouse environment to support financial data analysis and reporting needs. The...

  • Lead Data Engineer

    4 weeks ago

    Mississauga, Ontario, Peel region, Canada NexGen Tech Solutions Full time

    Position: Lead Data EngineerJob Type: Full Time Contract Location: Mississauga, ON (Hybrid)Must Have: Azure DevOps and azure data engineering, SSIS, Databricks Must HaveAs our new Data Engineer, you will play a role in designing, developing, and maintaining our data architecture on the Azure platform. You can expect to collaborate with cross-functional...

  • Piping Analysis Engineer

    4 days ago

    Mississauga, Ontario, C4W, Peel region, Canada OG7GES Indigenous Group Inc. Full time

    This position requires an individual who is self-motivated and capable of working independently or under the instruction from a senior technical engineer and develop his/her career and experience in nuclear field. The candidate shall be able to follow the instructions from a qualified professional engineer to complete the assigned mechanical related scopes...

  • Director, Engineering

    11 hours ago

    Mississauga, Ontario, C4W, Peel region, Canada AtkinsRéalis Full time

    AtkinsRéalis is one of Canada's largest private sector nuclear engineering groups. We have been providing a wide range of services to the nuclear industry in Canada and around the world since for more than 60 years. We are proud to be the steward of Canadian CANDU nuclear technology.The Engineering Director will play a pivotal role in the development...

  • Senior Java Software Engineer

    4 days ago

    Mississauga, Ontario, C4W, Peel region, Canada Synechron Full time

    We areAt Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron’s progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud & DevOps,...

  • Senior Project Engineer

    4 days ago

    Mississauga, Ontario, C4W, Peel region, Canada CHEP Full time

    Position PurposeDevelop and execute Non-Hire Stock Capital Engineering projects in support of CHEP North American Operations.Major/Key AccountabilitiesAssist in developing business cases for project sponsors.Develop project plans and successfully execute them in accordance with established schedule, cost and performance standards.Develop standards for CHEP...

  • Java Architect

    4 weeks ago

    Mississauga, Ontario, Peel region, Canada Themesoft Inc. Full time

    Position : Java Architect - Big DataEmp Type : Full-TimeLocation : Mississauga, ON (3 Days Onsite)Salary- 150K CAD Mandatory Skills- Java, Springboot, microservices, Kafka, Spark, AWS, Redhat OpenShift, Bigdata, Spark Job DescriptionSystem Design: Create and maintain efficient, reusable, and reliable Java code and big data solutions.Data Management: Design...

  • Production Support Engineer

    4 days ago

    Mississauga, Ontario, C4W, Peel region, Canada Orion Innovation Full time

    Job title: Senior [7+ years] Production Support Engineer Location: Mississauga, ONType: Hybrid [3 days on site]Job Description:Are you passionate about ensuring seamless Production operations and delivering top-notch application support?We are seeking a dedicated and skilled Senior [7+years]Production Support Engineer to join our dynamic team. In this role,...

  • CAD Engineer

    3 months ago

    Mississauga, Ontario, C4W, Peel region, Canada Synopsys Full time

    We are looking for an Engineer to join a growing, dynamic and multi-cultural Project Support Team who provides world-wide experienced design communities daily support on their R&D activities.We need people who are able to work in multi-functional teams, capable of understanding requirements, come up with creative solutions, are reliable in following existing...

  • Data Governance Specialist

    4 days ago

    Mississauga, Ontario, C4W, Peel region, Canada LTIMindtree Full time

    Job Title: Data Governance SpecialistWork Location: Mississauga,Ontario,(CA-ON)Job Description:ResponsibilitiesJob DescriptionWe are looking for a skilled Data Governance Specialist with handson experience in Data Lineage and Data Catalog as well as metadata management using industrystandard tools The ideal candidate will have a strong background in data...

  • ETL Developer

    4 days ago

    Mississauga, Ontario, C4W, Peel region, Canada Synechron Full time

    We areAt Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron’s progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud & DevOps,...

  • Data Governance Specialist

    4 days ago

    Mississauga, Ontario, C4W, Peel region, Canada LTIMindtree Full time

    Job Title: Data Governance Specialist Work LocationMississauga, Ontario CanadaJob Description:We are looking for a skilled Data Governance Specialist with handson experience in Data Lineage and Data Catalog as well as metadata management using industry standard tools The ideal candidate will have a strong background in data governance and be proficient in...