Senior Data Pipeline Engineer

3 weeks ago


Vancouver BC, Canada Celestial Systems Full time

Celestial Systems, a name synonymous with Enterprise Softwaredevelopment since 2001. By designingreliable software,delivering quality services, and developing valuable partnerships, we are on a journey of continuousinnovation. Ourenterprise-level experience in engineering enables us to offer critical development services fororganizations to besuccessful in the digital ages. Moreover, at Celestial, we focus on adhering to the complete softwaredevelopmentlifecycle process. Our experts are curious to work with the latest technologies and are extremelypassionate aboutsoftware development. Whether its Frontend Development, Backend, DevOps, QA and Testing or CloudHosting, ourtailor-made technology solutions, provide answers to the problems.We are ISO certified enterprise-grade software company.

Roles and Responsibilities

Celestial is rapidly expanding or Enterprise Data Services team to offer our customers a single point for all requirements including, Data Engineering, Data Analytics, Data Protection and Data Ethics/Governance.

We are looking for Senior Data Pipeline Engineers (multiple positions) for ongoing projects, within the Data Services and ML/AI Group. If you are motivated and willing to work in a startup mode on exciting NextGen projects do read on.

Requirements

• 4+ years of experience in big data engineering, with a focus on Spark and SQL.

• 4+ Years Experience in developing Large-scale ETL Pipelines using AWS services not limited to IAM, Glue, Lambda, S3, Athena, SNS, SQS, DynamoDB, RDS, EMR, ECS, Route 53 and Redshift.

• Highly skilled in Python, PostgreSQL/RDS/Redshift, and writing complex SQL queries and analysis of data correlations.

• Design, construct, and manage Data Lake environments including Data ingestion, staging, Data quality monitoring, and business modeling, Data Schema Normalization, etc.

• Develop, construct, secure, test and maintain architectures (databases and large-scale processing systems)

• Design, develop and implement technical solutions in SQL environment (databases, SQL, data marts, integrations)

• Experience in tools like Jenkins, GIT and/or AWS CodePipeline/CodeBuild/CodeDeploy

• Experience building data-driven unit test suites for data platforms and modeling highly dimensional datasets

• Strong functional understanding of Sagemaker, Jupyter and other Data Science tools on AWS

PREFERRED SKILLS

Experience designing and building big data architectures using tools such as Hadoop, Hive, or Kafka.

ADDITIONAL NOTES

If you are applying for Part-time/Contract role please email your resume with contact details, and "Hourly Consulting Rate Expectations".

Apply for this Job

Drop your Resume *


Attach or Drop Your resume

#J-18808-Ljbffr

  • Vancouver, BC, Canada Celestial Systems Full time

    Celestial Systems, a name synonymous with Enterprise Softwaredevelopment since 2001. By designingreliable software,delivering quality services, and developing valuable partnerships, we are on a journey of continuousinnovation. Ourenterprise-level experience in engineering enables us to offer critical development services fororganizations to besuccessful in...

  • Senior Data Engineer

    2 weeks ago


    Vancouver, BC, Canada Zortechsolutions Full time

    Role: Data Engineer-SQL, Python, ETL Duration: Full Time We are looking for Senior Data Engineering Candidates based out of the Canada. Expertise in SQL, Python and ETL flows is a must have. Medium expertise/working knowledge in Scala is also needed, as our data pipelines are written in both Scala and Hive/Spark SQL, with new pipelines being written in...


  • Vancouver, BC, Canada railtown.ai Full time

    As a Senior Machine Learning Engineer, you will work with our growing AI team in developing AI and machine learning (ML) solutions for problems in the software development domain. This position will provide a mix of research and engineering problems as you will be involved in the development of the end-to-end machine learning pipeline, ranging from data...

  • Senior Data Engineer

    3 weeks ago


    Vancouver, BC, Canada Knewin Full time

    We are at the forefront of geolocation, cybersecurity, and anti-fraud innovation, developing and delivering cutting-edge technologies to help ensure regulatory compliance, combat bad online actors, alleviate user friction, and protect businesses from fraud. Achieving significant business and revenue growth over the past three years and dubbed a tech...


  • Surrey, BC, Canada Fraser Health Full time

    Salary range The salary range for this position is CAD $45.46 - $65.35 / hour Why Fraser Health? Fraser Health is responsible for the delivery of hospital and community-based health services to over 1.9 million people in 20 diverse communities from Burnaby to Fraser Canyon on the traditional territories of the Coast Salish and Nlaka’pamux Nations. Our...

  • Data Engineer

    3 weeks ago


    Vancouver, BC, Canada Fraser Health Full time

    Salary range The salary range for this position is CAD $37.01 - $53.20 / hour Why Fraser Health? Fraser Health is responsible for the delivery of hospital and community-based health services from Burnaby to Fraser Canyon on the traditional, ancestral and unceded territories of the Coast Salish and Nlaka’pamux Nations and is home to six Métis Chartered...


  • Vancouver, Canada Fraser Health Full time

    Salary range The salary range for this position is CAD $45.46 - $65.35 / hour Why Fraser Health? Fraser Health is responsible for the delivery of hospital and community-based health services to over 1.9 million people in 20 diverse communities from Burnaby to Fraser Canyon on the traditional territories of the Coast Salish and Nlaka'pamux Nations....


  • Vancouver, Canada Amplitude Full time

    Amplitude is a leading digital analytics platform that helps companies unlock the power of their products. More than 2,300 customers, including Atlassian, Jersey Mike’s, NBCUniversal, Shopify, and Under Armour, rely on Amplitude to gain self-service visibility into the entire customer journey. Amplitude guides companies every step of the way as they...


  • Vancouver, BC, Canada Zortechsolutions Full time

    Role: Data Engineer-SQL, Python, ETL Location: Vancouver/CANADA Duration: Full Time Job Description Key Skills required: We are looking for Senior Data Engineering Candidates based out of the Canada. Expertise in SQL, Python and ETL flows is a must have. Medium expertise/working knowledge in Scala is also needed, as our data pipelines are written in...

  • Lead Engineer

    1 week ago


    Vancouver, BC, Canada Zonda Full time

    Job Title: Lead Engineer (Data) Location: Vancouver, BC / GVRD Reports To: Director of Data Engineering Job Summary: We are seeking a highly skilled and motivated individual to join our team as a Lead Engineer in Data Operation. As a Lead Engineer in Data Operations, you will be responsible for driving the development of modern data pipelines and...

  • Data Engineer

    3 weeks ago


    Vancouver, BC, Canada MineSense Full time

    Metals are necessary for our world, but mining historically faced issues with waste, sustainability, and inefficiencies. Our solutions reduce mining waste and environmental impacts while increasing profits and efficiencies. Backed by the best in venture capital and government funds, and recognized by tech industry and sustainability programs, MineSense is...

  • Project Manager

    3 weeks ago


    Vancouver, BC, Canada ICES: Data, Discovery, Better Health Full time

    OVERVIEW The Project Manager is responsible for completing, planning, coordinating and managing all project related for Engineering or EPC projects. The incumbent will provide project management to promote the successful execution of projects with a focus on cost efficiency, constructability and schedule. The incumbent will provide project management to...


  • Vancouver, BC, Canada Fraser Health Full time

    The salary range for this position is CAD $37.Why Fraser Health? Fraser Health is responsible for the delivery of hospital and community-based health services from Burnaby to Fraser Canyon on the traditional, ancestral and unceded territories of the Coast Salish and Nlaka’pamux Nations and is home to six Métis Chartered Communities. Our team of 45,000...


  • Vancouver, BC, Canada Klue Labs Full time

    We are looking for Senior Machine Learning Engineers to work with our team to deliver high quality products in the most efficient way. We build machine learning services and data pipelines to automatically extract insights about competitors from both public and internal data sources. Every day, our services process millions of data points, including...


  • Vancouver, BC, Canada Visier Inc. Full time

    Our co-op experience is unique and designed to prepare you for professional success as you work on real, impactful work from the beginning. Our ultimate goal is to give you the mentorship, training, and work experience you need to start your career. A number of our students return to us after they graduate and start developing their professional career at...


  • Vancouver, BC, Canada Vancouver Airport Authority Full time

    YVR is made up of a team of diverse people who are working collaboratively to Connect BC Proudly to the World. Safety is at the core of everything we do; we’re innovative, fun, and we invest in our people. We’re a BC Top Employer for 14 years standing, with high engagement scores, an abundance of learning and development opportunities, and a holistic...

  • Data Engineer II

    3 weeks ago


    Vancouver, BC, Canada Brooksource Full time

    Data Engineer II Vancouver, Canada – hybrid (3 days a week onsite) Long Term Contract $45 - $60 /hr. Who We Are: Our client is an innovative performance apparel company for yoga, running, training, and other athletic pursuits. Setting the bar in technical fabrics and functional design, we create transformational products and experiences that support...

  • Software Engineer

    2 weeks ago


    Vancouver, BC, Canada UrbanLogiq Full time

    Join UrbanLogiq - Where Data Empowers Communities! At UrbanLogiq, we're not just developing a platform; we're revolutionizing how government agencies leverage data to enhance community life. Since our inception in 2016, we've been at the forefront of the govtech industry's innovation, growing rapidly and making waves globally. Our focus?...


  • Vancouver, BC, Canada Visier, Inc Full time

    Visier Co-op Opportunity Our innovative technology transforms the way that organisations make decisions, allowing them to elevate their employees and drive better business outcomes. Embarking on an exciting new chapter in our growth story, we are looking for talented individuals who can help both Visier and our customers grow, evolve, and win! Our co-op...

  • Data Engineer

    3 weeks ago


    Vancouver, BC, Canada Sandman Hotel Group Full time

    Tuesday, February 20, 2024 Proud to be 100% Canadian-owned,Northland Properties' are recognized as one of the most trusted names in hotels, restaurants, resorts, sports, construction, and asset management. Our well-known and loved brands have been bringing people together to celebrate unforgettable experiences across Canada, the US, Ireland, and the UK for...