Big Data Developer

3 weeks ago

Greater Vancouver Metropolitan Area, Canada Capgemini Full time

The Senior Data Developer Lead- Digital Analytics is joining our growing hybrid team in Vancouver, not looking to relocate. We are interested in your data implementation experience.

Must Have Skills:

7 to 10 years working in ETL, Databricks, Python, Spark, SQL Databases with data implementation experience.

Responsibilities:

  • Strong proficiency in Python (especially data packages like pandas, Numpy, etc.) and SQL for analytics, database development, and data modelling
  • Professional experience with data ingestion, ETL, and ELT for structured and unstructured data.
  • Experience with DevOps and CICD for data.
  • Python - Strong proficiency, unit testing expertise and good knowledge of packages for data, like Pandas, SQL Alchemy, and Alembic.
  • SQL - strong proficiency in DDL and DML, including window function (ex.: lag), CTEs, sub-queries, joins, optimization, and performance profiling. Know how SQL behaves in different platforms, like Spark, PostgreSQL,
  • Spark - Pyspark, Spark SQL, Batch and Streaming processing, partitioning/liquid clustering, delta tables, parquet
  • Databricks - Workflows/Jobs, Clusters, SQL Warehouse, Unity Catalog, Performance profiling, log analysis.
  • Azure EventHub or a similar streaming solution understand how to best consume data from streaming for aggregations and parallelization/scaling.
  • PostgreSQL - Queries, indexes (different types of indexes), performance profiling, JSON columns.
  • Data Modeling: Dimensional Modeling (and experience with this model being used in BI tools), Normal Forms.
  • Docker containerization
  • DBT - models, seeds, multiple environments, parameters, macros, unit tests, data tests, incremental materialization, snapshots
  • Infra as code/ CICD tools: Kubernetes, Argo, Cross Plane, Terraform or similar.
  • Migrations - versioning for database, for example, Alembic (preferred), Flyway, Liquibase
  • SQL and NoSQL databases and selecting the best fit for different use cases.
  • Logging and being able to query logs using KQL(Azure) or similar.
  • Support, maintain, optimize, and create ETL/ELT pipelines, both batch and streaming, in Databricks (Pyspark, Databricks SQL), Python, SQL and/or DBT.
  • Proficient in Dimensional Modeling and Database normalization (Normal Form)

Life at Capgemini

Capgemini supports all aspects of your well-being throughout the changing stages of your life and career. For eligible employees, we offer:

  • Collaborating with teams of creative, fun, and driven colleagues
  • Flexible work options enabling time and location-based flexibility
  • Company-provided home office equipment
  • Virtual collaboration and productivity tools to enable hybrid teams
  • Comprehensive benefits program (Health, Welfare, Retirement and Paid time off)
  • Other perks and wellness benefits like discount programs, and gym/studio access.
  • Paid Parental Leave and coaching, baby welcome gift, and family care/illness days
  • Back-up childcare/elder care, childcare discounts, and subsidized virtual tutoring
  • Tuition assistance and weekly hot skill development opportunities
  • Experiential, high-impact learning series events
  • Access to mental health resources and mindfulness programs
  • Access to join Capgemini Employee Resource Groups around communities of interest

About Capgemini

Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, cloud and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of €22.5 billion

  • Big Data Developer for Amazon Promotions

    2 weeks ago

    Vancouver, British Columbia, Canada Amazon Full time

    We are seeking a big data developer to join our team of experts in Amazon promotions. As a big data developer, you will play a critical role in designing and implementing big data architectures that enable us to analyze and optimize our promotional strategies. Your expertise in Hadoop, Spark, NoSQL databases, and data engineering will enable us to build...

  • Cloud-Based Big Data Developer

    4 days ago

    Vancouver, British Columbia, Canada Wipro Technologies Full time

    About This OpportunityWe are seeking a highly skilled Big Data/Analytics professional to join our team as a Developer - L3.You will be responsible for designing and developing batch and streaming solutions using cutting-edge big data technologies.To succeed in this position, you should have hands-on experience with big data technologies, excellent...

  • Big Data Professional

    1 week ago

    Vancouver, British Columbia, Canada SAP SE Full time

    SAP is seeking a highly skilled Big Data Professional to join our team at SAP SE. As a key member of our infrastructure team, you will play a crucial role in the design, development, and deployment of our cloud-based analytics platform.Responsibilities include collaborating with cross-functional teams to design and implement new features, working closely...

  • Big Data Platform Specialist

    2 days ago

    Vancouver, British Columbia, Canada Autodesk, Inc. Full time

    Responsibilities:As a Big Data Platform Specialist, you will be responsible for designing and developing API-driven platforms with thorough documentation. You will also automate and administer big data applications and services on cloud, interface with data engineers, data scientists, product managers, and all data stakeholders to understand their needs and...

  • Big Data Analytics Professional

    4 days ago

    Vancouver, British Columbia, Canada Wipro Technologies Full time

    Job DescriptionThis dynamic role offers a unique opportunity to work on real-time or near real-time data ingestion using various technologies. As a key member of the development process, you will collaborate closely with architects to ensure the design of secure and scalable solutions.Your primary responsibilities will involve:Designing and developing batch...

  • Data Architect for Big Data Solutions

    1 week ago

    Vancouver, British Columbia, Canada Amazon Full time

    The Business Data Technology team at Amazon is responsible for developing innovative big data solutions that enable our enterprise data sets, the Amazon data lake/data marketplace, and our core infrastructure supporting data analytics and machine learning.About UsWe are a group of talented software engineers who work together to design and architect scalable...

  • Big Data Technologist

    4 days ago

    Vancouver, British Columbia, Canada Rivian and Volkswagen Group Technologies Full time

    Rivian and Volkswagen Group Technologies Job DescriptionSkill Requirements:Pyspark Pipeline Development:Databricks Platform Ownership:Ci/Cd & Infrastructure Management:Performance Optimization:Troubleshooting & Issue Resolution:Collaborative Development:Staying Ahead of the Curve:QualificationsBachelor's or Master's degree in Computer Science, Data Science,...

  • Big Data Technology Leader

    2 weeks ago

    Vancouver, British Columbia, Canada Amazon Full time

    About the RoleAmazon is committed to a diverse and inclusive workplace. We are an equal opportunity employer and do not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, disability, age, or other legally protected status.We are seeking talented and experienced software engineers to join our group. In this role,...

  • Big Data Production Specialist

    4 days ago

    Vancouver, British Columbia, Canada Apex Systems Full time

    Apex Systems is a world-class IT services company that serves thousands of clients across the globe. We are currently seeking a Big Data Production Specialist to join our team.About the RoleThis role involves working closely with our client's database support team to deliver exceptional big data production solutions. The ideal candidate will have deep...

  • Big Data Architect

    1 week ago

    Vancouver, British Columbia, Canada Behavox Full time

    About UsBehavox's VisionWe aim to organize all corporate communications and productivity data on Earth, making it useful to human organizations worldwide. Our data operating platform presents opportunities to build multilingual, AI/ML-based solutions that activate data for every function within a global enterprise.The PositionWe are looking for a Big Data...

  • Big Data Solutions Expert

    3 days ago

    Vancouver, British Columbia, Canada Capgemini Full time

    **Big Data Solutions Expert - Join Our Team**We are seeking a highly skilled Big Data Solutions Expert to join our team at Capgemini. The ideal candidate will have 7 to 10 years of experience in big data technologies, including Hadoop, Spark, and NoSQL databases.The successful candidate will have a strong understanding of data ingestion, transformation, and...

  • Big Data Engineering Specialist

    1 week ago

    Vancouver, British Columbia, Canada Rivian and Volkswagen Group Technologies Full time

    About UsRivian and Volkswagen Group Technologies is a cutting-edge joint venture driving innovation in the automotive sector. Our goal is to shape the future of mobility through groundbreaking technology that combines the strengths of two industry leaders. We're addressing complex challenges in electric vehicle development by harnessing expertise in...

  • Software Development Engineer

    1 week ago

    Vancouver, British Columbia, Canada Amazon Full time

    **About Us**The Business Data Technology (BDT) team at Amazon is responsible for developing innovative solutions to power the company's data analytics and machine learning capabilities. Our team is seeking talented software engineers to join our group and help shape the future of big data solutions.Job Description:Lead, design, and architect large-scale...

  • Technical Program Director

    1 week ago

    Vancouver, British Columbia, Canada T-Net British Columbia Full time

    **About the Role:**We are looking for a Lead Developer Advocate for Enterprise Systems to join our team at T-Net British Columbia. As a Technical Program Director, you will be responsible for leading our development efforts in Big Data technology, overseeing development resources, ensuring environment management and monitoring, championing the introduction...

  • Big Data Production Specialist

    1 week ago

    Vancouver, British Columbia, Canada Apex Systems Full time

    Job DescriptionApex Systems is seeking a support engineer to join our global IT services provider team. As a support engineer, you will be responsible for supporting our client's database support team with deep knowledge in databases (Mongo DB, MS SQL, No SQL) and big data production support (Azure/Cloud).Key Responsibilities:Providing technical support and...

  • Principal Big Data Engineer

    5 days ago

    Vancouver, British Columbia, Canada Mastercard Full time

    About Our MissionOur mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart, and accessible.Job RoleWe're seeking a Principal Big Data Engineer to join our team to lead the data platform growth across our product suite of solutions and technology landscape. This senior role...

  • Big Data Promotions and Optimization Software Engineer

    1 week ago

    Vancouver, British Columbia, Canada Amazon Full time

    Big Data Promotions and Optimization Software Engineer JobAmazon is seeking a talented Software Development Engineer to join our team and help us develop innovative solutions for big data promotions and optimization. In this role, you will be responsible for designing and developing software components that drive business growth and improve customer...

  • Big Data Engineering Expert

    2 weeks ago

    Vancouver, British Columbia, Canada Capgemini Full time

    Job DescriptionThe primary responsibility of this position involves architecting, designing, and implementing ETL processes using Microsoft Azure Data Factory, Databricks, and Synapse Analytics. Our ideal candidate should possess hands-on experience with these tools and related technologies.We seek an individual with excellent problem-solving, communication,...

  • Big Data Engineering Lead for Amazon

    2 weeks ago

    Vancouver, British Columbia, Canada Amazon Full time

    Amazon's Sales Promotions team is dedicated to supporting new members and fostering a culture of knowledge sharing and mentorship. As a Data Science Engineer, you will have the opportunity to work with a broad mix of experience levels and skills, including Software Engineering, Big Data Engineering, and Applied Science. We are building an environment that...

  • Cloud Data Architect

    1 week ago

    Greater Vancouver Metropolitan Area, Canada Capgemini Full time

    Job Summary: We are seeking an experienced Azure Fabric Data Engineer to work on hybrid projects from our offices in Vancouver or Calgary. As a key member of our team, you will be responsible for architecting, designing, and implementing ETL processes to ingest data from various sources into Unity Catalog. Your expertise in using notebooks to write code in...