Big Data Developer

2 weeks ago

Greater Vancouver Metropolitan Area, Canada Capgemini Full time

The Senior Data Developer Lead- Digital Analytics is joining our growing hybrid team in Vancouver, not looking to relocate. We are interested in your data implementation experience.

Must Have Skills:

7 to 10 years working in ETL, Databricks, Python, Spark, SQL Databases with data implementation experience.

Responsibilities:

  • Strong proficiency in Python (especially data packages like pandas, Numpy, etc.) and SQL for analytics, database development, and data modelling
  • Professional experience with data ingestion, ETL, and ELT for structured and unstructured data.
  • Experience with DevOps and CICD for data.
  • Python - Strong proficiency, unit testing expertise and good knowledge of packages for data, like Pandas, SQL Alchemy, and Alembic.
  • SQL - strong proficiency in DDL and DML, including window function (ex.: lag), CTEs, sub-queries, joins, optimization, and performance profiling. Know how SQL behaves in different platforms, like Spark, PostgreSQL,
  • Spark - Pyspark, Spark SQL, Batch and Streaming processing, partitioning/liquid clustering, delta tables, parquet
  • Databricks - Workflows/Jobs, Clusters, SQL Warehouse, Unity Catalog, Performance profiling, log analysis.
  • Azure EventHub or a similar streaming solution understand how to best consume data from streaming for aggregations and parallelization/scaling.
  • PostgreSQL - Queries, indexes (different types of indexes), performance profiling, JSON columns.
  • Data Modeling: Dimensional Modeling (and experience with this model being used in BI tools), Normal Forms.
  • Docker containerization
  • DBT - models, seeds, multiple environments, parameters, macros, unit tests, data tests, incremental materialization, snapshots
  • Infra as code/ CICD tools: Kubernetes, Argo, Cross Plane, Terraform or similar.
  • Migrations - versioning for database, for example, Alembic (preferred), Flyway, Liquibase
  • SQL and NoSQL databases and selecting the best fit for different use cases.
  • Logging and being able to query logs using KQL(Azure) or similar.
  • Support, maintain, optimize, and create ETL/ELT pipelines, both batch and streaming, in Databricks (Pyspark, Databricks SQL), Python, SQL and/or DBT.
  • Proficient in Dimensional Modeling and Database normalization (Normal Form)

Life at Capgemini

Capgemini supports all aspects of your well-being throughout the changing stages of your life and career. For eligible employees, we offer:

  • Collaborating with teams of creative, fun, and driven colleagues
  • Flexible work options enabling time and location-based flexibility
  • Company-provided home office equipment
  • Virtual collaboration and productivity tools to enable hybrid teams
  • Comprehensive benefits program (Health, Welfare, Retirement and Paid time off)
  • Other perks and wellness benefits like discount programs, and gym/studio access.
  • Paid Parental Leave and coaching, baby welcome gift, and family care/illness days
  • Back-up childcare/elder care, childcare discounts, and subsidized virtual tutoring
  • Tuition assistance and weekly hot skill development opportunities
  • Experiential, high-impact learning series events
  • Access to mental health resources and mindfulness programs
  • Access to join Capgemini Employee Resource Groups around communities of interest

About Capgemini

Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, cloud and data, combined with its deep industry expertise and partner ecosystem. The Group reported 2023 global revenues of €22.5 billion

  • Big Data Platform Developer

    4 days ago

    Vancouver, British Columbia, Canada Rivian and Volkswagen Group Technologies Full time

    About UsRivian and Volkswagen Group Technologies is a pioneering joint venture in the automotive industry. Our company combines the strengths of two industry leaders to drive innovation and shape the future of software-defined vehicles. Through cutting-edge technology, we address the challenges of electric vehicles and set the standards for software-defined...

  • Big Data Architect

    4 days ago

    Vancouver, British Columbia, Canada Amazon Full time

    The Business Data Technology (BDT) team at Amazon is responsible for delivering innovative Big Data solutions that power our enterprise data sets, the Amazon data lake/data marketplace, and core infrastructure supporting data analytics and machine learning.We are looking for software developers who can design and build large-scale systems with global impact....

  • Big Data DBT Developer

    6 days ago

    Vancouver, British Columbia, Canada Two Circles Full time

    Who we areWe are Two Circles. We are a Sports & Entertainment Marketing business. We grow audiences and revenues. We do that by knowing fans best. We work with clients to help them understand & influence what their fans are doing – the way fans spend their money, the events that fans attend, the channels fans respond to, the content fans watch and more....

  • Data Engineer for Big Data Solutions

    3 days ago

    Vancouver, British Columbia, Canada Amazon Full time

    Company OverviewThe Business Data Technology (BDT) team at Amazon is responsible for developing innovative big data solutions that support the company's enterprise data sets, data lake/data marketplace, and core infrastructure. We are a dynamic group of professionals who continuously push the boundaries of technology to deliver high-quality data to our users...

  • Big Data Engineer Manager

    2 days ago

    Vancouver, British Columbia, Canada T-Net British Columbia Full time

    Company Overview">T-Net British Columbia is a renowned innovator that thrives on technical excellence and collaboration. Our organization seeks exceptional talent to drive its success.">Job Description">We are looking for a seasoned Big Data Engineer Manager to lead our agile development team, leveraging expertise in software architecture and Big Data...

  • Big Data Solutions Expert

    2 days ago

    Vancouver, British Columbia, Canada Rivian and Volkswagen Group Technologies Full time

    About Rivian and Volkswagen Group TechnologiesWe're a pioneering company that combines the expertise of two industry leaders to create innovative technology for software-defined vehicles.Our company's vision is to shape the future of electric vehicles through cutting-edge technology that sets global standards. We're committed to creating a diverse and...

  • Big Data and Machine Learning Developer

    17 hours ago

    Vancouver, British Columbia, Canada Amazon Full time

    About the TeamWe're a diverse group of experienced professionals with a broad mix of skills, including software engineering, big data engineering, and applied science. We're passionate about innovation and constantly seek new ways to improve our systems.

  • Big Data Technology Architect

    6 days ago

    Vancouver, British Columbia, Canada Amazon Full time

    Company OverviewAmazon's Big Data solutions team is responsible for developing innovative technologies to deliver data to users in a consistent and accurate manner. We are constantly pushing the boundaries of technology to meet our customers' needs.Job DescriptionWe are seeking talented software engineers to join our group. In this role, you will lead,...

  • Big Data Technology Strategist

    3 days ago

    Vancouver, British Columbia, Canada Databricks Inc. Full time

    **Job Summary:**We are looking for a highly skilled Big Data Technology Strategist to join our team at Databricks. As a Big Data Technology Strategist, you will partner with our customers to design scalable data architectures using Databricks technology and services.You will work closely with our Account Executives, technical leads, architects, engineers,...

  • Senior Big Data Specialist

    6 days ago

    Vancouver, British Columbia, Canada Avanciers Inc. Full time

    About Avanciers Inc.We're seeking a highly skilled Senior Big Data Specialist to join our team of experienced professionals. As a key member of our GCP cloud engineering group, you will be responsible for designing, building, and maintaining large-scale data systems on Google Cloud Platform.Your Role: You will work closely with our development team to...

  • Data architect

    4 weeks ago

    Greater Ottawa Metropolitan Area, Canada Altis Technology Full time

    Location: Hybrid - Toronto or Ottawa We are seeking a skilled Data Architect to lead the design and evolution of modern data architectures, driving best practices in data ingestion, pipeline development, and cloud-native data solutions. This role is critical in shaping our organization's data strategy, ensuring scalability, efficiency, and innovation. The...

  • Product Manager, Data Products

    3 weeks ago

    Greater Toronto Area, Canada Data Axle Full time

    **Responsibilities**: - Create, refine, and implement a strategic, long-term vision and roadmap for data products. - Identify, track, understand, and ultimately improve core business KPIs that ladder up to long-term success of the company. - Develop powerful insights and recommendations informed by market research and data analysis. - Build and present...

  • Senior Data Engineer

    3 weeks ago

    Greater Winnipeg Metropolitan Area, Canada Lotlinx Full time

    Since our founding in 2012, Lotlinx has consistently pioneered advancements in the automotive landscape. We specialize in empowering automobile dealers and manufacturers by providing cutting-edge data and technology, delivering a distinct market advantage for every single vehicle transaction. Today, we stand as the foremost automotive AI and machine learning...

  • Senior Data Analyst

    6 days ago

    Greater Toronto Area, Canada Trader Interactive Full time

    Are you ready to be a big part of something big?At Trader Interactive, we make buying and selling a great experience. We're a group of go-getters who decided they didn't want to settle for the status quo. We come together as one team to build value and drive innovation across our industries - but we have fun while we do it and make sure our people are always...

  • Senior Data Analyst

    2 days ago

    Greater Toronto Area, Canada Trader Interactive Full time

    Are you ready to be a big part of something big?At Trader Interactive, we make buying and selling a great experience. We're a group of go-getters who decided they didn't want to settle for the status quo. We come together as one team to build value and drive innovation across our industries - but we have fun while we do it and make sure our people are always...

  • Big Data Scientist

    6 days ago

    Vancouver, British Columbia, Canada TTT Studios Full time

    Company InformationTTT Studios is a pioneering digital consulting and software development company serving a diverse range of clients worldwide.We have built a strong reputation for delivering innovative applications and patented technologies, making us a trusted partner for ambitious projects.About the PositionWe are seeking skilled Data Engineers to...

  • Big Data Engineer

    2 days ago

    Vancouver, British Columbia, Canada Samsung Electronics America Full time

    **Responsibilities**- Design and build advanced machine learning and AI models to address business challenges and generate reports.- Analyze data to understand usage and performance of the Samsung Health service.- Apply and innovate latest advances in LLM, NLP, and recommender systems to build innovative solutions.- Collaborate with data engineers to develop...

  • DevOps, Site Reliability

    3 weeks ago

    Vancouver, Canada Global Relay Full time

    **Who we are**: For over 20 years, Global Relay has set the standard in enterprise information archiving with industry-leading cloud archiving, surveillance, eDiscovery, and analytics solutions. We securely capture and preserve the communications data of the world's most highly regulated firms, giving them greater visibility and control over their...

  • Data Entry Clerk

    3 weeks ago

    Greater Toronto Area, Canada Think Big Solutions Full time

    **URGENT HIRING ENTRY-LEVEL DATA ENTRY** Do you have good data entry skills and can fluently speak/type in English? **What’s in it for you?** - Full-time only, 8 hours shift. - Shifts timings **Monday to Friday, 7:00 am to 3:00 pm and 12.00 pm to 8.00 pm** **Key** **Responsibilities**: - Ability to type accurate data from paper to computer software. -...

  • Data Platform Director

    3 weeks ago

    Greater Toronto Area, Canada VTRAC Consulting Corporation (WBE) Full time

    VTRAC Consulting Corporation Intelligent SolutionsThank you for applying to VTRAC opportunities. Please e-mail your resume as an MS-WORD document in confidence Subject: Data Platform Director, Attention: mask@vtrac.com or call: (647) 254-0770.Position#: 251140Position: Data Platform DirectorPosition Type: PermanentNo. of Positions: 1Location: GTA,...