Log Data Engineer for Scalable Solutions

4 weeks ago


Toronto, Canada Capgemini Full time
Job Title:

Log Data Engineer for Scalable Solutions

Overview:

We are seeking a skilled Log Data Engineer to join our team at Capgemini. This role involves designing, implementing, and maintaining scalable log management solutions using the ELK stack (Elasticsearch, Logstash, Kibana) in cloud-based environments.

Key Responsibilities:
  • Design and Implement ELK Solutions: Develop efficient log ingestion pipelines using Logstash or Beats agents to collect and process large volumes of log data from various sources.
  • Maintain ELK Stack Performance: Monitor, configure, and troubleshoot the ELK stack to ensure optimal performance, reliability, and scalability in production environments.
  • Data Analysis and Visualization: Create and maintain dashboards and visualizations in Kibana for different teams (e.g., development, security, operations) to track system performance, errors, and other metrics.
  • Search Engine Optimization: Optimize Elasticsearch queries, mappings, and indexing to ensure fast search performance and efficient data storage.
  • Automation and Security: Automate the setup, configuration, and scaling of the ELK stack using tools like Ansible, Terraform, or Kubernetes, and implement security configurations in Elasticsearch, including access control and encryption.
  • Collaboration and Support: Collaborate with internal teams to understand their data needs and provide support in leveraging ELK stack capabilities for various use cases.
Requirements:
  • Strong ELK Stack Experience: Hands-on experience with Elasticsearch, Logstash, and Kibana is essential.
  • Programming Skills: Proficiency in scripting languages such as Python, Bash, or Groovy is required.
  • Cloud Platforms: Familiarity with cloud services such as AWS, Azure, or GCP is a plus.
  • Performance Optimization: Ability to optimize Elasticsearch performance for large-scale data sets is crucial.
  • Version Control: Proficiency with Git and other version control systems is necessary.
  • Problem-Solving and Troubleshooting: Strong problem-solving skills, with an ability to debug and troubleshoot issues in a complex environment.
Estimated Salary Range:

$120,000 - $160,000 per year



  • Toronto, Ontario, Canada Strategic Solutions Technology Staffing Inc. Full time

    Job Title: Data Engineering Leader for Scalable Data SolutionsAbout Us:Strategic Solutions Technology Staffing Inc. is a leading provider of data-driven solutions, serving top-tier clients across the globe.Job Summary:We are seeking an experienced Data Engineer to lead our data engineering efforts and design scalable data pipelines that support our client's...


  • Toronto, Ontario, Canada Data Engineer Jobs Full time

    About OxaOxa is revolutionizing transportation by developing autonomous solutions for goods and people. Our cutting-edge technology is being deployed across the UK and US, with a growing ecosystem of partners. Based in Oxford, our team of experts is dedicated to making transportation safer, more efficient, and explainable.Job DescriptionWe're seeking a...


  • Toronto, Ontario, Canada Data Engineer Jobs Full time

    Data Engineer Job DescriptionAt Lyft, our purpose is to serve and connect people with the world around them. We strive to create an open, inclusive, and diverse organization that values every individual's unique perspective and skills.We are looking for a highly skilled Data Engineer to join our Experimentation and Metric Platform team. As a key member of...


  • Old Toronto, Canada Jobber Full time

    Senior Software Engineer - Data and Streaming TeamAt Jobber, we're on a mission to help small businesses succeed. We believe in fostering a collaborative environment where teams make decisions autonomously and contribute directly to shaping the future of our company.We're looking for a Senior Software Engineer to join our Data and Streaming team. As a key...

  • Data Engineer

    5 days ago


    Toronto, Ontario, Canada Strategic Solutions Technology Staffing Inc. Full time

    About Strategic Solutions Technology Staffing Inc.We are a global leader in delivering cutting-edge technology solutions to our clients. Our team is comprised of innovative professionals who share a passion for data-driven decision-making and collaborative work environments.Job DescriptionData Engineer - Scalable Data Pipelines SpecialistWe are seeking an...


  • toronto, Canada Encore Technical Solutions Inc. Full time

    Principal Engineer, Data Engineering (Director Level)Contract to Full-time PermanentHybrid/Remote - 2-3 days per week in officeGreater Toronto Area, ONWe are seeking a Principal Engineer to lead our client's data engineering initiatives on the Microsoft Azure stack. This is a hands-on, director-level role that combines technical leadership with strategic...

  • Principal Engineer

    5 hours ago


    Greater Toronto Area, Canada Encore Technical Solutions Inc. Full time

    Principal Engineer, Data Engineering (Director Level)Contract to Full-time PermanentHybrid/Remote - 2-3 days per week in officeGreater Toronto Area, ONWe are seeking a Principal Engineer to lead our client's data engineering initiatives on the Microsoft Azure stack. This is a hands-on, director-level role that combines technical leadership with strategic...

  • Principal Engineer

    5 hours ago


    Greater Toronto Area, Canada Encore Technical Solutions Inc. Full time

    Principal Engineer, Data Engineering (Director Level)Contract to Full-time PermanentHybrid/Remote - 2-3 days per week in officeGreater Toronto Area, ONWe are seeking a Principal Engineer to lead our client's data engineering initiatives on the Microsoft Azure stack. This is a hands-on, director-level role that combines technical leadership with strategic...


  • Greater Toronto Area, Canada NextGen Consulting Inc. Full time

    We are looking for a skilled Cloud Data Engineering Leader to join our team at NextGen Consulting Inc. in the Greater Toronto Area, Canada.As a Cloud Data Engineering Leader, you will play a key role in designing and implementing scalable data pipelines and cloud architectures for our clients. This is an excellent opportunity to work with state-of-the-art...


  • Old Toronto, Calgary, Vancouver, Canada Themis Solutions Inc. Full time

    About UsAt Themis Solutions Inc., we're transforming the legal experience by bettering the lives of professionals while increasing access to justice.We're a global leader united by our mission, dedicated to diversity, equity, and inclusion. Our teams feel included, valued, and empowered to do their best work, wherever they choose to log in from.Your...

  • Principal Engineer

    24 hours ago


    Greater Toronto Area, Canada Encore Technical Solutions Inc. Full time

    Principal Engineer, Data Engineering (Director Level)Contract to Full-time PermanentHybrid/Remote - 2-3 days per week in officeGreater Toronto Area, ONWe are seeking a Principal Engineer to lead our client's data engineering initiatives on the Microsoft Azure stack. This is a hands-on, director-level role that combines technical leadership with strategic...

  • Principal Engineer

    4 hours ago


    Greater Toronto Area, Canada Encore Technical Solutions Inc. Full time

    Principal Engineer, Data Engineering (Director Level)Contract to Full-time PermanentHybrid/Remote - 2-3 days per week in officeGreater Toronto Area, ONWe are seeking a Principal Engineer to lead our client's data engineering initiatives on the Microsoft Azure stack. This is a hands-on, director-level role that combines technical leadership with strategic...

  • Principal Engineer

    1 hour ago


    Greater Toronto Area, Canada Encore Technical Solutions Inc. Full time

    Principal Engineer, Data Engineering (Director Level) Contract to Full-time Permanent Hybrid/Remote - 2-3 days per week in office Greater Toronto Area, ON We are seeking a Principal Engineer to lead our client's data engineering initiatives on the Microsoft Azure stack. This is a hands-on, director-level role that combines technical leadership with...


  • Toronto, Ontario, Canada Amazon Full time

    Amazon Advertising is a rapidly growing team building a unique, world-class advertising group with a relentless focus on the customer. We're looking for a talented Software Development Engineer II to join our team and contribute to the design and implementation of scalable data solutions that drive real-time data processing.As a key member of our team, you...

  • Principal Engineer

    19 hours ago


    Greater Toronto Area, Canada, Ontario Encore Technical Solutions Inc. Full time

    Principal Engineer, Data Engineering (Director Level)Contract to Full-time PermanentHybrid/Remote - 2-3 days per week in officeGreater Toronto Area, ONWe are seeking a Principal Engineer to lead our client's data engineering initiatives on the Microsoft Azure stack. This is a hands-on, director-level role that combines technical leadership with strategic...


  • Toronto, Canada Luxoft Full time

    Job DescriptionWe are seeking a highly skilled Senior Data Engineer to join our team in Toronto. As a key member of our organization, you will play a critical role in designing and developing scalable software solutions that drive business growth.The ideal candidate will have a strong background in data engineering, with expertise in technologies such as...


  • Toronto, Canada Mindlance Full time

    About MindlanceMindlance is a company that prides itself on its commitment to inclusivity and accessibility. We believe in fostering a work environment where everyone feels valued and respected.Job Title: Cloud Data Engineer for Scalable ArchitecturesWe are currently seeking an experienced Cloud Data Engineer with expertise in designing and implementing...


  • Toronto, Ontario, Canada Highbrow LLC Full time

    At Highbrow LLC, we are seeking an experienced Data Engineering Specialist to join our team and help us build scalable systems.About the RoleThe successful candidate will be responsible for ensuring data quality and integrity across various data sources and systems. They will optimize data pipelines for performance and scalability, providing technical...


  • Old Toronto, Canada Amazon Full time

    Job OverviewWe're seeking a skilled Data Science Engineer to join our team and help build scalable measurement solutions at Amazon.About the RoleYou will design, develop, and maintain large-scale data pipelines and software systems that compute estimated impact of Amazon ads. Your expertise in machine learning and data science will be crucial in defining how...


  • Toronto, Ontario, Canada Tbwa ChiatDay Inc Full time

    As a seasoned software engineer with 7+ years of experience, you have the unique opportunity to join our Ads Insights team as a Lead Data Architect for Scalable AI Solutions. Our mission is to rapidly innovate and grow like never before, shaping the future of Reddit's monetization strategy by harnessing the power of artificial intelligence and rich...