Big Data Developer

1 week ago

Toronto, Ontario, Canada fc71e8c5-deda-48e9-b38c-23c0611c543b Full time $70,000 - $120,000 per year

Job Overview

We are seeking a dynamic and innovative Big Data Developer to join our data engineering team. In this role, you will harness the power of large-scale data processing and analytics to drive strategic insights and support business growth. Your expertise in big data technologies, cloud platforms, and data integration will be crucial in designing, developing, and maintaining scalable data solutions. This position offers an exciting opportunity to work with cutting-edge tools and methodologies to transform complex data into actionable intelligence, empowering decision-makers across the organization.

Duties

  • Design, develop, and optimize large-scale data pipelines using Hadoop, Spark, and Apache Hive to handle vast datasets efficiently.
  • Implement ETL (Extract, Transform, Load) processes utilizing tools like Informatica and Talend to ensure seamless data flow from diverse sources into centralized data warehouses.
  • Develop and maintain data models and database schemas for data warehouses built on platforms such as Azure Data Lake, Oracle, and Microsoft SQL Server.
  • Collaborate with cross-functional teams using Agile methodologies to gather requirements, deliver solutions, and iterate rapidly on project goals.
  • Create RESTful APIs for secure data access and integration with external applications or services.
  • Utilize programming languages like Java and Python for scripting, automation, model training, and analysis tasks.
  • Conduct analysis of large datasets to identify trends, anomalies, or insights that support business strategies.
  • Design database structures with a focus on scalability, performance optimization, and efficient query execution.
  • Develop shell scripts using Bash or Shell Scripting for automation of routine tasks in Unix/Linux environments.
  • Leverage Looker for visualization and reporting to communicate complex findings clearly to stakeholders.

Skills

  • Proven experience working with big data technologies such as Hadoop, Spark, Apache Hive, and Azure Data Lake.
  • Strong proficiency in programming languages including Java and Python for development and analytics purposes.
  • Hands-on knowledge of cloud platforms like AWS and Azure for deploying scalable data solutions.
  • Expertise in SQL-based databases including Microsoft SQL Server, Oracle, and related data warehouse architectures.
  • Familiarity with ETL tools such as Informatica and Talend for efficient data integration workflows.
  • Experience designing database schemas optimized for large datasets with a focus on performance tuning.
  • Ability to develop RESTful APIs for secure external access to data resources.
  • Knowledge of Linked Data principles for integrating diverse datasets across platforms.
  • Skills in model training techniques for predictive analytics within big data environments.
  • Strong analysis skills to interpret complex datasets and generate actionable insights.
  • Experience with Shell Scripting (Bash), VBA automation is a plus for process automation tasks.
  • Familiarity with Agile development practices to promote iterative project delivery. Join us if you're passionate about transforming massive amounts of raw data into strategic assets Bring your expertise in big data ecosystems combined with your enthusiasm for innovative problem-solving — together we'll unlock new possibilities through advanced analytics

Job Type: Full-time

Pay: $70,000.00-$120,000.00 per year

Work Location: In person

  • Big Data Developer

    4 days ago

    Toronto, Ontario, Canada Triton Digital Full time $120,000 - $180,000 per year

    We are open to candidates in Ontario or Quebec.Are you passionate about new technologies and looking to be part of a software development team that is constantly striving for innovation in a warm, relaxed, and stimulating environment?We are looking for aBig Data Developer / Machine Learning Engineerto join our "Audience" team.Our success depends on our...

  • Big Data Developer

    2 weeks ago

    Toronto, Ontario, Canada Tekshapers Full time $80,000 - $120,000 per year

    Job Title: Big data DeveloperLocation: Toronto, ONFull TimeTotal Experience: 8-10 yearsRole Description:• We are looking for a big data developer with hands on experience on developing APIs.• The candidate should also have knowledge of Kubernetes and worked as a Java developer.• The candidate should be able to exhibit strong coding skills in above tech...

  • Big Data Developer

    7 days ago

    Toronto, Ontario, Canada Accuro Full time $90,000 - $120,000 per year

    Job Tite: Big Data DeveloperLocation: Toronto, ON (Hybrid)Job Type: FTEJob Description:Primary Skill:• Min 5+ years in developing and optimizing Big Data applications using Java/Scala and Spark on Cloudera/HDP.• Experience in developing/designing micro-service architecture.• Proven hands-on experience in containerization – Docker, Kubernetes,...

  • Big Data Developer

    7 days ago

    Toronto, Ontario, Canada ehub global Full time $72,000 - $96,000 per year

    Role-Big-Data developerLocation- Toronto, ON (Onsite from day 1)JD:Experience: 8+ yearsBig Data: Apache Spark, Hadoop ecosystem(HDFS, Hive, Yarn).Programming : Expert in Scala and JavaBackend :Spring Boot, RESTful APIsDatabase : Microsoft SQL Server(MSSQL)Nice to have : Experience in migrating on-premise applications to cloud (Azure/AWS), familiarity with...

  • Big Data Developer

    2 weeks ago

    Toronto, Ontario, Canada Info Way Solutions Full time $80,000 - $120,000 per year

    Big Data Developer: (No of position – 1)JD:• Experience: 8+ years• Big Data: Apache Spark, Hadoop ecosystem(HDFS, Hive, Yarn).• Programming : Expert in Scala and Java• Backend :Spring Boot, RESTful APIs• Database : Microsoft SQL Server(MSSQL)• Nice to have : Experience in migrating on-premise applications to cloud (Azure/AWS), familiarity with...

  • Senior big Data Developer

    3 hours ago

    Toronto, Ontario, Canada BULL-IT SOLUTIONS LTD Full time $100,000 - $150,000 per year

    Tips: Provide a summary of the role, what success in the position looks like, and how this role fits into the organization overall.ResponsibilitiesTotal Experience: 7-12 yearsRole Description:• Design high quality deliverables adhering to business requirements with defined standards and design principles, patterns• Develop and maintain highly scalable,...

  • Lead Big Data Developer

    2 weeks ago

    Toronto, Ontario, Canada Vybog Full time $120,000 - $180,000 per year

    OverviewAre you interested in building the next-generation platform that handlespetabytes of dataand powersdata insight creationusing cutting-edgedata science technologies?Are you excited about leveragingBig Data,AI / Machine Learning, andData Analyticsto deliver actionable insights?We are adynamic team of top-notch engineersdedicated to making a dent in...

  • Big Data Developer

    2 weeks ago

    Toronto, Ontario, Canada Smart IT Frame LLC Full time $80,000 - $120,000 per year

    Role :Bigdata(Spark,Scala), Java,Sql,Unix,OCPLocation :TorontoJD:6 - 0 years of experience in Big Data technologies, especially Spark and Scala.Strong programming skills in Java and Scala.Proficient in writing complex SQL queries and working with relational databases.Hands-on experience with Unix/Linux environments and shell scripting.Oracle Certified...

  • Lead Big Data Developer, GFT

    7 hours ago

    Toronto, Ontario, Canada RBC Full time $120,000 - $180,000 per year

    Job DescriptionWhat is the opportunity?As the Lead Big Data Developer lead the technical design and development of application systems on GFT - DaaS team. Apply extensive, in-depth knowledge, skills, and practices to perform complex assignments.Are you a talented, creative, and results-driven professional who thrives on delivering high-performing...

  • Consultant, Big Data Development

    7 days ago

    Toronto, Ontario, Canada CIBC Full time $80,000 - $120,000 per year

    We're building a relationship-oriented bank for the modern world. We need talented, passionate professionals who are dedicated to doing what's right for our clients.At CIBC, we embrace your strengths and your ambitions, so you are empowered at work. Our team members have what they need to make a meaningful impact and are truly valued for who they are and...