Senior Big Data Engineer

1 month ago

Port Perry, Canada Doran Jones Full time
Responsibilities
  1. Technical Leadership: Provide technical leadership and guidance to junior data engineers, mentoring them on best practices, design principles, and emerging technologies in the field of data engineering.
  2. Data Architecture: Design and implement scalable, robust, and efficient data architectures, including data lakes, data warehouses, and data pipelines, to support the organization's data requirements.
  3. Data Pipeline Development: Develop and maintain ETL (Extract, Transform, Load) processes and data pipelines to ingest, process, and transform large volumes of data from various sources, ensuring data quality and reliability.
  4. Data Modeling: Design and implement data models and schemas to organize and structure data for efficient storage, retrieval, and analysis, considering business requirements and data governance principles.
  5. Data Integration: Integrate data from diverse sources, such as databases, APIs, files, and streaming platforms, ensuring seamless data flow and interoperability across systems.
  6. Data Transformation and Enrichment: Implement data transformation and enrichment processes to derive insights, perform aggregations, and create derived datasets for analytics, reporting, and machine learning.
  7. Data Governance and Compliance: Establish and enforce data governance policies, standards, and best practices to ensure data integrity, security, privacy, and compliance with regulatory requirements.
  8. Performance Optimization: Optimize data pipelines and queries for performance, scalability, and efficiency, leveraging techniques such as partitioning, indexing, and query optimization.
  9. Collaboration: Collaborate with cross-functional teams, including data scientists, analysts, software engineers, and business stakeholders, to understand data requirements and deliver data solutions that meet business needs.
Required Experience
  1. 7+ Years of overall software development experience in enterprise systems.
  2. 6+ years of Data Management experience in an enterprise environment.
  3. 6+ years of experience with SQL.
  4. 5+ years of experience with AWS S3, Spark, and Glue using PySpark (Scala is nice to have).
  5. 5+ years working with Snowflake, Databricks.
  6. Strong knowledge of metadata management, data governance, and data lineage.
  7. In-depth knowledge of Agile Software Development practices and how to leverage these to streamline project execution.
  8. Possess the ability to decompose business problems into smaller parts to correctly solution, taking into consideration the impact on other subsystems.
  9. Team player with excellent problem-solving and troubleshooting skills.
  10. Familiar with software engineering practices, including CI/CD pipelines, Automated Testing, and Performance Engineering.
  11. Working knowledge of SDLC, especially regarding improving agile delivery metrics.
  12. Strong communication and team/technical leadership skills.
  13. Bachelor’s degree in Computer Science, MIS, or a related field.
Preferred Experience
  1. AWS certification preferred.
  2. Financial services experience.
  3. Consulting experience.

Doran Jones Inc. (DJI) is the leading financial services Data Engineering and Application Development firm, specializing in Capital Markets, Risk, and Regulatory Compliance. We are US-based, helping our clients fill gaps in capacity and expertise, reduce risk, and accelerate change. Our leaders average over 25 years of experience in Financial Services and Technology. DJI is repeatedly engaged by the largest and most complex clients for our Agile transformation expertise and practical understanding of the critical relationship between data, architecture, and application development.

DJI has a mission to place more people from non-traditional backgrounds into sustainable technology careers. Through partnerships with non-profit technology programs in underserved communities and Veteran organizations, candidates transition from tech training programs into real IT careers at DJI. Our unique recruitment policy allows us to create exceptional teams, bringing a broad spectrum of experience to our company and creating anything but a traditional consulting firm.

DJI is a wholly-owned subsidiary of McLaren Strategic Ventures, a private equity firm, that invests in start-ups and scale-ups. They provide a full range of highly trusted domain consulting, advisory services, and innovative technologies globally. Doran Jones is able to offer a broad range of technology, deeper AI and machine learning tools, data intelligence, alternative resourcing capabilities, and capital access to invest and digitize at scale. Our clients benefit from this unique portfolio allowing them to accelerate the pace of their own digital transformations through innovative products and services designed around industry-wide issues.

Visit Doran Jones to learn more

#J-18808-Ljbffr

  • Sr. Associate, Platform Engineer

    1 month ago

    Port Perry, Canada Pfizer Full time

    Pfizer’s Chief Digital Office (CDO) leads the transformation of Pfizer into a digital powerhouse that will generate patient superior experiences which results in better health outcomes. The Analytics Experience team, which is part of the Artificial Intelligence, Data and Advanced Analytics (AIDA) organization, is responsible for creating a seamless...

  • Sr. Associate, Platform Engineer

    1 month ago

    Port Perry, Canada Pfizer Full time

    Pfizer’s Chief Digital Office (CDO) leads the transformation of Pfizer into a digital powerhouse that will generate patient superior experiences which results in better health outcomes. The Analytics Experience team, which is part of the Artificial Intelligence, Data and Advanced Analytics (AIDA) organization, is responsible for creating a seamless...

  • Senior Data Visualization Architect

    1 month ago

    Port Coquitlam, Canada Tekshapers Inc Full time

    Position : Senior Data Visualization Architect Location : Vancouver, BC (Onsite from Day 1) Duration : Full Time / Contract Job Description: Mandatory Skillset: Commercial Software Mastery: Expert-level proficiency in visualization tools such as Apache ECharts, D3.JS and Three.js for 3D visualization. Ability to customize and extend these tools through...

  • Senior Data Visualization Architect

    1 month ago

    Port Coquitlam, Canada Tekshapers Inc Full time

    Position : Senior Data Visualization Architect Location : Vancouver, BC (Onsite from Day 1) Duration : Full Time / Contract Job Description: Mandatory Skillset: Commercial Software Mastery: Expert-level proficiency in visualization tools such as Apache ECharts, D3.JS and Three.js for 3D visualization. Ability to customize and extend these tools through...

  • Senior Civil/Industrial Engineering Consultant

    3 weeks ago

    Port Perry, Canada Prescient Edge Full time

    Prescient Edge is seeking a Senior Civil/Industrial Engineering Consultant to deliver expert support to a federal government client. The successful candidate will possess a master's degree in a pertinent discipline and a minimum of 20 years of extensive experience in analytical roles. Exceptional capabilities in communication, analytical thinking, and...

  • Senior Commissioning Specialist

    3 weeks ago

    Port Perry, Canada Affiliated Engineers Full time

    Position OverviewAffiliated Engineers, Inc. (AEI) stands as a premier, multidisciplinary consulting engineering firm, comprising a collaborative community of over 800 professionals dedicated to planning, designing, and delivering high-performance, innovative engineering solutions that positively impact society and advance human progress.Consistently...

  • Senior Data Engineer, Ads Data

    4 weeks ago

    Port Coquitlam, British Columbia, V3B, Greater Vancouver, Canada New Miles Immigration, Recruitment & Consulting Services Full time

    Career Opportunities with New Miles Immigration, Recruitment & Consulting Services Careers At New Miles Immigration, Recruitment & Consulting Services Current job opportunities are posted here as they become available. A leading social media company is looking to fill the following position at its Vancouver branch: Responsibilities: Collaborate with...

  • Director of Engineering

    2 weeks ago

    Port Perry, Canada Adamson Systems Engineering Full time

    We seek an exceptional senior engineering director to add to our rapidly growing company. A good candidate will possess an MBA along with significant management deployment in high technology product manufacturing, that extends well beyond their engineering studies and experience. The Adamson bran

  • New Graduate, Engineering Intern, Land/Site Development

    1 month ago

    Port Perry, Canada BGE Full time

    BGE, Inc. is a nationwide consulting firm that provides services in civil engineering, construction management, land planning, landscape architecture, survey and environmental services for public and private clients. At BGE, we seek out challenges and apply our community-minded approach to see the big picture and uncover opportunities. We are adept at...

  • Senior Data Engineer

    3 weeks ago

    Port Alberni, Canada Lumenalta (formerly Clevertech) Full time

    Experience Remote done Right. With over 20 years of remote experience, all 500+ staff are 100% remote, and we still grow vibrant relationships and provide exceptional opportunities for career growth while working with stellar clients on ambitious projects.What we're working on:Enterprise companies turn to us to help them launch innovative digital products...

  • Senior Data Engineer

    3 weeks ago

    Port Alberni, Canada Lumenalta (formerly Clevertech) Full time

    Experience Remote done Right. With over 20 years of remote experience, all 500+ staff are 100% remote, and we still grow vibrant relationships and provide exceptional opportunities for career growth while working with stellar clients on ambitious projects.What we're working on:Enterprise companies turn to us to help them launch innovative digital products...

  • Director of Engineering

    2 weeks ago

    Port Perry, Canada Adamson Systems Engineering Full time

    We seek an exceptional senior engineering director to add to our rapidly growing company. A good candidate will possess an MBA along with significant management deployment in high technology product manufacturing, that extends well beyond their engineering studies and experience.The Adamson brand is globally acclaimed, and a best-in-class industry leader....

  • Director of Engineering

    2 weeks ago

    Port Perry, Canada Adamson Systems Engineering Full time

    We seek an exceptional senior engineering director to add to our rapidly growing company. A good candidate will possess an MBA along with significant management deployment in high technology product manufacturing, that extends well beyond their engineering studies and experience.The Adamson brand is globally acclaimed, and a best-in-class industry leader....

  • Director of Engineering

    2 weeks ago

    Port Perry, Canada Adamson Systems Engineering Full time

    We seek an exceptional senior engineering director to add to our rapidly growing company. A good candidate will possess an MBA along with significant management deployment in high technology product manufacturing, that extends well beyond their engineering studies and experience.The Adamson brand is globally acclaimed, and a best-in-class industry leader....

  • Director of Engineering

    2 weeks ago

    Port Perry, Canada Adamson Systems Engineering Full time

    We seek an exceptional senior engineering director to add to our rapidly growing company. A good candidate will possess an MBA along with significant management deployment in high technology product manufacturing, that extends well beyond their engineering studies and experience. The Adamson brand is globally acclaimed, and a best-in-class industry...

  • Senior Level Electrical Engineer

    1 month ago

    Port Perry, Canada Sargent & Lundy Full time

    DescriptionProject teams in our Energy & Industrial Group (EIG) support industry-leading projects in power generation, plant design, environmental compliance, mid/downstream oil & gas, and multi-disciplined industrial projects. We’re leading efforts to decarbonize the industry by utilizing advanced technologies in renewable energy, carbon capture, and...

  • Engineering Team Lead

    4 hours ago

    Port Perry, Canada Adamson Systems Engineering Full time

    Job Summary:The Product Development Manager will lead and coordinate the engineering teams that develop our products and prepare them for manufacturing. They will liaise with other department heads and senior management to ensure product development schedules are met, manufacturing processes and resources are in place, propose remedies to constraints, and...

  • Engineering Technician

    1 month ago

    Port Perry, Canada Hillsborough County - Florida Full time

    Job OverviewThis is responsible technical work involving engineering planning, construction and/or compliance inspections. Incumbents in this job may also perform duties of identifying, inspecting, and condition coding county utilities assets and provide an accurate inventory and condition of assets.SalaryMin $41,808.00 annuallyMid $55,400.80...

  • Infrastructure Engineering Specialist

    3 weeks ago

    Port Perry, Canada Hillsborough County - Florida Full time

    Position Summary This role involves performing technical engineering tasks related to the analysis, evaluation, management, planning, documentation, coordination, and inspection of public infrastructure and systems. The position operates under the guidance of a licensed Professional Engineer and includes project management responsibilities for infrastructure...

  • Engineering Associate

    1 month ago

    Port Perry, Canada Hillsborough County - Florida Full time

    Job Overview This position performs paraprofessional engineering related work involving the investigation, evaluation, assessment, asset management, planning, writing reports, coordinating work, or inspection of public infrastructure and systems. This position works under the supervision of a licensed Professional Engineer. Project management responsibility...