Big Data Engineer
3 weeks ago
Toronto, Ontario, Canada
Recutify
Full-time
Free with email or Google
Save this job and keep your search organized
Create a free account to save jobs, create alerts and return to this listing from your dashboard.
Free with email or Google
By continuing, you agree to our Terms & Privacy Policy.
Role Name: Big Data Engineer / Toronto hybrid 4 days a week on site
JOB DESCRIPTION:
"Technical Skills:
• Data Structures and Algorithms: Expertise in designing efficient data structures and algorithms for big data processing.
• Distributed Systems: Experience with distributed computing frameworks and parallel processing.
• Data Integration: Proficiency in integrating data from various sources and ensuring data consistency.
• Performance Optimization: Skills in optimizing data processing and storage for high performance.
• Containerization: Experience with Docker and Kubernetes for deploying big data applications.
• Version Control: Proficiency in using Git for collaborative development.
• Security: Knowledge of data security practices and compliance with regulatory standards.
• Machine Learning: Familiarity with machine learning techniques and their application in big data environments
Responsibilities:
• Design, develop, and maintain scalable big data architectures and systems.
• Implement data processing pipelines using technologies such as Java, Spring Batch, Spring Boot, Scala ,Hadoop, Spark, and Kafka
• Optimize data storage and retrieval processes to ensure high performance and reliability.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Perform data modeling, mining, and production processes to support business needs.
• Ensure data quality, governance, and security across all data systems.
• Stay updated with the latest trends and advancements in big data technologies.
• Experience with real-time data processing and stream analytics.
• Knowledge of advanced analytics and data visualization tools.
• Knowledge of DevOps practices and tools for continuous integration and deployment
• Experience in managing big data projects and leading technical teams."
JOB DESCRIPTION:
"Technical Skills:
• Data Structures and Algorithms: Expertise in designing efficient data structures and algorithms for big data processing.
• Distributed Systems: Experience with distributed computing frameworks and parallel processing.
• Data Integration: Proficiency in integrating data from various sources and ensuring data consistency.
• Performance Optimization: Skills in optimizing data processing and storage for high performance.
• Containerization: Experience with Docker and Kubernetes for deploying big data applications.
• Version Control: Proficiency in using Git for collaborative development.
• Security: Knowledge of data security practices and compliance with regulatory standards.
• Machine Learning: Familiarity with machine learning techniques and their application in big data environments
Responsibilities:
• Design, develop, and maintain scalable big data architectures and systems.
• Implement data processing pipelines using technologies such as Java, Spring Batch, Spring Boot, Scala ,Hadoop, Spark, and Kafka
• Optimize data storage and retrieval processes to ensure high performance and reliability.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Perform data modeling, mining, and production processes to support business needs.
• Ensure data quality, governance, and security across all data systems.
• Stay updated with the latest trends and advancements in big data technologies.
• Experience with real-time data processing and stream analytics.
• Knowledge of advanced analytics and data visualization tools.
• Knowledge of DevOps practices and tools for continuous integration and deployment
• Experience in managing big data projects and leading technical teams."