Spark Developer

Occubee is a cutting-edge SaaS platform, which enhances retailers and manufacturers impact on their financial and operational performance by optimizing their supply chain processes. Occubee provides demand forecasting, demand management and replenishment by using artificial intelligence and machine learning.

The mission of a Spark Developer is developing and maintaining data processing applications using Apache Spark, a powerful open-source big data processing framework. Spark Developers are responsible for harnessing the capabilities of Spark to handle large-scale data processing tasks efficiently.

Your responsibilities

Documentation: Maintain technical documentation throughout developmentProgramming Skills: Proficiency in PySpark and Python, with a deep understanding of Python syntax, data structures, and algorithms.Problem Solving: Ability to solve complex technical problems using PySpark and Python, analyze requirements, and design software solutions.Collaboration: Ability to collaborate with developers, testers, designers, and stakeholders to ensure software meets requirements.Agile Development: Familiarity with agile methodologies like Scrum or Kanban.Quality Assurance: Ensure high quality of provided solutions.Communication: Communicate application status, updates, and issues to team, management, and stakeholders, facilitating collaboration.Optimization: Analyze and optimize performance, identify and eliminate bottlenecks.Code Review: Review and provide feedback on code to maintain code quality and consistency within the project.

Key requirements

Good knowledge of OOP paradigms and SOLID principles using Python and experience with the most common PySpark Python API for Apache Spark.Good knowledge of application development environments like: Git.Experience with Zeppelin.Proficiency in Agile/Scrum methodologies and a commitment to driving Agile practices within the team.Knowledge of Linux.Experience with Azure Cloud Platform.Analytical thinking, and problem-solving skills.You have good qualities in communication in Polish & English (B1-B2): verbally and written.Best practices for designing scalable data processing systems, including data pipelines, advanced ETL processes and data lakes.

Key Skills

• Python
• Machine Learning
• OOP
• Agile