Position Overview:
We are seeking an experienced and dynamic Lead Python Developer – Data Engineer with a strong focus on data engineering. The ideal candidate will possess excellent leadership qualities and deep expertise in Python and data engineering technologies. This role requires hands-on experience in cloud technologies, data processing, and modern software practices. As the Lead Python Developer, you will lead a team of developers, drive data architecture, and ensure the successful delivery of high-quality data-driven solutions.
Key Responsibilities:
Leadership and Mentorship: o Lead, mentor, and inspire a team of Python developers and data engineers to achieve technical excellence. o Provide guidance on best practices, code reviews, and technical decision-making. o Collaborate with cross-functional teams, including data scientists, product managers, and analysts, to align on project goals and deliverables.
Technical Development: o Architect, design, and develop scalable data processing pipelines using Python and modern frameworks such as FastAPI or similar. o Lead the design and implementation of RESTful APIs for data access and manipulation, ensuring high performance, scalability, and security. o Oversee the deployment and management of data applications using cloud services such as AWS, focusing on data storage, processing, and retrieval. o Implement robust data testing strategies, including unit/integration testing, data validation, and quality checks. o Apply design patterns and software architecture principles to build maintainable and reusable data pipelines.
Data Engineering: o Develop and manage ETL (Extract, Transform, Load) processes to ensure efficient data processing and integration. o Work with large datasets, implementing data storage solutions, data lakes, and data warehouses using cloud platforms like AWS S3 and Redshift. o Optimize data workflows for performance and scalability, leveraging tools like Apache Spark, Hadoop, or similar technologies. o Collaborate with data scientists to deploy machine learning models into production, ensuring seamless data integration and processing.
Project Management: o Coordinate with stakeholders to define project scope, timelines, and deliverables. o Monitor project progress and ensure timely delivery of high-quality solutions. o Facilitate regular team meetings, retrospectives, and status updates.
Required Skills:
Technical Expertise: o Proficiency in Python with a strong understanding of Python frameworks (e.g., FastAPI, Flask, Django). o Hands-on experience with AWS, including Lambda, S3, Redshift, and other key services related to data engineering. o Expertise in containerization technologies like Docker. o Strong understanding of data processing, ETL processes, and big data technologies (e.g., Apache Spark, Hadoop). o Experience with design patterns and best practices in software and data development.
Leadership Skills: o Proven experience in leading and mentoring development and data engineering teams. o Excellent communication and collaboration skills. o Ability to manage multiple projects and prioritize tasks effectively.
Soft Skills: o Strong analytical and problem-solving abilities. o High level of initiative and a proactive approach to challenges. o Adaptability to work in a fast-paced and dynamic environment.