We are looking for a skilled Python developer with extensive experience in mobile app automation and data extraction. The ideal candidate will be responsible for developing sophisticated crawlers to collect and analyze data from Android mobile applications. This role requires deep knowledge of mobile automation frameworks, data handling, and the ability to maintain and adapt to app updates.
Responsibilities:- Develop and maintain automated crawlers to extract data from specific Android mobile apps, ensuring the collection is comprehensive, accurate, and efficient.- Integrate with Celery for task scheduling and manage queues to handle large-scale data extraction tasks.- Implement mechanisms to bypass app security measures and avoid detection while maintaining the integrity of data collection.- Organize, store, and preprocess the collected data for easy retrieval and analysis.- Continuously monitor and update the crawler to adapt to changes in the mobile app environments, ensuring ongoing functionality.- Collaborate with the team to design and implement robust automation workflows using tools like UIAutomator2 or Appium.- Ensure the crawler’s performance is optimized and capable of handling high volumes of data extraction without errors or bottlenecks.
Requirements:- Proven experience in mobile app automation, specifically using frameworks like UIAutomator2, Appium, or similar tools.- Strong proficiency in Python, with experience in asynchronous programming and task scheduling using Celery.- Expertise in web scraping, data extraction, and handling large datasets efficiently.- Familiarity with Android development, including understanding APK structure and mobile app security mechanisms.- Experience with Git, Docker, and continuous integration/continuous deployment (CI/CD) pipelines.- Knowledge of techniques to avoid detection and bypass anti-scraping measures in mobile applications.- Strong problem-solving skills, attention to detail, and the ability to adapt to rapidly changing environments.- Experience with logging, monitoring, and error handling to ensure the crawler’s reliability and maintainability.