Public summary
A remote freelance opportunity is available for a Senior Python Data Scraping Engineer to manage complex data extraction workflows within a hybrid AI-human system. The role requires expertise in Python web scraping, handling dynamic web content, data cleaning and validation, and experience with cloud infrastructure and containerization. Contributors can earn up to $37 per hour depending on experience and contribution pace. English proficiency at upper-intermediate level or higher is required. This role offers flexible hours and the chance to gain experience in a unique collaboration between AI and human expertise in a growing technology field.
Salary
USD 37.00 - 37.00 hour
Responsibilities
Manage end-to-end data extraction workflows on complex websites ensuring accuracy and data quality. Utilize internal tools like Apify and OpenRouter alongside custom methods to accelerate data collection and validation. Adapt scraping approaches for dynamic and interactive web content, enforce data quality standards including validation and consistency checks, and scale operations for large datasets using batching or parallelization. Monitor operations for failures and maintain scraper stability despite site changes.
Qualifications
Minimum of 5 years of relevant experience in data engineering, web scraping, automation, or software development. Bachelor's or Master's degree in Engineering, Applied Mathematics, Computer Science, or related fields preferred. Strong skills in Python web scraping libraries such as BeautifulSoup and Selenium, including handling dynamic content like JavaScript and AJAX. Proven ability in extracting data from complex and inconsistent web structures, experience with data cleaning and validation to deliver structured datasets. Knowledge of anti-bot techniques and handling dynamic site layouts at scale. Familiarity with cloud services like AWS and containerization tools such as Docker. Hands-on experience with large language model frameworks like LangChain or OpenRouter applied to automation tasks. High attention to detail, independent troubleshooting skills, and upper-intermediate English proficiency required. A GitHub portfolio is a plus.