Public summary
A freelance opportunity for a Senior Python Data Scraping Engineer to join a cutting-edge project focusing on specialized data scraping workflows within a hybrid AI and human system. The role involves end-to-end data extraction, processing, validation, and delivery of structured datasets using tools like Apify and OpenRouter, requiring strong technical skills in Python, web scraping, automation, and experience with dynamic web content and cloud infrastructure. This part-time remote position expects 10-20 hours per week during active phases, offering compensation up to $45 per hour based on contribution and expertise.
Location and work setup
- Location
- Berlin
- Remote status
- Remote
- German requirement signal
- No German Required Detected
- Detected job language
- English
Salary
USD 45.00 - 45.00 hour
Responsibilities
Lead data scraping workflows across complex and dynamic websites ensuring complete and accurate extraction of data. Utilize internal tools and custom solutions to enhance data collection, validation, and task execution. Adapt scraping methods for JavaScript-rendered and interactive content. Maintain high data quality through validation and consistency checks. Scale operations for large datasets using efficient batching or parallelization techniques. Monitor and troubleshoot failures while ensuring stability against web structure changes.
Qualifications
Minimum 5 years of experience in data engineering, web scraping, automation, or software development. Strong proficiency in Python web scraping tools like BeautifulSoup and Selenium, including handling dynamic content (JavaScript, AJAX, infinite scroll) and APIs via proxies. Proven experience in data cleaning, normalization, and delivering structured datasets. Knowledge of anti-bot techniques and scalable scraping operations. Familiarity with cloud infrastructure such as AWS and containerization with Docker. Hands-on experience with LLM frameworks like LangChain or OpenRouter for automation. Detail-oriented with excellent problem-solving skills and ability to work independently. Upper-intermediate English proficiency (B2) or above required. A relevant degree and GitHub portfolio are advantageous.