Public summary
Seeking an experienced Senior Python Data Scraping Engineer for a freelance, part-time remote role contributing around 10-20 hours weekly to specialized data scraping workflows supporting AI and human hybrid systems. The role involves managing end-to-end data extraction from complex and dynamic websites, utilizing tools like Apify and OpenRouter alongside custom techniques, ensuring data accuracy, validation, and delivery of structured datasets such as CSV and JSON. Candidates should have 5+ years in data engineering or web scraping, strong Python skills with experience in handling JavaScript-rendered content, anti-bot mechanisms, cloud infrastructure (AWS), containerization (Docker), and AI-assisted workflows involving large language models.
Location and work setup
- Location
- Frankfurt
- Remote status
- Remote
- German requirement signal
- No German Required Detected
- Detected job language
- English
Salary
USD 45.00 - 45.00 hour
Responsibilities
- Manage complete data extraction workflows from complex, dynamic websites ensuring full coverage and accurate results. - Use internal and external tools, including Apify and OpenRouter, to accelerate and validate data collection. - Adapt scraping strategies for JavaScript-heavy and interactive web content. - Enforce strict data quality standards through validation, consistency checks, and formatting adherence. - Scale scraping operations efficiently for large datasets, monitoring failures and maintaining scraping stability. - Collaborate with AI systems and human agents to deliver actionable, structured data outputs.
Qualifications
- Minimum 5 years experience in data engineering, web scraping, automation, or software development. - Bachelor's or Master's degree in Engineering, Applied Mathematics, Computer Science, or related fields is a plus. - Proficient in Python-based web scraping tools and techniques, including handling dynamic content and complex site structures. - Experience with data cleaning, normalization, validation, and delivering structured datasets (CSV, JSON, Google Sheets). - Skilled in dealing with anti-bot measures and dynamic site changes at scale. - Hands-on experience with cloud services (AWS) and containerization (Docker). - Practical knowledge of large language model frameworks such as LangChain or OpenRouter. - Strong analytical skills, attention to detail, and ability to work independently. - Upper-intermediate (B2) or higher English proficiency. - GitHub profile is helpful but not mandatory.