Public summary
A freelance remote opportunity for a senior-level Python Data Scraping Engineer to lead complex web data extraction and processing workflows for a cutting-edge AI and human hybrid project. The role requires expertise in Python-based web scraping, handling dynamic web content, ensuring data quality and accuracy, and working with automation tools and LLM frameworks. Candidates should have 5+ years of relevant experience, strong technical skills, and good English proficiency (B2 or above). The position involves approximately 10-20 hours per week with pay rates up to $45 per hour, depending on contribution level and project activity.
Location and work setup
- Location
- Hamburg
- Remote status
- Remote
- German requirement signal
- No German Required Detected
- Detected job language
- English
Salary
USD 45.00 - 45.00 HOUR
Responsibilities
Lead end-to-end data extraction workflows on complex websites, ensuring accuracy and full dataset delivery; utilize and integrate internal and custom tools for efficient data collection, validation, and task execution; adapt scraping techniques to handle dynamic and interactive web sources including JavaScript-rendered content; maintain data quality through thorough validation and consistency checks; scale scraping processes with batching or parallelization while monitoring and mitigating failures; maintain robustness against changes in website structures.
Qualifications
At least 5 years of experience in data engineering, web scraping, automation, or software development; proficiency in Python web scraping tools such as BeautifulSoup and Selenium; experience handling dynamic content (JavaScript, AJAX) and using APIs with proxies; solid skills in data cleaning, normalization, and validation; experience overcoming anti-bot and dynamic site challenges at scale; familiarity with cloud services like AWS and containerization via Docker; hands-on experience with LLM frameworks like LangChain or OpenRouter; strong analytical and problem-solving abilities; self-motivated and able to work independently; English proficiency at upper-intermediate level or higher; a relevant degree is a plus; GitHub portfolio is advantageous.