Back
Specialized Scrapy skillset plus mid-level experience amplifies competition despite smaller company brand.
Skills are transferable to other backend/data engineering roles but remain somewhat domain-specific to scraping.
Mandatory 2+ years, mandatory Scrapy expertise and anti-bot skills create strict shortlisting filters.
Login to See Your Match Score
Create a free account or log in to unlock your CV match score across:
Design, build, deploy, and maintain scalable, high-performance web scraping spiders using the Scrapy Framework.
Implement anti-bot bypass strategies including proxy rotation, user-agent spoofing, CAPTCHA solving, and session management to ensure reliable data extraction.
Optimize scraper performance, ensure concurrency and scalability, handle JavaScript-heavy sites using headless browsers when necessary, and maintain data accuracy and consistency.
Minimum 2 years of professional experience in web scraping, crawling, or data extraction.
Strong hands-on experience with Scrapy framework including spiders, middlewares, pipelines, and concurrency tuning.
Proficiency in Python including OOP, debugging, and modular coding.
Experience with anti-bot techniques, proxies, CAPTCHA solving, HTTP headers and cookies management, and working with HTML, DOM, CSS selectors, XPath, and JavaScript-rendered content.
Experience managing large-scale, high-throughput scraping projects with frequent site structure changes and anti-bot challenges.
Demonstrated ability to integrate scraping with various databases (MySQL, PostgreSQL, MongoDB) and REST APIs to optimize data collection workflows.
Familiarity with asynchronous Python frameworks, headless browsers (Playwright, Selenium), and scalable, maintainable codebases in a production environment.