Skip to content Skip to sidebar Skip to footer

Why Scrapy Returns An Iframe?

i want to crawl this site by Python-Scrapy i try this class Parik(scrapy.Spider): name = 'ooshop' allowed_domains = ['http://www.ooshop.com/courses-en-ligne/Home.aspx']

Solution 1:

The website is protected by Incapsula, a website security service. It's providing your "browser" with a challenge that it must perform before being given a special cookie that gives you access to the website itself.

Fortunately, it's not that hard to bypass. Install incapsula-cracker and install its downloader middleware:

DOWNLOADER_MIDDLEWARES = {
    'incapsula.IncapsulaMiddleware': 900
}

Post a Comment for "Why Scrapy Returns An Iframe?"