Skip to content Skip to sidebar Skip to footer

Scrapy Scrapes Data But No Output To File

I've been getting blank json files despite successfully being able to execute most of the lines in scrapy shell. When I run the command scrapy crawl courses with my courses bot bei

Solution 1:

You write a wrong word. In parse function change items -> item.

defparse(self, response):
    soup = BeautifulSoup(response.body_as_unicode(), 'lxml')
    items = []
    for ...
        item = CoursesItem()
        item['title'] = course_info.text
        item['description'] = course_desc.text
        item['requirements'] = course_req.text
        yield items # -> item

`

Post a Comment for "Scrapy Scrapes Data But No Output To File"