How to solve the problem of Scrapy starting error invalid syntax
Today, I will talk to you about how to solve the problem of invalid syntax in Scrapy startup error. Many people may not know much about it. In order to let you know more, Xiaobian summarizes the following contents for you. I hope you can gain something according to this article.
Scrapy start error invalid syntax error message
Traceback (most recent call last): File "d:\python\lib\runpy.py",
line 193, in _run_module_as_main
"main", mod_spec) File "d:\python\lib\runpy.py", line 85, in run_code
exec(code, run_globals) File "D:\python\Scripts\scrapy.exe_main.py", line 9, in File
"d:\python\lib\site-packages\scrapy\cmdline.py", line 150, in execute
_run_print_help(parser, _run_command, cmd, args, opts) File "d:\python\lib\site-packages\scrapy\cmdline.py", line 90, in
_run_print_help
func(*a, **kw) File "d:\python\lib\site-packages\scrapy\cmdline.py", line 157, in
_run_command
cmd.run(args, opts) File "d:\python\lib\site-packages\scrapy\commands\crawl.py", line 57, in
run
self.crawler_process.crawl(spname, **opts.spargs) File "d:\python\lib\site-packages\scrapy\crawler.py", line 170, in crawl
crawler = self.create_crawler(crawler_or_spidercls) File "d:\python\lib\site-packages\scrapy\crawler.py", line 198, in
create_crawler
return self._ create_crawler(crawler_or_spidercls) File "d:\python\lib\site-packages\scrapy\crawler.py", line 203, in
create_crawler
return Crawler(spidercls, self.settings) File "d:\python\lib\site-packages\scrapy\crawler.py", line 55, in init
self.extensions = ExtensionManager.from_crawler(self) File "d:\python\lib\site-packages\scrapy\middleware.py", line 58, in
from_crawler
return cls.from_settings(crawler.settings, crawler) File "d:\python\lib\site-packages\scrapy\middleware.py", line 34, in
from_settings
mwcls = load_object(clspath) File "d:\python\lib\site-packages\scrapy\utils\misc.py", line 44, in
load_object
mod = import_module(module) File "d:\python\lib\importlib_init.py", line 127, in import_module
return _bootstrap._ gcd_import(name[level:], package, level) File "
The activation commands are as follows:
scrapy crawl first_spider -o first_spider.csv Solution
After querying, it is found that async is a reserved word in python 3.7, so the problem can be solved by modifying async in twisted\conch\manhole.py to isasync.
Use scrapy to do some of the pits encountered by crawlers
Some invalid syntax encountered using xpath and CSS
1. Ambiguity of quotation marks: single quotation marks should be used for the outermost part
Error:
Correct: Place the positioned xpath in single quotes
2. using keywords as variables.
Error:
Correct: Note that keywords cannot be variables
3. When entering the shell window, url added ""
Error:
Correct: Remove the quotation marks from url
4. Use xpath to match relevant information and return an empty list
Error: Because the URL is wrong at the beginning, no matter whether the matching xpath is correct or not, it will return []
Correct:
After reading the above, do you have any further information on how to solve the problem of invalid syntax when Scrapy starts? If you still want to know more knowledge or related content, please pay attention to the industry information channel, thank you for your support.
