Shulou(Shulou.com)06/01 Report--

The following content mainly brings you python3 multi-process and collaborative processing of MySQL data handouts, the knowledge here is slightly different from books, are professional and technical personnel in the process of contact with users, summed up, has a certain experience sharing value, hope to bring help to the majority of readers.

The multi-process + co-program of python3 processes the data of MySQL. The main logic is to pull the data of MySQL, and then use flashtext to match keywords. When saving back to MySQL, the code is as follows (async_mysql.py):

Import timeimport asyncioimport randomfrom concurrent.futures import ProcessPoolExecutor as Poolimport aiomysqlfrom flashtext import KeywordProcessorimport clickclass AttrDict (dict): "you can use". " Get attribute Return None dictionary "def _ _ getattr__ (self, name): try: return self [name] except KeyError: return None def _ _ setattr__ (self, name) without this attribute" Value): self [name] = valueclass AttrDictCursor (aiomysql.DictCursor): "inheriting aiomysql dictionary cursor"dict_type = AttrDictclass MultiProcessMysql (object):" processing MySQL data with multiprocess and co-programming "def _ init__ (self, workers=2, pool=10, start=0)" End=2000): "" the parameters of the first paragraph need to follow the demand "self.host =" 192.168.0.34 "self.port = 3306 self.user =" root "self.password =" root "self.db =" mydb "self.origin_table =" judgment_main_etl "# main self.dest_table =" laws_finance1 " Self.s_sql = f "select uuid Court_idea, judge_result, reason, plt_claim, dft_rep, crs_exm from {self.origin_table} where s

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.