Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to realize two-way communication between Node and Python". In daily operation, I believe many people have doubts about how to realize two-way communication between Node and Python. Xiaobian consulted all kinds of materials and sorted out simple and easy operation methods. I hope to help you answer the doubts about "how to realize two-way communication between Node and Python"! Next, please follow the small series to learn together!

Third-party data vendors encapsulate data and Python together, and can only query data by calling Python methods. If Python method calls can be implemented under Node simple encapsulation, they can be quickly launched and save development costs.

The simplest and crudest way to communicate is for Nodejs to call a Python script and then get the output of the child process, but since Python starts and loads the package every time, it is a long process, so optimize the process.

process communication

index.py

#Encapsulated Python package, huge from mb import MB#Query mbe.get("1.0.1.0")from the package

index.js

const { spawn } = require("child_process");const ls = spawn("python3", ["index.py"]);ls.stdout.on("data", (data) => { console.log(`stdout: ${data}`);});ls.stderr.on("data", (data) => { console.error(`stderr: ${data}`);});ls.on("close", (code) => { console.log(`child process exited with code $[code]`);});

Spawn Python child processes via child_process.spawn, listening for stdout output. The above method is also an example in the official document. Currently, there are two problems with this example:

Nodejs does not send data to Python

After Nodejs is called, the Python child process will exit; the next query needs to call Python command again to load files and query data; it is impossible to load memory once and use it multiple times.

process bidirectional communication

The premise of ensuring one-time data loading and multiple use is that Python processes cannot exit after starting. Python process exit is because there is nothing to do, so common means are loop, sleep, listening port, these means can be translated into synchronous blocking tasks, synchronous non-blocking tasks, the least expensive of which is synchronous non-blocking tasks, and then you can think of Linux select, epoll, simple search Python epoll, it seems that there are native packages.

index.py Listen for stdin with epoll

import sysimport fcntlimport selectfrom mb import MBimport jsonmbe = MB("./ data")# epoll mod fd = sys.stdin.fileno()epoll = select.epoll()epoll.register(fd, select.EPOLLIN)try: while True: events = epoll.poll(10) #synchronous non-blocking data = "" for fileno, event in events: data += sys.stdin.readline() #Get data via standard input if data == "" or data == "": continue items = xxx #number processing for item in items: result = mbe.get(item) sys.stdout.write(json.dumps(result, ensure_ascii=False) +"") #Write to standard output sys.stdout.flush() #buffer flush finally: epoll.unregister(fd) epoll.close()

index.js -Send data via stdin

const child_process = require("child_process");const child = child_process.spawn("python3", ["./ base.py"]);let callbacks = [], chunks=Buffer.alloc(0), chunkArr = [], data = "", onwork = false; // buffer cannot be dynamically expanded child.stdout.on("data", (chunk) => { chunkArr.push(chunk) if (onwork) return; onwork = true; while(chunkArr.length) { chunks = Buffer.concat([chunks, chunkArr.pop()]); const length = chunks.length; let trunkAt = -1; for(const [k, d] of chunks.entries()) { if (d == "0x0a") { // 0a End data += chunks.slice(trunkAt+1, trunkAt=k); const cb = callbacks.shift(); cb(null, data === "null" ? null : data ) data = ""; } } if (trunkAt

{ if (callbacks.length) child.stdin.write(``); // Nodejs standard input and output have no flush method, only hack, python cannot get the latest in time after writing}, 500)exports.getMsg = function getMsg(ip, cb) { callbacks.push(cb) child.stdin.write(`${ip}`); //Write data to standard input of child process}

Python communicates with Nodejs via stdio; Python resides in memory via epoll listening to stdin, running for a long time.

existing problems

Nodejs takes the standard output as the execution result, so Python can only write the execution result to the standard output, and there can be no additional print information.

Nodejs standard input does not have flush method, so Python side event trigger is not timely enough, currently through the Nodejs side regularly send null information to hack implementation

Buffer can not be dynamically expanded, no C language pointer is easy to use, write ugly when parsing stdout

At this point, the study of "how to realize two-way communication between Node and Python" is over, hoping to solve everyone's doubts. Theory and practice can better match to help everyone learn, go and try it! If you want to continue learning more relevant knowledge, please continue to pay attention to the website, Xiaobian will continue to strive to bring more practical articles for everyone!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.