Shulou(Shulou.com)06/02 Report--

This article introduces the knowledge of "how to use ZeroMQ message library to share data between C and Python". Many people will encounter this dilemma in the operation of actual cases, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

ZeroMQ provides a simpler process:

Write a small piece of C code, read the data from the hardware, and then send out what you find as a message.

Python is used to write interfaces to realize the docking between new and old infrastructure.

Pieter Hintjens is one of the sponsors of the ZeroMQ project. He is an extraordinary person with interesting perspectives and works.

Prepare for

In this tutorial, you need to:

A C compiler (such as GCC or Clang)

Libzmq library

Python 3

Python package of ZeroMQ

Installation method on Fedora system:

$dnf install clang zeromq zeromq-devel python3 python3-zmq

Installation method on Debian and Ubuntu systems:

$apt-get install clang libzmq5 libzmq3-dev python3 python3-zmq

If you have any questions, refer to the installation guide for the corresponding project (with a link above).

Write hardware interface library

Because this is an imaginary scenario, this tutorial makes up an operation library that contains two functions:

Fancyhw_init () is used to initialize (imagined) hardware

Fancyhw_read_val () is used to return data read from hardware

Save the complete code of the library to the file libfancyhw.h:

# ifndef LIBFANCYHW_H#define LIBFANCYHW_H# include # include / / This is the fictitious hardware interfacing library void fancyhw_init (unsigned int init_param) {srand (init_param);} int16_t fancyhw_read_val (void) {return (int16_t) rand ();} # endif

This library can simulate the data you want to exchange between components implemented in different languages, with a random number generator in the middle.

Design C interface

Let's start with the library that contains the management data transfer, and gradually implement the C interface.

Required libraries

Start by loading the necessary libraries (see code comments for the role of each library):

/ / For printf () # include / / For EXIT_*#include / / For memcpy () # include / / For sleep () # include # include # include "libfancyhw.h" necessary parameters

Define the main function and the necessary parameters in subsequent procedures:

Int main (void) {const unsigned int INIT_PARAM = 12345; const unsigned int REPETITIONS = 10; const unsigned int PACKET_SIZE = 16; const char * TOPIC = "fancyhw_data"; Initialization

All libraries need to be initialized. The fictional one requires only one parameter:

Fancyhw_init (INIT_PARAM)

The ZeroMQ library requires solid initialization. First, define the object context, which is used to manage all sockets:

Void * context = zmq_ctx_new (); if (! context) {printf ("ERROR: ZeroMQ error occurred during zmq_ctx_new ():% s\ n", zmq_strerror (errno)); return EXIT_FAILURE;}

The socket used to send data is then defined. ZeroMQ supports several sockets, each of which has its own use. Using publish sockets (also known as PUB sockets), messages can be copied and distributed to multiple receivers. This allows you to have multiple receivers receive the same message. Messages without recipients will be discarded (that is, they will not be queued). The usage is as follows:

Void * data_socket = zmq_socket (context, ZMQ_PUB)

The socket needs to be bound to a specific address so that the client knows where to connect. In this example, the TCP transport layer is used (of course, there are other options, but TCP is a good default):

Const int rb = zmq_bind (data_socket, "tcp://*:5555"); if (rb! = 0) {printf ("ERROR: ZeroMQ error occurred during zmq_ctx_new ():% s\ n", zmq_strerror (errno)); return EXIT_FAILURE;}

Next, calculate some values to be used later. Notice the TOPIC in the following code, because the message sent by the PUB socket needs to be bound to a topic. Topics are used by recipients to filter messages:

Const size_t topic_size = strlen (TOPIC); const size_t envelope_size = topic_size + 1 + PACKET_SIZE * sizeof (int16_t); printf ("Topic:% s; topic size:% zu; Envelope size:% zu\ n", TOPIC, topic_size, envelope_size); send messages

Start a loop that sends messages and cycle REPETITIONS times:

For (unsigned int i = 0; I < REPETITIONS; iTunes +) {...

Before sending a message, populate a buffer of length PACKET_SIZE. This library provides 16-bit signed integers. Because the size of the space occupied by the int type in the C language is platform-dependent and is not a definite value, use the int variable of the specified width:

Int16_t buffer [pack _ SIZE]; for (unsigned int j = 0; j < PACKET_SIZE; jacks +) {buffer [j] = fancyhw_read_val ();} printf ("Read% u data values\ n", PACKET_SIZE)

The first step in preparing and sending a message is to create an ZeroMQ message, allocating the necessary memory space for the message. Blank messages are used to encapsulate the data to be sent:

Zmq_msg_t envelope; const int rmi = zmq_msg_init_size (& envelope, envelope_size); if (rmi! = 0) {printf ("ERROR: ZeroMQ error occurred during zmq_msg_init_size ():% s\ n", zmq_strerror (errno)); zmq_msg_close (& envelope); break;}

Now that the memory space is allocated, the data is saved in the ZeroMQ message "envelope". The function zmq_msg_data () returns a pointer to the top of the encapsulated data cache. The first part is the theme, followed by a space, and finally a binary number. The delimiter between the subject and binary data is a space character. If you need to traverse the cache, use type conversion and pointer algorithms. Thanks to the C language, it makes things straightforward. ) the practice is as follows:

Memcpy (zmq_msg_data (& envelope), TOPIC, topic_size); memcpy ((void*) ((char*) zmq_msg_data (& envelope) + topic_size), ", 1); memcpy ((void*) ((char*) zmq_msg_data (& envelope) + 1 + topic_size), buffer, PACKET_SIZE * sizeof (int16_t))

Send a message over data_socket:

Const size_t rs = zmq_msg_send (& envelope, data_socket, 0); if (rs! = envelope_size) {printf ("ERROR: ZeroMQ error occurred during zmq_msg_send ():% s\ n", zmq_strerror (errno)); zmq_msg_close (& envelope); break;}

Unencapsulate the data before using it:

Zmq_msg_close (& envelope); printf ("Message sent; i:% u, topic:% s\ n", I, TOPIC); cleanup

The C language does not provide garbage collection, so remember to clean it up after you use it. After sending the message, before you finish the program, you need to run the sweep code to free the allocated memory:

Const int rc = zmq_close (data_socket); if (rc! = 0) {printf ("ERROR: ZeroMQ error occurred during zmq_close ():% s\ n", zmq_strerror (errno)); return EXIT_FAILURE;} const int rd = zmq_ctx_destroy (context); if (rd! = 0) {printf ("Error occurred during zmq_ctx_destroy ():% s\ n", zmq_strerror (errno)); return EXIT_FAILURE;} return EXIT_SUCCESS; complete C code

Save the following complete interface code to the file with the local place name hw_interface.c:

/ / For printf () # include / / For EXIT_*#include / / For memcpy () # include / / For sleep () # include # include # include "libfancyhw.h" int main (void) {const unsigned int INIT_PARAM = 12345; const unsigned int REPETITIONS = 10; const unsigned int PACKET_SIZE = 16; const char * TOPIC = "fancyhw_data"; fancyhw_init (INIT_PARAM); void * context = zmq_ctx_new () If (! context) {printf ("ERROR: ZeroMQ error occurred during zmq_ctx_new ():% s\ n", zmq_strerror (errno)); return EXIT_FAILURE;} void * data_socket = zmq_socket (context, ZMQ_PUB); const int rb = zmq_bind (data_socket, "tcp://*:5555") If (rb! = 0) {printf ("ERROR: ZeroMQ error occurred during zmq_ctx_new ():% s\ n", zmq_strerror (errno)); return EXIT_FAILURE;} const size_t topic_size = strlen (TOPIC); const size_t envelope_size = topic_size + 1 + PACKET_SIZE * sizeof (int16_t); printf ("Topic:% s; topic size:% zu" Envelope size:% zu\ n ", TOPIC, topic_size, envelope_size); for (unsigned int i = 0; I < REPETITIONS; iTunes +) {int16_t buffer [packet _ SIZE]; for (unsigned int j = 0; j < PACKET_SIZE; jacks +) {buffer [j] = fancyhw_read_val ();} printf (" Read% u data values\ n ", PACKET_SIZE) Zmq_msg_t envelope; const int rmi = zmq_msg_init_size (& envelope, envelope_size); if (rmi! = 0) {printf ("ERROR: ZeroMQ error occurred during zmq_msg_init_size ():% s\ n", zmq_strerror (errno)); zmq_msg_close (& envelope); break } memcpy (zmq_msg_data (& envelope), TOPIC, topic_size); memcpy ((void*) ((char*) zmq_msg_data (& envelope) + topic_size), ", 1); memcpy ((void*) ((char*) zmq_msg_data (& envelope) + 1 + topic_size), buffer, PACKET_SIZE * sizeof (int16_t)) Const size_t rs = zmq_msg_send (& envelope, data_socket, 0); if (rs! = envelope_size) {printf ("ERROR: ZeroMQ error occurred during zmq_msg_send ():% s\ n", zmq_strerror (errno)); zmq_msg_close (& envelope); break;} zmq_msg_close (& envelope) Printf ("Message sent; i:% u, topic:% s\ n", I, TOPIC); sleep (1);} const int rc = zmq_close (data_socket); if (rc! = 0) {printf ("ERROR: ZeroMQ error occurred during zmq_close ():% s\ n", zmq_strerror (errno)); return EXIT_FAILURE;} const int rd = zmq_ctx_destroy (context) If (rd! = 0) {printf ("Error occurred during zmq_ctx_destroy ():% s\ n", zmq_strerror (errno)); return EXIT_FAILURE;} return EXIT_SUCCESS;}

Compile with the following command:

$clang-std=c99-I. Hw_interface.c-lzmq-o hw_interface

If there are no compilation errors, you can run this interface. Sweetly, ZeroMQ PUB sockets can run without any application sending or receiving data, which simplifies the complexity of usage because it does not limit the order in which processes are started.

Run the interface:

$. / hw_interfaceTopic: fancyhw_data; topic size: 12; Envelope size: 45Read 16 data valuesMessage sent; i: 0, topic: fancyhw_dataRead 16 data valuesMessage sent; i: 1, topic: fancyhw_dataRead 16 data values.

The output shows that the data has been sent through ZeroMQ, and what we need to do now is to let a program read the data.

Writing Python data processor

You are now ready to transfer data from the C program to the Python application.

Library

Two libraries are needed to help with data transfer. The first is the Python encapsulation of ZeroMQ:

$python3-m pip install zmq

The other is the struct library, which decodes binary data. This library is part of the Python standard library, so you don't need to install it using the pip command.

The first part of the Python program is to import these libraries:

Important parameters of import zmqimport struct

When using ZeroMQ, messages can only be sent to receivers with the same constant TOPIC definition:

Topic = "fancyhw_data" .encode ('ascii') print ("Reading messages with topic: {}" .format (topic)) initialization

Next, initialize the context and sockets. Use subscribe sockets (also known as SUB sockets), which are natural companions of PUB sockets. This socket also needs to match the theme when it is sent.

With zmq.Context () as context: socket = context.socket (zmq.SUB) socket.connect ("tcp://127.0.0.1:5555") socket.setsockopt (zmq.SUBSCRIBE, topic) I = 0. Receive messages

Starts an infinite loop waiting to receive a new message sent to the SUB socket. This loop ends when you press the Ctrl+C key combination or when an internal error occurs:

Try: while True:... # we will fill this in next except KeyboardInterrupt: socket.close () except Exception as error: print ("ERROR: {}" .format (error)) socket.close ()

The loop waits for the new message obtained by the recv () method, and then splits the received content from the first space character to get the topic:

Binary_topic, data_buffer = socket.recv () .split (b', 1) decode the message

Python does not know at this time that the subject is a string and uses the standard ASCII codec to decode it:

Topic = binary_topic.decode (encoding = 'ascii') print ("Message {: d}:" .format (I)) print ("\ ttopic:' {}'" .format (topic))

The next step is to use the struct library to read binary data, which converts binary segments into explicit values. First, the number of groups of values in the packet is calculated. The 16-bit signed integer used in this example corresponds to the h: in the struct format character

Packet_size = len (data_buffer) / / struct.calcsize ("h") print ("\ tpacket size: {: d}" .format (packet_size))

Once you know how many sets of data there are in the packet, you can define the format (such as "16h") by building a string containing the number of data groups and data types:

Struct_format = "{: d} h" .format (packet_size)

Convert a binary data string to a series of digits that can be printed directly:

Data = struct.unpack (struct_format, data_buffer) print ("\ tdata: {}" .format (data)) complete Python code

Here is the complete receiver of the Python implementation:

#! / usr/bin/env python3 import zmqimport struct topic = "fancyhw_data" .encode ('ascii') print ("Reading messages with topic: {}" .format (topic)) with zmq.Context () as context: socket = context.socket (zmq.SUB) socket.connect ("tcp://127.0.0.1:5555") socket.setsockopt (zmq.SUBSCRIBE, topic) I = 0 try: while True: binary_topic Data_buffer = socket.recv () .split (b'' 1) topic = binary_topic.decode (encoding = 'ascii') print ("Message {: d}:" .format (I)) print ("\ ttopic:' {}'" .format (topic)) packet_size = len (data_buffer) / / struct.calcsize ("h") print ("\ tpacket size: {: d}" .format (packet_size) ) struct_format = "{: d} h" .format (packet_size) data = struct.unpack (struct_format) Data_buffer) print ("\ tdata: {}" .format (data)) I + = 1 except KeyboardInterrupt: socket.close () except Exception as error: print ("ERROR: {}" .format (error)) socket.close ()

Save the above to a file named online_analysis.py. The Python code does not need to be compiled, you can run it directly.

The running output is as follows:

/ online_analysis.pyReading messages with topic: b'fancyhw_data'Message 0: topic: 'fancyhw_data' packet size: 16 data: (20946,-23616, 9865, 31416,-15911,-10845,-5332, 25662, 10955,-32501,-18717,-24490,-16511,-28861,24205,26568) Message 1: topic:' fancyhw_data' packet size: 16 data: (12505,31355) 14083,-19654,-9141, 14532,-25591, 31203, 10428,-25564,732,-79799529,-279822961030475). " This is the end of how to use the ZeroMQ message library to share data between C and Python. Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.