Shulou(Shulou.com)06/03 Report--

This article introduces the relevant knowledge of "how to understand code objects and pyc files in Python compilation results". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

1. Python program execution process

Similar to java, Python compiles .py into bytecode and then executes it through a virtual machine. Both the compilation process and the virtual machine execution process are in python25.dll. The Python virtual machine is more abstract than java and farther away from the underlying layer.

The compilation process not only generates bytecode, but also includes constants, variables, space occupied by the stack and so on. The compilation process in Pyton generates code object PyCodeObject. Write PyCodeObject to the binary file, which is .pyc.

If necessary, writing A.pyc means whether the .py is only run once, and if the module of import is, it will definitely generate .pyc.

2. PyCodeObject object and .pyc file

The Python interpreter compiles a .py program into a PyCodeObject object, which is similar to the compilation principle.

The number of parameters of typedef struct {PyObject_HEAD int co_argcount; / / Code Block, such as the number of local variables in the parameter int co_nlocals; / / Code Block of a function int co_stacksize; / / the stack space int co_flags; / / N PyObject * co_code needed to execute this segment of Code Block / / the byte code compiled by Code Block exists the PyObject * co_consts; / / PyTupleObject object in the form of PyStringObject, saves the constant PyObject * co_names; / / PyTupleObject object in Code Block, and saves the local variable name set PyObject * co_freevars in all symbols PyObject * co_varnames; / / Code Block in Code Block. / / what is needed to implement the closure PyObject * co_cellvars; / / the name of the local variable name referenced by the nested function inside PyObject * co_filename; / / Code Block corresponds to the full path to the .py file PyObject * co_name; / / Code Block, usually the function name or class name int co_firstlineno / / the corresponding relationship between the starting line PyObject * co_lnotab; / / byte code of Code Block in the corresponding .py file and the line number of source code in the .py file. There is void * co_zombieframe; PyObject * co_weakreflist;} PyCodeObject in the form of PyStringObject.

A Code Block generates a PyCodeObject and enters a namespace to become a Code Block. After the following .py file is compiled, three PyCodeObject will be generated, one corresponding to the entire .py file, one corresponding to Class An and one corresponding to def Fun. The three code objects are actually nested, and the last two code objects are in the co_consts property of the first code object. In fact, the bytecode is located in co_code.

Class A: passdef Fun (): passa = A () Fun ()

The pyc file consists of three parts:

(1) four-byte Magic int, indicating pyc version information

(2) the four-byte int is the time when the pyc is generated. If it is different from the py file time, it will be regenerated.

(3) serialized PyCodeObject object.

3. Generation of pyc file

The functions that write to the pyc file include the following steps:

PyMarshal_WriteLongToFile (pyc_magic, fp, Py_MARSHAL_VERSION); / write version information PyMarshal_WriteLongToFile (0L, fp, Py_MARSHAL_VERSION); / / write time information PyMarshal_WriteObjectToFile ((PyObject *) co, fp, Py_MARSHAL_VERSION); / / write PyCodeObject object

The key is the writing of the code object:

{WFILE wf; wf.fp = fp;. W_object (x & wf);}

A WFILE structure is used, which can be thought of as an encapsulation of FILE * fp:

Typedef struct {FILE * fp; int error; int depth; PyObject * strings; / / stores strings, written in dict form and read out in list form} WFILE

The key lies in the w_object () function:

Static void w_object (PyObject * v, WFILE * p) {if (v = = NULL). Else if (PyInt_CheckExact (v))... Else if (PyFloat_CheckExact (v))... Else if (PyString_CheckExact (v))... Else if (PyList_CheckExact (v))... }

The essence of w_code is to choose different strategies according to different object types, such as tuple objects:

Else if (PyTuple_CheckExact (v)) {w_byte (TYPE_TUPLE, p); n = PyTuple_Size (v); W_SIZE (n, p); for (I = 0; I

Fp) / / for writing type static void w_long (long x, WFILE * p) {/ / for writing digits w_byte ((char) (x & 0xff), p); / / essentially storing a digital w_byte with four bytes ((char) ((x > 8) & 0xff), p); w_byte ((char) ((x > 16) & 0xff), p) W_byte ((char) ((x > 24) & 0xff), p);} static void w_string (char * s, int n, WFILE * p) {/ / is used to write the string fwrite (s, 1, n, p-> fp);}

Because structure information is lost after serialization writes to the file, type information w_byte is written when each object is written:

# define TYPE_INT 'i'#define TYPE_LIST' ['# define TYPE_DICT'{'# define TYPE_CODE'c'

Because Python is an object, w_object (PyObject*) can choose different writing methods for different types, subdivide it into PyInt_Object or PyString_Object, and write using w_long or w_string.

The numbers are relatively simple:

Else if (PyInt_CheckExact (v)) {w_byte (TYPE_INT, p); w_long (x, p);}

Strings are more complex:

Else if (PyString_CheckExact (v)) {if (p-> strings & & PyString_CHECK_INTERNED (v)) {PyObject * o = PyDict_GetItem (p-> strings, v); / / get the sequence number if (o) {/ / inter object's non-first write long w = PyInt_AsLong (o) in strings First write of w_byte (TYPE_STRINGREF, p); w_long (w, p); goto exit;} else {/ / intern object int ok; ok = o & & PyDict_SetItem (p-> strings, v, o) > = 0 Py_XDECREF (o); w_byte (TYPE_INTERNED, p);}} else {/ / write normal string w_byte (TYPE_STRING, p);} n = PyString_GET_SIZE (v); W_SIZE (n, p) W_string (PyString_AS_STRING (v), n, p);}

(1) if you write an ordinary string, write the string type information "S", and then write the string length and string value.

(2) if you write an inter string, first look it up in the strings of WFILE:

(a) if found, write the reference type information "R" and then write the serial number

(B) if not found, the created object is put into strings, and the intern type information "t" is written, and then the string length and string value are written.

If you write "efei", "snow" and "efei" in turn, it will be as follows:

When reading from a pyc file, rely on list, so the serial number can be used.

This is the end of the content of "how to understand code objects and pyc files in Python compilation results". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.