Shulou(Shulou.com)06/01 Report--

This article mainly shows you "how to use POSTGRESQL CDC". The content is simple and easy to understand, and the organization is clear. I hope it can help you solve your doubts. Let Xiaobian lead you to study and learn this article "how to use POSTGRESQL CDC".

First of all, POSTGRESQL itself has two replication methods: streaming replication and logical replication. What we want to talk about today is the CDC function in logical replication. The implementation of CDC POSTGRESQL needs to have two functions: 1 to obtain the change log (WAL) of the database and obtain the data consistency snapshot before the change log, and finally to save the offset of the consumer. Logical replication is a new feature starting with PG10.

1 First, let's take a look at what PG CDC is called through a practice (to ignore some database infrastructure)

First we need to create a logical replication slot and use a decoding plug-in for test_decoding given by the system.

After creating the replication slot, we start some related database operations. From the following figure, we can see that the DDL operation we want is not parsed, only the begin commit is recorded.

Let's do a DML operation, obviously related records can be decoded and see what the database just did.

Of course, we also need to use pg_recvlogical to capture data and more applications. If we output these outputs directly to dynamic files, then the CDC records of a database will be available. Later in the database found which table records are not right, you can use this method to do early log records.

At the same time, we can also replace the decoding tools in the logical copy slot. Let's see, the data input in PG is directly decoded into JSON data after passing through the logical copy slot.

Of course, if we use other parsing plug-ins, we can also translate the statements directly. If we keep these statements, we can do something similar to UNDO to roll back the executed statements. From this, we can better understand why the primary key should be set for the traditional database in the data pipleline. This may facilitate the operation of updating the captured data or determining whether to insert or update it.

Through the above figure, in fact, we can build a simple CDC system of our own, by recording that if the design of the table is more complete, a logging system of CDC will be POSTGRESQL database data at any point in time UNDO, as a strong support.

The above is "POSTGRESQL CDC how to use" all the content of this article, thank you for reading! I believe that everyone has a certain understanding, hope to share the content to help everyone, if you still want to learn more knowledge, welcome to pay attention to the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.