Shulou(Shulou.com)06/01 Report--

This article will explain what Pulsar Schema means for you in detail. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

Basic concepts of Pulsar Schema

Schema is defined on topic.

The update strategy for Schema is defined in the namespace.

The following picture is an example of schema JSON format

Field description nameSchema name (found to be the same as topic name) typeSchema type, commonly used JSON format, other details see official document schemaSchema data is a sequence of 8-bit unsigned bytes and schema type specific, this attribute may be the internal use of schemaDefinition specific business data structure properties to store user-defined attributes

Generation of two Schema

Generate schema on Topic in three ways

1 producer producer

2 consumer messengers

3 manage API

There is no schema on topic by default. Whoever accesses Topic first in the above three ways will be the first version according to his / her schema structure.

Three JAVA management API

1 establish a connection

PulsarAdmin admin = PulsarAdmin.builder () .serviceHttpUrl (url) .build ()

Schemas schemas=admin.schemas (); / / get the schemas management object

2 View schmea

Specify all versions of schema on topic

List allSchemaInfo=schemas.getAllSchemas ("topic name")

Specify the latest version of schema on topic

SchemaInfo snedschemaInfo=schemas.getSchemaInfo ("topic name")

3 delete schmea

Schemas.deleteSchema ("topic name")

4 create schema

Schemas.createSchema ("topic name", schemaInfo)

About the Generation of schemaInfo object

1) you can use the RecordSchemaBuilder type

RecordSchemaBuilder recordSchemaBuilder = SchemaBuilder.record ("schemaName")

RecordSchemaBuilder.field ("ww") .type (SchemaType.INT32)

SchemaInfo schemaInfo = recordSchemaBuilder.build (SchemaType.JSON)

2) you can also use Schema

SchemaInfo schemaInfo=Schema.JSON (UserModel.class) .getSchemaInfo ()

Four schema update policies

Set schema policy method

Namespaces namespaces=admin.namespaces ()

Namespaces.setSchemaAutoUpdateCompatibilityStrategy (Namespace name, SchemaAutoUpdateCompatibilityStrategy. Attributes)

Whether to force schema validation

Namespaces.setSchemaValidationEnforced (Namespace name, false)

Suppose you have a theme with three scenarios (V1, V2, and V3). V1 is the oldest and V3 is the latest.

Property description checks which schemaFull schema V3 and V2 are backward and forward compatible (default policy) the latest versions of FullTransitive schemas V3, V2 and V1 are backward and forward compatible with all previous versions of AlwaysCompatible can change the schema structure Different structures result in new versions all previous versions AutoUpdateDisabled disabled updating schema structure No Backward consumers using schema V3 can handle data written by producers using schema V3 or V2 the latest version of BackwardTransitive using schema V3 consumers can handle data written by producers using schema V3, V2 or V1. Consumers of all previous versions of Forward using schema V3 or V2 can handle data written by producers using schema V3. The latest version of ForwardTransitive using schema V3, V2 or V1 consumers can deal with the data prepared by producers using schema V3 all the previous versions of the article on "what is the meaning of Pulsar Schema" is shared here, I hope the above content can be of some help to you, so that you can learn more knowledge, if you think the article is good, please share it out for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.