Shulou(Shulou.com)05/31 Report--

This article mainly introduces "what are the three core configuration files of Mycat". In daily operation, I believe many people have doubts about what the three core configuration files of Mycat are. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the doubts of "what are the three core configuration files of Mycat?" Next, please follow the editor to study!

Three core configuration files of Mycat: server.xml, schema.xml and rule.xml.

Server.xml

Server.xml saves almost all the system configuration information needed by mycat. Its mapping class directly within the code is the SystemConfig class. Now let's introduce the configuration in this file.

User tag

Test TESTDB true

There are not many tags in server.xml, and this tag is mainly used to define the users and permissions to log in to mycat. For example, in the above example, I defined a user with the user name test and password test, and only TESTDB is accessible to the schema. If I define more than one schema in schema.xml, then this user cannot access other schema. From the point of view of the mysql client, it is not possible to switch to this other database using use. If you use the use command, mycat will report this error: ERROR 1044 (HY000): Access denied for user 'test' to database' xxx' this tag nested property tag is a specific declared attribute value, as in the example above. We can modify the name attribute of the user tag to specify the user name, modify the text in the password to change the password, change the readOnly to true or false to restrict whether the user is only readable, modify the text in the schemas to control the schema; that the user can ask, modify the text in the schemas to control the schema that the user can access, and access multiple schema at the same time, separated, for example:

TESTDB,db1,db2

System tag

All the property tags nested within this tag are related to the system configuration. Please note that I will omit the tag property to directly use the value in the tag's name attribute to describe the purpose of this attribute.

DefaultSqlParser attribute

Because mycat was originally the time for Foundation DB's sql parser, and then added Druid's parser. So this property is used to specify the default parser. Currently available values are: druidparser and fdbparser. You can choose one of them when using it, and druidparser is generally used at present.

Processors attribute

This property is mainly used to specify the number of threads available to the system, and the default value is the value returned by the Runtime.getRuntime (). AvailableProcessors () method. It mainly affects the attributes of processorBufferPool, processorBufferLocalPercent and processorExecutor. The number of NIOProcessor is also defined by this property, so you can increase this property appropriately when tuning.

ProcessorBufferChunk attribute

This property specifies the size of each allocated Socket Direct Buffer, which defaults to 4096 bytes. This property also affects the length of the buffer pool.

ProcessorBufferPool attribute

This property specifies the bufferPool calculation scale value. Since buffer is needed for every read and write operation of NIO, a certain length of buffer pool will be established during system initialization to speed up the efficiency of reading and writing and reduce the time of establishing buffer.

Two major buffer pools

BufferPool and ThreadLocalPool

BufferPool is composed of ThreadLocalPool. Every time you get a buffer from BufferPool, you will first get the buffer in ThreadLocalPool, and then get the buffer in BufferPool after you miss. In other words, ThreadLocalPool is used as a secondary cache for BufferPool, which is used internally by each thread. Of course, there are some restrictions that require the name of the thread to start with $_. However, buffer on BufferPool is shared by every NIOProcessor. The default value of this property is:

The default bufferChunkSize (4096) * processors attribute * total length of 1000 BufferPool = bufferPool / bufferChunk.

If bufferPool is not an integral multiple of bufferChunk, then the total length is the quotient + 1 calculated previously, assuming that the number of system threads is 4, and the rest is the default value of the property:

Total length of bufferPool = 4096 * 4 * 1000 BufferPool: 4000 = 16384000 / 4096

ProcessorBufferLocalPercent attribute

ThreadLocalPool was mentioned earlier. This property is used to control the size of the pool allocated, but it is not an exact value, but also a proportional value. The default value for this property is 100.

Percentage of thread cache = bufferLocalPercent / processors attribute.

For example, if the system can run four threads at the same time, using the default value, the percentage of each thread according to the formula is 25. Finally, according to this percentage to calculate the specific.

The length formula of ThreadLocalPool is as follows:

Length of ThreadLocalPool = percentage of thread cache * BufferPool length / 100

Assume that the length of the BufferPool is 4000, and the rest remains the default value. Then the final length of the ThreadLocalPool on each thread establishment is: 1000 = 25 * 4000 / 100.

ProcessorExecutor attribute

This property is mainly used to specify the size of the businessExecutor fixed thread pool shared on the NIOProcessor. Mycat commits tasks to this thread pool when it needs to deal with some asynchronous logic. This connection pool is not used very frequently in the new version, so you can set a smaller value.

SequnceHandlerType attribute

Specifies the type that uses the Mycat global sequence. 0 is the local file mode, 1 is the database mode. The default is to use local files, which are mainly used for testing.

TCP connection related properties

StandardSocketOptions.SO_RCVBUF StandardSocketOptions.SO_SNDBUF StandardSocketOptions.TCP_NODELAY

The above three attributes are respectively:

FrontSocketSoRcvbuf default value: 1024 * 1024

Default value of frontSocketSoSndbuf: 4 * 1024 * 1024

FrontSocketNoDelay default value: 1

Default value of backSocketSoRcvbuf: 4 * 1024 * 1024

BackSocketSoSndbuf default value: 1024 * 1024

BackSocketNoDelay default value: 1

Set the front and back end TCP connection parameters separately. Mycat uses these parameters to initialize the connection each time the front-end and back-end connections are established. You can resize these buffer appropriately according to the requirements of the system. For the definition of TCP connection parameters, you can see Javadoc. Properties related to MySQL connections some of the properties involved in initializing mysql front-and back-end connections:

PacketHeaderSize: specifies the header length in the Mysql protocol. The default is 4.

MaxPacketSize: specifies the maximum length of data that the Mysql protocol can carry. The default is 16m.

IdleTimeout: specifies the idle timeout for the connection. When a connection initiates an idle check, it is found that the idle time has exceeded since the last use, then the connection will be recycled and will be closed directly. The default is 30 minutes.

Charset: the initialization character set of the connection. The default is utf8.

TxIsolation: the initialization transaction isolation level of the front-end connection, which is only used during initialization. Later, the back-end database connection is synchronized according to the attributes passed by the client. The default is REPEATED_READ.

The timeout of sqlExecuteTimeout:SQL execution, Mycat will check the last time SQL was executed on the connection, and if it exceeds this time, the connection will be closed directly. The default time is 300 seconds.

Cycle interval related attributes

There are several periodic tasks in mycat to handle some of the work I need asynchronously. These attributes are also indispensable in the process of system tuning.

ProcessorCheckPeriod: the interval between cleaning up the front and rear idle, timeout, and closing connections on the NIOProcessor. The default is 1 second.

DataNodeIdleCheckPeriod: the interval between idle and timeout checks for backend connections. The default is 60 seconds.

DataNodeHeartbeatPeriod: the interval between heartbeats for all backend read and write libraries. The default is 10 seconds.

Service related attributes

Here is a service-related attribute that mainly affects the perception of myact by external systems.

BindIp: the IP address that the mycat service listens to. The default value is 0.0.0.0.

ServerPort: defines the usage port of mycat. The default value is 8066.

ManagerPort: defines the management port of mycat. The default value is 9066.

Schema.xml

As one of the important configuration files in MyCat, schema.xml manages the logic library, table, slicing rules, DataNode and DataSource of MyCat. Understanding these configurations is a prerequisite for the correct use of MyCat. Here, the file is parsed layer by layer.

Select user ()

Schema tag

DataNode

This attribute is used to bind a logical library to a specific database. If this attribute is defined, the logical library will not work in sub-library and sub-table mode. In other words, all operations on this logic library will be directly applied to the bound dataNode, and the schema can be used for read-write separation and master-slave switching, as follows:

So now the USERDB is bound to the specific database configured by dn1, and you can access the database directly. Of course, this property can only be configured to bind to one database, not multiple dn.

CheckSQLschema

When this value is set to true, if we execute the statement select * from TESTDB.travelrecord;, MyCat will change the statement to select * from travelrecord;. That is, remove the characters that represent schema and avoid sending them to the back-end database Executive Times (ERROR 1146 (42S02): Table 'testdb.travelrecord' doesn't exist). However, even if the value is set to true, if the statement contains a name other than the name specified by schema, for example: select * from db1.travelrecord;, then MyCat will not delete the db1 field, and if the library is not defined, it will report an error, so it is best not to use this field in the SQL statement.

SqlMaxLimit

When the value is set to a numeric value. For each executed SQL statement, if no limit statement is added, the MyCat will automatically add the corresponding value. For example, if you set the value to 100, the effect of executing select * from TESTDB.travelrecord; is the same as executing select * from TESTDB.travelrecord limit 100;. If this value is not set, MyCat will display all the information queried by default, resulting in too much output. Therefore, in normal use, it is recommended to add a value to reduce excessive data returns. Of course, the size of the limit is also explicitly specified in the SQL statement, which is not constrained by this attribute.

Table tag

The Table tag defines the logical tables in MyCat, and all tables that need to be split need to be defined in this tag.

Name attribute

Define the table name of the logical table, which is the same as the name I specified by executing the create table command in the database. The name defined in the same schema tag must be unique.

DataNode attribute

Define the dataNode to which this logical table belongs, and the value of this attribute needs to correspond to the value of the name attribute in the dataNode tag. If you need to define too many dn, you can use the following methods to reduce the configuration:

It is important to note that the real database name specified by the database attribute needs to be added later. For example, in the above example, I need to create a database named dbs0 to dbs99 on the real mysql.

Rule attribute

This attribute is used to specify the name of the rule to be used by the logical table, which is defined in rule.xml and must correspond to the value of the name attribute attribute in the tableRule tag.

PrimaryKey attribute

The logical table corresponds to the primary key of the real table. For example, if the rule of sharding uses a non-primary key for sharding, then when using the primary key query, the query statement will be sent to all configured DN. If you use this attribute to configure the primary key of the real table. Is it difficult for MyCat to cache the information of the primary key and the specific DN, so when you use the non-primary key to query again, there will be no broadcast query, and the statement will be sent directly to the specific DN? however, although the attribute is configured, if the cache does not hit, the statement will be sent to the specific DN to get the data.

Type attribute

This attribute defines the type of logical table, which currently has only two types: global table and normal table. Corresponding configuration: global table: global. Regular tables: all tables that do not specify this value as globla.

AutoIncrement attribute

MySQL for non-self-growing primary keys, using last_insert_id () will not return a result, only 0. Therefore, only tables that define a self-growing primary key can return the primary key value with last_insert_id (). Mycat currently provides self-growing primary key functionality, but if there is no auto_increment defined in the data table on the corresponding mysql node, then calling last_insert_id () on the mycat layer will not return a result. Since the insert operation does not include the sharding key, mycat will first take down the global sequence corresponding to the table and then assign a value to the sharding key. Only in this way can it be inserted into the database normally, and finally the inserted sharding key value will be returned using last_insert_id (). If you want to use this feature, it is best to use the global sequence of the database schema. Use autoIncrement= "true" to specify that the table uses a self-growing primary key so that mycat does not throw exceptions that cannot be found by the sharding key. Use autoIncrement= "false" to disable this feature, of course, you can also delete this attribute directly. The default is disabled.

NeedAddLimit attribute

Specify whether the table needs to automatically add limit restrictions to each statement. Due to the use of sub-database and sub-tables, the amount of data can sometimes be very large. Execute the query statement at this time, if you happen to forget to add a quantity limit. Then it will be enough to wait a little while to query all the data. So, mycat automatically adds LIMIT 100s for us. Of course, if there is a limit in the statement, it will not be added again. This property defaults to true, or you can set it to false` to disable the default behavior.

ChildTable tag

The childTable tag is used to define the child table of the Emurr shard. Associate with the parent table through the attributes on the tag

Name attribute

Defines the table name of the child table.

JoinKey attribute

When inserting a child table, the value of this column is used to find the data node stored in the parent table.

ParentKey attribute

Property is typically the name of the column that is associated with the parent table. The program first gets the value of joinkey, then generates a query statement through the column name specified by the parentKey attribute, and executes the statement to get which slice the parent table is stored on. Thus determine the location where the child table is stored.

PrimaryKey attribute

As described in the table tag.

NeedAddLimit attribute

As described in the table tag.

DataNode tag

The dataNode tag defines the data node in MyCat, which is what we usually call data shredding. A dataNode tag is a separate data shard. The meaning expressed in the example is: use the db1 physical database on the lch4307 database instance, which forms a data shard, and finally, we use the name dn1 to identify the shard.

Name attribute

Define the name of the data node, which needs to be unique, and we need to apply this name to the table tag to establish the relationship between the table and the shard.

DataHost attribute

This attribute is used to define which database instance the shard belongs to, and the attribute value refers to the name attribute defined on the dataHost tag.

Database attribute

This attribute is used to define the specific library on the specific database instance of the shard attribute, because here two latitudes are used to define the shard, namely: instance + concrete library. Because the table and table structure created on each library are the same. So this makes it easy to split the table horizontally.

DataHost tag

As the last tag in Schema.xml, this tag also exists as the lowest tag in the mycat logic library, directly defining the specific database instance, read-write separation configuration and heartbeat statements. Now let's analyze the label.

Select user ()

Name attribute

Uniquely identifies the dataHost tag for use by the upper tag.

MaxCon attribute

Specifies the maximum connection for each read-write instance connection pool. In other words, the writeHost and readHost tags nested within the tag will use the value of this attribute to instantiate the maximum number of connections to the connection pool.

MinCon attribute

Specify the minimum connection for each read-write instance connection pool and initialize the size of the connection pool.

Balance attribute

There are three types of load balancer:

Balance= "0", all reads are sent to the currently available writeHost.

Balance= "1", all read operations are randomly sent to readHost.

Balance= "2", all read operations are randomly distributed on writeHost and readhost.

WriteType attribute

There are three types of load balancer:

WriteType= "0", all writes are sent to the available writeHost.

WriteType= "1", all write operations are randomly sent to readHost.

WriteType= "2", all write operations are randomly sent on writeHost and readhost scores.

DbType attribute

Specifies the type of database for the back-end connection, which currently supports the binary mysql protocol, as well as other databases that use JDBC connections. For example: mongodb, oracle, spark, etc.

DbDriver attribute

Specify the Driver used to connect to the backend database. Currently, the optional values are native and JDBC. If you use native, because this value implements the binary mysql protocol, you can use mysql and maridb. Other types of databases need to be supported by JDBC drivers. If you use JDBC, you need to put the driver JAR package that conforms to the JDBC 4 standard into the MYCAT\ lib directory and check that the driver JAR package includes the following directory structure files: META-INF\ services\ java.sql.Driver. Write the specific Driver class name in this file, for example: com.mysql.jdbc.Driver.

Heartbeat tag

This tag indicates the statement used for heartbeat checking with the back-end database. For example, MYSQL can use select user (), Oracle can use select 1 from dual, and so on. This tag also has a connectionInitSql attribute, which is mainly put into the initialization SQL statement that needs to be executed when using the Oracla database. For example: alter session set nls_date_format='yyyy-mm-dd hh34:mi:ss'

WriteHost tag, readHost tag

Both tags specify the relevant configuration of the back-end database to mycat for instantiating the back-end connection pool. The only difference is that writeHost specifies write instances, readHost specifies read instances, and groups these read and write instances to meet the requirements of the system. Multiple writeHost and readHost can be defined within a dataHost. However, if the back-end database specified by writeHost goes down, then all readHost bound to this writeHost will be unavailable. On the other hand, due to the writeHost outage system will automatically detect and switch to the backup writeHost. The attributes of these two tags are the same, which are described together here.

Host attribute

Used to identify different instances, generally writeHost we use M1 ReadHost we use S1.

Url attribute

The connection address of the backend instance is usually in the form of address:port if the dbDriver of native is used. If you use JDBC or other dbDriver, you need a special specification. When using JDBC, you can write: jdbc:mysql://localhost:3306/.

User attribute

User name required by the backend storage instance

Password attribute

The backend stores the password required by the instance

Rule.xml

Rule.xml defines the rules involved in splitting the table. We can flexibly use different slicing algorithms for the table, or use the same algorithm for the table but with different specific parameters. There are mainly two tags tableRule and function in this file. In the specific use process, you can add tableRule and function as needed.

TableRule tag

This tag defines the table rules. The table rules defined in rule.xml:

Id func1

The name attribute specifies a unique name that identifies different table rules. The embedded rule tag specifies which column in the physical table to split and which routing algorithm to use. Specify the name of the column to split within the columns. Algorithm uses the name attribute in the function tag. Connection table rules and specific routing algorithms. Of course, multiple table rules can be connected to the same routing algorithm. Used within the tag. Let logical tables use this rule for slicing.

Function tag

Partition-hash-int.txt

Where the attributes are shown below.

Name specifies the name of the algorithm.

Class defines the specific class name of the routing algorithm.

Property is some of the attributes that the specific algorithm needs to use.

At this point, the study on "what are the three core configuration files of Mycat" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.