Shulou(Shulou.com)06/02 Report--

Sharding-jdbc how to configure and analyze Configuration, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain for you in detail, people with this need can come to learn, I hope you can gain something.

The core configuration of Sharding is as follows (official website):

Fragmentation rule

The total entry for the sharding rule configuration. Including data source configuration, table configuration, binding table configuration and read-write separation configuration, etc.

Data source configuration

List of real data sources

Table configuration

Configuration of logical table names, data nodes and sub-table rules

Data node configuration

Used to configure the mapping between logical tables and real tables. It can be divided into two forms: uniform distribution and custom distribution.

Sharding policy configuration

For sharding strategy, there are two dimensions: data source slicing strategy and table sharding strategy.

Data source sharding strategy:

Corresponds to DatabaseShardingStrategy. The target data source to which the configuration data is assigned

Table slicing strategy

Corresponds to TableShardingStrategy. The target table used to configure the data to be assigned, and the target table exists in the target data source with the data. Therefore, the table fragmentation strategy is dependent on the result of the data source fragmentation strategy.

Self-increasing primary key generation strategy

By generating the self-increasing primary key on the client side and replacing the primary key with the original self-increasing primary key in the database, the distributed primary key is not repeated.

Next, analyze each core configuration:

Take multi-master and multi-slave reading and writing separation and table slicing as examples.

Public final class ShardingMasterSlaveConfigurationPrecise implements ExampleConfiguration {@ Override public DataSource getDataSource () throws SQLException {ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration (); / / sharding table rule configuration shardingRuleConfig.getTableRuleConfigs () .add (getOrderTableRuleConfiguration ()); shardingRuleConfig.getTableRuleConfigs () .add (getOrderItemTableRuleConfiguration ()); / / bind sharding table, which is mainly used to route shardingRuleConfig.getBindingTableGroups () .add ("t_order, t_order_item") / / set the default data source sharding policy shardingRuleConfig.setDefaultDatabaseShardingStrategyConfig (new StandardShardingStrategyConfiguration ("user_id", new PreciseModuloShardingDatabaseAlgorithm (); / / set the default table sharding policy shardingRuleConfig.setDefaultTableShardingStrategyConfig (new StandardShardingStrategyConfiguration ("order_id", new PreciseModuloShardingTableAlgorithm (); / / Master-slave configuration to support multi-master and multi-slave shardingRuleConfig.setMasterSlaveRuleConfigs (getMasterSlaveRuleConfigurations ()) / / create ShardingDataSource data source return ShardingDataSourceFactory.createDataSource (createDataSourceMap (), shardingRuleConfig, new Properties ());} private static TableRuleConfiguration getOrderTableRuleConfiguration () {/ / sharded table configuration TableRuleConfiguration result = new TableRuleConfiguration (/ * logical table * / "t_order", / * data source name. Real table * / "ds_$ {0.1} .t _ order_$ {[0,1]}"); / / Custom primary key generation configuration result.setKeyGeneratorConfig (new KeyGeneratorConfiguration ("SNOWFLAKE", "order_id", getProperties (); return result } private static TableRuleConfiguration getOrderItemTableRuleConfiguration () {TableRuleConfiguration result = new TableRuleConfiguration ("t_order_item", "ds_$ {0.1} .t _ order_item_$ {[0,1]}"); result.setKeyGeneratorConfig (new KeyGeneratorConfiguration ("SNOWFLAKE", "order_item_id", getProperties (); return result } private static List getMasterSlaveRuleConfigurations () {MasterSlaveRuleConfiguration masterSlaveRuleConfig1 = new MasterSlaveRuleConfiguration ("ds_0", "demo_ds_master_0", Arrays.asList ("demo_ds_master_0_slave_0", "demo_ds_master_0_slave_1")) MasterSlaveRuleConfiguration masterSlaveRuleConfig2 = new MasterSlaveRuleConfiguration ("ds_1", "demo_ds_master_1", Arrays.asList ("demo_ds_master_1_slave_0", "demo_ds_master_1_slave_1"); return Lists.newArrayList (masterSlaveRuleConfig1, masterSlaveRuleConfig2);} private static Map createDataSourceMap () {final Map result = new HashMap () Result.put ("demo_ds_master_0", DataSourceUtil.createDataSource ("demo_ds_master_0")); result.put ("demo_ds_master_0_slave_0", DataSourceUtil.createDataSource ("demo_ds_master_0_slave_0")); result.put ("demo_ds_master_0_slave_1", DataSourceUtil.createDataSource ("demo_ds_master_0_slave_1")) Result.put ("demo_ds_master_1", DataSourceUtil.createDataSource ("demo_ds_master_1")); result.put ("demo_ds_master_1_slave_0", DataSourceUtil.createDataSource ("demo_ds_master_1_slave_0")); result.put ("demo_ds_master_1_slave_1", DataSourceUtil.createDataSource ("demo_ds_master_1_slave_1")); return result } private static Properties getProperties () {Properties result = new Properties (); result.setProperty ("worker.id", "123"); return result;}}

Core configuration of ShardingRuleConfiguration sharding rules

@ Getter@Setterpublic final class ShardingRuleConfiguration implements RuleConfiguration {/ / Table rule configuration private Collection tableRuleConfigs = new LinkedList (); / binding table configuration private Collection bindingTableGroups = new LinkedList (); / / broadcast table configuration private Collection broadcastTables = new LinkedList (); / / default data source name private String defaultDataSourceName; / / default sharding policy private ShardingStrategyConfiguration defaultDatabaseShardingStrategyConfig; / / default sharding policy private ShardingStrategyConfiguration defaultTableShardingStrategyConfig / / default primary key generation tool class private KeyGeneratorConfiguration defaultKeyGeneratorConfig; / / Master-slave rule configuration private Collection masterSlaveRuleConfigs = new LinkedList (); / / data desensitization rule configuration private EncryptRuleConfiguration encryptRuleConfig;}

Analyze ShardingRule configuration in ShardingDataSourceFactory#createDataSource

/ * Sharding data source factory. * * @ author zhangliang * / @ NoArgsConstructor (access = AccessLevel.PRIVATE) public final class ShardingDataSourceFactory {/ * * Create sharding data source. * * @ param dataSourceMap data source map * @ param shardingRuleConfig rule configuration for databases and tables sharding * @ param props properties for data source * @ return sharding data source * @ throws SQLException SQL exception * / public static DataSource createDataSource (final Map dataSourceMap, final ShardingRuleConfiguration shardingRuleConfig, final Properties props) throws SQLException {/ / create a ShardingDataSource data source Also create sharding rule configuration return new ShardingDataSource (dataSourceMap, new ShardingRule (shardingRuleConfig, dataSourceMap.keySet ()), props) }}

ShardingRule slicing rule

Public ShardingRule (final ShardingRuleConfiguration shardingRuleConfig, final Collection dataSourceNames) {Preconditions.checkArgument (null! = shardingRuleConfig, "ShardingRuleConfig cannot be null."); Preconditions.checkArgument (null! = dataSourceNames & &! dataSourceNames.isEmpty (), "Data sources cannot be empty."); this.shardingRuleConfig = shardingRuleConfig; / / sharding data source name shardingDataSourceNames = new ShardingDataSourceNames (shardingRuleConfig, dataSourceNames); / / create table rule set tableRules = createTableRules (shardingRuleConfig) / / create a grouping binding table / / 1. Find / / 2 from tableRules. Is it a broadcast table? if it is a broadcast table, dataSourceNames.size () = = 1? DataSourceNames.iterator (). Next (): shardingRuleConfig.getDefaultDataSourceName (); / / 3 If the above two situations are not satisfied, create tableRule bindingTableRules = createBindingTableRules (shardingRuleConfig.getBindingTableGroups ()) using the default data source; / / broadcast table broadcastTables = shardingRuleConfig.getBroadcastTables (); / / default sublibrary policy defaultDatabaseShardingStrategy = createDefaultShardingStrategy (shardingRuleConfig.getDefaultDatabaseShardingStrategyConfig ()); / / default table slicing policy defaultTableShardingStrategy = createDefaultShardingStrategy (shardingRuleConfig.getDefaultTableShardingStrategyConfig ()) / / default primary key generation tool class defaultShardingKeyGenerator = createDefaultKeyGenerator (shardingRuleConfig.getDefaultKeyGeneratorConfig ()); / / create master-slave rule masterSlaveRules = createMasterSlaveRules (shardingRuleConfig.getMasterSlaveRuleConfigs ()); / / data desensitization rule encryptRule = createEncryptRule (shardingRuleConfig.getEncryptRuleConfig ());}

TableRule rules

Public TableRule (final TableRuleConfiguration tableRuleConfig, final ShardingDataSourceNames shardingDataSourceNames, final String defaultGenerateKeyColumn) {/ / get logical table logicTable = tableRuleConfig.getLogicTable () .toLowerCase () / / the inline expression parses out the real table For example: ds_$ {0.1} .t _ order_$ {[0,1]} / / parsed as: / / ds_0.t_order_0 / / ds_0.t_order_1 / / ds_1.t_order_0 / / ds_1.t_order_1 List dataNodes = new InlineExpressionParser (tableRuleConfig.getActualDataNodes ()) .splitAndEvaluate () DataNodeIndexMap = new HashMap (dataNodes.size (), 1); / / real table data node / / 1. If the real table is not configured, the corresponding real table data node / / 2 is generated according to the logical table and data source. Otherwise, the real data node actualDataNodes = isEmptyDataNodes (dataNodes) is generated from the real table parsed by the inline expression. GenerateDataNodes (tableRuleConfig.getLogicTable (), shardingDataSourceNames.getDataSourceNames ()): generateDataNodes (dataNodes, shardingDataSourceNames.getDataSourceNames ()); / / real table actualTables = getActualTables (); / / data source-> sharding policy databaseShardingStrategy = null = = tableRuleConfig.getDatabaseShardingStrategyConfig ()? Null: ShardingStrategyFactory.newInstance (tableRuleConfig.getDatabaseShardingStrategyConfig ()); / / Table-> sharding policy tableShardingStrategy = null = = tableRuleConfig.getTableShardingStrategyConfig ()? Null: ShardingStrategyFactory.newInstance (tableRuleConfig.getTableShardingStrategyConfig ()); / / primary key field generateKeyColumn = getGenerateKeyColumn (tableRuleConfig.getKeyGeneratorConfig (), defaultGenerateKeyColumn); / / generate primary key tool classes, such as SNOWFLAKE shardingKeyGenerator = containsKeyGeneratorConfiguration (tableRuleConfig)? New ShardingKeyGeneratorServiceLoader () .newService (tableRuleConfig.getKeyGeneratorConfig () .getType (), tableRuleConfig.getKeyGeneratorConfig () .getProperties ()): null;}

Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.