Shulou(Shulou.com)06/01 Report--

This article mainly introduces "how to change Skywalking into an Elasticsearch service that adapts to Http Basic". In daily operation, I believe many people have doubts about how to change Skywalking to Elasticsearch service that adapts to Http Basic. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the question of "how to change Skywalking to Elasticsearch service that adapts to Http Basic". Next, please follow the editor to study!

Preface

Recently, the company's skywalking service often has a blank market. It has been found out that it is due to the thread blocking caused by the writing bottleneck of ES and the fact that the data has not landed on the ES. After considering the comprehensive operation and maintenance costs, we are ready to use the Elasticsearch service provided by Aliyun. Aliyun's ES has added Http Basic authentication to both internal and external networks, but the RestHighLevelClient client provided by skywalking6.x does not adapt to the ES service with Http Basic basic authentication, so the skywalking source code needs to be slightly changed.

Skywalking project structure

After the project is pulled down from the github, first understand the project structure. In the skywalking2.x version, I studied the plug-in mechanism of skywalking, and now the 6.x version has made great changes in the project structure and design. 6.x the project structure is as follows. We mainly focus on the module referred to by the arrow and the implementation of the es storage plug-in.

Location code change

1. Locate StorageModuleElasticsearchConfig.java and add users and passwords required for Http Basicr authentication

@ Setter @ Getter private String nameSpace; @ Setter @ Getter private String clusterNodes; private int indexShardsNumber; private int indexReplicasNumber; private boolean highPerformanceMode; private int traceDataTTL = 90; private int minuteMetricDataTTL = 90; private int hourMetricDataTTL = 36; private int dayMetricDataTTL = 45; private int monthMetricDataTTL = 18; private int bulkActions = 2000; private int bulkSize = 20; private int flushInterval = 10; private int concurrentRequests = 2; private String user; private String password

two。 Locate the ElasticSearchClient.java, add user and password attributes, change the construction method, input the user and password, and determine whether the user has configured authentication information when creating the connect. If not, use the authenticated connection. If not, use the default connection without authentication. The specific changes are as follows:

Private static final String TYPE = "type"; private final String clusterNodes; private final String namespace; private final String user; private final String password; private RestHighLevelClient client; public ElasticSearchClient (String clusterNodes, String namespace, String user, String password) {this.clusterNodes = clusterNodes; this.namespace = user; this.password = password;} @ Override public void connect () {ListpairsList = parseClusterNodes (clusterNodes) RestClientBuilder builder; if (StringUtils.isNotBlank (user) & & StringUtils.isNotBlank (password)) {final CredentialsProvider credentialsProvider = new BasicCredentialsProvider (); credentialsProvider.setCredentials (AuthScope.ANY, new UsernamePasswordCredentials (user, password)) Builder = RestClient.builder (pairsList.toArray (new HttpHost [0])) .setHttpClientConfigCallback (new RestClientBuilder.HttpClientConfigCallback () {@ Override public HttpAsyncClientBuilder customizeHttpClient (HttpAsyncClientBuilder httpClientBuilder) {return httpClientBuilder.setDefaultCredentialsProvider (credentialsProvider)) );} else {builder = RestClient.builder (pairsList.toArray (new HttpHost [0]));} client = new RestHighLevelClient (builder);}

Official documentation of Elasticsearch's basic_authentication connection: https://www.elastic.co/guide/en/elasticsearch/client/doc

3. Locate the input parameters of StorageModuleElasticsearchProvider.java and ElasticSearchClientTestCase.java to modify the construction method. Basically, the above changes will enable skywalking to support Elasticsearch services with basic_authentication authentication. Add user and password during configuration, such as:

Storage: elasticsearch: nameSpace: ${SW_NAMESPACE: ""} clusterNodes: ${SW_STORAGE_ES_CLUSTER_NODES:localhost:9200} user: ${SW_ES_USER: ""} password: ${SW_ES_PASSWORD: ""} indexShardsNumber: ${SW_STORAGE_ES_INDEX_SHARDS_NUMBER:2} indexReplicasNumber: ${SW_STORAGE_ES_INDEX_REPLICAS_NUMBER:0} # Batch process setting Refer to https://www.elastic.co/guide/en/elasticsearch/client/java-api/5.5/java-docs-bulk-processor.html bulkActions: ${SW_STORAGE_ES_BULK_ACTIONS:2000} # Execute the bulk every 2000 requests bulkSize: ${SW_STORAGE_ES_BULK_SIZE:20} # flush the bulk every 20mb flushInterval: ${SW_STORAGE_ES_FLUSH_INTERVAL:10} # flush the bulk every 10 seconds whatever the number of requests concurrentRequests: $ {SW_STORAGE_ES_CONCURRENT_REQUESTS:2} # the number of concurrent requests considerations

Skywalking project from individual open source to international open source project is closely related to its code quality and project management. There are strict code compilation checks in skywalking projects, such as no spaces before commas, spaces after commas, etc.

And don't forget to modify the assembly\ application.yml file by adding user and password, otherwise the printed package does not have these two configuration attributes by default

Compilation, because it involves the GPRC part of the dependency, and the structure of the front-end project, need to do some processing.

Because of the problem with the system character set, the package typed under the Windows system and the shell footsteps under the bin directory may not run. This is caused by the problem with the Windows character set, but it doesn't matter. You can copy the footsteps in the official package to replace.

After the revision and compilation is successful, a dist directory is produced under the root directory of the project, which contains two compressed packages, corresponding to the linux system tar and the zip package under Windows.

At this point, the study on "how to change Skywalking to Elasticsearch services adapted to Http Basic" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.