Shulou(Shulou.com)06/03 Report--

I. Preface

In intelligent operation and maintenance, related information such as components and JAR packages used by application services are very important. These information can clearly describe the skeleton of an application service, which we call application portraits. In UAVStack, the InterceptFramework under the Middleware Enhancement Framework (MOF) can obtain portrait information during the startup of the application. This paper mainly introduces the architecture principle of InterceptFramework and the application portrait data acquisition and storage based on it.

II. Overall structure

Key technology JavaassistHook2.2 key classes and functions Profile data and client monitoring indicators are mainly captured and monitored through InterceptSupport and hookProxy mechanisms. Profile is also installed into the DataObserver object and exposes the JMX/HTTP interface. 2.3 the preparation before interception is to rewrite the bytecode and insert specific logic processing code to intercept within the specific life cycle of the application startup. Start intercepting the Supporter corresponding to the framework, obtain the Listener that should be loaded from the configuration file, build the InterceptSupport instance, and add the Listener to the InterceptSupport instance. 2.4 specific interception logic

During a specific life cycle during the startup of the application, the previously implanted code blocking logic begins to execute the doIntercept method of InterceptSupport, traversing the registered Listener listeners. Each listener inherits from the InterceptEventListener class. Different sub-Listener collects and processes corresponding types of portraits: AppProfilingListener is responsible for listening to all service portrait events, and AppFrkHookFactoryListener is responsible for listening to client-side portrait events.

Third, architecture description InterceptSupport: singleton as a unified Profile/Client hook capture entry point InterceptContext: capture context interface, all capture parameters are encapsulated by Context InterceptEventListener: event hijacking listener, all listeners need to inherit and register to listen AppProfilingListener: service Portrait listener, listener for all Service Portrait events AppFrkHookFactoryListener: client Portrait listener Agent IV, key class description StandardProfile: implements the Profile interface, is a portrait data capture implementation class The doProfiling method is provided to be responsible for crawling behavior and generating crawling results. ProfileHandler: common interface for different crawling logic and points to implement different crawling logic; doProfiling method for crawling behavior; StandardProfileRepository: storing portrait data crawling data structure; DataObserver: exposing JMX/HTTP interface data; Profile real-time monitoring installs the Profile object into the DataObserver object when calling startProfiling from the AppProfilingListener class DataObserver provides JMX/HTTP services for subsequent crawling, in which the Http service registers HttpJEEProfileObserver and is mainly responsible for capturing portrait data; each Handler class specifically deals with all kinds of captured data. 5. Portrait data

Portrait data, that is, static data, is the data that can be obtained when the process starts, including operating system-related information, program-related information, and initialization information after program startup.

The portrait data mainly includes three parts: service portrait, traceability perception portrait and client portrait, as shown above. The following introduces the collection of portraits from these three aspects:

5.1 Service Portrait Collection

The service portrait describes the information of the service ontology, including the application unique identification (AppID), the service name (Service ID), the URI of the service instance, the URI of the service interface, and the metadata of the service interface (classes, methods, input and output parameters, annotations, deployment descriptors).

Take Tomcat as an example, the startup process of the application needs to go through the start method of StandardContext. We can intercept by inserting code logic at the end of this method, and the following information can be obtained through interception:

Container context information such as application name, webworkdir (working directory), servletcontext, contextpath, basepath (application actual path); obtain portrait data according to corresponding technical specifications (Servlet, JAXRS, JAXWS, Spring, etc.), mainly including: obtaining class and method names and input parameter information by reflection, scanning annotation classes, obtaining annotation information, parsing deployment descriptor file (web.xml, etc.) information.

In addition to the above information, the service portrait also obtains information such as the log path of the application and the JAR package used.

5.2 traceability Portrait Collection

The traceability portrait collection mainly traces the source with the help of the fields in the HTTP protocol Header, and intercepts the Invoke method of StandardEngineValve in Tomcat through the middleware hijacking technology to obtain the traceability data.

The following types of data are collected:

Client Address: directly connected client IP address; X-Forwarded-For: if it exists, it is proxy routing address chain, then directly connected client is proxy service; Host: indicates remote host or even port information; if directly connected client is proxy service, Host is proxy IP address and port; User-Agent: proxy description, which can be used to distinguish between browser and program client, and can also extract a lot of browser terminal information. The UAV-Client-Src:HTTP client hijacks the added Header field, which is used for fitting calls between UAV applications. 5.3 client Portrait Collection

Client portrait is realized by hijacking a series of commonly used middleware clients. At present, synchronous / asynchronous http, database jdbc, redis (jedis,aredis,lettuce), mongodb, rocketmq/rabbitmq/Kafka, ESClient and so on are supported.

Through AOP technologies such as javaassist bytecode rewriting and dynamic proxy, hijacking embeds specific processing logic in the client call code to obtain call-related information, such as call address, call protocol, call result and so on.

The portrait data determines the target service based on the feature extraction of the call address, access protocol and call result.

1) call address: expressed in URI-like format

Http/https service (business / proxy service): http:// relational database (data source service): jdbc: non-relational database or cache (data source service): message queue (message service): mq:

2) access protocol: some kind of access action. For example, POST of HTTP, SQL insert, send / subscribe messages, hgethashall of Redis, Collection operation of Mongo, etc.

3) access result characteristics: the basic stack type of the service, whether it is clustered, such as Nginx, Tomcat, Apache, etc.

Taking ESClient as an example, we can hijack the sendRequest method of TransportService and populate our interception code logic.

Unlike the service profile, the client profile occurs in the specific client call process, not in the startup process of the application.

VI. Storage structure of portraits

Portrait storage is layered in the following structure:

StandardProfileRespository stores the whole portrait of an application; StandardProfileElement stores certain component portraits (server, client, log, jar, etc.); ProfileElementInstance stores a certain component portrait in a certain kind of component, taking the service component portrait as an example, for example, the components JAXRS and Servlets are each a ProfileElementInstance, but they all belong to the StandardProfileElement of the service component.

DataObserver provides two modes, JMX and HTTP, to expose interface profile data, which is similar to real-time data, which has been described in detail in our brother article in the MOF series, "adding Framework to Middleware CaptureFramework", and will not be discussed here.

Open source address: open source address: https://github.com/uavorg

Author: Li Xingsheng

Source: Yixin Institute of Technology

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.