Shulou(Shulou.com)05/31 Report--

This article mainly introduces how to achieve the Wifi equipment supervision system architecture based on Tablestore, which has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, let the editor take you to understand it.

Wifi equipment supervision

The company maintains the attributes of Wifi equipment and collects Wifi equipment monitoring data through the supervision system. When the Wifi equipment is needed to be online or offline, the equipment can be added and offline through the supervision system operation, and the equipment attribute information can be modified and added through the system, such as device mac address, equipment model, equipment geographical location and so on. After the equipment is online, the monitoring data will be pushed to the system regularly, so as to complete the collection of equipment monitoring data. The collected data include: cpu, memory, number of connections, flow and flow rate of Wan port, channel data of 2.4G and 5G modules, etc.

By analyzing the monitoring data index and analyzing the running status of the equipment, the running status of the problem equipment is dynamically modified to: early warning and alarm. With the help of the system, the network department can quickly obtain the list of problem equipment, understand the distribution of equipment, and query historical monitoring indicators. At the same time, it can also accurately lock the old equipment to facilitate equipment upgrade, or provide data basis for the location with high long-term load rate to expand Wifi equipment.

Functional requirements

1. Manage Wifi equipment, launch new equipment and offline old equipment through the system.

2. The system has the ability of grouping management and tag retrieval.

3. High concurrent mass monitoring data collection ability

4. Manage the geographical distribution of all devices

5. Query the location of all devices in a certain area

6. Query the monitoring data of [a device] in [a certain period of time] [different indicators]

7, low-cost persistence of all data, mining data potential value and so on.

System sample, as shown below: official website console address: project sample

Technical requirements

In general, users will focus on the following four main technical requirements in the design:

First, we need to have strong query and statistical capabilities to realize the management of Wifi equipment.

Second, support the high concurrent monitoring data collection of the equipment, and the database needs strong writing ability.

Third, the demand for data persistence leads to data expansion, but most of the historical monitoring data are cold data, and the storage cost needs to be as low as possible; fourth, the potential value of monitoring data mining is high in the future, and a better computing ecology is needed downstream of the product.

Table storage scheme

Table storage (Tablestore) fully meets four important technical requirements:

First, the newly commercialized multiple indexing (SearchIndex) function of table storage supports multi-dimensional retrieval, GEO query and other functions, which fully meets the needs of metadata management.

Second, the distributed NoSQL database based on LSM tree can easily cope with massive and high concurrency, zero operation and maintenance can easily cope with the continuous expansion of the amount of data, and there is no upper limit in theory.

Third, table storage is billed by quantity, providing two instance types: capacity type and high performance type. Capacity type is more suitable for cold data and provides lower storage cost.

Fourth, more importantly, table storage has a relatively perfect computing ecology, provides full and incremental channel services, provides a streaming and batch computing system, and provides channels for future monitoring data value mining.

The table storage has a very high matching at the technical point of the timing scenario requirements, and the timing model based on the timing scenario (Timestream) encapsulates the general functions of the timing scene into an easy-to-use interface, which makes it easier for users to build a Wifi equipment supervision system based on table storage.

Data structure design

First of all, we abstract two types of data in the table storage, namely meta data (equipment metadata) and data data (monitoring data); the following two types of data are briefly introduced.

WiFi device metadata

Meta data manages the attribute information of user timeline and supports indicators, tags, attributes, geographical location, update time and other parameters. The model will create corresponding indexes for all attributes and provide multi-dimensional conditional combination queries (including GEO queries). Identifier is the identification of the timeline, which consists of two parts: name part (monitoring indicator identification) and tags part (inherent immutable parameter set).

In this example, we classify "wifi" as metrics, mac addresses as immutable tag, and other attributes as variable Attributes as attribute information.

Equipment monitoring data

Data data manages the monitoring status data of each timeline, and can express any type of quantitative data, geographical location and text. Data data are arranged in order of +, so all data of the same timeline are ordered based on time. This data storage method greatly improves the query efficiency of the timeline.

We store more than a dozen monitoring data of the equipment at a certain point in time as a row of data, with different attributes corresponding to different columns. According to different monitoring dimensions, users only need to provide different columnToGet fields and obtain part of the index data of different monitoring dimensions to correspond to different monitoring indicators, such as WAN port traffic: corresponding to wan_total_in and wan_total_out fields.

Read and write interface

Write data

Write data provides two types of interfaces: Wifi device addition and monitoring data writing.

Wifi device addition: if you add a new Wifi device, you need to insert a piece of device meta data into the meta table first, and create or modify meta information through metaTable.put (Meta)

Monitoring data writing: after the meta is created, the wifi device can collect the monitoring data regularly and periodically, and push and write the data to the data table. The model design can support multi-precision table management, and users can manage multiple precision data data according to their own needs.

Read data

Like writing data, two types of read interfaces are provided for two types of data: Wifi device query and monitoring data reading.

Wifi device query: according to the combination of equipment grouping, device status, geographical location and other multi-dimensional conditions, obtain the corresponding wifi equipment list, and grasp the latest status of the equipment.

Monitoring data reading: based on the Identifier of a single meta, obtain the monitoring data of the device within a certain period of time and a certain index

Core code

SDK and sample code

SDK: the time series model Timestream model is integrated into the SDK of table storage and has been supported in version 4.11.0:

Com.aliyun.openservices tablestore 4.11.0

Open source code: https://github.com/aliyun/tablestore-examples/tree/master/demos/WifiMonitor

Create a datasheet

After creating an instance, users need to create corresponding meta tables and data tables through the sdk of the time series model: different precision monitoring data are stored in different tables and distinguished by table names. Monitoring data with different precision is required according to the query of different range. Only one precision is used in the instance, and users can design multiple tables according to their own needs.

Private void init () {AsyncClient asyncClient = new AsyncClient (endpoint, accessKeyId, accessKeySecret, instance); TimestreamDBConfiguration conf = new TimestreamDBConfiguration ("metaTableName"); TimestreamDBClient db = new TimestreamDBClient (asyncClient, conf) } public void createTable () {db.createMetaTable (Arrays.asList (new AttributeIndexSchema ("group", AttributeIndexSchema.Type.KEYWORD), new AttributeIndexSchema ("id", AttributeIndexSchema.Type.KEYWORD), new AttributeIndexSchema ("status", AttributeIndexSchema.Type.KEYWORD), new AttributeIndexSchema ("version", AttributeIndexSchema.Type.KEYWORD), new AttributeIndexSchema ("location", AttributeIndexSchema.Type.GEO_POINT)); db.createDataTable ("dataTableName");}

Data writing

Data writing is mainly divided into two parts: adding new Wifi equipment to meta table and collecting equipment monitoring data in data table.

Add a new Wifi device (meta write)

/ / metaWriter corresponds to the meta table and provides the read / write interface TimestreamMetaTable metaWriter = db.metaTable (); / / identifier is used as the unique of the timeline, including: Name, Tags,TimestreamIdentifier identifier = new TimestreamIdentifier.Builder ("wifi") .addTag ("mac", "mock:mac:1:1") .build () / create a meta object based on identifier and set more attributes for meta. Attributes is the attribute parameter TimestreamMeta meta = new TimestreamMeta (identifier) .addAttribute ("group", "group-1") .addAttribute ("id", "id-1") .addAttribute ("version", "v1.0") .addAttribute ("status", "normal") .addAttribute ("location", "30120"); / / create a new timeline and then write to the monitoring data metaWriter.put (meta)

Collect monitoring data of Wifi equipment (write data table)

/ / dataWriter corresponds to data table, respectively, and provides read and write interfaces TimestreamDataTable dataWriter = db.dataTable ("dataTableName"); TimestreamMeta meta / / meta has built / / created a new timeline in the previous step, and then write the monitoring data dataWriter.asyncWrite (meta.getIdentifier (), / / Identifier identifier new Point.Builder (I, TimeUnit.SECONDS) .addField ("cpu", 30) .addField ("ram", 29) .addField ("flash_used", 20) .addField ("flash_total", 1048576) .build ())

Data reading

Data reading is divided into two categories: Wifi equipment list query and equipment monitoring data query.

Query Wifi device list (meta table read)

/ / reader corresponds to the meta table and provides read and write interfaces. Here the name is TimestreamMetaTable metaReader = db.metaTable (); / / build filter condition Filter filter = new AndFilter (Arrays.asList (Name.equal ("wifi"), Tag.equal ("mac", "mock:mac:1:1"), Attribute.inGeoDistance ("location", "30120", 100000)); Iterator iterator = metaReader .filter (filter) .fetchAll () While (iterator.hasNext ()) {TimestreamMeta meta = iterator.next (); / / deal with metas}

Get the monitoring data of Wifi devices (data table read)

/ / dataWriter corresponds to data table, respectively, and provides read and write interfaces TimestreamDataTable dataReader = db.dataTable ("dataTableName"); TimestreamMeta meta / / based on the acquired meta list, get the ordered monitoring data Iterator iterator = reader.get (meta.getIdentifier ()) .select ("flash_used", "flash_total") / / set the returned column .timeRange (TimeRange.range (0, Long.MAX_VALUE, TimeUnit.SECONDS)) .fetchAll (); while (iterator.hasNext ()) {Point point = iterator.next (); / / deal with points long timestamp = point.getTimestamp (TimeUnit.MILLISECONDS) / / millisecond unit timestamp long flashUsed = point.getField ("flash_used") .asLong (); / / get the long type data size monitoring long flashUotal = point.getField ("flash_total") .asLong () / / get this point of long type data size monitoring} Thank you for reading this article carefully. I hope the article "how to implement the Wifi equipment Supervision system Architecture based on Tablestore" shared by the editor will be helpful to you. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.