Get the App
SLTechnology News&Howtos  ›  Development  › 

What is the NIO optimization in active web page detection?

Shulou Source: shulou.com Published: 2022-06-02 05:07:51 09月22日 Update

What this article shares with you is about the NIO optimization in active detection of web pages. The editor thinks it is very practical, so I share it with you to learn. I hope you can get something after reading this article.

Build a table sentence:

CREATE SEQUENCE seq_probe_id INCREMENT BY 1 START WITH 1 NOMAXvalue NOCYCLE CACHE 2000: create table probe (host varchar (40) not null,state int not null,type varchar (10) not null,)

Using NIO to optimize this program and further squeeze the utilization rate of resources has been thought about for a long time.

But NIO+ multithreading, online examples are not very reliable. Their own learning is also a very headache, procrastination is more than a year.

New program, using three-stage process

First, a thread pool is used to continuously send connection requests, but no reception is processed. Just register a key for SelectionKey.OP_READ

Another single-threaded program constantly select qualified channels and then assigns them to another thread pool to receive and parse data. (the receiving and parsing processes are merged.)

Finally, a single-threaded program is used to constantly brush the results into the database in batches. This piece is also an optimization. Change from a single Insert to batch storage. This block saves the processing power of at least one CPU core.

The persistence and parsing processes basically reuse the original code.

Tags: Programs threads procedures web pages constantly data initiative detection code that is total more knowledge articles queues remaining processing headaches utility utilization Apple Docker Huawei Linux macOS MariaDB Microsoft MySQL NVidia OPPO Reno NVidia Docker Apple macOS Xiaomi