Shulou(Shulou.com)06/01 Report--

This article mainly explains the "parallelStream pit case analysis", the article explains the content is simple and clear, easy to learn and understand, now please follow the editor's train of thought slowly in depth, together to study and learn "parallelStream pit case analysis"!

Many students like to use lambda expressions, which allow you to define short functions that reflect your superb coding level. Of course, this feature suffers from some companies that measure the amount of work in terms of lines of code.

For example, the following code snippet is like reading a poem. But if it is not used well, it will be fatal.

List transactionsIds = widgets.stream () .filter (b-> b.getColor () = = RED) .sorted ((XMagi y)-> x.getWeight ()-y.getWeight ()) .mapToInt (Widget::getWeight) .sum ()

There is a key function in this code, which is stream. With it, a normal list can be converted into a stream, and then the list can be manipulated in a pipe-like manner. In short, everything that has been used is agreed.

Here comes the problem.

What happens if we replace stream with parallelStream?

Literally, the stream changes from serial to parallel.

Since it is parallel, if you think about it with your ass, you will know that there must be a thread safety problem. However, what we are talking about here is not for you to use thread-safe collections, this topic is too low-level. At this stage, it is a basic skill to know how to use thread-safe collections in a thread-safe environment.

The place where we step on the pit this time is the performance problem of parallel flow.

We talk in code.

The following code starts eight threads, all of which are using parallel streams for data calculations. In the logic of execution, we sleep each task for 1 second, so that we can simulate the time-consuming wait of some Imax O requests.

With stream, the program will return in 30 seconds, but we expect the program to return in more than 1 second, because it is a parallel flow, worthy of the title.

The test found that we waited for a long time before the task was completed.

Static void paralleTest () {List numbers = Arrays.asList (0,1,2,3,4,5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29); final long begin = System.currentTimeMillis () Numbers.parallelStream () .map (k-> {try {Thread.sleep (1000); System.out.println ((System.currentTimeMillis ()-begin) + "ms = >" + k + "\ t" + Thread.currentThread ());} catch (InterruptedException e) {e.printStackTrace ();} return k;}) .destroy (Collectors.toList ()) } public static void main (String [] args) {/ / System.setProperty ("java.util.concurrent.ForkJoinPool.common.parallelism", "20"); new Thread (()-> paralleTest ()). Start (); new Thread (()-> paralleTest ()). Start (); new Thread (()-> paralleTest ()). Start (); new Thread (()-> paralleTest ()). Start () New Thread (()-> paralleTest ()). Start (); new Thread (()-> paralleTest ()). Start (); new Thread (()-> paralleTest ()). Start (); new Thread (()-> paralleTest ()). Start ();} pit

In fact, this code takes different times to execute on different machines.

Since it is parallel, there must be a degree of parallelism. It is too low to reflect the ability of parallelism; it is too large, and it wastes the time of context switching. I am very frustrated to find that many advanced research and development, familiar with various parameters of the thread pool and various tuning, dare to turn a blind eye to the use of parallelStream in the intensive business of Imax O.

To understand this parallelism, we need to look at the specific construction method. Find such code in the ForkJoinPool class.

Try {/ / ignore exceptions in accessing/parsing properties String pp = System.getProperty ("java.util.concurrent.ForkJoinPool.common.parallelism"); if (pp! = null) parallelism = Integer.parseInt (pp); fac = (ForkJoinWorkerThreadFactory) newInstanceFromSystemProperty ("java.util.concurrent.ForkJoinPool.common.threadFactory"); handler = (UncaughtExceptionHandler) newInstanceFromSystemProperty ("java.util.concurrent.ForkJoinPool.common.exceptionHandler") } catch (Exception ignore) {} if (fac = = null) {if (System.getSecurityManager () = = null) fac = defaultForkJoinWorkerThreadFactory; else / / use security-managed default fac = new InnocuousForkJoinWorkerThreadFactory ();} if (parallelism)

< 0 && // default 1 less than #cores (parallelism = Runtime.getRuntime().availableProcessors() - 1) MAX_CAP) parallelism = MAX_CAP; 可以看到，并行度到底是多少，是由下面的参数来控制的。如果无法获取这个参数，则默认使用 CPU个数-1 的并行度。 可以看到，这个函数是为了计算密集型业务去设计的。如果你喂给它一大堆任务，它就会由并行执行退变成类似于串行的效果。 -Djava.util.concurrent.ForkJoinPool.common.parallelism=N 即使你使用-Djava.util.concurrent.ForkJoinPool.common.parallelism=N设置了一个初始值大小，它依然有问题。 因为，parallelism这个变量是 final 的，一旦设定，不允许修改。也就是说，上面的参数只会生效一次。 张三可能使用下面的代码，设置了并行度大小为20。 System.setProperty("java.util.concurrent.ForkJoinPool.common.parallelism", "20"); 李四可能用同样的方式，设置了这个值为30。那实际在项目中用的是哪个值，那就得问 JVM 是怎么加载的类信息了。 这种方式并不太非常靠谱。 一种解决方式 我们可以通过提供外置的forkjoinpool，也就是改变提交方式，来实现不同类型的任务分离。 代码如下所示，通过显式的代码提交，即可实现任务分离。 ForkJoinPool pool = new ForkJoinPool(30);final long begin = System.currentTimeMillis();try { pool.submit(() ->

Numbers.parallelStream () .map (k-> {try {Thread.sleep (1000); System.out.println ((System.currentTimeMillis ()-begin) + "ms = >" + k + "\ t" + Thread.currentThread ());} catch (InterruptedException e) {e.printStackTrace () } return k;}) .e.printStackTrace (Collectors.toList ()) .get ();} catch (InterruptedException e) {e.printStackTrace ();} catch (ExecutionException e) {e.printStackTrace ();}

In this way, different scenes can have different degrees of parallelism. This approach is similar to CountDownLatch, where we need to manage resources manually.

Thank you for your reading, the above is the content of "parallelStream pit case analysis", after the study of this article, I believe you have a deeper understanding of the pit case analysis of parallelStream, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.