Shulou(Shulou.com)06/01 Report--

This article mainly introduces the example analysis of perl multithreading, which is very detailed and has certain reference value. Friends who are interested must finish it!

Perl multithreading

1. Parallel series 1

Thread is a single execution flow, which is the smallest control unit in the execution of all programs, that is, the smallest task unit that can be scheduled by CPU. There is a connection and a completely different relationship between a thread and a process. To put it simply, a thread must belong to a process, and a process contains at least one or more threads. Early computer systems could only run one program at a time, so when there were multiple programs to be executed, the only way was to queue them up and execute them sequentially. The emergence of processes breaks this pattern, and CPU resources are divided according to time slices and allocated to different processes. In this way, although the execution of the process is still serial from a micro point of view, from a macro point of view, different programs are already executing in parallel. If we apply the same idea to a process, it is natural to subdivide the process into smaller units of execution, that is, threads. Because a process often needs to perform multiple similar tasks at the same time, these subdivided threads can share the same resources such as code segments, data segments and file handles. With the process, we can run multiple programs such as Firefox and Microsoft Office Word on a single CPU computer system at the same time; with threads, we can make Firefox load multiple different pages in different tags at the same time, and check syntax errors while editing documents in Office Word. Therefore, threads bring us higher CPU utilization, faster program response, more economical use of resources and better adaptability to multi-CPU architecture.

For a detailed explanation of multithreading, see: perl Thread Model explanation (http://it.chinawin.net/softwaredev/article-124a1.html)

Multithreaded Module in perl

For the multithreaded module, see perldoc (http://perldoc.perl.org/threads.html).

Multithreaded example of perl:

Language involved: Perl

Module used: threads

The method in the module: threads- > create () to create a new thread

Threads- > join () to harvest the created thread

Threads- > list (threads::all), returning all threads that have been created

Threads- > is_joinable (), returns whether the target thread has completed, and waits for join

The rest is on perldoc. The letters are so long. There will be no translation.

# beginuse threads; # declaration module; use warnings;use strict;print localtime (time), "\ n"; # output system time; my $jyog0bot my $thread;while () {last if ($j > = 10); here control the number of tasks, a total of 10; # control the number of threads created, here is the number of threads- > list () elements in the list returned by the 5M scalar function While (scalar (threads- > list ()) new (\ & ss,$j,$j);} foreach $thread (threads- > list (threads::all)) {if ($thread- > is_joinable ()) # determines whether the thread is running; {$thread- > join (); # outputs the intermediate result Print scalar (threads- > list ()), "\ tfolj\ t", localtime (time), "\ n";} # join drops the remaining threads (because 4 threads are still running when while is 10, but at this time the program will exit while, so you need an additional program join to drop the remaining 4 threads) foreach $thread (threads- > list (threads::all)) {$thread- > join () Print scalar (threads- > list ()), "\ tcoolj\ t", localtime (time), "\ n";} # output the end time of the program, compared with the start time of the program, and see the performance of the program. Print localtime (time), "\ n"; # here are the functions referenced by each thread; sub ss () {my ($t recording s) = @ _; sleep ($t); # sleep function, sleep; in seconds; print "$s\ t";}

Results:

The first column represents the number of tasks the program has completed, the second column represents the number of running threads-1 (one join is missing), the third column represents the newly added tasks after a thread has been harvested, and the last column represents the system time when a thread is completed.

Multithreaded performance

If you run these 10 tasks separately, the time required is: 1 / 2 / 3 / 4 / 10 / 55s

If you use multi-thread to run (5), the time required is 54-391s.

Running process

Briefly describe the running process of the program, in order to better understand the concept of multithreading, hehe

The program runs a total of 10 tasks, the first task is to pause the program 1s (sleep (1)), the second task is to pause the program 2s (sleep (2)), and so on, the tenth task is to suspend the program for 10s

Time (s) Task

At the beginning of the program, 5 threads are running at the same time, and thread 5 (5s) takes the longest time.

1 / 2 / 3 / 4 / 5 / 5 / 6 (after 1 second, the first task has been completed and is dropped by join, while adding a new task 6)

2, 3, 4, 5, 6, 7 (ibid.)

3 4,5,6,7,8

4 5,6,7,8,9

5 6,7,8,9,10

7-end join all remaining threads (all tasks have been added, while loop exits in the program)

The function of the method $thread- > is_joinable ()

As mentioned earlier, this method is used to determine whether the thread has finished running and is waiting for join. This approach is especially important when you need to deal with multiple tasks that take a different amount of time to complete.

Let's take the above procedure as an example. Five threads are created when the program is initially run. The first thread takes the shortest time, 1s. The fifth thread takes up to 5s. If you do not apply $thread- > is_joinable () and directly join these five threads, it is as follows:

Foreach $thread (threads- > list (threads::all)) {$thread- > join ();}

The result is that the main program is waiting. After 1s, the first thread is join, the main program is still waiting, the second thread is join after 2s, and the main program is waiting. Knowing that the fifth thread is join after 5s, the main program is unobstructed, and recreate the next set of threads (5). It is clear that this process does not maximize the use of CPU resources. When the first thread is join, although there are only 4 threads running in the program, the new thread will not be created because the main program is in a waiting state.

The best way is to determine whether a thread can be join. As written in the above program. This ensures that there are always 5 threads in the process of running the program, maximizing the use of CPU resources.

Example

Having said so much, here is a simple example of how multithreading can be used in bioinformatics. Search for homologous sequences from the KEGG database (http://www.genome.jp/kegg/).

Required file: seqname.txt (used to store the sequence KEGG name to be searched)

Source code:

Use strict;use warnings;use threads;use SOAP::Lite;use Cwd;my $path=getcwd;my $wsdl = 'http://soap.genome.jp/KEGG.wsdl';my $serv = SOAP::Lite- > service ($wsdl); open (F, "K00006.txt"); my @ names=;chomp @ names;close (F); my $ionometermy $thread;print localtime (time); while ($ilist ()) new (\ & orgfile,$names [$]); $iTunes + } foreach $thread (threads- > list (threads::all)) {if ($thread- > is_joinable ()) {$thread- > join ();} foreach $thread (threads- > list (threads::all)) {$thread- > join ();} print localtime (time); sub orgfile {my ($seq) = @ _; my $offset = 1; my $limit = 100 My $top5 = $serv- > get_best_neighbors_by_gene ($seq, $offset,$limit); $seq=~s/://; open (F, "> $seq.txt"); foreach my $hit (@ {$top5}) {print F "$hit- > {genes_id1}\ tintit-> {genes_id2}\ tintit-> {sw_score}\ n";} close (F); print "$seq\ n";}

END

2. Series 2 of parallel development

In today's mainstream situation of multi-core CPU, multi-core parallel programs provide the possibility of making maximum use of CPU. Perl has been providing ithread module since version 5.6 to provide technical support for multithreading of perl. In the article multithreading of perl, we preliminarily discussed the multithreading technology of perl. The is_joinable method is used to determine whether the target thread has completed execution and is waiting for join. The program source code is as follows (http://blog.sina.com.cn/s/blog_7ea1e7fc0100r61x.html):

Use threads;use warnings;use strict;print localtime (time), "\ n"; my $jacuzzo my $thread;while () {last if ($j > = 10) While (scalar (threads- > list ()) new (\ & ss,$j,$j);} foreach $thread (threads- > list (threads::all)) {if ($thread- > is_joinable ()) {$thread- > join (); print scalar (threads- > list ()), "\ tpragj\ t", localtime (time), "\ n" } foreach $thread (threads- > list (threads::all)) {$thread- > join (); print scalar (threads- > list ()), "\ t list\ t", localtime (time), "\ n";} print localtime (time), "\ n"; sub ss () {my ($t) = @ _; sleep ($t); print "$s\ t";}

The above method has a great flaw. If none of the five threads that are executing finishes, the outermost while loop will "idle" until one thread is dropped by join, and the while loop controls the creation of new threads. In this process, the main thread will always consume system resources because of the existence of this while loop. In fact, you can see in the task manager that our program consumes 50% of the CPU (dual-core CPU), which is actually spent on the outer while loop that does not perform any function.

There is no hint at the end of the thread created by perl to tell the master thread that it (the slave thread) has ended. So you must use an add-on to determine whether the target thread has completed execution and immediately join the thread to release system resources. But this judgment process consumes system resources. Just like the program above.

To this end, in a google, thanks to Yunshu provided the method (http://icylife.net/yunshu/show.php?id=617), and finally learned.

Semaphore

The Thread::Semaphore package provides semaphore support for threads. Semaphores can be used to control the number of parallel threads.

Object declaration:

My $semaphore = Thread::Semaphore- > new (5)

Or my $semaphore = new Thread::Semaphore (5); controls the maximum number of threads in parallel

Object method:

$semaphore- > down

Get a semaphore, then the available semaphore minus 1, when $semaphore=0, it means that all threads have been created, cannot get a new semaphore, and the main thread is waiting. Until a new semaphore is obtained.

$semaphore- > up

Release a semaphore, after which you can add 1. When a thread finishes executing, this method is called to release a semaphore. At this point, the $semaphore- > down method obtains the semaphore successfully. The waiting main thread wakes up again to create a new thread.

Harvesting thread

The last program used to harvest the thread uses the join method. The characteristic of the join method is that if the thread has finished execution, it is only natural to call the join method at this time, but if the join call is too early and the thread has not finished execution, the main thread will be in a congested state. Until the thread finishes execution and the join method finishes. This greatly limits the performance of the program.

Perl provides another way to harvest threads: detach. Detach is characterized by the direct separation of the slave thread from the main thread, which is not related to each other in the future. Occupied resources are automatically released when execution from the thread is complete. It takes a lot of effort and effort. Here we will try to harvest threads using the detach method and use semaphores to control the number of threads.

Example:

Use threads;use Thread::Semaphore;my $thread;my $max_threads=5;my $semaphore=new Thread::Semaphore ($max_threads); print localtime (time), "\ n"; while () {if ($j > 10) {last;} $jackers 1; # get a semaphore; when the number of threads executed is 5, the fetch fails and the main thread waits. The new semaphore is available until a thread ends. Return to normal operation; $semaphore- > down (); my $thread=threads- > new (\ & ss,$j,$j); # create thread; $thread- > detach (); # split thread;} # what must exist to control the completion of all split threads before the end of the main thread. Otherwise, when the master thread ends and the slave thread does not finish, the slave thread will have to be forcibly kill off. & waitquit; print localtime (time), "\ n"; sub ss () {my ($tgrammes) = @ _; sleep ($t); print "$s\ t", scalar (threads- > list ()), "\ tSecretj\ t", localtime (time), "\ n"; $semaphore- > up (); # release semaphore when thread execution is complete. } # from http://icylife.net/yunshu/show.php?id=617; sub waitquit {print "Waiting to quit...\ n"; my $num=0; while ($numdown (); $num++; print "$num thread quit...\ n";} print "All $max_thread thread quit\ n";}

The running result of the program is consistent with that of the previous article. Do not describe (http://blog.sina.com.cn/s/blog_7ea1e7fc0100r61x.html).

The biggest difference from the code used last time is that an outer loop is missing to determine whether the slave thread has completed execution. It greatly reduces the utilization rate of CPU.

The above is all the contents of the article "sample Analysis of perl Multithreading". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.