Shulou(Shulou.com)06/03 Report--

The main content of this article is to explain "what are the ways to use Shell scripts to arrange the contents of files out of order", interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Next let the editor to take you to learn "what are the ways to use Shell scripts to disorderly arrange the contents of files?"

Shuffle question: shuffle a pair of cards, is there any good way? Can you wash evenly and quickly? That is, relative to a file, how to efficiently achieve disordered arrangement?

ChinaUnix is indeed a place full of Shell experts, as long as you want to get the question, you can find the answer there. R2007 gives a trick by using the $RANDOM variable of Shell to add a random line number to each line of the original file and then sort it according to that random line number, and then filter out the temporarily added line number, so that the new file obtained after the operation is equivalent to being randomly "washed" once:

The code is as follows:

While read I Tian do echo "$I $RANDOM"; the content of the line done

That is, the random function constructed.

Thus, there are:

The code is as follows:

Awk 'BEGIN {srand ()} {b [rand () NR] = $0} END {for (x in b) print b [x]}' data

In fact, you don't have to worry too much about the problem of using too much memory, you can do a test:

Test environment:

PM 1.4GHz CPU,40G hard disk with 256m LAPTOP memory

SUSE 9.3 GNU bash version 3.00.16 GNU Awk 3.1.4

Generate a random file of more than half a million lines, about 38m:

The code is as follows:

Od / dev/urandom | dd count=75000 > data

Take the less efficient method one:

Time spent in a shuffle:

The code is as follows:

Time awk-v N = `sed-n'$= 'data`'

BEGIN {

FS= "\ n"

RS= ""

}

{

Srand ()

While (tweezers) {

X=int (N*rand () + 1)

A [x] + +

If (a [x] = = 1)

{

Print $xx _ tcm +

}

}

}

'data

Result (contents of the file are omitted):

The code is as follows:

Real 3m41.864s

User 0m34.224s

Sys 0m2.102s

So efficiency is barely acceptable.

The test of method 2:

The code is as follows:

Time awk-f awkfile datafile

Result (contents of the file are omitted):

The code is as follows:

Real 2m26.487s

User 0m7.044s

Sys 0m1.371s

The efficiency is obviously better than the first one.

Then examine the efficiency of method 3:

The code is as follows:

Time awk 'BEGIN {srand ()} {b [rand () NR] = $0} END {for (x in b) print b [x]}' data

Result (contents of the file are omitted):

The code is as follows:

Real 0m49.195s

User 0m5.318s

Sys 0m1.301s

It's pretty good for a 38m file.

A disordered code from the python version written by flyfly is attached:

The code is as follows:

# coding:gb2312

Import sys

Import random

Def usage ():

Print "usage:program srcfilename dstfilename"

Global filename

Filename = ""

Try:

Filename = sys.argv [1]

Except:

Usage ()

Raise ()

# open the phonebook file

F = open (filename,'r')

Phonebook = f.readlines ()

Print phonebook

F.close ()

# write to file randomly

Try:

Filename = sys.argv [2]

Except:

Usage ()

Raise ()

F = open (filename,'w')

Random.shuffle (phonebook)

F.writelines (phonebook)

F.close ()

At this point, I believe you have a deeper understanding of "what are the ways to use Shell scripts to arrange the contents of files out of order". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.