What are the methods of using Shell scripts to disorderly arrange the contents of files?
The main content of this article is to explain "what are the ways to use Shell scripts to arrange the contents of files out of order", interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Next let the editor to take you to learn "what are the ways to use Shell scripts to disorderly arrange the contents of files?"
Shuffle question: shuffle a pair of cards, is there any good way? Can you wash evenly and quickly? That is, relative to a file, how to efficiently achieve disordered arrangement?
ChinaUnix is indeed a place full of Shell experts, as long as you want to get the question, you can find the answer there. R2007 gives a trick by using the $RANDOM variable of Shell to add a random line number to each line of the original file and then sort it according to that random line number, and then filter out the temporarily added line number, so that the new file obtained after the operation is equivalent to being randomly "washed" once:
The code is as follows:
While read I Tian do echo "$I $RANDOM"; the content of the line done
That is, the random function constructed.
Thus, there are:
The code is as follows:
Awk 'BEGIN {srand ()} {b [rand () NR] = $0} END {for (x in b) print b [x]}' data
In fact, you don't have to worry too much about the problem of using too much memory, you can do a test:
Test environment:
PM 1.4GHz CPU,40G hard disk with 256m LAPTOP memory
SUSE 9.3 GNU bash version 3.00.16 GNU Awk 3.1.4
Generate a random file of more than half a million lines, about 38m:
The code is as follows:
Od / dev/urandom | dd count=75000 > data
Take the less efficient method one:
Time spent in a shuffle:
The code is as follows:
Time awk-v N = `sed-n'$= 'data`'
BEGIN {
FS= "\ n"
RS= ""
}
{
Srand ()
While (tweezers) {
X=int (N*rand () + 1)
A [x] + +
If (a [x] = = 1)
{
Print $xx _ tcm +
}
}
}
'data
Result (contents of the file are omitted):
The code is as follows:
Real 3m41.864s
User 0m34.224s
Sys 0m2.102s
So efficiency is barely acceptable.
The test of method 2:
The code is as follows:
Time awk-f awkfile datafile
Result (contents of the file are omitted):
The code is as follows:
Real 2m26.487s
User 0m7.044s
Sys 0m1.371s
The efficiency is obviously better than the first one.
Then examine the efficiency of method 3:
The code is as follows:
Time awk 'BEGIN {srand ()} {b [rand () NR] = $0} END {for (x in b) print b [x]}' data
Result (contents of the file are omitted):
The code is as follows:
Real 0m49.195s
User 0m5.318s
Sys 0m1.301s
It's pretty good for a 38m file.
A disordered code from the python version written by flyfly is attached:
The code is as follows:
# coding:gb2312
Import sys
Import random
Def usage ():
Print "usage:program srcfilename dstfilename"
Global filename
Filename = ""
Try:
Filename = sys.argv [1]
Except:
Usage ()
Raise ()
# open the phonebook file
F = open (filename,'r')
Phonebook = f.readlines ()
Print phonebook
F.close ()
# write to file randomly
Try:
Filename = sys.argv [2]
Except:
Usage ()
Raise ()
F = open (filename,'w')
Random.shuffle (phonebook)
F.writelines (phonebook)
F.close ()
At this point, I believe you have a deeper understanding of "what are the ways to use Shell scripts to arrange the contents of files out of order". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!
