Example Analysis of Mdrill Test data Writing Program
Mdrill test data writing program example analysis, in view of this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.
Sql of the table to which the data needs to be written:
CREATE TABLE tv (
Thedate string
Tv string
)
Java code written:
Import java.io.BufferedWriter
Import java.io.IOException
Import java.io.OutputStreamWriter
Import java.text.SimpleDateFormat
Import java.util.Calendar
Import java.util.Date
Import org.apache.hadoop.conf.Configuration
Import org.apache.hadoop.fs.FSDataOutputStream
Import org.apache.hadoop.fs.FileSystem
Import org.apache.hadoop.fs.Path
Public class ImportData {
Static Configuration conf = null
Static FileSystem fs = null
Public static void main (String [] args) throws IOException {
Conf = ConfigurationUtil.getConf ()
Fs = FileSystem.get (conf)
/ / TODO Auto-generated method stub
String pathStirng = "/ group/tbdp-etao-adhoc/p4padhoc/tablelist/tv"
/ / generate one-year folders (365) and write data to the folder name: dt=20140201
Calendar calendar = Calendar.getInstance (); / / create a calendar object
Calendar.setTime (new Date ())
Calendar.set (2013, 1,1)
SimpleDateFormat sf = new SimpleDateFormat ("yyyyMMdd")
While (calendar.get (Calendar.YEAR) < 2014) {
System.out.println (sf.format (calendar.getTime ()
String dateStr = sf.format (calendar.getTime ())
String path = pathStirng + "/ dt=" + dateStr
Mkdir (path)
WriteTVData (path, dateStr)
Calendar.add (Calendar.DAY_OF_YEAR, 1)
}
Fs.close ()
}
/ * *
* write 100000 digits
* @ param dir
* @ throws IOException
, /
Public static void writeTVData (String dir, String dateStr)
Throws IOException {
FSDataOutputStream fos = null
OutputStreamWriter osw = null
BufferedWriter bw
Fos = fs.create (new Path (dir + "/ data.txt"), false)
Osw = new OutputStreamWriter (fos)
Bw = new BufferedWriter (osw)
/ / append data to data.txt row by row
For (int I = 0; I < 1000000; iTunes +) {
StringBuffer strBuffer = new StringBuffer ()
StrBuffer.append (dateStr)
StrBuffer.append ((char) 1)
StrBuffer.append (I)
StrBuffer.append (System.lineSeparator ())
Bw.write (strBuffer.toString ())
}
Bw.close ()
Osw.close ()
Fos.close ()
}
Public static void mkdir (String dir) throws IOException {
Path p = new Path (dir)
If (! fs.exists (p)) {
Fs.mkdirs (p)
}
}
}
The answers to the sample analysis questions about the Mdrill test data writing program are shared here. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel for more related knowledge.
