[筆記] Hadoop 2.x YARN Test Case如何執行？放在哪裡？

Hadoop 2.x 開始有太多指令和API 都不太一樣了，而目前網路上能搜尋到的幾乎都是 1.x 版本的資訊，所以如果遇到無法執行的，還是回到官網看文件最實在。關於Hadoop 內建的Benchmark 程式原本釋放在hadoop-common-test，那現在搬到哪裡去了呢？答案是：

/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.2.0.2.0.6.0-101-tests.jar



執行以下指令，就可以看到有哪些Benchmark 的方法可以使用：

# hadoop jar  hadoop-mapreduce-client-jobclient-2.2.0.2.0.6.0-101-tests.jar

An example program must be given as the first argument.
Valid program names are:

• DFSCIOTest: Distributed i/o benchmark of libhdfs.
• DistributedFSCheck: Distributed checkup of the file system consistency.
• JHLogAnalyzer: Job History Log analyzer.
• MRReliabilityTest: A program that tests the reliability of the MR framework by injecting faults/failures
• SliveTest: HDFS Stress Test and Live Data Verification.
• TestDFSIO: Distributed i/o benchmark.
• fail: a job that always fails
• filebench: Benchmark SequenceFile(Input|Output)Format (block,record compressed and uncompressed), Text(Input|Output)Format (compressed and uncompressed)
• loadgen: Generic map/reduce load generator
• mapredtest: A map/reduce test check.
• minicluster: Single process HDFS and MR cluster.
• mrbench: A map/reduce benchmark that can create many small jobs
• nnbench: A benchmark that stresses the namenode.
• sleep: A job that sleeps at each map and reduce task.
• testbigmapoutput: A map/reduce program that works on a very big non-splittable file and does identity map/reduce
• testfilesystem: A test for FileSystem read/write.
• testmapredsort: A map/reduce program that validates the map-reduce framework's sort.
• testsequencefile: A test for flat files of binary key value pairs.
• testsequencefileinputformat: A test for sequence file input format.
• testtextinputformat: A test for text input format.
• threadedmapbench: A map/reduce benchmark that compares the performance of maps with multiple spills over maps with 1 spill

除了 hadoop-mapreduce-client-jobclient 所顯示的test case 外，另外一個有名的Tera sort 則仍然是放在hadoop-mapreduce-examples*.jar裡面，指令如下：

hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.2.0.2.0.6.0-101.jar teragen

hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.2.0.2.0.6.0-101.jar terasort