In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-01-30 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >
Share
Shulou(Shulou.com)05/31 Report--
Xiaobian to share with you how to implement random forest algorithm in spark mllib, I believe most people still do not know how, so share this article for your reference, I hope you have a lot of harvest after reading this article, let's go to understand it together!
Random forest machine: a decision tree forest composed of several decision trees
An algorithm widely used in computer vision and other fields. It can be used not only for classification, but also for regression, that is, prediction. Random forest machine is composed of multiple decision trees. Compared with a single decision tree algorithm, it has better classification and prediction effects and is not easy to overfit.
Run the code as follows package spark.DTimport org.apache.spark.mllib.tree.RandomForestimport org.apache.spark.mllib.util.MLUtilsimport org.apache.spark. {SparkConf, SparkContext}/** * Random Rainforest Decision Tree * Decision tree forest composed of several decision trees, * The essence of random rainforest is to build multiple decision trees and then obtain the average value of all decision trees * ps: A dataset includes one score, assuming a total of 5 scores, and in practical application, dichotomy * 1 2 3 | 4 5 * That is, there are 2 bins containing data sets {1,2,3},{4,5} * respectively. split is set to 3 * * Created by eric on 16-7-20. */object RFDTree { val conf = new SparkConf() //Create environment variables .setMaster("local") //Set localization processing .setAppName("ZombieBayes") //setName val sc = new SparkContext(conf) def main(args: Array[String]) { val data = MLUtils.loadLibSVMFile(sc, "./ src/main/spark/DT/DTree.txt") val numClasses = 2//Number of Classes val categoricalCharacteriesInfo = Map[Int, Int]()//Set input format val numTrees = 3//Number of decision trees in random rainforest val featureSubSetStrategy = "auto" //Sets the number of attributes calculated on a node, automatically determining the number of attributes per node val impurity = "entropy" //Set information gain calculation method val maxDepth = 5 //Max Depth val maxBins = 3 //Set split dataset val model = RandomForest.trainClassifier( data, numClasses, categoricalFeaturesInfo, numTrees, featureSubSetStrategy, impurity, maxDepth, maxBins )//Create a model model.trees.foreach(println)//Print information about each tree println(model.numTrees) }} The result is as follows
Each time the tree has a different depth and nodes
The above is "how to implement random forest algorithm in spark mllib" all the contents of this article, thank you for reading! I believe that everyone has a certain understanding, hope to share the content to help everyone, if you still want to learn more knowledge, welcome to pay attention to the industry information channel!
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.