Shulou(Shulou.com)05/31 Report--

This article mainly introduces "how to use sbt to package the spark project written by scala". In the daily operation, I believe many people have doubts about how to use sbt to package the spark project written by scala. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful to answer the doubt of "how to use sbt to package scala project written by spark". Next, please follow the editor to study!

Project root directory build.sbt

/ / Project name name: = "sparkScala" / / Project version version: = "1.0" / / scala version scalaVersion: = "2.11.7" / / jdk version javacOptions + + = Seq ("- source", "1.7", "- target", "1.7") / / dependency,% indicates that it is required for testing, general% % "provided" means that this jar does not enter into the final jar file libraryDependencies + = "com.alibaba"% "fastjson"% "1.2.4" libraryDependencies + = "org.apache.spark"% "spark-core"% "1.4.1"% "provided" libraryDependencies + = "org.apache.spark"% "spark-sql"% "1.4.1"% "provided" / / main function mainClass in Compile : = Some ("main.MergePartition") / / when packing Exclude the scala class library assemblyOption in assembly: = (assemblyOption in assembly) .value.copy (includeScala = false)

Add assembly plug-in to the project root directory project/plugins.sbt

AddSbtPlugin ("com.eed3si9n"sbt-assembly"0.13.0")

Finally, it is executed under the project root.

Sbt clean assembly

You can see the delicious jar package file.

When you unzip it, you can see that fastjson has been entered into the jar package, and everything else to be excluded has been excluded.

Main-Class: main.MergePartition, the main function execution class, is also written in the META-INF/MANIFEST.MF file.

At this point, the study on "how to use sbt to package a spark project written by scala" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.