Shulou(Shulou.com)06/01 Report--

This article mainly introduces how Spark defines schema dynamically. It is very detailed and has certain reference value. Friends who are interested must finish it!

Define schema dynamically

Different schema needs to be defined according to different data

Val schemaInfo = "name age" val fields = schemaInfo.map (item= > item.split (") .map (item= > StructField (item,StringType,nullable=true)) val schema = StructType (fields) val rowRDD = peopleRDD.map (_ .split (") .map (attributes= > Row (attributes (0), attributes (1)) val peopleDF = spark.createDataFrame (rowRDD,schema) peopleDF.show () these are all the contents of the article "how Spark defines schema dynamically". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.