Shulou(Shulou.com)05/31 Report--

This article mainly introduces "what to do if the namenode of hdfs is hung up". In the daily operation, I believe that many people have doubts about what to do when the namenode of hdfs is hung up. The editor has consulted all kinds of materials and sorted out a simple and easy-to-use method of operation. I hope it will be helpful to answer the doubts of "what to do if hdfs's namenode is hung up?" Next, please follow the editor to study!

Q1: how to start with spark for beginners?

Learn the contents of Scala first, and "learn Scala quickly" is highly recommended.

Then follow our free release of "Cloud Computing distributed big data Spark practice Master Road (a total of 3 books)" step by step learning, in which "Cloud Computing distributed big data Spark practice Master Road-from scratch" covers all Spark1.0 topics: including the construction of Spark cluster, Spark architecture design, Spark kernel analysis, Shark, Spark SQL, Spark Streaming, graphical computing GraphX, machine learning, Spark on Yarn, JobServer, etc. In order to make it convenient for you to learn Spark, the web release version is released in a graphic and textual way, so that when you read it, it will be like Jia Lin doing a demonstration and explanation around you. Take the construction of Spark cluster as an example, for more than 90% of people who want to learn Spark, how to build Spark cluster is one of the biggest difficulties. In order to solve all the difficulties of building Spark cluster, Jia Lin divides the construction of Spark cluster into four steps in the first chapter of "Cloud Computing distributed big data Spark practical experts-starting from scratch." each step is divided into a section, starting from scratch. Do not need any pre-knowledge, cover every detail of the operation, build a complete Spark cluster.

Q2: I would like to ask, hdfs's namenode is dead, how to deal with it?

Use ZooKeeper

Use Mesos

Use Yarn

Q3: is there a big difference between python and scala?

It's not big in terms of the style of the code.

There are also many people in the world who use python to develop Spark programs

But Scala is the most recommended, because the Spark framework is written in Scala, and the support for Scala in API is the best.

Q4: how does SPARK support hundreds of terabytes of data now?

Spark can handle tens or hundreds of T data very well.

Just as Spark can easily handle PB-level data

Q5: can the results be output directly to a relational database?

Not right now.

The result is usually output directly to HDFS.

After using Sqoop and other tools to import data into Oracle, MySQL and other databases

What is required for a Q6:SPARK environment? In addition to the HDFS foundation, is there a SPARK integration package?

The installation of Spark requires the HDFS of Hadoop

Spark has its own integration package, but it still needs the cooperation of HDFS

At the same time, Spark can also be deployed to the Amazon cloud

What are the disadvantages of Q7:spark? What is not suitable for doing?

At present, the main disadvantage is that the support for fine granularity of data is not good enough.

Can Q8:spark sql replace hive and hbase?

Spark SQL can replace Hive

Spark SQL can accomplish most of the functions of HBase.

Spark Streaming with Spark SQL can replace HBase.

Q9: can I look at scala language directly without java foundation?

You can look at Scala language directly.

At this point, the study of "what to do if the namenode of hdfs is hung up" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.