Shulou(Shulou.com)06/03 Report--

Small mistakes easy to make in Storm Development

Storm is a distributed and fault-tolerant real-time computing system. Following Eclipse Public License 1.0 Storm, you can easily write and expand complex real-time computing in a computer cluster. Storm is to real-time processing, just like Hadoop to batch processing. Storm ensures that every message is processed, and it is fast-- in a small cluster, millions of messages can be processed per second. You can use any programming language for development.

Advantages of Storm

1. A simple programming model. Similar to MapReduce, which reduces the complexity of parallel batch processing, Storm reduces the complexity of real-time processing.

two。 Service-oriented, a service framework that supports hot deployment, instant online or offline App.

3. A variety of programming languages are available. You can use various programming languages on top of Storm. Clojure, Java, Ruby, and Python are supported by default. To increase support for other languages, you only need to implement a simple Storm communication protocol.

4. Fault tolerance. Storm manages worker processes and node failures.

5. Scale horizontally. Computing is performed in parallel among multiple threads, processes, and servers.

6. Reliable message processing. Storm ensures that each message is fully processed at least once. When a task fails, it is responsible for retrying the message from the message source.

7. fast. The design of the system ensures that messages can be processed quickly, and ZeroMQ is used as its underlying message queue.

8. Local mode. Storm has a "local mode" that completely simulates the Storm cluster during processing. This allows you to do development and unit testing quickly.

However, during the development process, the kill Topology of Storm often leads to some inexplicable errors, such as the following error:

The cause of the exception is

There is no topology running by kill, but the storm process of kill nimbus and supervisor comes first.

Solution:

Deleted the contents of the folder of "storm.local.dir" in the configuration (this is the main cause of ERROR)

Delete / storm/assignments under Zookeeper client

Delete / storm/workerbeats under Zookeeper client

Delete the workdir under the storm directory (that is, the folder where TOPOLOGY serialization is configured in the configuration file)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.