Shulou(Shulou.com)06/03 Report--

Local mode operation

The local mode is similar to the storm cluster, which is a process used to write and test topology. Running topology in local mode is similar to running topology on a cluster. Create a local cluster:

Import backtype.storm.LocalCluster

LocalCluster cluster = new LocalCluster ()

Submit the cluster to use submitTopology

Kill the cluster using killTopology

Shut down a local cluster using cluster.shutdown ()

Public configuration in local mode:

Config.TOPOLOGY_MAX_TASK_PARALLELISM

Config.TOPOLOGY_DEBUG:

Production cluster operation

I. steps

1. Define topology. If you use Java, use TopologyBuilder to define it.

2. Use StormSubmitter to submit topology to the cluster. The required parameters need topology name, parameter configuration of topology, and topology itself.

Example:

Config conf = new Config ()

Conf.setNumWorkers (20)

Conf.setMaxSpoutPending (5000)

StormSubmitter.submitTopology ("mytopology", conf, topology)

3. Use storm client commands to create the jar package, and the jar package contains all your code.

4. Submit topology and specify parameters for jar package

Storm jar path/to/allmycode.jar org.me.MyTopology arg1 arg2 arg3

Example:

1 public static void main (String [] args) throws Exception {

2 TopologyBuilder builder = new TopologyBuilder ()

3 builder.setSpout ("random", new RandomWordSpout (), 2)

4 builder.setBolt ("transfer", new TransferBolt (), 4). ShuffleGrouping ("random")

5 builder.setBolt ("writer", new WriterBolt (), 4) fieldsGrouping ("transfer", new Fields ("word"))

6 Config conf = new Config ()

7 conf.setNumWorkers (4); / / set to start 4 Worker

8 conf.setNumAckers (1); / / set up an ack thread

9 conf.setDebug (true); / / set up to print all sent messages and system messages

10 StormSubmitter.submitTopology ("test", conf, builder.createTopology ())

11}

II. Public allocation

Config.TOPOLOGY_WORKERS: sets the number of worker that executes topology

Config.TOPOLOGY_ACKER_EXECUTORS:

Config.TOPOLOGY_MAX_SPOUT_PENDING

Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS: the default is 30s

Config.TOPOLOGY_SERIALIZATIONS

3. Kill topology

Storm kill stormname

Update a running topology

Kill a running topology, submit a new topology, and plan to use the storm swap command

5. Monitor the work logs of topology using Storm UI or cluster

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.