Shulou(Shulou.com)05/31 Report--

This article shows you how to encapsulate Protocol in Storm, which is concise and easy to understand, which will definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

Encapsulation for Protocol

Package com.digitalpebble.storm.crawler.fetcher;import com.digitalpebble.storm.crawler.util.Configuration;public interface Protocol {public ProtocolResponse getProtocolOutput (String url) throws Exception; public void configure (Configuration conf);}

Encapsulation for ProtoclFactory

Package com.digitalpebble.storm.crawler.fetcher;import java.net.URL;import java.util.WeakHashMap;import com.digitalpebble.storm.crawler.fetcher.asynchttpclient.AHProtocol;import com.digitalpebble.storm.crawler.util.Configuration;/** * @ author Yin Shuai * * / public class ProtocolFactory {private final Configuration config; private final WeakHashMap cache = new WeakHashMap (); public ProtocolFactory (Configuration conf) {config = conf } / * * Returns an instance of the protocol to use for a given URL * * / public synchronized Protocol getProtocol (URL url) {/ / get the protocol String protocol = url.getProtocol (); Protocol pp = cache.get (protocol); if (pp! = null) return pp; / / yuk! Hardcoded for now pp = new AHProtocol (); pp.configure (config); cache.put (protocol,pp); return pp;}}

Encapsulation for ProtocolResponse

Package com.digitalpebble.storm.crawler.fetcher;import java.util.HashMap;public class ProtocolResponse {final byte [] content; final int statusCode; final HashMap metadata; public ProtocolResponse (byte [] c, int s, HashMap md) {content = c; statusCode = s; metadata = md;} public byte [] getContent () {return content;} public int getStatusCode () {return statusCode } public HashMap getMetadata () {return metadata;}} the above content is how Protocol is encapsulated in Storm. Have you learned any knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.