Shulou(Shulou.com)06/01 Report--

This article mainly introduces python how to achieve Simhash algorithm, the article is very detailed, has a certain reference value, interested friends must read it!

1. Simhash steps

Simhash includes five steps: word segmentation, hash, weighting, merging and dimensionality reduction.

The simhash code is as follows:

Import jiebaimport jieba.analyseimport numpy as npclass SimHash (object): def simHash (self, content): seg = jieba.cut (content) # jieba.analyse.set_stop_words ('stopword.txt') # jieba based on TF-IDF extraction keywords keyWords = jieba.analyse.extract_tags ("|" .join (seg), topK=10, withWeight=True) keyList = [] for feature Weight in keyWords: # print ('feature:' + feature) print (' weight: {} '.format (weight)) # weight = math.ceil (weight) weight = int (weight) binstr = self.string_hash (feature) print (' feature:% s, string_hash% s'% (feature) Binstr)) temp = [] for c in binstr: if (c = ='1'): temp.append (weight) else: temp.append (- weight) keyList.append (temp) listSum = np.sum (np.array (keyList)) Axis=0) if (keyList = = []): return '00' simhash =' 'for i in listSum: if (I > 0): simhash = simhash +' 1' else: simhash = simhash +'0' return simhash def string_hash (self) Source): if source = "": return 0 else: temp = source [0] temp1 = ord (temp) x = ord (source [0])

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.