Shulou(Shulou.com)06/02 Report--

It is believed that many inexperienced people don't know what to do about how to use Faiss library to realize vector nearest neighbor search in Python. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

1. Read the Embedding data trained in advance.

Import pandas as pdimport numpy as np

Df = pd.read_csv (". / datas/movielens_sparkals_item_embedding.csv")

# extract the ID list of movies to use and convert it to int64ids = df ["id"] .values.astype (np.int64)

# record the size of the ID list ids_size = ids.shape [0]

# read content embedding data and convert it to 2D arrayimport jsonimport numpy as npdatas = [] for x in df ["features"]: datas.append (json.loads (x)) # becomes 2D arraydatas = np.array (datas) .astype (np.float32)

# record data dimension dimension = datas.shape [1]

2. Use faiss to build the index

Import faiss

Index = faiss.IndexFlatL2 (dimension) index2 = faiss.IndexIDMap (index)

Index2.add_with_ids (datas, ids)

3. Realize nearest neighbor search.

# read user embedding data df_user = pd.read_csv (". / datas/user_embedding.csv") df_user.head ()

# pick an embedding of user and convert it to a row of N-column 2D arrayuser _ embedding = np.array (df_ user [DF _ user ["id"] = = 10] ["features"]. Iloc [0]) user_embedding = np.expand_dims (user_embedding, axis=0) .astype (np.float32) user_embedding

# to search, where I is the nearest neighbor ID list tok = 30D, I = index.search (user_embedding, topk) # actual search

4. Stitching to get the content list

# change the searched ID to Seriestarget_ids = pd.Series (I [0], name= "MovieID")

# read movie data df_movie = pd.read_csv (". / datas/ml-1m/movies.dat", sep= "::", header=None, engine= "python", names = "MovieID::Title::Genres" .split ("::")) # implementation content JOINdf_result = pd.merge (target_ids, df_movie) finish reading the above content Have you mastered how to use Faiss library to achieve vector nearest neighbor search in Python? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.