Shulou(Shulou.com)05/31 Report--

This article mainly introduces Python how to use the MapReduce programming model to count sales related knowledge, the content is detailed and easy to understand, the operation is simple and fast, has a certain reference value, I believe that after reading this Python how to use the MapReduce programming model to calculate sales will have something to gain, let's take a look.

MapReduce is a programming model for parallel operations on large datasets (larger than 1TB). The concepts "Map" and "Reduce" are their main ideas, both borrowed from functional programming languages and features borrowed from vector programming languages. It greatly facilitates programmers to run their programs on distributed systems without distributed parallel programming. The current software implementation is to specify a Map (mapping) function, which is used to map a set of key-value pairs into a new set of key-value pairs, and to specify concurrent Reduce (reduction) functions, which are used to ensure that each of the mapped key-value pairs shares the same key group.

The following is simulated by manually implementing the MapReduce code to count the number of sales.

1. Generate simulation data #! / usr/bin/python#-*-coding: utf-8-*-import random# Simulation products stocks = ["HUAWEI Mate40", "Apple iphone13", "Apple MacBook Pro14", "ThinkBook 14p", "RedmiBook Pro14", "Feihe Xing Feifan Baby Milk Powder", "Love Tamei Baby Milk Powder", "Li Ning Sports Men's Pants", "Xiaomi Stepper Elliptic Machine", "L'Or é al Mask", "Royal Nai Fang Mask" "L'Or é al Men's suit", "Jin Liufu Liquor", "Niulanshan 42 degrees", "Maotai Feitian"] # sales orders sales_list = list () # generate 100 buyer orders Three items per order for i in range (100): sstocks = list () for j in range (3): sstocks.append (stock.randint (0jin14)]) a = "buyer" + str (item1) + ":" +, ".join (sstocks) print (a)

2. Mapper implementation

Use the result of the first step as the input of the second step.

#! / usr/bin/python#-*-coding: utf-8-*-import sys# reads data from the console Send each line of data for line in sys.stdin: # split the order orders = line.strip (). Split (":") if len (orders) = 2: # split the goods in the order stocks = orders [1] .split (",") for stock in stocks: # output each item as key,value print ('% s % s'% (stock,1))

3. Reducer implementation

Use the result of step 2 as the input of step 3.

#! / usr/bin/python#-*-coding: utf-8-*-import sys# creates an empty dictionary for the sales data of each item stock_dict = dict () for line in sys.stdin: if len (line.strip ()) > = 1: # split each line of goods Sales volume stock, sales = line.split (',') # determine whether the current item is stored in the dictionary if stock in stock_dict: # if so, take out the item and sales volume in the dictionary, add the current sales volume and put it in stock_ [stock] = stock_ inventory [stock] + int (sales) else: # if not Directly put the goods and sales data into the dictionary stock_ [stock] = int (sales) # traversal dictionary list to get the sales volume of each item for stock, sales in stock_dict.items (): print ('% s\ t% s'% (stock, sales)

This is the end of the article on "how Python uses the MapReduce programming model to count sales". Thank you for reading! I believe that everyone has a certain understanding of the knowledge of "how Python uses the MapReduce programming model to count sales". If you want to learn more, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.