Shulou(Shulou.com)06/01 Report--

This article mainly analyzes how to analyze the relevant knowledge points of the dmp user portrait project in big data, the content is detailed and easy to understand, the operation details are reasonable, and has a certain reference value. If you are interested, you might as well follow the editor and learn more about "how to analyze the dmp user profile project in big data".

First, the introduction of accurate Internet advertising (1) the display principle of dsp:

① users browse the media website, which initiates an advertisement request to the AdExchange through the added SSP code.

② AdExchange sends the key information of this request (such as domain names URL, IP, Cookie, etc.) to multiple DSP at the same time. We call this request Bid Request.

After receiving the request, ③ DSP uses Cookie, IP, URL and other information to decide whether to participate in the bidding. DSP can use Cookie to query the user's historical behavior in its own system to calculate population attributes and interests. If DSP does not have this ability, it can judge the user's characteristics through the assistance of a third party DMP, so as to bid more reasonably. If you bid, return the price, advertisement to be displayed, jump link and other information to AdExchange. We call this information return Bid Response.

④ AdExchange selects the DSP with the highest bid, informs the DSP that it has won the bid, and tells it the cost of the display (because the second-order pricing is used in RTB, the second highest bid, so DSP does not know the actual cost and needs to be notified by AdExchang again). At the same time, AdExchange returns the html content to the media to display the advertisement.

The static resources (pictures, Flash, etc.) of ⑤ advertisements are generally stored on the DSP server, so when loading the advertising code, you need to go to DSP to request static resources.

⑥ DSP returns static resources to complete the rendering and display of ads.

(2) explanation of related nouns:

   DSP:DSP is a system and an online advertising platform. It serves advertisers and helps advertisers to place advertisements on the Internet or mobile Internet. DSP can make it easier for advertisers to follow a unified bidding and feedback mode, and buy high-quality advertising inventory in real time at reasonable prices for online advertisements located on multiple advertising trading platforms.

   Ad Exchange:Ad Exchange is the Internet advertising trading platform, which connects DSP (buyer platform) and SSP (seller platform). It collects a large amount of media traffic through access to SSP, so as to collect and process data belonging to advertising target customers. Ad Exchange is the trading place to achieve accurate marketing.

   SSP:SSP (Suply Side Platform), the supplier platform, that is, the media platform, that is, the medium through which consumers see advertisements.

   DMP: the data management platform helps all parties involved in the purchase and sale of advertising inventory manage their data, make it easier to use third-party data, enhance their understanding of all of this data, send back data, or transfer customized data to a platform for better positioning.

(3) DMP details:   1) user data classification:

   -first party data: the demand side is the advertiser's own user data, including website / APP monitoring data, CRM (Custom Relation Management) data, e-commerce transaction data, etc.

   -second-party data: business data accumulated by demand-side service providers in the process of advertising, such as audience browsing advertising, clicking advertising and other related data accumulated in DSP platform business.

   -third-party data: data owned by non-direct partners, such as operator data, etc.

  2) data analysis capabilities:

In   , user profile is the basis, that is, through the tagging of user information, a complete picture of user information is abstracted perfectly, and sufficient data basis is provided for further accurate and rapid analysis of user behavior habits, consumption habits and other important information. As the name implies, the focus of the user's portrait is to tag the user, and a tag is usually considered to be highly refined features, such as age, gender, region, user preference, and so on. Finally, taking a comprehensive view of all the tags of the user, you can outline the user's three-dimensional portrait.

  3) the role of DMP:

  -ability to query, feedback and render results quickly

  -helps customers enter the market cycle faster

  -enables collaboration between business users and partners

  -ability to predict, analyze and respond in depth

  -can bring competitive advantage in all aspects

  -reduces information access and labor costs

2. Project requirements (1) requirements realized:

 -simulates reading data from a file, uses spark to clean the data, distributes it to dataframe, and after compression, outputs it to a file in the form of parquet.

 -read the parquet file, use sparksql to etl it on demand, and output it to MySQL (report)

 -read the parquet file, through userid, according to each user, type the appropriate tag, and finally put it into hbase

(2) Field introduction of the original file:

Serial number attribute name description

1 Sessionid:String session ID

2 Advertisers:Int advertiser id

3 Adorderid:Int advertisement id

4 Adcreativeid:Int Advertising creativity id (> = 200000:dsp)

5 Adplatformproviderid:Int Advertising platform id (> = 100000:rtb)

6 Sdkversion:String Sdk version

7 Adplatformkey:String platform business key

8 Putinmodeltype:Int: according to the advertiser's delivery mode, 1: display quantity, 2: click

9 Requesmode:Int data request method (1: request, 2: show, 3: click)

10 Adprice:Double advertising price

11 Adpprice:Double platform price

12 Requestdate:String request time format: yyyy-m-dd hh:mm:ss

13 the real ip address of the Ip:String device user

14 Appid:String Application IP

15 Appname:String application name

16 unique identification of Uuid:String device

17 Device:String device model, such as htc,iphone

18 Client:Int device type (e. G.: 1 Client:Int device type, 2 Client:Int device type, 3 device type)

19 Osversion:String device operating system version

20 Density:String device screen density

21 Pw:Int device screen width

22 Ph:Int device screen height

23 longitude of Long:string equipment

24 Dimensions where Lat:String devices are located

25 Province name of the Provincename:String device

26 name of the city where the Cityname:String device is located

27 Ispid:Int operator id

28 Ispname:String operator name

29 Networkmannerid:Int networking mode id

Name of 30 Networkmannername:String networking method

31 Iseffective:Int valid logo (valid refers to normal billing) (0: invalid, 1: valid)

Whether to charge for 32 Isbilling:Int (0: no charge, 1: charge)

33 Adspacestype:Int ad space type (1:banner2: plug-in 3: full screen)

34 Adspacetypename:String ad space type name (banner banner, insert screen, full screen)

35 Devicetype:Int device type (1: mobile: 2: tablet)

36 Processnode:Int process node (1: number of requests ktp2: valid requests 3: advertising requests)

37 Apptype:Int Application Type id

38 name of the county where the District:String equipment is located

39 Paymode:Int 's payment model for platform merchants 1: display volume (CPM) 2: click (cpc)

Whether 40 Isbid:Int is rtp (1 participates in bidding 0 does not participate in bidding)

41 Bidprice:Double Rtp bidding price

42 Winprice:Double Rtp successful bidding price

43 whether the Iswin:Int bid is successful or not

44 Cur:String Values:umd | rmb, etc.

45 Rate:Double exchange rate

46 Cnywinprice:Double Rtp bid successfully converted into RMB price

47 Imei:String Imei

48 Imac:string Mac

49 Idfa:String Idfa

50 Openudid:String Openudid

51 Androidid:String Androidid

52 Rtbprovice:String Rtb Province

53 Rtbcity:String Rtb City

54 Rtbdistrict:String Rtb area

55 Rtbstreet:String Rtb Street

Market download address of 56 Storeurl:String App

57 Realip:String Real ip

58 Isqualityapp:Int preferred logo

59 Bidfloor:Double low price

The width of 60 Aw:Int advertising space

61 Ah:Int advertising space is high

62 Imeimd5:String Imei_md5

63 Macmd5:String Mac_md5

64 Idfamd5:String Idfa_md5

65 Openudidmd5:String Openudid_md5

66 Androididmd5:String Androidid_md5

67 Imeisha1:String Imei_sha1

68 Macsha1:String Mac_sha1

69 Idfasha1:String Idfa_sha1

70 Openudidsha1:String Openudid_sha1

71 Androididsha1:String Androidid_sha1

72 Uuidunknow:String Uuid_unknow tanx ciphertext

73 tanx plaintext decrypted by Decuuidunknow:String

74 Userid:String platform user id

75 Reqdate:String date

76 Reqhour:String hours

77 Iptype:Int represents the ip type

78 Initbidprice:Double initial bid

79 Adpayment:Double Advertising consumption after conversion

80 Agentrate:Double agent profit margin

81 Lomarkrate:Double agent profit margin

82 Adxrate:Double media profit margin

83 Title:String title

84 Keywords:String keyword

85 Tagid:String advertising space logo (when the video traffic value is the video ID number)

86 Callbackdate:String callback time in YYYY/mm/dd hh:mm:ss format

87 Channeid:String Channel ID

88 Megratype:Int Media Type 1: long tail Media 2: video Media 3: independent Media, default: 1

(3) report:

Geographical distribution:

End equipment:

Operating system

Media Analysis:

Channel report:

(4) user portraits:

Tag 1: advertising space type (tag format: LC03- > 1 or LC16- > 1) xx is a number, less than 10 plus 0

Tag 2: APP name (tag format: APPxxxx- > 1) xxxx is the name of APP, which is converted using the cache file appname_dict

Label 3: Channel (label format: CNxxxx- > 1) xxxx is Channel ID

Tag 4: device: operating system | networking method | operator

Device operating system

1 Android D0001001

2 IOS D0001002

3 Winphone D0001003

4 other D0001004

Equipment networking mode

WIFI D0002001

4G D0002002

3G D0002003

2G D0002004

Equipment operator scheme

Move D0003001

Unicom D0003002

Telecom D0003003

OPERATOROTHER D0003004

Tag 5: keyword (tag format: Kxxx- > 1) xxx is the keyword. The number of keywords cannot be less than 3 characters and not more than 8 characters. If the keyword contains "|", it is divided into an array and converted into multiple keyword tags.

Tag 6: regional label (provincial label format: ZPxxx- > 1, prefectural and municipal label format: ZCxxx- > 1) xxx is the name of the province or city

Tag 7: 6) context tag: tag the data with the above six types, and merge the current file according to [user ID]. The data is saved in the format: userid K × × Zhi: 3D00030002Vl1.

This article mainly analyzes how to analyze the relevant knowledge points of the dmp user portrait project in big data, the content is detailed and easy to understand, the operation details are reasonable, and has a certain reference value. If you are interested, you might as well follow the editor and learn more about "how to analyze the dmp user profile project in big data".

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.