Shulou(Shulou.com)06/03 Report--

Search for comparison testing tools

Catalogue

1. Introduction to tools. 1

two。 Design ideas. 1

3. Tool practicability. 3

4. Tool integration extension. 6

1. Introduction to tools

Because the company's search service and underlying index creation and maintenance are maintained by colleagues in the Infrastructure Department, business access students are basically the basic functional verification of business-related search tests. in order to more intuitively evaluate the effect and impact of search strategy optimization from multiple dimensions, we have developed this search comparison testing tool. The tool parses the search logs of real users online, invokes pre-online and online search services respectively, and gives the visual comparison results of the number of search results, sorting, service time and other dimensions. The functions and statistics that have been implemented so far include:

Compare the number of pre-launched and online search results for the same search query, and output their respective numbers.

Compare the pre-online and online sorting of the same search results, and output the same sort and different sort results respectively.

Repetitive judgment and repetitive quantity output of search results

For gradient data, that is, a comparative analysis of search terms with a number of search results in the range of one, one hundred, ten thousand, ten thousand and ten thousand.

Statistics of search terms with few search results

Statistics of search time corresponding to search terms

According to the configured floating red threshold, the abnormal data is displayed in the test report, which is convenient for viewing and analysis.

Configuration control: configurable items include business to be tested, global sampling quantity, gradient search word sampling quantity, differential proportion floating red threshold in the report, convenient for other business line access and flexible use of tools.

Flexible expansion, you can simply add other business lines of search results to achieve classes, support more business lines.

(see screenshot below for detailed statistics and presentation format of the report)

two。 Design ideas

The main purpose of the search comparison test is to compare whether there are obvious fluctuations in the number of search results, whether the search results meet the expected results, and whether there is a significant difference in service performance before and after changing the search strategy. Our comparison testing tool is mainly designed and implemented from three dimensions: the number of search results, the ranking of search results and the time of service invocation. The design ideas of the main function points are explained below.

In order to simulate real users, we extract the search query of real users from online search logs for comparison and testing.

Comparison of the number of search results: we take online query to call pre-online and online services respectively, and compare the current number of pre-online and online search results and the number of results found by real users in the last time.

Search service invocation time comparison: we use stopwatch to calculate the time of calling pre-online / online services respectively

Comparison of the ranking of search results: because both pre-online and online services are connected to online databases, and online data are real-time and dynamic, coupled with policy adjustments, the ranking of pre-online / online results corresponding to the same query is not comparable, so we re-splice the query that calls the pre-online service. Considering that the search service can search according to the index id, and the index id does not participate in the scoring of search sorting, we take a certain number of index id of online search results, splice them to query, and re-invoke the pre-online service to search, so that we can compare the ranking of the same position / resume in the two environments.

Gradient data comparison: to a certain extent, the magnitude of search results will also affect the service performance, the number of results and the relative change of ranking, so we sampled and compared the query of different orders of magnitude of results to increase the rationality of the evaluation of the comparison results.

Statistics with few results: scenarios with few results are an important reference for search optimization.

Statistics are made on the lack of results in online and online environments to support subsequent optimization.

Repetitive result statistics: repetitive results are meaningless to users, so we judge whether there are repeated results.

The basic flow chart of the comparison tool is shown in the above figure.

3. Tool practicability

At present, our search comparison test tool has been applied to the comparison test of talent position and resume search.

The comparison of multi-dimensional search results can provide valuable reference for R & D, products and testers.

It is convenient to expand. For each business line that calls the vertical search service of the platform department, you only need to add the specific search result implementation class and the corresponding service address to access it.

Visual comparison test report, the comparison results can be known to the relevant personnel in the form of e-mail, and the abnormal data are processed in the report to facilitate the relevant personnel to investigate, and the detailed differences and comparison contents are output in the form of attachments. A sample test report is shown in the following figure:

(the attached content shows the index data that only exists in the pre-launch environment)

(the attached content shows the index data that exists only online)

(attached contents show different sorted index data in search results)

4. Tool integration extension

In the future, it is planned to integrate with jenkins and realize the automation of the whole process through interface configuration.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.