Shulou(Shulou.com)06/02 Report--

Today, I will talk to you about the use of Python and how to do digital operation. Many people may not know much about it. In order to make you understand better, the editor has summarized the following contents for you. I hope you can get something according to this article.

Digital operation is a necessary subject to improve profits, reduce costs, optimize operational efficiency and maximize financial returns. As one of the key tools in the field of data science, Python can be applied to almost all scenarios of data operation analysis and practice.

First, use Python for digital operation.

What is Python? What is the digital operation? Why use Python for digital operations? Let's answer these questions first in this section.

1. What is Python?

Python is an object-oriented interpretive computer programming language invented by the Dutch Guido van Rossum in 1989. The first public version was released in 1991. Python was originally developed as a programming language, not a data processing or modeling program dedicated to data work and scientific computing (of course, it is now).

Why should we choose Python over other languages (such as R) for data processing, analysis, and mining? This is because Python has some special conditions and capabilities that make it the most suitable tool for enterprises (especially in the field of big data) to do data operation.

Open source / free: there is no product, license, or license fee for using Python (and its third-party libraries), both for individuals and for businesses.

Portability: Python programs can run across Windows, Linux, Mac and other platforms, which determines its portability is very strong, one-time development, multi-platform applications.

Rich structured and unstructured data work libraries and tools: Python not only comes with its own mathematical calculation libraries, but also includes rich third-party libraries and tools, such as connection libraries for connecting Oracle, MySQL, SQLite and other databases, data science computing libraries Numpy, Scipy, Pandas, text processing libraries NLTK, machine learning libraries Scikit-Learn, Theano, graphics and video analysis processing and mining libraries PIL and Opencv, and open source computing framework TensorFlow.

Powerful data acquisition and integration capabilities: Python can not only support various types of files (image, text, log, voice, video, etc.) and database integration, but also obtain external data through API, network crawling and other ways. Internal and external data source integration, multi-source data integration, heterogeneous data coexistence, multi-type data interleaving is the basic form of the current enterprise data operation.

Computing power and efficiency of massive data: when faced with massive data beyond the scale of GB or even TB, traditional data tools are usually unable to support, let alone computational efficiency. The computing power and efficiency of Python for this scale of data is much higher than that of other data working languages.

Integration with other languages: Python has "glue" capability and can be integrated with Java, C, C++, MATLAB, R and other languages, which means that scripts written in other languages can be embedded in Python as well as Python scripts in other languages.

Powerful learning communication and training resources: Python has become one of the most mainstream programming languages and core tools for data processing in the world. There are many communities, blogs, forums, training institutions and educational institutions that provide communication and learning opportunities.

Efficient development: the Python language is concise and standardized, making it easier to use when developing programs using Python. This is critical for efficiency-first program work or verification projects, where efficiency determines business opportunities.

Easy to learn: Python syntax is simple, and even people without any code base can master basic Python programming skills in a few hours, which is very important for beginners, because it shows that programmatic data analysis is no longer out of reach, and they can use Python like Excel.

All in all, with a certain amount of Python experience and skills, there are few work scenarios that Python is not up to! If so, you will be competent if you call other languages with Python or Python with other languages.

2 what is the digital operation

Digital operation refers to the scientific analysis, guidance and application of all aspects of the operation process through digital tools, technologies and methods, so as to achieve the purpose of optimizing operation effect and efficiency, reducing costs and improving benefits.

Operation is a concept with a very large scope of "flexibility", which can be extended to the transaction management of all companies at the maximum, and may only include website operation and management at least. The scope of operation includes four aspects: member operation, commodity operation, traffic operation and content operation.

1. The significance of Digital Operation

The core of digital operation is operation, and all data work is carried out around the operation work chain, gradually strengthening the driving role of data for operation work. The value of digital operation is reflected in the assistance, promotion and optimization of operation, and even some operations have been gradually digitized, automated and intelligent.

Specifically, the significance of digital operations is as follows:

1) improve the efficiency of operational decisions. In the era of ever-changing information, it is very important for enterprises to seize fleeting opportunities. More efficient decision-making means that decisions can be made in a shorter period of time to keep up with or even stay ahead of competitors. Digital operation can make auxiliary decision-making more convenient, make data intelligence lead to active decision-making thinking, so as to predict the timing of decision-making in advance and improve decision-making efficiency.

2) improve the correctness of operation decisions. The intelligent data working way can carry on the data drill based on the data scientific method, and get the quantifiable expected results, combined with the rich experience of the decision-making layer, will improve the correctness of the operation decision.

3) optimize the operation and execution process. Through standard caliber data, information and conclusions, digital operation can provide KPI management with unified standards and clear objectives for the operation department, and optimize the execution links in the operation process combined with digital working methods and ideas, so as to reduce communication costs, improve work efficiency and improve implementation effect.

4) improve the return on investment. In the process of data operation, through the establishment of continuous correct work goals, the improvement of work efficiency and the implementation of optimal work methods, it can effectively reduce the redundant expenditure of enterprises and improve the return on investment per unit cost.

two。 Two ways of digital operation

From the point of view of the role of data, data operation can be divided into auxiliary decision-making data operation and data-driven data operation.

(1) Auxiliary decision-making data operation

Auxiliary decision-making data operation is the decision support of operation, which centers on the theme of decision-making, and assists decision-makers to make business decisions through data, models, knowledge and so on with the help of computer-related technology. the purpose of helping, assisting and assisting decision-makers. For example, by providing information on sales promotion for decision makers, support on ordering, sales and other aspects of the promotion activities of enterprises is provided.

(2) data-driven digital operation

Data-driven data operation means that the whole operation process aims at maximizing the results, triggers and optimizes the key data, encapsulates the workflow, logic and skills of the operation business into specific applications, and forms an integrated data workflow with the help of computer technology and combined with the internal processes and mechanisms of the enterprise. For example, personalized recommendation is a data-driven digital operation.

Auxiliary decision-making data operation and data-driven data operation are two levels of data applications. Compared with auxiliary decision-making, data-driven data-driven operation is more difficult and embodies more data value.

The auxiliary decision-making data operation is the service of the business decision maker, the whole process is controlled by the operator, and the data is the auxiliary role.

The process of data-driven data operation is controlled by data, and data is the main body. The realization of this process needs the support of IT, automation system, algorithm and so on. Data-driven is self-oriented, self-driven and effect-oriented.

Note: due to the flaws in the data and process itself, and the need for mandatory rules in the operating business, human intervention will be added even in the data-driven data operation. But even so, the data as the core of the data-driven is unchanged, that is to say, the data is the decision-maker itself.

3. Workflow of digital operation

Above, we introduced two ways of data operation: auxiliary decision-making data operation and data-driven data operation. Among them, the data-driven data operation depends on the application scenario, and the specific workflow is different in different scenarios. This section focuses on the workflow of data-driven data operations.

Data-driven data operation includes two main bodies: data and operation, which need to be coordinated in the actual work process. In some large-scale work projects, it may also involve linkage with IT departments, information centers and other departments. The workflow is divided into three stages, as shown in figure 1-1.

▲ figure 1-1 data-driven data operation workflow

(1) Phase 1: data requirements communication

This stage mainly includes two steps: demand generation and demand communication.

1) demand generation: some digital operational requirements generated by the operation department, such as forecasting commodity sales, finding abnormal orders, determining the list of marketing target groups, etc.

2) demand communication: face-to-face communication and communication according to the needs put forward by the operation department, which mainly includes three aspects:

One is the communication of business requirements, including the background of the demand, the problems to be solved, the expected results, etc.

Second, data status communication, including data storage environment, main fields, data dictionary, data volume, update frequency, data cycle, etc. If there is no data, it is necessary to formulate data collection rules and begin to collect data. The assistance of IT department may be needed in this process.

The third is the communication between data and analysis, according to the communication with operators, to understand which common data with business background, how different scenarios will lead to data changes, what key fields or scenario data will be involved in the analysis, and so on. The rich experience of business personnel will help data workers to take fewer detours.

(2) Phase 2: data analysis and modeling.

From this stage, we enter the formal data workflow, including four steps: data acquisition, data preprocessing, data analysis modeling and data conclusion output.

1) obtaining data: the data needed for digital operation analysis needs to be obtained from the database or file with specific authorization.

2) data preprocessing: in this process, data quality inspection, sample balance, classification summary, merging data sets, deleting duplicates, partitioning, sorting, discretization, standardization, filtering variables, transposition, lookup conversion, desensitization, conversion, sampling, outliers and missing values are processed.

3) data analysis modeling: a variety of data analysis and mining methods are used to analyze and model the data. The methods include statistical analysis, OLAP analysis, regression, clustering, classification, correlation, anomaly detection, time series, collaborative filtering, topic model, path analysis, funnel analysis and so on.

4) data conclusion output: there are many ways to output data conclusions, the common way is data analysis or mining modeling report, in addition, it also includes Excel statistical results, data API output, data results returned to the database, data results directly integrated into the application for automatic operation (such as SMS marketing).

(3) Phase 3: application of data landing.

This stage is the key stage for the landing of the digital operation, through which all the preparatory and processing work can generate value. This stage includes three steps: data conclusion communication, data deployment and application, and follow-up effect monitoring and feedback.

Data conclusion communication: for the content output in the form of reports and Excel statistical results, it is usually necessary to communicate deeply with the operating objects. The main content of communication is to communicate the conclusions and results obtained through the data with the business, and to initially verify the correctness, reliability and feasibility of the conclusions through communication, and to modify the results. If it is not feasible, you need to return to phase 2 to restart the data analysis modeling process.

Data deployment application: after communicating feasible data conclusions, it can be directly applied to the operation and execution link. For example, take the forecast results as the KPI target for the next month, and the selected users as key customers for secondary marketing.

Follow-up effect monitoring and feedback: most of the data operation analysis is not "one-off", especially after the application has been deployed, it is necessary to verify the effectiveness of the previous data conclusions in practice. if necessary, it is necessary to revise the conclusions and give feedback.

Many people think that the work of digital operation should start after the generation of data, which is a misconception, because the beginning of digital operation is the generation of demand, and the generation of demand is often not necessarily related to the generation of data.

Three Python are used for digital operation

Python is used for digital operation and will make full use of the powerful functions and efficiency of Python to meet the complex needs of digital operation.

Python can effectively integrate massive, multi-type, heterogeneous and multi-data sources from inside and outside the enterprise in the process of data operation, and provide rich integration, development, analysis, modeling and deployment applications.

The efficient development efficiency of Python can help the digital operation to carry on the proof of concept in the shortest time, and provide scientific prediction results, which provides the basis for the speed and accuracy of the digital operation.

Python can seamlessly connect the data workflow and IT workflow, which is conducive to the integration of data work and operation work. This is also the working method of data-driven digital operation, which is conducive to the real realization of digital and intelligent operation.

Four Python programs

1. Python 2 or Python 3

At present, Python is still the coexistence of two series of versions, one is Python 2, the other is Python 3. The syntax of the two versions is not completely compatible, so it is likely that the two versions of the program will report an error when calling each other's execution script.

If there are no special requirements, choosing Python 3 is the right choice in most scenarios. However, combined with specific scenarios, the author still gives the following specific suggestions:

If you want a mature, reliable and stable program, you can choose either Python 2 or Python 3.

If you are just learning or learning about Python, Python 3 is preferred.

If it is an internal application, there are historical programs that need to be executed and developed, depending on which version the enterprise uses.

If you need a large number of third-party libraries in your work, and are relatively early libraries, then use Python 2. But more often, even if libraries that only support Python 2 are no longer updated, there are many alternative libraries that can be implemented, so if it is not necessary to use Python 2, it is recommended to use Python 3.

If your program needs to run on a Linux server and use its native programs, check out the Python version that comes with your Linux server (most of which come with Python 2 on Linux servers). Similarly, you can install and use Python 3 even if the version of the program that comes with Linux native is low.

If none of the above scenarios meets your needs, start with Python 3.

2. 32 bit or 64 bit

In most cases, if there are no special requirements, try to choose the 64-bit version.

The author chose the 64-bit Python,3.7 version. The reason for choosing 64-bit is that it can handle larger data applications. In addition to using Python 3 because it is a trend, another important reason is that Numpy has announced that the new features will only support Python 3 from January 1, 2019. I believe that many systems and tools are beginning to give up support for Python 2.

3. The construction of Python environment

In general, we can download the required version directly from the Python official website https://www.python.org, and then customize the installation of other related libraries and packages as needed after installing the Python program. But for most readers who are new to Python, it would be better to have "click-to-install". Here, we introduce Anaconda.

Anaconda is a Python distribution, which contains more than 180 science packages such as conda, Python and their dependencies. It is a very popular Python package and integrated environment management application in the field of scientific computing. Its advantages are mainly shown in the following aspects:

By default, it can help you install the Python main program without having to download and install it separately.

Commonly used data work packages, including data import, cleaning, processing, calculation, display and other aspects of the main packages have been installed, such as Pandas, Numpy, Scipy, Statsmodels, Scikit-Learn (sklearn), NetworkX, Matplotlib and so on. Common unstructured data processing tools are also available, such as beautifulsoup4, lxml, NLTK, pillow, scikit-image and so on.

Many packages have dependencies on installation, which is very common on Linux systems, and Anaconda has solved all of these dependencies. Especially in the offline environment to do the installation and deployment of Python and a large number of libraries, Anaconda greatly reduces the difficulty of implementation and is an indispensable and effective tool in the process of project development.

Provides a command conda similar to the package management function of pip, which can display, update, install, uninstall and other common operations of the package. Of course, if you prefer pip, you can still use this command, because it is also installed by default in Anaconda.

Multi-platform, multi-version versatility, and keep up with the pace of Python main program update. Anaconda supports Windows, Mac OS, and Linux systems and includes both 32-bit and 64-bit Python versions (both Python 2 and Python 3 are supported).

Provide IPyton, Jupyter, Spyder interactive environment, can directly guide the user operation through the interface, the degree of ease of use is very high, and even specific learning resources are ready.

To install the Anaconda environment, simply log in to https://www.anaconda.com/download to download the corresponding version of the installation package, as shown in figure 1-2. The latest Python versions currently released by Anaconda include 32-bit and 64-bit Python 2.7 and Python 3.7.

▲ figure 1-2 Anaconda download page

Take the Windows64 bit Python version 3.7 as an example, after the download is complete, the .exe file will exist on the local computer, and the installation process only needs to use the default configuration.

After reading the above, do you have any further understanding of the use of Python and how to do digital operation? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.