Shulou(Shulou.com)06/02 Report--

This article mainly introduces "introduction of 7 practical important libraries in Python". In daily operation, I believe that many people have doubts about the introduction of 7 practical important libraries in Python. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts of "introduction of 7 practical important libraries in Python". Next, please follow the editor to study!

01 NumPy

NumPy is the abbreviation of Numerical Python and the cornerstone of Python numerical calculation. It provides a variety of data structures, algorithms and most of the interfaces needed for Python numerical computation. NumPy also includes other contents:

Fast and efficient Multi-dimensional Array object ndarray

Element-based array calculation or mathematical operation function between arrays

A tool for reading and writing array-based datasets on a hard disk

Linear Algebraic Operation, Fourier transform and Random number Generation

The mature C language API allows Python extensions and native C or C++ code to access NumPy's data structures and computing facilities.

In addition to the fast array processing capabilities that NumPy gives Python, another main use of NumPy is to serve as a data container for data transfer between algorithms and libraries. For numerical data, NumPy array can store and manipulate data more efficiently than Python built-in data structure.

In addition, libraries written in the underlying language, such as those written in C or Fortran, can operate directly on the data stored in the NumPy array without having to copy the data to other memory. Therefore, many Python numerical calculation tools use the NumPy array as the underlying data structure, or seamlessly interoperate with NumPy.

02 pandas

Pandas provides advanced data structures and functions, which are designed to make the work of using structured and tabular data fast, simple and expressive. It emerged in 2010 to help Python become a powerful and efficient data analysis environment. The commonly used pandas object is DataFrame, which is a data structure used to implement tabular, column-oriented, and column tags; and Series, an one-dimensional label array object.

Pandas combines the flexible data manipulation capabilities of tables and relational databases such as SQL with NumPy's concept of high-performance array computing. It provides complex index functions that make data reorganization, slicing, slicing, aggregation, and subset selection easier. As data manipulation, preprocessing and cleaning are important skills in data analysis, pandas will be an important topic.

To introduce some background knowledge, as early as 2008, I started the development of pandas when I worked in AQR Capital Management, a quantitative investment company. At that time, I had some unique needs that could not be met by any single tool on the tool list:

Data structures with tag axes that support automated or explicit data alignment-this prevents common errors caused by unaligned data and different index data from different data sources

Integrated time series function

A unified data structure that can process time series data and non-time series data at the same time

Arithmetic operations and simplification of metadata can be saved

Deal with missing data flexibly

Relational operations such as merging in popular databases (such as SQL-based databases)

I want to do the above work in the same place, preferably in a language with general software development capabilities. Python is a good alternative, but at that time there was no integrated set of such data structures, and there was no tool to provide relevant functionality. The result is that pandas was originally developed to solve financial and business analysis problems, and pandas is particularly good at deep time series and processing time index data generated in business processes.

Users who do statistical calculations in R will be familiar with the name of DataFrame because the object is named after a similar R data.frame object. Unlike Python, data boxes are part of the standard library in the R language. Therefore, many of the features in pandas are usually consistent with the functions provided by the implementation of the R core or the additional libraries of R.

The name pandas comes from panel data, an econometric term for multidimensional structured datasets. Pandas is also an abbreviated phrase of Python data analysis (Python data Analysis) itself.

03 matplotlib

Matplotlib is the most popular Python library for mapping and other two-dimensional data visualization. It was created by John D. Hunter and is currently maintained by a large team of developers. Matplotlib is designed as a cartographic tool suitable for publication.

There are other visualization libraries for Python programmers, but matplotlib is still the most widely used and integrates well with other libraries in the ecosystem. I think it is a safe choice to use it as the default visualization tool.

For a more detailed explanation of matplotlib, please mark: pure practical information: hand-in-hand to teach you to use Python to do data visualization (with code)

04 IPython and Jupyter

The IPython project, launched in 2001 by Fernando P é rez, aims to develop a more interactive Python interpreter. Over the past 16 years, it has become one of the most important tools in the Python data technology stack.

Although it does not provide any computing or data analysis tools itself, it is designed to maximize productivity in both interactive computing and software development. It uses an execute-explore workflow to replace the typical edit-compile-run workflow in other languages. It also provides easy-to-use interfaces for operating system command lines and file systems. Because data analysis and coding involves a lot of exploration, trial and error, and traversal, IPython allows you to get the job done faster.

In 2014, the Fernando and IPython teams released the Jupyter project. The Jupyter project aims to design an interactive computing tool for more languages. IPython web notebook becomes Jupyter notebook and can support more than 40 programming languages. The IPython system can currently be used as a kernel (a programming language mode) to use Python in Jupyter.

IPython itself has become a component of the Jupyter open source project, which provides an interactive, exploratory and efficient environment. IPtyhon's oldest and simplest "mode" is an enhanced version of the Python command line to improve the speed of writing, testing, and debugging Python code.

You can also use the IPython system through Jupyter Notebook, a Web-based, multilingual code "notebook". The IPython command line and Jupyter notebook are very useful for data exploration and visualization.

The Jupyter notebook system allows you to use Markdown and HTML to create rich documents containing code and text. Other programming languages also implement kernels for Jupyter, allowing you to use multiple languages in Jupyter rather than just Python.

Personally, IPython involves most of my work, including running, debugging, and testing code.

05 SciPy

SciPy is a collection of packages for different standard problem domains in the field of scientific computing. Here are some packages included in SciPy:

Scipy.integrate

Numerical integral routines and differential equation solvers

Scipy.linalg

Linear Algebra routines and Matrix decomposition based on numpy.linalg

Scipy.optimize

Function optimizer (minimizer) and root algorithm

Scipy.signal

Signal processing tool

Scipy.sparse

Sparse Matrix and Solver of sparse Linear system

Scipy.special

Wrapper for SPECFUN. SPECFUN is a package that implements general data functions, such as gamma functions, in Fortran.

Scipy.stats

Standard continuous and discrete probability distribution (density function, sampler, continuous distribution function), various statistical tests, all kinds of descriptive statistics.

Together with NumPy, SciPy provides a reasonable, complete and mature computing foundation for many traditional scientific computing applications.

06 scikit-learn

The scikit-learn project was born in 2010 and has become the machine learning toolkit of choice for Python programmers. In just seven years, scikit-learn has 1,500 code contributors worldwide. It contains the following sub-modules.

Classification: SVM, nearest neighbor, random forest, logical regression, etc.

Regression: Lasso, Ridge regression, etc.

Clustering: k-means, spectral clustering, etc.

Dimensionality reduction: PCA, feature selection, matrix decomposition, etc.

Model selection: grid search, cross-validation, index matrix

Preprocessing: feature extraction, normalization

Scikit-learn, together with pandas, statsmodels and IPython, makes Python an efficient data science programming language.

07 statsmodels

Statsmodels is a statistical analysis package. It comes from various analytical models implemented by Jonathan Taylor, a professor of statistics at Stanford University, using R language. Skipper Seabold and Josef Perktold created a new statsmodels project as early as 2010. The project has grown rapidly since then, with a large number of active users and contributors.

Nathaniel Smith developed the Patsy project to provide a formula and model specification framework for the statsmodels package driven by the R language formula system.

Compared with scikit-learn, statsmodels contains classical (high frequency vocabulary) statistical and economic algorithms. The model it contains is as follows.

Regression models: linear regression, general linear model, robust linear model, linear mixed effect model, etc.

Analysis of variance (ANOVA)

Time series analysis: AR, ARMA, ARIMA, VAR and other models

Nonparametric methods: kernel density estimation, kernel regression

Visualization of statistical model results

Statsmodels focuses more on statistical reasoning, providing uncertainty evaluation and p-value parameters. Instead, scikit-learn is more focused on forecasting.

At this point, the study of "introduction to 7 practical important libraries in Python" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.