Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "what are the popular Python libraries". In the operation of actual cases, many people will encounter such a dilemma. Next, let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

1 、 Typer

Https://github.com/tiangolo/typer

You don't always need to write CLI applications, but doing so can save you a lot of work. After the great success of FastAPI, tiangolo used the same principle to bring us Typer: a new library that allows you to take advantage of the type prompt of Python 3.6 + to write a command line interface.

The design does make Typer stand out. In addition to ensuring that the code is recorded correctly, you can easily validate the CLI interface. By using type hints, you can get automatic completion in Python editors such as VS Code, which will increase your productivity.

To enhance its functionality, the Typer kernel is based on Click, while Click is well known and has been rigorously tested. This means that it can take advantage of all its benefits, such as communities and plug-ins, while starting simple with less boilerplate code and becoming complex as needed.

Typer documentation is really helpful and should be a model for other projects. You can't miss it!

2. Rich

Https://github.com/willmcgugan/rich

Following the theme of CLI, who says that terminal applications must be pure white, or if you are a real hacker, they must be green or black?

Do you want to add colors and styles to the terminal output? Effortlessly display a beautiful progress bar? Markdown? Emoticons? Rich can achieve all of the above functions. See the screenshot of the following example to learn more:

Dear PyGui can be used without a steep learning curve and can be run on Windows 10 (DirectX 11), Linux (OpenGL 3), and MacOS (Metal).

4. PrettyErrors

Https://github.com/onelivesleft/PrettyErrors

From Avenue to Jane, this is a library worth thinking about: no one has ever thought about what this is all about before.

PrettyErrors only does one thing and does it well. In terminals that support color output, it converts hidden stack tracks into something more suitable for analysis with faint human eyes. There is no need to scan the entire screen to find the cause of the exception. You can see at a glance now!

5. Diagrams

Https://github.com/mingrammer/diagrams

We programmers like to solve problems with code. But sometimes we need to explain complex architectural designs to other colleagues. Traditionally, we use the GUI tool, where we can work with charts and visualization to fit into presentations and documents. But it's not the only way.

Diagrams allows you to draw cloud system architectures directly in Python code without any design tools. It contains icons that support multiple cloud providers (including AWS,Azure,GCP). This makes it easy to create arrows and groups. Really, there are only a few lines of code!

What's the best thing about code-based diagrams? You can use version control through git to control progress!

6. Hydra and OmegaConf

Https://hydra.cc/ https://github.com/omry/omegaconf

In the research and experiment of machine learning projects, there are always countless settings to try. In non-trivial applications, configuration management can become quite complex and fast. Wouldn't it be nice to have a structured way to deal with this complexity?

Hydra is a tool that allows you to build configurations in a composable manner and overwrite parts from the command line or configuration files.

To illustrate some common tasks that can be simplified through the library, assume that there is a basic architecture of the model we are trying, as well as many variations of it. Using Hydra, you can define a basic configuration, then run multiple jobs and make the following changes:

Python train_model.py variation=option_a,option_b ├── variation │ ├── option_a.yaml │ └── option_b.yaml ├── base.yaml └── train_model.py

Hydra's cousin OmegaConf provides a consistent API for the foundation of the hierarchical configuration system and supports different sources such as YAML, configuration files, objects, and CLI parameters.

This is a necessary condition for configuration management in the 21st century!

7. PyTorch Lightning

Https://github.com/PyTorchLightning/PyTorch-lightning

Every tool to improve the productivity of data science teams should be encouraged. There is no reason for people working on data science projects to reinvent the wheel every time, ponder over how to better organize the code in their projects, use poorly maintained "PyTorch templates", or use higher levels of abstraction.

PyTorch Lightning helps to increase productivity by separating science from engineering. In a sense, it makes your code more concise, a bit like TensorFlow's Keras. But it's still PyTorch, and you can access all the commonly used API.

The library helps teams leverage good practices and clear component responsibilities around the organization to build easily extensible to train high-quality code on multiple GPU,TPU and CPU.

A library that can help junior members of the data science team produce better results, but more experienced members will like it because of the increase in overall productivity and without giving up control.

8. Hummingbird

Https://github.com/microsoft/hummingbird

Not all machine learning is deep learning. Typically, your model consists of more traditional algorithms implemented in scikit-learn (for example, Random Forest), or you use gradient enhancement methods such as popular LightGBM and XGBoost.

However, many advances are taking place in the field of deep learning. Frameworks like PyTorch are evolving at an alarming rate, and hardware devices have been optimized to run tensor computing faster and reduce power consumption. Wouldn't it be nice if we could use all this work to run traditional methods faster and more efficiently?

This is an opportunity for Hummingbird to use his talents. This new library provided by Microsoft can compile well-trained traditional ML models into tensor computing. This is great because it eliminates the need to redesign the model.

So far, Hummingbird supports conversion to PyTorch,TorchScript,ONNX and TVM, as well as various ML models and vectorizers. Inferential API is also very similar to the Sklearn paradigm, allowing you to reuse existing code, but change the implementation to code generated by Hummingbird. This is a tool worth paying attention to because it has support for schema models and formats!

9. HiPlot

Https://github.com/facebookresearch/hiplot

Almost every data scientist has dealt with high-dimensional data at some point in his career. Unfortunately, the human brain does not have enough connections to process this data intuitively, so we have to resort to other technologies.

Earlier this year, Facebook released HiPlot, a library that uses parallel drawings and other graphics to represent information to help discover dependencies and patterns in high-dimensional data. This concept has been explained in its blog post, but basically, it is a good way to visualize and filter high-dimensional data.

HiPlot is interactive and extensible, and you can use it from a standard Jupyter laptop or through its own server.

10. Scalene

Https://github.com/emeryberger/scalene

As the Python library ecosystem becomes more and more complex, we find ourselves writing more and more code that relies on C extensions and multithreading. This becomes a problem when measuring performance because CPython's built-in profiler does not properly handle multithreading and native code.

That's when Scalene came to the rescue. Scalene is the CPU and memory profiler for Python scripts that correctly handles multithreaded code and distinguishes the time it takes to run Python and native code. There is no need to modify the code, just run the script using scalene on the command line, and the script generates a text or HTML report for you, showing the CPU and memory usage of each line of code.

Honorary nominations:

Norfair

Https://github.com/tryolabs/norfair

Norfair is a customizable lightweight Python library for real-time 2D object tracking. With Norfair, you can add tracking capabilities to any detector in just a few lines of code.

Quart

Https://gitlab.com/pgjones/quart/

An asynchronous Web framework compatible with Flask API. Some existing Flask extensions can even be used directly!

Alibi-detect

Https://github.com/SeldonIO/alibi-detect

Monitor outliers and distribution drift in the production model to obtain tabular data, text, images and time series.

Einops

Https://github.com/arogozhnikov/einops

Einops became popular in 2020, allowing you to write tensor operations to get readable and reliable code, and support numpy,PyTorch,TensorFlow and so on.

Stanza

Https://github.com/stanfordnlp/stanza

Natural language processing tools provided by Stanford that support more than 60 languages. Multiple pre-training models are available for different tasks.

Datasets

Https://github.com/huggingface/datasets

Lightweight extensible library from HuggingFace to easily share and access evaluation metrics such as datasets and natural language processing (NLP)

Pytorch-forecasting

Https://github.com/jdb78/pytorch-forecasting

With the help of neural network, the time series prediction for actual cases and studies is simplified.

Sktime

Https://github.com/alan-turing-institute/sktime

Dedicated time series algorithms and scikit-learn compatible tools are provided for building, adjusting and evaluating composite models. Also check its accompanying sktime-dl software package for a deep learning-based model.

Netron

Https://github.com/lutzroeder/netron

Visualization tools for neural networks, deep learning and machine learning models. There are more supported formats than I know.

Pycaret

Https://github.com/pycaret/pycaret

Several common ML libraries have been packaged to greatly improve your productivity and save hundreds of lines of code.

Tensor-sensor

This is the end of the content of https://github.com/parrt/tensor-sensor" 's popular Python libraries. Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.