Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to use Loki and fzf to upgrade your Shell history, which has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

Problem: a persistent centralized Shell history

I like my shell history and have always been an avid user of CTRL+R. About a year ago, my terminal life changed dramatically, and my colleague Dieter Plaetinck introduced me to the command-line fuzzy finder fzf.

All of a sudden, the search in the command starts from this:

Before Loki and fzf

Turned out to be like this:

After Loki and fzf

Although fzf has greatly improved my quality of life, there are still some pieces missing around my shell history:

The shell history will be lost when the terminal shuts down suddenly, the computer crashes, crashes, and the whole disk encryption key is forgotten.

Want to access my shell history from all my computers.

I think my shell history is a file: it's an important story that I don't want to lose. Combining Loki with my shell history helps to solve these and more problems.

About Loki

Loki adopts the intuitive label model that the open source Prometheus project uses for metrics and extends it to the world of log aggregation. This allows developers and operators to switch seamlessly between their metrics and logs using the same set of tags. Even if you don't use Prometheus, there are many reasons why Loki may be well suited to your log storage needs:

Low overhead: Loki does not do full-text log indexing; it only creates indexes of tags you put on the log. Keeping the index small greatly reduces the operation and maintenance requirements of Loki. I run my loki-shell project on raspberry pie, which uses Loki to store shell history and uses only a little more memory from 50MB.

* low cost: * * Log content is compressed and stored in object storage, such as Amazon S3, Google cloud storage, Azure Blob, and even directly in the file system. Our goal is to use cheap and durable storage.

Flexibility: Loki is provided as a single binary file that can be downloaded and run directly or as an Docker image in any container environment. You can quickly get started with a Helm chart in Kubernetes. If you have high requirements for logging tools, you can take a look at the production environment running in the Grafana lab. It uses open source Jsonnet and Tanka to deploy the same Loki image as discrete components to achieve large-scale horizontal expansion, high availability, replication, separate expansion of read and write paths, highly parallel queries and so on.

All in all, Loki's approach is to keep a small index (tag) of your log metadata and store unindexed, compressed log content in cheap object storage to make it easier and cheaper to operate. The application is built to run as a single process and can easily evolve into a highly available distributed system. You can achieve higher query performance on larger log workloads through parallelization and query fragmentation-a bit like MapReduce designed for your log.

In addition, this feature can be used by anyone for free. Like its Grafana open observation platform, Grafana Labs is committed to building Loki into a fully functional, fully open log aggregation software that anyone can use.

Let's go

I run Loki on raspberry pie and store my shell history remotely in S3 bucket.

When I press CTRL+R,Loki 's LogCLI command line interface, several batch requests are made and transferred to fzf. Here is an example. The top half shows the Loki server log on the raspberry pie.

Raspberries send logs from the Loki server

Ready to try? The following guidelines will help you set up and run Loki and integrate with your shell history. To keep this tutorial concise, this setting will run Loki locally on your computer and store all files on the file system.

In loki-shell 's GitHub version library, you can find all of this, as well as information on how to set up a more complex installation.

Please note that this tutorial will not change any existing behavior around your history, so your existing shell history commands and history settings will not be touched. Instead, this will copy the command history to Loki with $PROMPT_COMMAND in Bash and precmd in Zsh. On the CTRL+R side, it overloads the functions that fzf uses to access CTRL+R commands. So it's safe to give it a try, and if you don't like it, just follow the uninstall steps in the GitHub version library to remove all traces. Your shell history will not be touched.

Step 1: install fzf

There are several ways to install fzf, but I prefer the Git method:

Git clone-depth 1 https://github.com/junegunn/fzf.git ~ / .fzf ~ / .fzf/install

Prompt yes for all questions.

If you have installed fzf, make sure you have enabled key binding (that is, make sure that fzf pops up when you type CTRL+R). If necessary, you can rerun the fzf installation process to enable key binding.

Step 2: install loki-shell

Like fzf, loki-shell has a Git version library and installation scripts:

Git clone-- depth 1 https://github.com/slim-bean/loki-shell.git ~ / .loki-shell~/.loki-shell/install

First, the script creates the ~ / .loki-shell directory, where all files will be saved (including Loki data). Next, it will download the binaries for Promtail, LogCLI, and Loki.

And then it asks:

Do you want to install Loki? ([y] / n)

If you have run a centralized Loki for Loki-shell, you can answer n; however, for this tutorial, answer y or press enter.

There are two ways to run Loki locally: as a Docker image or as a single binary (which can be added as a systemd service). I recommend using Docker if you can, because I think it simplifies the operation a little bit, but both work.

Run using Docker

Run Loki as a Docker image:

[y] to run Loki in Docker, [n] to run Loki as a binary ([y] / n) yError: No such object: loki-shellError response from daemon: No such container: loki-shellError: No such container: loki-shell54843ff3392f198f5cac51a6a5071036f67842bbc23452de8c3efa392c0c2e1e

If this is the first time you have run this installer, you can ignore the error message. This script will stop and replace the running Loki container. If the version does not match, you can rerun the script to upgrade Loki.

okay! Loki now runs as a Docker container.

Loki data will be stored in ~ / .loki-shell/data.

Since the image is run with the-restart=unless-stopped flag, it will restart the service when the system is restarted, but will remain stopped if you run docker stop loki-shell.

(if you are using Docker, you can skip to the "Shell Integration" section. )

Run as a binary file

There are many ways to run binaries on Linux systems. This script installs a systemd service. If you don't have systemd, you can also use binary installation:

[y] to run Loki in Docker, [n] to run Loki as a binary ([y] / n) n Run Loki with systemd? ([y] / n) n This is as far as this script can take youYou will need to setup an auto-start for LokiIt can be run with this command: / home/username/.loki-shell/bin/loki-config.file=/home/username/.loki-shell/config/loki-binary-config.yaml

The script will output the commands you need to run Loki, and you can set up an init script or other method to start it automatically.

If you want, you can run the command directly, running Loki from your current shell.

If you have systemd, you can choose to have the script install the systemd service or show your own command to run it:

Run Loki with systemd? ([y] / n) y Installing the systemd service requires root permissions. [y] to run these commands with sudo [n] to print out the commands and you can run them yourself ([y] / n) nsudo cp / home/ed/.loki-shell/config/loki-shell.service / etc/systemd/system/loki-shell.servicesudo systemctl daemon-reloadsudo systemctl enable loki-shellsudo systemctl start loki-shellCopy these commands and run them when the script finishes (press enter to continue) Shell integration

No matter how you install Loki, you should now see a prompt:

Enter the URL for your Loki server or press enter for default (http://localhost:4100)

If you have set up a centralized Loki, you should enter its URL here. However, this demo only uses the default URL, so you can press enter.

It will output a lot of text to explain all the entries added to your ~. Bashrc or ~. Zshrc (or both).

Okay!

Finished. Restart your shell or reload config file. Source ~ / .bashrc # bash source ~ / .zshrc # zsh step 3: try it!

Start using your shell and use CTRL+R to view your commands.

Open multiple terminal windows, enter commands in one window, type CTRL+R in another window, and you will see that your commands are available immediately.

In addition, please note that when you switch between terminals and enter commands, you can use them immediately with CTRL+R, but the up arrow operation is not affected between terminals. If you install Oh My Zsh, the situation may be different, because it automatically appends all commands to the history. )

Press CTRL+R multiple times to switch between sorting by time and sorting by correlation.

Please note that this configuration will only display the query history of the current host, even if you are sending shell data from multiple hosts to Loki. I think this is the most reasonable by default. If you want to change this behavior, there are many things you can adjust; see the loki-shell version Library for more information.

It also installs an alias called hist.

Alias hist= "$HOME/.loki-shell/bin/logcli-- addr=$LOKI_URL"

LogCLI can be used to query and search your history directly on Loki, as well as allow you to search for other hosts. Check out LogCLI's getting started guide to learn more about queries.

Loki's log query language (LogQL) provides metrics queries that allow you to do some interesting things, for example, I can see how many kc commands I have issued in the past 30 days (my alias for kubectl).

Count the number of times a command is used

Extra Enhancement

Install Grafana and fiddle with your shell history.

Docker run-d-p 3000UR 3000-name=grafana grafana/grafana

Open a Web browser, visit http://localhost:3000, and log in with the default admin/admin username and password.

On the left, navigate to configure Configuration- > data Source Datasources, click the add data Source Add Datasource button, and then select Loki.

For URL, you should be able to use http://localhost:4100 (however, on my WSL2 machine, I have to use the actual IP address of the computer).

Click Save and Test Save and Test. You should see that the data source is connected and the label is found.

Click the "Manager Explore" icon on the left, make sure that the Loki data source is selected, and try the query:

{job= "shell"}

If there are a large number of hosts sending shell commands, you can use the "host Host" label to limit the results to a host:

{job= "shell", hostname= "myhost"}.

You can also use filter expressions to find specific commands:

{job= "shell"} | = "docker"

Or you can explore the world of metrics from the log and see how often you use shell:

Rate ({job= "shell"} [1m])

Calculate shell usage in the past 20 days

Want to reconstruct a timeline from an event? You can filter through a specific command to see its running time:

Thank you for reading this article carefully. I hope the article "how to use Loki and fzf to upgrade your Shell History" shared by the editor will be helpful to everyone. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.