Shulou(Shulou.com)05/31 Report--

This article is a detailed introduction to "how to use docker compose to deploy golang's Athens private agent". The content is detailed, the steps are clear, and the details are properly handled. I hope this article "how to use docker compose to deploy golang's Athens private agent" can help you solve your doubts. Let's go deeper and learn new knowledge together with the ideas of Xiaobian.

Go Private Agent Build Why Choose Avens

The selection criteria for privatization agents are nothing more than the following

1. Hosting private modules;

2. Excluding access to public modules;

3. Store public modules;

Features of athens:

Athens can be configured to access private repositories first;

Athens will store each pull packet. If the module has not passed through athens before, athens will request data from the target address. When returning to the client, it will store the module in storage. This realizes that go mod download will only happen once.

Athens 'strategy for dealing with storage is to add only, a module is saved, it will never change, even if the developer pushes the tag, it will not be deleted;

Athens can also configure download policies to filter packets that pose security risks.

Athens supports disk, mongo, gcs, s3, minio, external storage/custom, but disk is generally recommended.

Deployment using docker-compose

The official website has already provided a solution for deployment through docker and binary system. Here, the principle of good memory is better than bad writing is adhered to. Here, I have also made a record.

Configure authentication information for private repositories

Configuration through the. netrc file, which can put their own private repository address, and user, password authentication information

# cat .netrcmachine gitlab.test.com login test-name password test-pass

There are several private repositories, and it is OK to configure a few

Configure download mode

Through The download mode (download mode configuration policy) is now more respected in ATHENS, the previous method through Filtering modules (filtering mode) has been abandoned.

See how to configure

# DownloadMode defines how Athens behaves when a module@version# is not found in storage. There are 4 options:# 1. "sync" (default): download the module synchronously and# return the results to the client.# 2. "async": return 404, but asynchronously store the module# in the storage backend.# 3. "redirect": return a 301 redirect status to the client# with the base URL as the DownloadRedirectURL from below.# 4. "async_redirect": same as option number 3 but it will# asynchronously store the module to the backend.# 5. "none": return 404 if a module is not found and do nothing.# 6. "file:": will point to an HCL file that specifies# any of the 5 options above based on different import paths.# 7. "custom:" is the same as option 6# but the file is fully encoded in the option. This is# useful for using an environment variable in serverless# deployments.# Env override: ATHENS_DOWNLOAD_MODEDownloadMode = "sync"

The environment variable ATHENS_DOWNLOAD_MODE can be specified or modified to configure config.dev.toml, default is sync

ATHENS_DOWNLOAD_MODE What can be specified:

1. Specify an hcl file through file: , which can set download mode for different repositories;

2, through custom: specify a base64 encoded HCL file;

3. Specify a specific global policy, sync, async, none, redirect, or async_redirect, which is a global setting. The above two can customize the policy group.

Take a look at the specific download mode

sync: Download modules go mod download from VCS by synchronizing them, persist them to storage, and immediately return them to the user. Note that this is the default behavior;

async: returns 404 to the client and asynchronously downloads module@version and persists it to storage;

none: returns 404 and does nothing;

redirect: redirect to an upstream proxy (e.g. proxy.golang.org) and do nothing thereafter;

async_redirect: redirect to upstream proxies (e.g. proxy.golang.org) and asynchronously download module@version and persist it to storage;

Let's take a look at the hcl file that configures the policy

# cat download.hcl downloadURL = "https://goproxy.cn"mode = "async_redirect"download "gitlab.test.com/*" { mode = "sync"} Deployment

Docker-composer deployment is used here

version: '2'services: athens: image: gomods/athens:v0.11.0 restart: always container_name: athens_proxy ports: - "3000:3000" volumes: - ./. netrc:/root/.netrc - ./ athens-storage:/var/lib/athens - ./ download.hcl:/root/download.hcl environment: - ATHENS_NETRC_PATH=/root/.netrc - ATHENS_STORAGE_TYPE=disk - ATHENS_DISK_STORAGE_ROOT=/var/lib/athens - ATHENS_GOGET_WORKERS=100 - ATHENS_DOWNLOAD_MODE=file:/root/download.hcl - ATHENS_GONOSUM_PATTERNS=gitlab.test.com

ATHENS_GONOSUM_PATTERNS: configured as a private repository address. The configured repository address will not be checked for security.

go is for security considerations. In order to ensure that the developer's dependency library is not maliciously hijacked and tampered with, the GOSUMDB environment variable is introduced to set the verification server.

When you make changes (updates/additions) to dependencies locally, Go will automatically go to this server for data validation to ensure that your code base is the same as everyone else's code base in the world. If there's a problem, there's a big safety alert. Of course, these operations are already integrated into Go, and developers do not need to perform additional operations.

For our private repository, go to the public security verification library, it must not pass the verification, we can set the code repository without verification through the ATHENS_GONOSUM_PATTERNS environment variable, it can set multiple matching paths, separated by commas.

Start docker-compose up -d

Client Settings Proxy export GOPROXY=http://xxxx:3000

so we can use our proxy services.

Because the selected ATHENS_STORAGE_TYPE is disk, the athens service will pull the resource package and download it into the configured ATHENS_DISK_STORAGE_ROOT.

Authentication of private repositories using secret keys

The above method authenticates the private repository through. netrc, because the account password is always not very good, you can use the secret key to authenticate

1. Configure the secret key

First check if the computer has a secret key

# cd .ssh# lsid_rsa id_rsa.pub

If not, generate by following command

# ssh-keygen -t rsa -C "youremail@example.com"

Change the mailbox to your own, and enter all the way.

Then add the contents of id_rsa.pub public key to your own private repository, how to add your own google bar, relatively simple

# cat gitconfig [url "ssh://git@gitlab.test.com"] insteadOf = https://gitlab.test.com3, Configure SSH to bypass host SSH key verification # cat config Host gitlab. test.comHostname gitlab.test. comStrictly HostKeyChecking noIdentityFile /root/.ssh/id_rsa

Map the authentication information configured above to the container

version: '2'services: athens: image: gomods/athens:v0.11.0 restart: always container_name: athens_proxy ports: - "3000:3000" volumes: - ./ athens-storage:/var/lib/athens - ./ download.hcl:/root/download.hcl - ./ gitconfig:/root/.gitconfig - ./ ssh-keys:/root/.ssh environment: - ATHENS_STORAGE_TYPE=disk - ATHENS_DISK_STORAGE_ROOT=/var/lib/athens - ATHENS_GOGET_WORKERS=100 - ATHENS_DOWNLOAD_MODE=file:/root/download.hcl - ATHENS_GONOSUM_PATTERNS=gitlab.test.com

This allows authentication of the secret key.

You need to pay attention to the permissions of the private key. At first, you didn't pay attention. The execution reported the following error.

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: UNPROTECTED PRIVATE KEY FILE! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ Permissions 0644 for '/root/.ssh/id_rsa' are too open. It is required that your private key files are NOT accessible by others. This private key will be ignored. Load key "/root/.ssh/id_rsa": bad permissions git@gitlab.test.com: Permission denied (publickey). fatal: Could not read from remote repository.

Read the error can be inferred that the permissions are too large, the need for private key files can not be accessed by others.

Just modify the permissions.

ssh-keys # chmod 600 id_rsa Read here, this article "How to use docker compose to deploy golang's Athens private proxy" article has been introduced, want to master the knowledge of this article also need to do your own practice to understand, if you want to know more about the content of the article, welcome to pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.