Shulou(Shulou.com)06/01 Report--

In this issue, the editor will bring you about how to use gosec to check the security problems in Go code. The article is rich in content and analyzes and describes for you from a professional point of view. I hope you can get something after reading this article.

Code written in the Go language is becoming more and more common, especially in container, Kubernetes, or cloud ecology-related development. Docker was one of the first projects to adopt Golang, followed by Kubernetes, and then a large number of new projects chose Go among many programming languages.

Like other languages, Go has its strengths and weaknesses (such as security flaws). These defects can be caused by defects in the language itself and improper coding by programmers, such as memory security issues in C code.

Regardless of the cause of their occurrence, security issues should be fixed early in the development process to avoid appearing in packaged software. Fortunately, static analysis tools can help you deal with these problems in a more repeatable way. Static analysis tools find problems by parsing code written in a programming language.

Many of these tools are called linter. Traditionally, linter has focused more on examining coding problems, bug, code style, and so on, in the code, which may not find security problems in the code. For example, Coverity is a popular tool that can help you find problems in the code in C _ hand Cure +. However, there are some tools specifically designed to check for security issues in the source code. For example, Bandit can check for security flaws in Python code. Gosec is used to search for security flaws in the Go source code. Gosec scans Go's AST (Abstract Syntax Tree (abstract syntax tree)) to check for security issues in the source code.

Start using gosec

Before you start learning and using gosec, you need to prepare a project written in Go. There are so many open source software, I believe this is not a problem. You can find one in GitHub's popular Golang warehouse.

In this article, I randomly selected the Docker CE project, but you can choose any Go project.

Install Go and gosec

If you haven't installed Go, you can pull it from the warehouse first. If you are using Fedora or other RPM-based Linux distributions:

$dnf install golang.x86_64

If you are using another operating system, please refer to the Golang installation page.

Use the version parameter to verify that Go is installed successfully:

$go version go version go1.14.6 linux/amd64

You can easily install gosec by running the go get command:

$go get github.com/securego/gosec/cmd/gosec

The above command downloads the source code of gosec from GitHub, compiles it, and installs it to the specified location. You can also see other ways to install the tool in the README of the repository.

The source code for gosec will be downloaded to $GOPATH, and the compiled binaries will be installed in the bin directory set up on your system. You can run the following command to view the $GOPATH and $GOBIN directories:

$go env | grep GOBIN GOBIN= "/ root/go/gobin" $go env | grep GOPATH GOPATH= "/ root/go"

If the go get command is executed successfully, the gosec binary should be ready to use:

$ls-l ~ / go/bin/ total 9260-rwxr-xr-x. 1 root root 9482175 Aug 20 04:17 gosec

You can add the bin directory under $GOPATH to $PATH. This allows you to use the gosec command line tool (CLI) just like any other command on the system.

$which gosec / root/go/bin/gosec $

Use the-help option of the gosec command line tool to see if it runs as expected:

$gosec-help gosec-Golang security checker gosec analyzes Go source code to look for common programming mistakes that can lead to security problems. VERSION: dev GIT TAG: BUILD DATE: USAGE:

After that, create a directory and download the source code to this directory as an example project (in this case, I use Docker CE):

$mkdir gosec-demo $cd gosec-demo/ $pwd / root/gosec-demo $git clone https://github.com/docker/docker-ce.git Cloning into 'docker-ce'... Remote: Enumerating objects: 1271, done. Remote: Counting objects: 100% (1271 + 1271), done. Remote: Compressing objects: 100% (722 amp 722), done. Remote: Total 431003 (delta 384), reused 981 (delta 318), pack-reused 429732 Receiving objects: 100% (431003amp 431003), 166.84 MiB | 28.94 MiB/s, done. Resolving deltas: 100% (221338 shock 221338), done. Updating files: 10861 (10861), done.

The code statistics tool (cloc in this case) shows that most of the project is written in Go, which caters to the functionality of gosec.

$. / cloc / root/gosec-demo/docker-ce/ 10771 text files 8724 unique files. 2560 files ignored. -Language files blank comment code- -Go 7222 190785 230478 1574580 YAML 37 4831 817 156762 Markdown 529 21422 0 67893 Protocol Buffers 149 5014 16562 10071

Run gosec with default options

Run gosec with the default options in the Docker CE project, and execute gosec. /... Orders. There will be a lot of output on the screen. At the end you will see a short "Summary" listing the number of files viewed, the total number of lines of all files, and the number of problems found in the source code.

$pwd / root/gosec-demo/docker-ce $time gosec. / [gosec] 2020-08-20 04:44:15 Including rules: default [gosec] 2020-08-20 04:44:15 Excluding rules: default [gosec] 2020-08-20 04:44:15 Import directory: / root/gosec-demo/docker-ce/components/engine/opts [gosec] 2020-08-20 04:44:17 Checking package: opts [gosec] 2020-08-20 04:44:17 Checking file: / root/gosec-demo/docker-ce/components/engine/opts/address_ Pools.go [gosec] 2020-08-20 04:44:17 Checking file: / root/gosec-demo/docker-ce/components/engine/opts/env.go [gosec] 2020-08-20 04:44:17 Checking file: / root/gosec-demo/docker-ce/components/engine/opts/hosts.go # End of gosec run Summary: Files: 1278 Lines: 173979 Nosec: 4 Issues: 644 real 0m52.019s user 0m37.284s sys 0m12.734s $

Scroll through the screen and you will see lines highlighted in different colors: red indicates high priority issues that need to be reviewed as soon as possible, and yellow indicates medium priority issues.

About misjudgment.

Before I start reviewing the code, I'd like to share a few basic principles. By default, static checking tools analyze the test code based on a series of rules and report any problems they find. Does this mean that every problem reported by the tool needs to be fixed? No. The best answer to this question is the person who designed and developed the software. They are most familiar with the code and, more importantly, they know in what environment the software will be deployed and how it will be used.

This knowledge point is very important to determine whether a piece of code marked by the tool is a security flaw. With the accumulation of working time and experience, you will gradually learn how to make static analysis tools ignore non-security defects and make the content of the report more enforceable. Therefore, to determine whether a problem reported by gosec needs to be fixed, it would be better to have an experienced developer audit the source code manually.

High priority problem

Judging from the output, gosec found a high priority problem with Docker CE, which uses a lower version of TLS (Transport layer Security (Transport Layer Security) ()). At any time, using the latest version of the software and library is the best way to ensure that it is up-to-date and free of security issues.

[/ root/gosec-demo/docker-ce/components/engine/daemon/logger/splunk/splunk.go:173]-G402 (CWE-295): TLS MinVersion too low. (Confidence: HIGH, Severity: HIGH) 172: > 173: tlsConfig: = & tls.Config {} 174:

It also found a weak random number generator. Whether it is a security flaw depends on how the generated random number is used.

[/ root/gosec-demo/docker-ce/components/engine/pkg/namesgenerator/names-generator.go:843]-G404 (CWE-338): Use of weak random number generator (math/rand instead of crypto/rand) (Confidence: MEDIUM, Severity: HIGH) 842: begin: > 843: name: = fmt.Sprintf ("% slots% s", left [rand.Intn (len (left))] Right [rand.Intn (len (right)]) 844: if name = = "boring_wozniak" / * Steve Wozniak is not boring * / {

Medium priority problem

The tool also found some medium priority issues. It marks a potential DoS threat achieved by decompressing bombs associated with tar, which could be exploited by malicious attackers.

[/ root/gosec-demo/docker-ce/components/engine/pkg/archive/copy.go:357]-G110 (CWE-409): Potential DoS vulnerability via decompression bomb (Confidence: MEDIUM, Severity: MEDIUM) 356: > 357: if _, err = io.Copy (rebasedTar, srcTar); err! = nil {358: w.CloseWithError (err)

It also found a problem accessing files through variables. If malicious users can access this variable, then they can change the value of the variable to read other files.

[/ root/gosec-demo/docker-ce/components/cli/cli/context/tlsdata.go:80]-G304 (CWE-22): Potential file inclusion via variable (Confidence: HIGH, Severity: MEDIUM) 79: if caPath! = "" {> 80: if ca, err = ioutil.ReadFile (caPath); err! = nil {81: return nil, err

Files and directories are usually the most basic elements of operating system security. Here, gosec reports a problem that may require you to check whether the permissions of the directory are secure.

[/ root/gosec-demo/docker-ce/components/engine/contrib/apparmor/main.go:41]-G301 (CWE-276): Expect directory permissions to be 0750 or less (Confidence: HIGH, Severity: MEDIUM) 40: / / make sure / etc/apparmor.d exists > 41: if err: = os.MkdirAll (path.Dir (apparmorProfilePath), 0755); err! = nil {42: log.Fatal (err)

You often need to start the command line tool in the source code. Go is implemented using the built-in exec library. A careful analysis of the variables used to invoke these tools reveals security flaws.

[/ root/gosec-demo/docker-ce/components/engine/testutil/fakestorage/fixtures.go:59]-G204 (CWE-78): Subprocess launched with variable (Confidence: HIGH, Severity: MEDIUM) 58: > 59: cmd: = exec.Command (goCmd, "build", "- o", filepath.Join (tmp, "httpserver") "github.com/docker/docker/contrib/httpserver") 60: cmd.Env = append (os.Environ (), [] string {

Low priority problem

In this output, gosec reports a low priority issue associated with a unsafe call that bypasses the memory protection provided by Go. Take a closer look at the way you call unsafe to see if it is possible to be used by others.

[/ root/gosec-demo/docker-ce/components/engine/pkg/archive/changes_linux.go:264]-G103 (CWE-242): Use of unsafe calls should be audited (Confidence: HIGH) Severity: LOW) 263: for len (buf) > 0 {> 264: dirent: = (* unix.Dirent) (unsafe.Pointer (& buf [0])) 265: bufbuf = buf [dirent.Reclen:] [/ root/gosec-demo/docker-ce/components/engine/pkg/devicemapper/devmapper_wrapper.go:88]-G103 (CWE-242): Use of unsafe calls should be audited (Confidence: HIGH Severity: LOW) 87: func free (p * C.char) {> 88: C.free (unsafe.Pointer (p)) 89:}

It also marks unhandled errors in the source code. You should deal with all errors in the source code.

[/ root/gosec-demo/docker-ce/components/cli/cli/command/image/build/context.go:172]-G104 (CWE-703): Errors unhandled. (Confidence: HIGH, Severity: LOW) 171: err: = tar.Close () > 172: os.RemoveAll (dockerfileDir) 173: return err

Custom gosec scan

Using the default option of gosec can cause a lot of problems. However, after manual audit, you will know which problems do not need to be marked over time. You can specify which tests to exclude and include.

As I mentioned above, gosec looks for problems from the Go source code based on a series of rules. Here is a complete list of rules it uses:

G101: find hard-coded credentials

G102: bind to all interfaces

G103: audit the use of unsafe blocks

G104: audit unchecked errors

G106: audit the use of ssh.InsecureIgnoreHostKey

G107: the url provided to the HTTP request is input as a stain

Automatically exposed profiling endpoints on G108: / debug/pprof

G109: potential integer overflow when strconv.Atoi converts to int16 or int32

G110: potential DoS by decompressing bombs

G201:SQL query construction uses format strings

G202:SQL query construction using string concatenation

G203: using unescaped data in HTML templates

G204: implementation of audit orders

G301: unreasonable allocation of file permissions when creating a directory

G302: unreasonable file permission assignment when using chmod

G303: create temporary files using predictable paths

G304: file path provided by smudge input

G305: traversing files when extracting zip/tar documents

G306: unreasonable file permission assignment when writing to a new file

G307: put the function that returns the error into defer

G401: detect the use of DES, RC4, MD5, or SHA1

G402: find incorrect TLS connection settings

G403: ensure that the minimum RSA key length is 2048 bits

G404: unsafe random number source (rand)

G501: import blacklist list: crypto/md5

G502: import blacklist list: crypto/des

G503: import blacklist list: crypto/rc4

G504: import blacklist list: net/http/cgi

G505: import blacklist list: crypto/sha1

G601: use implicit element aliases in range statements

Exclude specified tests

You can customize gosec to avoid scanning and reporting problems that are known to be secure. You can use the-exclude option and the rule number above to ignore the specified problem.

For example, if you don't want gosec to check for unhandled errors related to hard-coded credentials in the source code, you can run the following command to ignore these errors:

$gosec-exclude=G104. /... $gosec-exclude=G104,G101. /.

Sometimes you know that a piece of code is safe, but gosec still reports a problem. However, you don't want to completely rule out the whole check, because you want gosec to check the new code. You can avoid gosec scanning by adding a # nosec tag to a block of code that you know to be secure. In this way, gosec continues to scan the new code, ignoring the code block of the # nosec tag.

Run the specified check

On the other hand, if you only want to check for the specified problem, you can tell gosec which checks to run with the-include option and the rule number:

$gosec-include=G201,G202.

Scan test files

The Go language comes with support for testing, which uses unit tests to verify that an element meets expectations. In default mode, gosec ignores test files, and you can include them with the-tests option:

Gosec-tests.

Modify the format of the output

Finding problems is only half its function; the other half is to report the problems it checks in a user-friendly and tool-friendly way. Fortunately, gosec can be output in different ways. For example, if you want to see a report in JSON format, use the-fmt option to specify the JSON format and save the results to a results.json file:

$gosec-fmt=json-out=results.json. /... $ls-l results.json-rw-r--r--. 1 root root 748098 Aug 20 05:06 results.json ${"severity": "LOW", "confidence": "HIGH", "cwe": {"ID": "242"," URL ":" https://cwe.mitre.org/data/definitions/242.html"} " "rule_id": "G103", "details": "Use of unsafe calls should be audited", "file": "/ root/gosec-demo/docker-ce/components/engine/daemon/graphdriver/graphtest/graphtest_unix.go" "code": "304:\ t reflect.SliceHeader / Cast to [] byte\ n305:\ theader: = * (* reflect.SliceHeader) (unsafe.Pointer (\ u0026buf))\ n306:\ theader. Len * = 8\ n "," line ":" 305 "," column ":" 36 "}

Check for problems that are easy to find with gosec

Static checking tools cannot completely replace manual code auditing. However, when the contemporary code volume becomes larger and there are many developers, such tools often help to identify problems that are easy to find in a repeatable way. It is useful to help new developers identify and avoid introducing these security flaws when coding.

The above is the editor for you to share how to use gosec to check the security issues in Go code, if you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.