Shulou(Shulou.com)06/02 Report--

This article mainly explains "why strace doesn't work in Docker container", interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's take you to learn "why strace doesn't work in Docker containers"!

The problem here is-if I run strace in Docker container on my laptop, this happens:

$ docker run -it ubuntu:18.04 /bin/bash $ # ... install strace ... [email protected]:/# strace ls strace: ptrace(PTRACE_TRACEME, ...): Operation not permitted

Strace works through the ptrace system call, so if ptrace isn't allowed, it certainly won't work! This problem is easy to solve--on my machine, it works like this:

docker run --cap-add=SYS_PTRACE -it ubuntu:18.04 /bin/bash

But I'm not interested in how to fix it, I wonder why that happened. Why does strace not work and why--cap-add=SYS_PTRACE fix this problem?

Hypothesis 1: The container process lacks CAP_SYS_PTRACE capability.

I always thought it was because Docker container processes didn't have CAP_SYS_PTRACE capability by default. This is the same thing as it can be fixed by--cap-add=SYS_PTRACE, right?

But this is actually unreasonable for two reasons.

Reason 1: In the experiment, as a normal user, I could strace any process my user runs. But if I check if my current process has CAP_SYS_PTRACE capability, then there is no:

$ getpcaps $$ Capabilities for `11589': =

Reason 2: CAP_SYS_PTRACE is described in the manual page of capabilities:

CAP_SYS_PTRACE * Trace arbitrary processes using ptrace(2);

CAP_SYS_PTRACE allows you to ptrace any process owned by any user, just like root. You don't need to use it to ptrace a normal process owned only by your users.

I tested the third method--I ran a Docker container with docker run --cap-add=SYS_PTRACE -it ubuntu:18.04 /bin/bash and removed the CAP_SYS_PTRACE capability, but I could still track progress, even though I no longer had that capability. What? Why?!

Hypothesis 2: What about user namespaces?

My next (and not so well-founded) hypothesis is "Well, maybe this process is in a different user namespace, and strace doesn't work, doesn't work for some reason?" "The question is not really relevant, but it comes to mind when I observe.

Are container processes in different user namespaces? Well, in the container:

root@e27f594da870:/# ls /proc/$$/ns/user -l ... /proc/1/ns/user -> 'user:[4026531837]'

On the host:

bork@kiwi:~$ ls /proc/$$/ns/user -l ... /proc/12177/ns/user -> 'user:[4026531837]'

Because the user namespace ID(4026531837) is the same, root in the container and root on the host are exactly the same user. So there is absolutely no reason why it should not be able to strace the process it creates!

This assumption doesn't make sense, but I didn't realize that root in Docker containers is the same user as root on hosts, so I find it interesting.

Assumption 3: Calls to the ptrace system are blocked by the seccomp-bpf rule

I also know Docker uses seccomp-bpf to prevent container processes from running many system calls. And ptrace is on the list of system calls blocked by Docker's default seccomp profile! (Actually, the list of allowed system calls is a whitelist, so it's just that ptrace isn't in the default whitelist.) But the result is the same.)

It's easy to explain why strace doesn't work in Docker containers-if the ptrace system call is completely shielded, then of course you can't call it, and strace fails.

Let's test this hypothesis-if we disable all seccomp rules, will strace work in Docker containers?

$ docker run --security-opt seccomp=unconfined -it ubuntu:18.04 /bin/bash $ strace ls execve("/bin/ls", ["ls"], 0x7ffc69a65580 /* 8 vars */) = 0 ... it works fine ...

Yes, it works! Very good. The mystery is solved, except...

Why--cap-add=SYS_PTRACE solves the problem?

What we haven't explained is why--cap-add=SYS_PTRACE solves this problem.

The docker run man page explains the--cap-add parameter like this.

--cap-add=[] Add Linux capabilities

This has nothing to do with seccomp rules! What's going on?

Let's look at Docker source code.

When documentation doesn't help, the only thing to do is look at the source code.

The nice thing about Go is that because dependencies are usually in a Go repository, you can use grep to find out where the code to do something is. So I cloned github.com/moby/moby and grep something like rg CAP_SYS_PTRACE.

I think so. In the seccomp implementation of containerd, in contrib/seccomp/seccomp/seccomp_default.go, there is a bunch of code to ensure that if a process has a capability, it also gets access (via seccomp rules) to use system calls related to that capability.

case "CAP_SYS_PTRACE": s.Syscalls = append(s.Syscalls, specs.LinuxSyscall{ Names: []string{ "kcmp", "process_vm_readv", "process_vm_writev", "ptrace", }, Action: specs.ActAllow, Args: []specs.LinuxSeccompArg{}, })

There are other codes in profile/seccomp/seccomp.go in moby and the default seccomp configuration file that seem to do something very similar, so it's possible that this is the code doing it.

So I think we have an answer!

Docker--cap-add does more than it says.

It turns out that--cap-add isn't what the man page says, it's more like--cap-add-and-also-whiteelist-some-extra-system-calls-if-required. This makes sense! If you have an ability like--CAP_SYS_PTRACE that lets you use the process_vm_readv system call, but that system call is blocked by the seccomp configuration file, that won't help you!

So when you give the container CAP_SYS_PTRACE capability, allowing process_vm_readv and ptrace system calls seems like a reasonable choice.

At this point, I believe everyone has a deeper understanding of "why strace doesn't work in Docker containers", so let's actually operate it! Here is the website, more related content can enter the relevant channels for inquiry, pay attention to us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.