Shulou(Shulou.com)06/01 Report--

This article mainly explains the "macvtap use tutorial", the article explains the content is simple and clear, easy to learn and understand, the following please follow the editor's ideas slowly in depth, together to study and learn the "macvtap use tutorial" bar!

One-click installation of kubernetes

Macvtap is a common technology of virtual machine network virtualization, of course, containers can also be used. The implementation of MACVTAP is based on traditional MACVLAN. Like TAP devices, each MACVTAP device has a corresponding Linux character device and the same IOCTL interface as the TAP device, so it can be directly used by KVM/Qemu to easily complete the network data exchange. The goal of introducing MACVTAP devices is to simplify the switching network in the virtualized environment, replace the traditional combination of Linux TAP devices and Bridge devices, and support new virtualized network technologies, such as 802.1 Qbg.

For example, kata's virtual network uses this technology, the following practice will be more clear about the network principle of kata, it is recommended to compare with the tutorial hands-on practice.

Initialize the environment qemu libvirt environment

I have made a mirror image of qemu libvirt, you can use it directly: there are many benefits in the container, if the environment is messy, you can quickly restore a clean environment. The use of device pairs can also reduce the impact on the host network.

Docker run-d-- privileged-v / dev:/dev-v / home/fanux:/root-- name qemu-vm fanux/libvirt:latest init

Note:

Network and other operations require the container to have a privileged mode

The tap network needs to mount the / dev directory

/ home/fanux can be used as a working directory to mirror the libvirt configuration written by yourself and put it in it to prevent loss after deleting the container

Since libvirt requires systemd, we start the init process in the container

You can also build your own image. I provide a Dockerfile, and the-j parameter sets the number of threads at compile time according to your machine's CPU:

FROM centos:7.6.1810RUN yum install-y wget & & wget https://download.qemu.org/qemu-4.0.0.tar.xz & &\ tar xvJf qemu-4.0.0.tar.xz\ & & yum install-y automake gcc-c++ gcc make autoconf libtool gtk2-devel\ & & cd qemu-4.0.0\ & &. / configure\ & & make-j 72 & & make install\ & & yum install- Y bridge-utils & & yum install-y net-tools tunctl iproute & & yum-y install openssh-clients\ cd. & & rm qemu-4.0.0.tar.xz & & rm-rf qemu-4.0.0RUN yum install-y libvirt virt-manager gustfish openssh-clients virtual machine image

Enter the container

[root@compute84 libvirt] # docker exec-it qemu-vm bashbash-4.2# cdbash-4.2# lsCentOS-7-x86_64-GenericCloud.qcow2 centos.qcow2 image nohup.out start.sh vm3.xmlCentOS-7-x86_64-Minimal-1810.iso cloud-init-start.sh kernel qemu vm.xmlFedora-Cloud-Base-30-1.2.x86_64.qcow2 destroy.sh libvirt run.sh vm2.xml

Download the virtual machine image:

Openstack has provided many images with cloud-init installed, address: https://docs.openstack.org/image-guide/obtain-images.html

I used a relatively new qcow2 format image of centos:

Wget http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1905.qcow2

Modify the virtual machine root password:

Virt-customize-a CentOS-7-x86_64-GenericCloud.qcow2-- root-password password:coolpass starts the virtual machine

View container network information:

Bash-4.2# systemctl start libvirtdbash-4.2# ip addr1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6:: 1/128 scope host valid_lft forever preferred_lft forever2: virbr0: mtu 1500 qdisc noqueue state DOWN group default qlen 1000 link/ether 52:54:00:c6:59:47 brd ff:ff:ff:ff:ff:ff inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0 valid_lft forever preferred_lft forever3: virbr0-nic: mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000 link/ether 52:54:00:c6:59:47 brd ff:ff:ff:ff:ff:ff1310: eth0@if1311: mtu 1500 qdisc noqueue state UP group default link / ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 172.17.0.2/16 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe11:2/64 scope link valid_lft forever preferred_lft forever

Libvirt created the negligible, most important thing is eth0.

Write libvirt configuration

Vm3.xml:

Vm3 2048 2048 hvm destroy restart destroy / usr/local/bin/qemu-system-x86_64

The correct image address is configured here, and the place of interface is the configuration related to macvtap.

Start the virtual machine bash-4.2# virsh define vm3.xml Domain vm3 defined from vm3.xmlbash-4.2# virsh start vm3 Domain vm3 started

After booting, you can see that the macvtap device has been created.

Bash-4.2# ip addr7: macvtap0@eth0: mtu 1500 qdisc pfifo_fast state UP group default qlen 500 link/ether 52:54:00:56:e4:20 brd ff:ff:ff:ff:ff:ff inet6 fe80::5054:ff:fe56:e420/64 scope link valid_lft forever preferred_lft forever

Go to the virtual machine:

Virsh console vm3

If you get stuck at this step:

A start job is running for LSB: Bri... Networkingcloud-init [2253]: 2019-06-27 08 url_helper.py 37 url_helper.py [WARNING]: Calling 'http://192.168.122.1/latest/meta-data/instance-id' failed [87120s]: request error

Just wait for it to time out, because when we macvtap, we need to enter the virtual machine to configure the network. Then you can enter the virtual machine:

CentOS Linux 7 (Core) Kernel 3.10.0-957.1.3.el7.x86_64 on an x86_64localhost login: rootPassword: Last login: Thu Jun 27 07:19:32 from gateway

The password is the image password we set above: coolpass

Configure virtual machine IP [root@localhost ~] # ip addr1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00 scope host valid_lft forever preferred_lft forever2: eth0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ Ether 52:54:00:56:e4:20 brd ff:ff:ff:ff:ff:ff inet6 fe80::5054:ff:fe56:e420/64 scope link valid_lft forever preferred_lft forever [root@localhost ~] # ip addr add 172.17.0.2 ip route add default via 16 dev eth0 [root@localhost ~] # ip route add default via 172.17.0.1 dev eth0 [root@localhost ~] # ip addr1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6:: 1/128 scope host valid_lft forever preferred_lft forever2: eth0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:56:e4:20 brd ff:ff:ff:ff:ff:ff inet 172.17.0.2 / 16 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe56:e420/64 scope link valid_lft forever preferred_lft forever [root@localhost ~] # ip route default via 172.17.0.1 dev eth0 172.17.0.0 dev eth0 proto kernel scope link src 16 dev eth0 proto kernel scope link src 172.17.0.2 [root@localhost ~] # ping 172.17.0.1PING 172.17.0.1 (172.17.0.1) 56 (84 ) bytes of data.64 bytes from 172.17.0.1: icmp_seq=1 ttl=64 time=0.622 ms64 bytes from 172.17.0.1: icmp_seq=2 ttl=64 time=0.194 ms

After the configuration, you can ping the network off.

Modify DNS configuration

Not changing this may cause ssh to be very slow:

[root@localhost ~] # cat / etc/resolv.conf; Created by cloud-init on instance boot automatically, do not edit.;; generated by / usr/sbin/dhclient-scriptnameserver 114.114.114.114 modify sshd configuration

Modify the / etc/ssh/sshd-config file and change the PermitRootLogin no to yes,PubkeyAuthentication yes to no,AuthorizedKeysFile .ssh / authorized_keys preceded by # masking and PasswordAuthentication no to yes.

Launch the ssh client container docker run-- rm-it fanux/libvirt bash [root@ee18547e9ed2 /] # ssh root@172.17.0.2ssh: connect to host 172.17.0.2 port 22: Connection refused

It will not work because the eth0 in the container and the eth0 in the virtual machine are configured with the same address. You only need to delete the eth0 address in the container:

Bash-4.2# ip addr del 172.17.0.2/16 dev eth0

Ssh again to enter the virtual machine:

[root@ee18547e9ed2 /] # ssh root@172.17.0.2The authenticity of host '172.17.0.2 (172.17.0.2)' can't be established.ECDSA key fingerprint is SHA256:kTk3yy8588WQHNtwpzS+h7u0W3RELWC8hJQwIwLOkdc.ECDSA key fingerprint is MD5:0c:f3:b5:69:c6:08:05:14:f8:da:42:2f:85:29:51:d0.Are you sure you want to continue connecting (yes/no)? YesWarning: Permanently added '172.17.0.2' (ECDSA) to the list of known hosts.root@172.17.0.2's password: Last login: Thu Jun 27 08:38:00 2019 [root@localhost ~] # ip addr1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00 link/loopback 00 brd 00 00 inet 127.0.1 8 scope host lo valid_lft forever Preferred_lft forever inet6:: 1scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe56:e420/64 scope link valid_lft forever preferred_lft forever 128 scope host valid_lft forever preferred_lft forever2: eth0: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 52:54:00:56:e4:20 brd ff:ff:ff:ff:ff:ff inet 172.17.0.2 Universe 16 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::5054:ff:fe56:e420/64 scope link valid_lft forever preferred_lft forever repair Change the virtual machine mac address [root@localhost ~] # ip link set eth0 address 52:54:00:56:e4:23

You will find that you can't connect to the virtual machine.

Change back to:

[root@localhost ~] # ip link set eth0 address 52:54:00:56:e4:20

Can connect normally again, why?

This is because the mac address of the eth0 of the virtual machine must be the same as the mac address of macvtap0. The principle is very simple.

ARP asks what is the mac address of a machine whose IP address is 172.17.0.2.

The virtual machine responded with a 52:54:00:56:e4:20.

Macvtap0 can be understood as hanging on the bridge port, so the packet is sent to macvtap0 (because the mac address is the same, different will not be sent to macvtap)

Macvtap0 throws the packet to the qemu application process (eventually to the virtual machine eth0)

Naked qemu

The above is used through libvirt, which shields a lot of underlying details. If you use the qemu command directly, you need to do the following:

Create a macvtap device:

Ip link add link eth0 name macvtap0 type macvtap mode bridgeip link set macvtap0 address 1a:46:0b:ca:bc:7b upbash-4.2# cat / sys/class/net/macvtap0/ifindex # corresponds to the / dev/tap22bash-4.2# cat / sys/class/net/macvtap0/address # configuration of the following command consistent with the qemu mac address configuration 1a:46:0b:ca:bc:7b

Start qemu, and then the address configuration in the virtual machine is the same as libvirt, and you can enter the virtual machine configuration through the vnc client (vnc viewer). I will not repeat it:

Introduction to bash-4.2# qemu-system-x86_64-enable-kvm / root/CentOS-7-x86_64-GenericCloud.qcow2\-netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=4 30/dev/tap2 4/dev/vhost-net\-device virtio-net-pci,netdev=hostnet0,id=net0,mac=1a:46:0b:ca:bc:7b\-monitor telnet:127.0.0.1:5801,server,nowaitVNC server running on:: 1:5900cloud-init

As mentioned above, configuring the address of a virtual machine Nic through macvtap technology requires entering the virtual machine configuration. However, when we implement a virtual machine management system, we obviously do not enter it manually to configure it, which requires cloud-init.

It can help us configure the virtual machine when the virtual machine starts, such as configuring the password, configuring the network, executing commands and writing some files. First, create a user-data file with the following contents:

# cloud-configwrite_files:-content: | DEVICE=eth0 ONBOOT=yes TYPE=Ethernet USERCTL=no IPADDR=172.17.0.2 NETMASK=255.255.0.0 GATEWAY=172.17.0.1 BOOTPROTO=static DNS1=172.17.0.1 ONBOOT=yes path: / etc/sysconfig/network-scripts/ifcfg-eth0runcmd:-systemctl start networkgroups:-centos: [root]-cloud-usersssh_pwauth: yeschpasswd: expire: false list: | user1:222222 root:333333

Create a cloud-init image file

Yum install-y cloud-utilscloud-localds my-seed.img user-data

Use this image in libvirt

In order to prevent cloud-init from using the network to obtain metadata, because the network card is not set up, it will be stuck for five minutes, so we directly prohibit the network access to metadata:

Vi / etc/cloud/cloud.cfg.d/05_logging.cfg

Plus:

Network: config: disabled

Then start the virtual machine.

common problem

Inappropriate ioctl for device

Qemu-system-x86_64:-net tap,fd=5: TUNGETIFF ioctl () failed: Inappropriate ioctl for deviceTUNSETOFFLOAD ioctl () failed: Inappropriate ioctl for device

Because the container does not mount the / dev directory

KVM bios is banned.

[root@helix105 ~] # docker run busybox uname-aCould not access KVM kernel module: No such file or directoryqemu-lite-system-x86_64: failed to initialize KVM: No such file or directory/usr/bin/docker-current: Error response from daemon: oci runtime error: Unable to launch / usr/bin/qemu-lite-system-x86_64: exit status 1.ERRO [0001] error getting events from daemon: net/http: request canceled [root@helix105 ~] # lsmod | grep kvmkvm 598016 0 Irqbypass 16384 1 kvm [root@helix105 ~] # modprobe kvm-intelmodprobe: ERROR: could not insert 'kvm_intel': Operation not supportedYou have mail in / var/spool/mail/root [root@helix105 ~] # dmesg | grep kvm [8.239309] kvm: disabled by bios

This needs to be opened in bios.

KVM: Permission denied

Bash-4.2# virsh start vm_name1error: Failed to start domain vm_name1error: internal error: qemu unexpectedly closed the monitor: Could not access KVM kernel module: Permission denied2019-06-20T07:26:33.304320Z qemu-system-x86_64: failed to initialize KVM: Permission denied

Solution:

# chown root:kvm / dev/kvm modification / etc/libvirt/qemu.conf,#user= "root" user= "root" # group= "root" group= "root" restart the service # service libvirtd restart. Thank you for reading the problem. The above is the content of the "macvtap usage tutorial". After the study of this article, I believe you have a deeper understanding of the macvtap usage tutorial, and the specific usage needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.