How to solve the performance problem of using ceph qem in backend 04/27 Update SLTechnology News&Howtos

How to solve the performance problem of using ceph qem in backend

2025-04-27 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Servers >

Shulou(Shulou.com)05/31 Report--

How to solve the back-end ceph qem performance problem, I believe that many inexperienced people do not know what to do, so this article summarizes the causes of the problem and solutions, through this article I hope you can solve this problem.

Because there is no qemu-kvm module after upgrading libvirt and qemu. The configuration of the nova profile to virt-type=kvm will cause errors, and the configuration to qemu will result in poor performance.

This test environment configuration:

Ip of horizon 192.168.10.71

Other components ip 192.168.10.191

1. Virtual machines started with virt-type=qemu:

Root 24502 17.6 0.4 2530112 307192? Sl 21:51 1:23 / usr/bin/qemu-system-x86_64-name instance-00000001-S-machine pc-i440fx-1.6,accel=tcg,usb=off-m 512-realtime mlock=off-smp 1 Sl-uuid 8ee337e0-4808-49e7-9e66-45dc4a7b1f40-smbios type=1,manufacturer=Red Hat Inc.,product=OpenStack Nova,version=2014.1.1-3.el6 serializer 44454c4cMob 3600-1058-8033 Mustang b3c04f463232 b3c04f463232, uuidfang 8ee337e0-4808-49e7-9e66-45dc4a7b1f40-no-user-config-nodefaults-chardev socket,id=charmonitor Path=/usr/local/libvirt/var/lib/libvirt/qemu/instance-00000001.monitor,server,nowait-mon chardev=charmonitor,id=monitor,mode=control-rtc base=utc-no-shutdown-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2-drive file=rbd:liunova/8ee337e0-4808-49e7-9e66-45dc4a7b1f40_disk:id=admin:key=AQBlwSxU0HsWBxAAMfhhWANPIIDihk7JTFunKQ==:auth_supported=cephx\ None:mon_host=192.168.10.186\: 6789; 192.168.10.187\: 6789\; 192.168.10.188\: 6789\ 192.168.10.189\: 6789 49e7 9e66 9e66 45dc4a7b1f40/console.log device isa-serial,chardev=charserial0,id=serial0 chardev pty Id=charserial1-device isa-serial,chardev=charserial1,id=serial1-device usb-tablet,id=input0-spice port=5900,addr=0.0.0.0,disable-ticketing,seamless-migration=on-k en-us-device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,bus=pci.0,addr=0x2-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

It is obvious that / usr/bin/qemu-system-x86_64 is used above.

Look at: / usr/bin/qemu-system-x86_64-- help

2. Virt-type=kvm generates a virtual machine with openstack and reports an error:

2014-11-18 21 40cd-889d-0aee14afd7f8 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] Traceback (most recent call last):

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] File "/ usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1305, in _ build_instance

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] set_access_ip=set_access_ip)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] return function (self, context, * args, * * kwargs)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] LOG.exception (_ ('Instance failed to spawn'), instance=instance)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] File "/ usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py", line 68, in _ _ exit__

2014-11-18 21 40cd-889d-0aee14afd7f8 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] six.reraise (self.type_, self.value, self.tb)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] block_device_info)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] File "/ usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3656, in_ create_domain_and_network

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] power_on=power_on)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] File "/ usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3559, in _ create_domain

2014-11-18 21 40cd-889d-0aee14afd7f8 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] domain.XMLDesc (0)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] File "/ usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py", line 68, in _ _ exit__

2014-11-18 21 40cd-889d-0aee14afd7f8 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] six.reraise (self.type_, self.value, self.tb)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] File "/ usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 3554, in _ create_domain

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] domain.createWithFlags (launch_flags)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] File "/ usr/lib/python2.6/site-packages/eventlet/tpool.py", line 183, in doit

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] result = proxy_call (self._autowrap, f, * args, * * kwargs)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] File "/ usr/lib/python2.6/site-packages/eventlet/tpool.py", line 141, in proxy_call

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] rv = execute (f, * args, * * kwargs)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] six.reraise (c, e, tb)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 TRACE nova.compute.manager [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] File "/ usr/lib/python2.6/site-packages/eventlet/tpool.py", line 80, in tworker

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] rv = meth (* args, * * kwargs)

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] File "/ usr/lib64/python2.6/site-packages/libvirt.py", line 727, in createWithFlags

2014-11-18 21 40cd-889d-0aee14afd7f8 53 TRACE nova.compute.manager 55.498 25214 TRACE nova.compute.manager [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] if ret =-1: raise libvirtError ('virDomainCreateWithFlags () failed', dom=self)

2014-11-18 21 40cd-889d-0aee1tafd7f8 53 TRACE nova.compute.manager 55.498 25214 [instance: 5b50ed5b-c3d9-40cd-889d-0aee1tafd7f8] libvirtError: internal error: process exited while connecting to monitor: char device redirected to / dev/pts/4

2014-11-18 21 5b50ed5b-c3d9 53 TRACE nova.compute.manager 55.498 25214 TRACE nova.compute.manager [instance: 5b50ed5b-c3d9-40cd-889d-0aee14afd7f8] q%mu-kvm:-drive file=rbd:liunova/5b50ed5b-c3d9-40cd-889d-0aee14afd7f8_disk:id=admin:key=AQBlwSxU0HsWBxAAMfhhWANPIIDihk7JTFunKQ==:auth_supported=cephx\; none:mon_host=192.168.10.186\: 6789\; 192.168.10.187\: 6789\; 192.168.10.188\: 6789\ 192.168.10.189\: 6789 none:mon_host=192.168.10.186: 6789\; 192.168.10.188\: 6789\; 192.168.10.188\: 6789; 192.168.10.189\: 6789: No such file or directory

3. About qemu-system-x86_64, qemu-kvm, qemu-img

Figure 3.1

Check the versions and supported formats of the above two qemu-img:

/ usr/bin/qemu-img | grep-E "version | Supported"

Figure 3.2

/ usr/local/qemu/bin/qemu-img | grep-E "version | Supported"

Figure 3.3

After going back to the above error:

Figure 3.4

The qemu-kvm used to launch here:

Find qemu-kvm:

Figure 3.5

View the qemu-kvm version:

Figure 3.6

Then find the corresponding 0.12.1 version of qemu-img (figure 3.2), and find that rbd is not supported (rbd support is required for the use of ceph backend). (note: figure 3.3qemu-img supports rbd. )

Through the above analysis and summary:

/ usr/bin/qemu-system-x86_64 is matched with / usr/local/qemu/bin/qemu-img. Version: version 1.6.1 (after manually compiling the source package of 1.6.1 qemu)

/ usr/libexec/qemu-kvm is matched with / usr/bin/qemu-img. Version: version 0.12.1 (pre-upgrade version)

This explains the error in figure 3.4, because figure 3.4 uses V0.12.1, and the qemu-img of V0.12.1 does not compile the module that supports rbd, so the error in figure 3.4 occurs.

Solution:

Replace the qemu-kvm started in figure 3.4 with the / usr/bin/qemu-system-x86_64 of the new version of version 1.6.1 to start the virtual machine.

Procedure:

Mv / usr/libexec/qemu-kvm / usr/libexec/qemu-kvm.bak

Generate the virtual machine again:

It was found that the virtual machine was generated successfully.

Let's take a look at the bottom book startup parameters:

Ps aux | grep qemu

After discovering the virtual machine generated later and configuring nova.conf as virt-type=kvm. The virtual machine can be generated normally if the native qemu-kvm is removed. And the generated virtual machine uses kvm virtual machine (accel=kvm), which improves the performance.

Conclusion:

1. When nova.conf is configured as virt-type=qemu, the qemu software feature will be used to simulate and generate a virtual machine, resulting in a performance degradation.

2. When nova.conf is configured as virt-type=kvm, first go back to find the qemu-kvm module. If there is a qemu-kvm module, use this module directly to start the virtual machine. If the qemu-kvm module cannot be found, it looks for qemu-system-x86_64, starts the virtual machine with qemu-system-x86_64, and uses the kvm virtual machine feature. And enable-- enable-kvm characteristic parameters

Note: kvm Virtualization Module:

In the old version, there was a separate qemu-kvm module that combined with qemu to do virtual machine work. In subsequent new releases, the qemu-kvm module has been fully incorporated into qemu. So when you need to use the kvm feature, you only need to add the parameter enable-kvm to the qemu-system-x86_64 startup command to enable it.

After reading the above, have you mastered how to solve the performance problem of using ceph qem in the back end? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.

*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.