Shulou(Shulou.com)05/31 Report--

This article mainly introduces OpenStack how to deal with BUG, the article is very detailed, has a certain reference value, interested friends must read it!

one。 Discover Bug:

The virtual machine soft delete (soft-delete) function in   Nova means that only a virtual machine record in the database is marked (status='SOFT-DELETE') for a period of time, and then the corresponding virtual machine instance in the virtualization platform (kvm, etc.) is set to the shutdown state, and the virtual machine instance is not deleted until after a certain period of time. This feature provides "regret time" for cloud platform users, which can recover the misoperation to some extent. By default, the soft delete feature is turned off by adding the "reclaim_instance_interval" option to the nova profile and setting its value to the number of milliseconds of "regret time".

Before describing a specific Bug,   needs to briefly introduce the basic concepts of user management in openstack.

The figure above on   is a simplified version of the openstack user model, and the quota that is not managed by keystone is also taken for ease of understanding.

  Bug is related to soft deletion. The specific scenario is as follows: suppose there are two projects and two users in OpenStack: general project A, its user a, and administrator project Admin, whose user is admin (user management-related concepts can be found in keystone documents). User an inadvertently deletes one of his virtual machines. At this time, ask the system administrator to see if there is any way to restore it. Fortunately, the soft delete function enabled by the system And the deleted virtual machine is still in the recyclable time range, then the administrator logs in to the system as admin and restores the virtual machine for user a, but the careful administrator finds something wrong: there are no virtual machines under its Admin project, but its quota is used. does this have anything to do with the operation just now? Try again: the normal user deletes the virtual machine and the admin user recovers it, and the quota changes again, and sure enough, the quota of the restored virtual machine is incorrectly added to the Admin project. The Bug still exists in the latest version of kilo, so interested students can try it.

two。 Locate Bug:

  discovered the existence of Bug, so go a step further and find out why in the code.

How does   determine the location of the problem code? This requires a general understanding of the project structure of Nova, let's take a brief look at:

The figure above in   is a minimalist version of the nova architecture, and none of the components related to this problem are drawn. The operation process of restoring a virtual machine is roughly as follows:

Nova api receives the user request, queries the details of the virtual machine in the database, and sends the host and name of the virtual machine to the message queue.

After listening to the relevant messages, the nova compute service begins to perform specific operations, make some adjustments to the records of the virtual machine in the database, and call the underlying driver to restore the virtual machine.

Since there is no problem with the functional level of soft deletion in  , and the deletion and recovery process of the virtual machine is smooth, it can be seen that it will not be the problem of the driver. Follow the code call of the API layer to find it, and you can locate it soon. See the code snippet of the problem directly:

Def restore (self, context, instance): # the code deletes flavor = instance.get_flavor () # gets the quotas object num_instances, quotas = self._check_num_instances_quota (context, flavor, 1,1) self._record_action_start (context, instance Instance_actions.RESTORE) try: if instance.host: instance.task_state = task_states.RESTORING instance.deleted_at = None instance.save (expected_task_state= [none]) self.compute_rpcapi.restore_instance (context Instance) else: instance.vm_state = vm_states.ACTIVE instance.task_state = None instance.deleted_at = None instance.save (expected_task_state= [none]) # Update quotas quotas.commit ()

The code above   is the main method for virtual machine recycling at the API level, and you can see that there is an obvious quota operation (quotas). Before interpreting this code, it is necessary to introduce the concept of "context" in nova. Not only nova, but also this "context" can be seen everywhere in other openstack projects. It is an object that wraps user request information, including user project and authentication information. Through it, it can easily carry out API calls between projects and query user information. Every time the API service receives a user's HTTP request, it will create a new context.

When   comes back to this code, we focus on what we did to quotas: on the second line of the method, we got quotas through a method, and quotas.commit () was executed at the end of the method. There is not much information we can get. Let's take a look at the method to get quotas: _ check_num_instances_quota.

# only part of def _ check_num_instances_quota (self, context, instance_type, min_count, max_count) is intercepted here: req_cores = max_count * instance_type ['vcpus'] vram_mb = int (instance_type.get (' extra_specs', {}). Get (VIDEO_RAM) 0) req_ram = max_count * (instance_type ['memory_mb'] + vram_mb) try: quotas = objects.Quotas (context) quotas.reserve (context, instances=max_count, cores=req_cores, ram=req_ram). Return max_count, quotas

  you can see the process of obtaining quotas here: the quotas object is created through the current context, and the reserve operation is performed. We know that the context comes from the HTTP request, and the information of the requesting user is stored in it, so the "owner" of the quotas object here is the user in the context.

  combines the scenario of Bug: the administrator restores the virtual machine of user a, and the request is made by the administrator. The information of the administrator is recorded in the current context, where the quotas is taken for granted, and then the virtual machine of user an is operated, but the administrator's quotas is updated. Well, the truth comes out!

three。 Fix Bug:

The reason for   Bug is that the obtained quotas does not belong to the desired user, but it is obviously not appropriate to modify the context directly (it will affect subsequent operations). Let's take a look at the quotas object itself:

Class Quotas (base.NovaObject): # partial code def _ _ init__ (self, * args, * * kwargs): super (Quotas, self). _ _ init__ (* args, * * kwargs) # Set up defaults. Self.reservations = [] self.project_id = None self.user_id = None self.obj_reset_changes ()... Def reserve (self, context, expire=None, project_id=None, user_id=None, * * deltas): reservations = quota.QUOTAS.reserve (context, expire=expire, project_id=project_id, user_id=user_id * deltas) self.reservations = reservations self.project_id = project_id self.user_id = user_id self.obj_reset_changes () def commit (self, context=None): if not self.reservations: return if context is None: context= self._context quota.QUOTAS.commit (context Self.reservations, project_id=self.project_id, user_id=self.user_id) self.reservations = None self.obj_reset_changes ()

  pay attention to the parameters of the reserve method, which defaults to project_id and user_id of None, which is a convenient entrance to change the owner of quotas!

The modified code of   will not be posted here. Interested students can read it in this submission: Code Review

four。 Code submission and Review:

The   openstack community has a complete project management process, and here is a diagram that describes the workflow in more detail:

  as you can see from the diagram, bugfix is one of the simplest processes.

The above is all the content of the article "how OpenStack deals with BUG". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.