Shulou(Shulou.com)05/31 Report--

This article mainly introduces what OpenStack means. It is very detailed and has certain reference value. Friends who are interested must finish reading it.

1. What is OpenStack?

OpenStack is both a community, a project and open source software that provides an operating platform or toolset for cloud deployment. Its purpose is to help organizations run clouds for virtual computing or storage services, providing scalable and flexible cloud computing for public and private clouds, as well as large and small clouds.

OpenStack includes a set of open source projects maintained by the community, namely OpenStackCompute (Nova), OpenStackObjectStorage (Swift), and OpenStackImageService (Glance).

OpenStackCompute [1], the controller of the cloud organization, provides a tool to deploy the cloud, including running instances, managing the network, and controlling access to the cloud by users and other projects (thecloudthroughusersandprojects). Its underlying open source project is called Nova, and it provides software that controls the IaaS cloud computing platform, similar to AmazonEC2 and RackspaceCloudServers. In fact, it defines drivers that interact with potential virtualization mechanisms running on the host operating system, exposing WebAPI-based functionality.

OpenStackObjectStorage [2] is an extensible object storage system. Object storage supports a variety of applications, such as copying and archiving data, image or video services, storing secondary static data, developing new applications for data storage integration, storing data whose capacity is difficult to estimate, and creating cloud-based elastic storage for Web applications.

OpenStackImageService [1] is a virtual machine image storage, query and retrieval system, including RESTfulAPI services that allow users to query VM image metadata through HTTP requests and retrieve actual images. VM images can be configured in four ways: a simple file system, an object storage system like OpenStackObjectStorage, which is directly stored in Amazon'sSimpleStorageSolution (S3), and an indirect access to S3 with ObjectStore.

The basic relationship of the three projects is shown in figure 1-1 below:

The relationship between the three components of 1-1 OpenStack

two。 Conceptual Architecture of Cloud Service provider

OpenStack can help us set up our own IaaS and provide AmazonWebService-like services to our customers. To achieve this, we need to provide several advanced features:

A) allow application owners to register for cloud services to view usage and billing

B) allow Developers/DevOpsfolks to create and store custom images for their applications

C) allow them to start, monitor and terminate the instance

D) allow CloudOperator to configure and operate the infrastructure

All four points go straight to the core of providing IaaS. Now, assuming you agree to these four features, you can now put them into the conceptual architecture 2-1 shown below.

2-1 OpenStack Conceptual Framework

In this model, the author assumes four sets of users that need to interact with the cloud: developers,devops,ownersandoperators, and divides the functionality they need for each category of users. The architecture uses a very common hierarchical method (presentation,logicandresources), which has two orthogonal regions.

In the presentation layer, components interact with users, receive and present information. Webportals provides a graphical interface for non-developers and API endpoints for developers. If it is a more complex structure, load balancing, control agents, security and name services will also be at this layer.

The logic layer provides intelligence and control functions for the cloud. This layer includes deployment (workflow for complex tasks), scheduling (job-to-resource mapping), policies (quotas, etc.), mirror registration imageregistry (instance mirror metadata), and logging (events and metering).

Assume that the vast majority of service providers already have customer identity and billing systems. Any cloud architecture needs to integrate these systems.

In any complex environment, we will need a management layer to manipulate the environment. It should include an API access cloud management feature and some forms of monitoring (forms). It is likely that monitoring capabilities will be added to an existing tool in the form of integration. Monitoring and adminAPI have been added to the current architecture for our virtual service providers, and in a more complete architecture, you will see a range of supporting features, such as provisioning and configurationmanagement.

Finally, the resource layer. Since this is a compute cloud, we need actual compute, network, and storage resources to supply to our customers. This layer provides these services, whether they are servers, network switches, NAS (networkattachedstorage), or other resources.

3. OpenStack Compute architecture

3.1 OpenStack Compute logical architecture

In the OpenStack Compute logical architecture, the vast majority of components can be divided into two custom-written Python daemons (custom written python daemons).

A) WSGI applications (nova-api, glance-api, etc) that receive and coordinate API calls

B) the Worker daemon that performs the deployment task (nova-compute, nova-network, nova-schedule, etc.)

However, there are two important parts of the logical architecture, neither custom writing nor based on Python, they are message queues and databases. Both simplify the asynchronous deployment of complex tasks (tasks through messaging and information sharing).

Figure 3-1 of the logical architecture is as follows:

3-1 OpenStack Compute logical architecture

From the figure, we can sum up three points:

A) end users (DevOps, Developers, and other OpenStack components) interact with OpenStack Compute by talking to nova-api.

B) OpenStack Compute daemons exchange information through queues (behaviors) and databases (information) to execute API requests.

C) OpenStack Glance is basically an independent infrastructure, and OpenStack Compute interacts with it through Glance API.

The situation of its various components is as follows:

A) the nova-api daemon is the center of the OpenStack Compute. It provides endpoints for all API queries (OpenStack API or EC2 API), initializes most deployment activities (such as running instances), and enforces some policies (most quota checks).

B) the nova-compute process is primarily a Worker daemon that creates and terminates virtual machine instances. The process is quite complex, but the basic principle is simple: receive behaviors from the queue, and then execute a series of system commands to execute them when updating the status of the database.

C) nova-volume manages the creation, attachment, and cancellation of volumes mapped to computer instances. These volumes can come from many providers, such as ISCSI and AoE.

D) the Nova-network worker daemon is similar to nova-compute and nova-volume. It receives network tasks from the queue and then performs tasks to manipulate the network, such as creating a bridging interfaces or changing an iptables rules.

E) Queue provides a central hub to pass messages to the daemon. It is currently implemented in RabbitMQ. But in theory it could be any AMPQ message queue supported by python ampqlib.

F) SQL database stores the vast majority of compile-time and run-time state in the cloud infrastructure. This includes available instance types, in-use instances, available networks and projects. In theory, OpenStack Compute can support any database supported by SQL-Alchemy, but currently the widely used databases are sqlite3 (suitable for testing and development work only), MySQL, and PostgreSQL.

G) OpenStack Glance, which is a separate project, is an optional part of a compute architecture and is divided into three parts: glance-api, glance-registry and the image store. Where glance-api accepts API calls, glance-registry is responsible for storing and retrieving mirror metadata, and the actual Image Blob is stored in Image Store. Image Store can be a variety of different Object Store, including OpenStack Object Storage (Swift)

H) finally, user dashboard is another optional project. OpenStack Dashboard provides an OpenStack Compute interface for application developers and devops staff to provide API-like functionality. It is currently implemented as a Django web Application. Of course, there are other Web front ends available.

3.2 concept mapping

By mapping the logical schema to the conceptual schema (as shown in 3-2), you can see what we are missing.

Mapping from 3-2 logical Architecture to Conceptual Architecture

This kind of coverage is not unique, it is only the author's understanding here. The functional scope is represented by overriding the OpenStack Compute logical components, Glance and Dashboard. For each override, there is a corresponding name of the logical component that provides the function.

A) in this coverage, the biggest gaps are logging and billing. At the moment, OpenStack Compute does not have a Billing component that coordinates logging events, logs, and creates / renders bills. The real focus is the integration of logging and Billing. This can be remedied in the following ways. Such as code expansion, integration of commercial products or services, or custom log parsing.

B) Identity is also a possible addition in the future.

C) customer portal is also an integration point. User dashboard (see running instances, starting new instances) does not provide an interface to allow application owners to sign services, track their costs, and declare faulty lodge trouble tickets. Moreover, this is probably appropriate for the service provider we envisioned.

D) ideally, Admin API will replicate all the functionality we can do through the command line interface. It would be better in a Diablo release with Admin API work.

E) Cloud monitoring and operations will be the focus of service providers. The key to a good method of operation is good tools. Currently, OpenStack Compute provides nova-instancemonitor, which tracks the usage of compute nodes. We still need tripartite tools to monitor in the future.

F) Policy is an extremely important aspect, but it is very relevant to the vendor. Everything from quotas to QoS to privacy controls is under its jurisdiction. There is partial coverage on the current diagram, but it depends on the complex requirements of the supplier. For accuracy, OpenStack Compute provides quotas for instances, floating-point IP addresses, and metadata.

G) currently, Scheduling within OpenStack Compute is quite preliminary for large installations. The scheduler is designed as a plug-in and currently supports chance (random host allocation), simple (minimum load) and zone (random nodes in an availability zone.) A distributed scheduler and a scheduler that understands heterogeneous hosts are under development.

As you can see, OpenStack Compute provides a good foundation for our imaginary service provider, as long as the service provider is willing to do some integration.

3.3 OpenStack Compute system architecture

OpenStack Compute consists of some major components. "Cloud controller" contains many components, which represent global state and interact with other components. In fact, it provides Nova-api services. Its function is to provide an endpoint for all API queries, initialize most deployment activities, and implement some policies. The API server acts as a cloud controller web Service front end. Compute controller provides compute service resources, typically including compute service,Object Store component to optionally provide storage services. Auth manager provides authentication and authorization services, and Volume controller provides fast and persistent block-level storage for compute servers. Network controller provides a virtual network for compute servers to interact with each other and with the public network. Scheduler chooses the most appropriate compute controller to manage (host) an instance.

OpenStack Compute is based on a shared, message-based architecture. Cloud controller interacts with internal object store through HTTP and communicates through AMQP and scheduler, network controller, and volume controller. To avoid blocking each component while waiting to receive, OpenStack Compute uses asynchronous calls.

In order to obtain no shared attributes with multiple backups of a component, OpenStack Compute maintains all cloud system states in a distributed data store. Updates to the system state are written to this storage, using proton transactions if necessary.

Requests for system status are read out from the store. In a few cases, the controller also caches the read results for a short time.

3.4 OpenStack Compute physical Architectur

OpenStack Compute uses a shared, message-based architecture, which is very flexible, and we can install each nova- service on a separate server, which means that there are many possible ways to install OpenStack Compute. Perhaps the only federated dependency for multi-node deployment is that Dashboard must be installed on the nova-api server. Several deployment architectures are as follows:

A) single node: a server runs all nova- services and also drives virtual instances. This configuration is only for trying OpenStack Compute, or for development purposes

B) double nodes: one cloud controller node runs all nova-services,compute nodes except nova-compute runs nova-compute. A client computer probably needs to package the image and interact with the server, but it is not necessary. This configuration is mainly used for proof of concept and development environment.

C) Multi-node: by simply deploying nova-compute on an additional server and copying nova.conf files to this new node, you can add more compute nodes on the basis of the two nodes to form multi-node deployment. In more complex multi-node deployments, an volume controller and a network controller can be added as additional nodes. For running multiple virtual machine instances that require a lot of processing power, at least four nodes are the best.

A possible Openstack Compute multi-server deployment (the networked virtual servers in the cluster may change) is shown below:

3-3 OpenStack Compute physical architecture one

If you notice that a large number of replications in message queues cause performance problems, an optional architecture is to add more Messaging servers. In this case, in addition to extending the database server, an additional RabbitMQ server can be added. In the deployment, you can run any nova-service on any server, as long as the nova.conf is configured to point to the RabbitMQ server, and these servers can send messages to it.

Figure 3-4 below is another multi-node deployment architecture.

3-4 multi-node deployment architecture II

3.5 OpenStack Compute Services Architecture

Because Compute has multiple services and possibly multiple configurations, figure 3-5 below shows the overall service architecture and the communication system between the services.

3-5 OpenStack Compute service architecture

4. OpenStack Image Service

OpenStack Image Service consists of two main parts, API server and Registry server (s).

OpenStack Image Service is designed to fit as many back-end warehousing and registration database scenarios as possible. API Server (running the "glance api" program) acts as a communication hub. For example, various client programs, registration of mirror metadata, and storage systems that actually contain virtual machine mirror data communicate through it. API server forwards the client's request to the mirror metadata registry and its back-end repository. It is through these mechanisms that OpenStack Image Service actually saves the virtual machine image.

The backend repositories supported by OpenStack Image Service are:

A) OpenStack Object Storage. It is a highly available object storage project in OpenStack.

B) FileSystem. The default backend for OpenStack Image Service storage virtual machine images is the back-end file system. This simple backend writes the image file to the local file system.

C) S3. This backend allows OpenStack Image Service storage virtual machine images in the Amazon S3 service.

D) HTTP. OpenStack Image Service can read available virtual machine images on Internet through HTTP. This storage is read-only.

OpenStack Image Service registry servers is an OpenStack Image Service Registry API-compliant server.

According to the installation manual, the two services are installed on the same server. The mirror itself can be stored in OpenStack Object Storage, Amazon's S3 infrastructure,fileSystem. If you only need read-only access, you can store it on a Web server.

5. OpenStack Object Storage

5.1 key concepts

A) Accounts and Account Servers

OpenStack Object Storage systems are designed to be used by many different storage consumers or customers. Each user must identify himself through the authentication system. To this end, OpenStack Object Storage provides an authorization system (swauth).

The node that runs the Account service is a different concept from the individual account. The Account server is part of the storage system and must be configured with the Container server and the Object server.

B) Authentication and Access Permissions

You must authenticate through the authentication service to receive OpenStack Object Storage connection parameters and authentication tokens. Tokens must be passed for all subsequent container/object operations. Typically, language-specific API handles authentication, token passing, and HTTPS request/response communication.

By using X-Container-Read: accountname and X-Container-Write: accountname:username, you can perform access control on objects for users or accounts. For example, this setting allows any user from the accountname account to read, but only the user username in the accountname account to write. You can also grant public access to objects stored in OpenStack Object Storage, and you can restrict public access by blocking site-based content theft such as hot links through Referer headers. The public container setting is used as the default authorization on the access control list. For example, the setting of X-Container-Read: referer: any allows anyone to read from container, regardless of other authorization settings.

In general, each user has full access to his or her storage account. Users must authenticate with their certificates, and once authenticated, they can create or delete objects such as container and accounts. The only way a user can access the contents of another account is that they have an API access key or a session token provided by your authentication system.

C) Containers and Objects

A Container is a storage cubicle that provides you with a way to organize data that belongs to you. It is more similar to a folder or directory. The main difference between Container and other file system concepts is that containers cannot be nested. However, you can create countless containers in your account. But you must have a container on your account, because the data must be stored in Container.

The limitation on Container naming is that they cannot contain "/" and are less than 256bytes in length. The length limit also applies to URL-encoded names. For example, the Container name of Course Docs is "Course%20Docs" after URL encoding, so the length is 13 bytes instead of 11 bytes.

An object is the basic storage entity and any optional metadata that represents the files you store in the OpenStack Object Storage system. When you upload data to OpenStack Object Storage, it is stored as is, any metadata composed of a location (container), object name, and key/value pairs. For example, you can choose to store copies of your digital photos and organize them into an album. In this case, each object can use metadata Album:

Caribbean Cruise or Album: Aspen Ski Trip to mark.

The only limitation on object names is that they are less than 1024 bytes long after being encoded by URL.

The maximum allowed size of the uploaded storage object is 5GB, with a minimum of 0 bytes. You can use embedded large object support and St tools to retrieve objects above 5GB. For metadata, each object should not exceed 90 key/value pairs, and the total byte length of all key/value pairs should not exceed 4KB.

D) Operations

Operations is the behavior you perform on the OpenStack Object Storage system, such as creating or deleting containers, uploading or downloading objects, and so on. A complete list of Operations can be found in the development documentation. Operations can be executed through ReST web service API or language-specific API. It is worth emphasizing that all operations must include a valid authorization token from your authorization system.

E) language-specific API bindings

API bindings supported by some popular languages are available in RackSpace cloud file products. These bindings provide a layer of abstraction on top of the underlying ReST API, allowing people to deal directly with container and object models rather than HTTP requests and responses. These bindings can be downloaded, used and modified free of charge. They follow the MIT license agreement. For OpenStack Object Storage, the currently supported API bindings are PHP,Python,Java,C#/.NET and Ruby.

5.2 how Object Storage works

A) Ring

Ring represents the mapping of the names of entities stored on disk and their physical locations. Accounts, containers, and and objects all have separate Ring. For other components to do anything in one of these three, they all need to interact with the corresponding Ring to determine its location in the cluster.

Ring uses zones,devices,partitions, and replicas to maintain mappings, and each partition in Ring has three replicas by default in the cluster. The location of the partition is stored in the mapping maintained by Ring. Ring is also responsible for determining which device to replace in a failure scenario. This is similar to the replication of HDFS replicas. Copies of partitions should be stored in different zone. The partitions of Ring are distributed across all OpenStack Object Storage installation devices. When a partition needs to be moved, Ring ensures that the partition is moved the least at a time, and that only one copy of the partition is moved at a time.

Weights can be used to balance the distribution of partitions on disk drives. Ring is used in proxy servers and some background processes.

B) Proxy Server

The proxy server is responsible for bringing together the rest of the OpenStack Object Storage architecture. For each request, it queries the location of account, container, and or object in Ring and forwards the request. The public APIs is also exposed through a proxy server.

A large number of failures are also handled by proxy servers. For example, if a server is unavailable, it will ask Ring to find the next replacement server for it and forward the request there.

When objects flow into or out of the object server, they are all flowed to the user through a proxy server, or obtained from the user through it. The proxy server does not buffer them.

The functions of Proxy server can be summarized as: querying location, processing failure, and transferring objects.

C) Object Server

Object Server, a very simple blob storage server, can store, retrieve and delete objects on the local disk. It is stored in the file system as a binary file, and the metadata is stored as an extended attribute of the file.

The object is stored in a path derived from the hash of the object name and the timestamp of the operation. The last time you write is always successful, make sure that the latest version will be used. Deletion is also treated as a version of the file: this ensures that the deleted file is also copied correctly, and that the older file does not disappear strangely due to failure.

D) Container Server

Its main job is to deal with the list of objects, it doesn't know where the objects are, just which objects are in a particular container. The list is stored as a sqlite database file and replicated in a cluster in an object-like manner. Tracking statistics are also carried out, including the total number of objects and the total amount of storage used in the container.

E) Account Server

It is similar to Container Server, except that it is a list rather than an object responsible for containers.

F) Replication

The purpose of the design copy is to keep the system in a consistent state in the face of temporary error conditions such as network outages or driver failures.

The copy process compares local data with each remote copy to ensure that they all contain the latest version. Object replicas use a Hash list to quickly compare fragments of each partition, while containe and account replication use a combination of Hash and shared high watermarks.

Updates to the copy are based on push. For object copies, the update is to synchronize files to Peer remotely. Account and container replication push the entire database file to the lost record through HTTP or rsync.

The copy also ensures that the data is cleared from the system by setting the latest version of the tombstone.

G) updater (Updaters)

Sometimes, container or account data cannot be updated immediately, usually in a failure situation or during a period of high load. If an update fails, the update is queued locally on the file system, and the updater handles these failed updates. The event consistency window (eventual consistency window) is most likely to work. For example, suppose a container server is loading, a new object is being put into the system, and the object is readable as soon as the proxy server responds to the client's success. However, the container server does not update the Object list, so the update is queued to wait for later updates. Container list, so this object may not be included immediately.

In fact, the consistency window runs just as often as updater, and when the proxy server forwards the list request to the first container server in the response, it may not even be noticed. The server under load may not be the one requested by the subsequent list of services. One of the other two copies may handle this list.

H) Auditors

Auditors checks the integrity of objects, containers, and accounts. If a corrupted file is sent first, it will be quarantined and a good copy will replace the bad file. If other errors are found, they will be recorded in the log.

5.3 OpenStack Object Storage physical Architectur

Proxy Services tends to be CPU-and networkI/O-intensive, while Object Services, Container Services and Account Services tend to be disk and networkI/O-intensive.

All services can be installed on each server, and within Rackspace, they put Proxy Services on their own server, while all storage services are on the same server. This allows us to send 10G of network to the agent and 1G to the storage server, thus keeping the load balance of the proxy server better managed. We can expand the overall API throughput by adding more agents. If you need greater throughput for Account or Container Services, they can also be deployed to their own servers.

When deploying OpenStack Object Storage, you can install it on a single node, but it is only available for development and testing purposes. It can also be installed with multiple servers, which can achieve the high availability and redundancy required by distributed object storage systems.

There is such a sample deployment architecture, as shown in figure 5-1. One Proxy node, running agent service, one Auth node, running authentication service, five Storage nodes, running Account,Container and Object services.

5-1 OpenStack Object Storage physical architecture of five Storage nodes

The above is all the contents of this article "what does OpenStack mean?" Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.