Shulou(Shulou.com)06/02 Report--

August 10, 2017 JAY CHRISTOPHERSON

The following is the blog of Jay Christopherson, chief engineer of Spaceflight Industries. Spaceflight is revolutionizing the space shuttle business by providing a new mode of access to space. As a comprehensive launch service and mission management provider, the company offers a simple and cost-effective suite of products and services, including state-of-the-art satellite infrastructure, ride launch products and global communications networks that enable businesses or governments to achieve their mission goals on time and on budget. As a service provider for the Seattle aerospace industry, Spaceflight provides services through a network of global partners, ground stations and launch vehicle suppliers

As we determine how to design the computing infrastructure to support business applications, we face two major challenges: how we should handle distributed runtime changes and service discovery. We need to make distributed changes when deploying remote satellite communication ground stations (spokes) around the world, but they are all managed through a central location (hub). Changes made from the central location need to be automatically distributed to one or more remote ground stations. As for service discovery, we often build and deploy, and we need to ensure that service changes in our infrastructure are detected and updated as soon as possible without any manual updates. That's why we focus on HashiCorp Consul.

All of our applications are registered with Consul at deployment time, so we make heavy use of Consul-based health check, tagging, external service registration and load balancing (for some tools, you can register an "active" component). In addition, DNS allows us to dynamically update our applications and services and discover other services we need in a timely manner. For run-time configurations, we have invested in tools such as Consul templates to build dynamic configuration files that can be updated based on triggers, such as value changes in Consul. Finally, changes to the services we deploy are usually simple, just like making changes to the parameters saved in the source repository file. The build is triggered when the change occurs, and the update is pushed to Consul. Now, changes to the services we deploy can happen naturally.

Our system construction based on Consul and other HashiCorp products

The core of any deployment in Spaceflight Industries consists of four parts: HashiCorp Consul, HashiCorp Vault, HashiCorp Nomad, and HashiCorp Terraform. Focus on Consul, which is an essential part of everything we do, including key / value management, service registration, dynamic DNS, and external services. Any new service we design revolves around how to interact and configure with Consul. It also serves as a backup store for our Vault deployment. For any production service, we deploy Consul in the cluster.

We designed our setup in a hub-and-spoke architecture. We have a central hub where all changes from all ground stations are synchronized to remote sites. We only choose to keep some key local content in the central hub cluster. Previously, we made each remote ground station a read-only copy of the hub cluster. However, we found that the ability to mark some items as local (that is, out of sync) is available in the latest version of Consul, which makes it easier for us to get some content, and we prefer this more hybrid synchronization approach.

Our use of Consul provides us with important experience throughout the process. We used to use scripts to populate the Kramp V storage, but found that this was not actually long-term sustainable. What we really want is for developers to check in a change in source control and be able to "automatically" generate / test / deploy to track all changes without having an in-depth understanding of Consul or our DevOps infrastructure. Using Terraform resources to populate Consul with Kramp V or external services means that we can easily achieve this goal. Make changes, commit, trigger build / test, trigger Terraform plan / deployment in the source code, all in the background in a very small amount of time, and no interaction is required after the initial commit.

Our best practice advice here, which is probably the most obvious, is to build your system so that changes can be easily tracked from a single source. In addition, set up your system so that changes can only be made from this single source. Implement break-glass procedure, which allows someone to escalate their privileges so that they can make changes directly in an emergency.

Improve developer productivity and reduce management overhead and bottlenecks

Our implementation plan has several main benefits:

We need very little administrative overhead to manage the global infrastructure. No one (or group) needs to make changes to the system-there is no case of "if he is hit by a bus". Any authenticated individual developer with access to the source repository can make changes and can trace from commit to deployment through the system. Building our architecture around Hashicorp services (especially Consul) means that our developers have a common configuration language in different functional parts of the infrastructure, the ability to get help (whether community-based or contract-based if they need to), and the ability to easily narrow the production environment to a single development environment with some parameter changes.

All of this reduces the time it takes us to build the system and roll out changes. Then we can spend more time building our products instead of delivering the infrastructure.

HashiCorp Consul is a product that can connect to any application on any infrastructure. Consul is a distributed, highly available, and data center-aware solution that connects and configures applications on a dynamic distributed infrastructure. Consul is used for service discovery, runtime configuration and orchestration, and simplified advanced network topology management. To learn more about HashiCorp Consul, visit https://www.hashicorp.com/products/consul/.

[original] Spaceflight uses HashiCorp Consul for Service Discovery and Runtime Configuration in their Hub-and-Spoke Network Architecture

AUG 10 2017 JAY CHRISTOPHERSON

This is a guest post by Jay Christopherson, principal engineer, DevOps, at Spaceflight Industries. Spaceflight is revolutionizing the business of space flight by delivering a new model for accessing space. A comprehensive launch service and mission management provider, the company provides a straightforward and cost-effective suite of products and services including state-of-the-art satellite infrastructure, rideshare launch offerings, and global communications networks that enable commercial and government entities to achieve their mission goals, on time and on budget. A service offering of Spaceflight Industries in Seattle Washington, Spaceflight provides its services through a global network of partners, ground stations, and launch vehicle providers.

We had two main challenges facing us as we determined how to design the computing infrastructure to support our business applications: how we should handle distributed runtime changes and service discovery. We need distributed changes as we deploy remote satellite communications ground stations (spokes) around the world, but which are all managed from a central location (hub). Changes made from a central location need to be distributed out to one or more remote ground stations in an automated fashion. As for service discovery, we build and deploy quite often and we needed to make sure that changes to services in our infrastructure are detected and updated as quickly as possible without any manual updates. These are the reasons we looked at HashiCorp Consul.

Our applications all register on deployment with Consul and from there, we make heavy use of Consul-based health checks, tags, external services registration, and load balancing (for certain tools that can register an "active" component). Also, DNS allows all of our various applications and services to dynamically update and discover other required services. For runtime configuration we have invested in tools like consul-template to build dynamic configuration files that can update based on triggers, such as a value change in Consul. In the end, a change to our deployed services often becomes as simple as making a change to a parameter held in a file in our source repository; builds are triggered on change, the updates are pushed to Consul. Now, changes to our deployed services "just happen."

Our Architecture with Consul and other HashiCorp products

The core of any deployment at Spaceflight Industries is composed of four pieces: HashiCorp Consul, HashiCorp Vault, HashiCorp Nomad, and HashiCorp Terraform. Focusing on Consul, it's the base component of everything we do which includes key/value management, service registration, dynamic DNS, and external services. Any new service we design is designed around how it is going to interact and be configured with Consul. It also serves as the backing store for our Vault deployment. For any production level services, we deploy Consul in a cluster.

We've designed our setup in a hub-and-spoke architecture. We have a central hub where any changes that are common across all ground stations are synchronized out to remote sites. There are a few key things that we choose to keep locally in the hub cluster only. Previously, we made every remote ground station a read-only copy of the hub cluster. However, we have found that the ability to mark some items as local-only (i.e., not sync'd out), available in the more recent versions of Consul, has made certain things easier for us and we prefer a more hybrid approach to synchronization now.

Our implementation of Consul provided us with key learnings throughout the process. We used to populate the K/V store using scripts but found that this really wasn't sustainable long term. What we really wanted, was to have a developer check in a change to source control, and have that "automagically" build/test/deploy so that all changes are tracked, without requiring special knowledge of Consul or our DevOps infrastructure. Using a Terraform resource to populate Consul with K/V or external services means that we can easily achieve that goal. Make a change in source, commit, trigger a build/test, trigger a Terraform plan/deploy, and it all happens behind the scenes in very little time with zero interaction beyond the initial commit.

Our best practice recommendation here (which is probably obvious to most), is to build your system so that changes can be easily tracked from a single source. Also, set up your system so that changes can only be made from that single source. Implement a break-glass procedure which would allow someone to escalate their privileges such that they can make changes directly in an emergency.

Less management overhead and bottlenecks with increased developer productivity

There have been several key benefits from our implementation:

We require very little administrative overhead to manage a global infrastructure.There's not one single person (or group) that is required to make changes to the system-no "what if he got hit by a bus" scenario. Any single developer, with proper authenticated access to our source repo, can make a change that they can track through the system from commit to deploy.Building our architecture around Hashicorp services (Consul in particular) means that our developers have a common configuration language across different functional pieces of our infrastructure, the ability to receive help if they need it (both community- and contract-based), and the ability to easily scale down a production environment to a single development environment with a few parameter changes.

All of this reduces our time to build systems and roll out changes. We then get to spend more time on building our products rather than the process of delivering our infrastructure.

HashiCorp Consul is a product to connect any application across any infrastructure. Consul is a distributed, highly available, and data center-aware solution to connect and configure applications across dynamic, distributed infrastructure. Consul is used for service discovery, runtime configuration and orchestration, and simplified management of advanced network topologies. To learn more about HashiCorp Consul, visit https://www.hashicorp.com/products/consul/.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.