Shulou(Shulou.com)06/01 Report--

In this issue, the editor will bring you about how to take ScaleIO as an example to analyze the deployment of SERVER SAN. The article is rich in content and analyzed and described from a professional point of view. I hope you can get something after reading this article.

Preface

SERVER SAN that supports iSCSI includes SolidFire, an all-flash array vendor (which also supports software-only deployment), and Zadara Storage, which we shared earlier. In fact, Scale-out SAN with iSCSI generally cannot have too many nodes, because it is very difficult for iSCSI to support distribution, such as 8 for NetApp, 16 for high-end storage of Huawei / EMC, 24 for high-end Fujitsu, and 100 for SolidFire. But EMC ScaleIO can be expanded to 1024, and Huawei FusionStorage can support 4K. That is, from the perspective of industrial implementation, SERVER SAN that supports more than 1000 nodes basically needs to rely on dedicated clients to provide block services, rather than relying on the standard iSCSI protocol.

Network topology

In the deployment of the network, the topology should be considered first. There are two kinds of SERVER SAN network topologies, one is flat network (Flat), the other is leaf thorn network (Leaf-Spine), that is, there are backbones and branches.

Generally, small-scale deployments use flat networks, while large-scale deployments use Leaf-Spine networks. In addition to the node size, future expansion needs to be considered when deploying. If the super-convergence deployment also needs to consider the application bandwidth and security, for example, if the client comes from an untrusted domain, you need to consider isolating SDC and SDS.

Leaf-Spine is a two-tier architecture and an alternative implementation of the traditional three-tier network. Each Leaf switch needs to be connected to all Spine switches, but there is no interconnection between Leaf and Spine switches. Leaf switches control traffic between servers, while Spine switches move traffic between L2 nodes.

In most cases, EMC recommends a Leaf-Spine topology because:

ScaleIO can scale to hundreds of nodes

Leaf-Spine is oriented to the future, and there is no need to readjust the network for future expansion.

Non-blocking switching can be realized with correct planning.

The connection used has the same bandwidth

Predictable delay

Higher availability and performance.

But flat networks are not unusable, especially in some branches and small and medium-sized enterprises. A maximum of 4 switches are recommended for a flat network, and the topology recommended by EMC is shown in the following figure, which does not have a full mesh connection, which saves some ports. If there are more than 4 switches, it is better to use the Leaf-Spine network.

Network performance

Choose a good network topology, performance planning is the most important.

EMC recommends that the default settings of the switch be used in the initial configuration of the network, without enabling advanced features such as jumbo frames, flow control, and link aggregation. Only after confirming that the devices on the network support these features, and then slowly enable them, otherwise debugging will be repeated and inefficient.

As we all know, the metadata service MDM is the control center of the whole system. EMC suggests that the communication delay from SDS to MDM (which can be tested by ping) is less than 1ms.

What needs to be calculated is mainly the bandwidth of the network. There are many factors to consider for the bandwidth of the network:

Refactoring time. Time for a node to fail and reconstruct

Time to rebalance. Time to add and delete nodes and rebalance data distribution

Capability and performance of hard disk

Performance expectation of the application, bandwidth, delay, IOPS and so on.

No matter how you calculate, the bandwidth of 1Gbps is obviously not enough, so a 10-gigabit network is necessary. So, how to calculate how many 10 Gigabit network cards are needed for each node?

If you think of a node as a storage system, the total bandwidth of the hard disk is the bandwidth of the back end, and the total bandwidth of the 10 Gigabit network card is the total bandwidth of the front end. From a system perspective, if the total front-end bandwidth is equal to the back-end total bandwidth, then there is no bottleneck in the storage system.

For better understanding, two examples of estimates are provided.

Example 1:

A node has 10 HDD, and the continuous bandwidth of each HDD is 100MB/s. So the total backend bandwidth is 1000MB/s. It is about equivalent to the performance of 1 10 Gigabit network card, and redundancy is considered, so it is more appropriate to configure 2 10 Gigabit network cards.

Example 2:

If this node has six SSD, the persistent bandwidth of each SSD is 450MB/s (I understand that it should be calculated according to the maximum, that is, read bandwidth). Then the total bandwidth of the back-end system is 2700MB/s, and it is more appropriate to configure 3 10 Gigabit NICs (I understand that 3 wire-speed 10 Gigabit ports, physically two 2-port and three 1-port are all right).

In other words, for reliability, configure at least 2 physical network cards (different slots to improve reliability). However, the exact number of 10 Gigabit ports to be used still needs to be estimated. In general, the number of hard disk slots on a server node is 12. If you use HDD,2, 10 gigabytes is basically enough, but if you use SSD, you may need more than 4. But some high-density network cards, all ports may not be full, this requires more physical network cards.

It is important to note that the above only stores the bandwidth requirements of IWeibo O itself, and if you want to use super-converged deployment, you also need to consider the bandwidth of the application.

EMC best practices also recommend isolating front-end (SDC-SDS) and back-end (SDS-SDS) traffic, which may require more network cards or ports. For example, 4 network cards, 2 run vertical flow, 2 run horizontal flow. ScaleIO supports a maximum of 8 networks per node.

For better performance, EMC also recommends that the switch access port should be configured by default and that VLAN tagging should not be enabled for ScaleIO traffic (this should mean that all messages received by NIC are without VLAN ID).

As for the use of jumbo frames (Jumbo Frames), EMC recommends enabling it after the network deployment is complete. If the network supports it, and the typical write size is more than 1500 bytes, enabling jumbo frames can still improve performance. (generally, the read and write of a file system is 4K and 8K, so this condition should be met in most cases).

As for flow control, it is generally recommended that the performance tuning phase be enabled. Flow control is divided into global flow control and priority flow control (Priority Flow Control,DCB/PFC, only for specific traffic, requires switch and NIC support). In some cases, flow control may not be used, for example, global flow control is not suitable for super-convergence deployment (because there is not only ScaleIO traffic), and flow control may not be required if the storage part has a dedicated physical network.

As for link aggregation, if the network supports, it is recommended to enable, performance and reliability will be better.

If the switch supports teaming/bonding, try to use dynamic link detection (do not use static LACP configuration) and configure shorter timers for faster failover.

The above is how to use ScaleIO as an example to analyze the deployment of SERVER SAN. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.