Shulou(Shulou.com)05/31 Report--

This article focuses on "how to deploy the cluster of apache Ignite". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to deploy the cluster of apache Ignite.

Cluster deployment of Ignite

Ignite has very advanced clustering capabilities. This paper makes a brief introduction to the technical points related to clustering, and then explains and compares the possible deployment forms of practical applications, from which we can find that Ignite platform has great advantages in deployment flexibility.

1. Related concepts 1.1. Node equality

Ignite has no master node or server node, nor worker node or client node, and all nodes are equal from Ignite's point of view. However, developers can configure nodes as master,worker or client and data nodes.

1.2. Discovery mechanism

Ignite nodes will automatically perceive each other, the cluster is scalable, there is no need to restart the cluster, simply start the new nodes and then they will automatically join the cluster. This is achieved through a discovery mechanism, which enables nodes to discover each other. By default, Ignite uses TcpDiscoverySpi through TCP/IP protocol as the implementation of node discovery, and can also be configured to be based on multicast or static IP. These methods are suitable for different scenarios.

1.3. Deployment mode

Ignite can run independently or in a cluster, or several jar packages can be embedded inside the application to run in embedded mode. It can also run in Docker containers and environments such as Mesos and Yarn. It can run in physical machines or virtual machines. This extensive adaptability is one of its great advantages.

1.4. Configuration mode

Most of Ignite's configuration options support both Spring-based XML configuration and Java code programming, which is also an important advantage.

1.5. Client and server

Each node in Ignite is equal, but the node can be configured as a client or server as needed, and the server node participates in caching, computing, streaming and so on, while the native client node provides the ability to connect to the server remotely. Ignite native clients can use the full Ignite API, including near caching, transactions, computing, streams, services, and so on. All Ignite nodes are started in server mode by default, and client mode needs to be explicitly enabled, as follows:

two。 Create a cluster

An Ignite node can be launched from the command line, using the default configuration or passing a configuration file. You can start a lot of nodes and they will automatically find each other. To start a grid node based on the default configuration, open the command line and switch to IGNITE_HOME (installation folder), and enter the following command:

$bin/ignite.sh

Then you will see the output roughly as follows:

1. [02:49:12] Ignite node started OK (id=ab5d18a6) 2. [02:49:12] Topology snapshot [ver=1, nodes=1, CPUs=8, heap=1.0GB]

In embedded mode, you can also start a node with the following code:

Ignite ignite = Ignition.start (); 3. Cluster group

By design, all cluster nodes are equal, so there is no need to start any nodes in a particular order or give them specific rules. However, Ignite can create logical groups of cluster nodes due to the special needs of some applications, for example, you may want to deploy only one service on a remote node, or give some worker nodes a rule called worker to do job execution. For example, the following example broadcasts jobs only to remote nodes (except local nodes):

Final Ignite ignite = Ignition.ignite (); IgniteCluster cluster = ignite.cluster (); IgniteCompute compute = ignite.compute (cluster.forRemotes ()); compute.broadcast (()-> System.out.println ("Node Id:" + ignite.cluster (). LocalNode (). Id ()

Ignite has many predefined cluster groups built in, and also supports customization of cluster groups. A dynamic cluster group can be defined based on some predicates, and this cluster group will only contain nodes that match that predicate. In the following example, a cluster group will only include nodes whose CPU utilization is less than 50%. Note that the nodes in this group will change with the CPU load:

IgniteCluster cluster = ignite.cluster (); ClusterGroup readyNodes = cluster.forPredicate ((node)-> node.metrics (). GetCurrentCpuLoad () < 0.5); 4. Cluster configuration

In Ignite, DiscoverySpi nodes can discover each other and can be configured based on multicast or static IP. Ignite provides TcpDiscoverySpi as the default implementation of DiscoverySpi, which uses TCP/IP as the implementation of node discovery.

When multicast is disabled, TcpDiscoveryVmIpFinder uses a preconfigured IP address list and only needs to provide the IP address of at least one remote node, but to ensure redundancy, it is better to provide IP addresses of 2-3 grid nodes. If a connection is established to any of the provided IP addresses, Ignite automatically discovers all other nodes.

You can also use both multicast-based and static IP-based discovery, in which case TcpDiscoveryMulticastIpFinder can use a preconfigured static IP address list in addition to accepting addresses through multicast. The following example shows how to build a cluster with a predefined list of IP addresses:

TcpDiscoverySpi spi = new TcpDiscoverySpi (); TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder (); / / set the predefined IP address, note that the port or port range is optional. IpFinder.setAddresses (Arrays.asList ("1.2.3.4", "1.2.3.5 spi.setIpFinder 47500. 47509"); spi.setIpFinder (ipFinder); IgniteConfiguration cfg = new IgniteConfiguration (); cfg.setDiscoverySpi (spi); / / start cluster Ignition.start (cfg); 5. Zero deployment

Code related to computing, etc., can be arbitrary custom classes, which are automatically perceived by remote nodes in Ignite, without the need to explicitly deploy or move any jar files to any remote nodes. This behavior is realized by peer-to-peer class loading (P2P class loading), which is a special distributed class loader in Ignite, which realizes the bytecode exchange between nodes. When peer class loading is enabled, you do not need to manually deploy code on each node in the cluster, nor do you need to redeploy every time a change occurs.

You can enable peer-to-peer class loading by

6. Cloud deployment

For many cloud environments, multicast is usually disabled and IP addresses are not fixed. In this case, Ignite provides a discovery extension mechanism to solve this problem, and built-in support for common cloud services (such as AWS). Developers can refer to the relevant documentation.

Deployment of other environments such as 7.Docker

Ignite also supports Docker, Mesos, Yarn and other environments. Developers can refer to the relevant documentation.

8. Deployment practice

The deployment mode of Ignite is very flexible, and different deployment methods can be adopted according to the actual needs in actual scenarios. Here is a simple summary and comparison:

8.1. Stand-alone Ignite cluster

In this case, the deployment of the cluster is completely independent of the application, and the cluster can be used for distributed computing, distributed caching, distributed services, and so on. In this case, the application connects to the cluster in client mode for related operations, generally speaking, the deployment mode is as follows:

Advantages

It has little impact on the existing application running environment, and the cluster can be shared to provide services for multiple applications, which adds a lot of additional computing and load capacity to the whole application.

Shortcoming

A separate set of machines is required, and the relative cost is higher. If the concurrency of cache operations is not high or the computation is not saturated, the resource utilization is low. The overall architecture has also become complex and maintenance costs are higher.

8.2. Embedded Ignite cluster

In this case, you can embed the necessary jar package inside the existing application, and use the discovery mechanism of Ignite to build the cluster automatically, which is generally the following deployment mode:

Advantages

No additional machines are needed, the cost is the lowest, Ignite can be seamlessly integrated with applications, and all nodes are server nodes, which can make full use of the rich features of Ignite. This model has the best scalability, and the computing and load capacity of the whole system can be quickly expanded by simply adding nodes.

Shortcoming

Ignite takes up some of the resources of the server and has an impact on the overall performance of the application. Targeted optimization may be needed. When the application is updated, the cluster may need to be restarted. If Ignite needs to load a large amount of data, the restart time may become longer or even unbearable.

8.3. Hybrid Ignite cluster

In this case, the above two modes are mixed together, that is, separate clusters are deployed by machines at the same time, while Ignite is embedded inside the application to run in server mode, and resources are allocated through logical cluster groups to form a larger cluster as a whole, which is generally the following deployment mode:

This mode is more flexible, the balance of cost, function and performance can be achieved after optimization, and the comprehensive effect is the best. At this time, the cached data can be deployed to the nodes outside the application through the cluster group, which can avoid frequent cold start-up leading to frequent long-time loading of cached data, and for computing, it can also make full use of the resources of all computing nodes dynamically.

At this point, I believe you have a deeper understanding of "how to deploy the cluster of apache Ignite". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.