Shulou(Shulou.com)05/31 Report--

This article mainly explains "what are the new features of Apache Ignite1.7". The content of the explanation is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "what are the new features of Apache Ignite1.7"?

New feature of Apache Ignite 1.7-non-juxtaposed distributed association

Recently, Apache Ignite released version 1.7.0, and among the many changes, there is a killer feature that many Apache Ignite users and customers have been waiting for-SQL queries support non-collocated distributed associations. This article will focus on this feature, detailing how a non-juxtaposed distributed association works and how it differs from traditional (relationship-based juxtaposition) Apache Ignite associations.

Relationship based on relation juxtaposition

Previously, Apache Ignite supported SQL associative queries across different tables, but required that the associated cached data in a query be juxtaposed. In fact, in Ignite, juxtaposition can be easily enabled through relational keys, where the data of one business entity is stored on the same node as the data of another related business entity. For example, suppose you have two business entities, Organization and Person, and the Id of an Organization acts as a relational key to the Person from that Organization. Ignite then ensures that all Person data is stored on the same node as the Organization data to which they belong, a simple concept that executes a series of conceivable ANSI-99-compatible SQL queries, including associations across multiple caches. Basically, the execution process of a SQL query that uses an association is absolutely the same as that of a SQL query that is not associated. We can take a look at a very basic query execution process, which is defined using Organization and Person business entities as follows:

Organization (id, address) entity: this id will act as the ID of Organization, and its value will be used as a cache key when Organization is injected into the cache. The key used as a cache key will be regarded as a primary key in the SQL engine layer of Ignite, a concept that will continue throughout this article.

Person (name, salary) entity: located in the Persons cache, AffinityKey (id, orgId) is used as the cache key, where AffinityKey is a special object in Ignite, he defines a unique Id of a Person (the first parameter) and his relational key (the second parameter). Here, Organization ID (orgId) is selected as a Person relational key, which means that the Persons data will be on the same node as the Organizations data to which they belong.

After defining these business entities and preloading the cached data, you are free to execute a SQL query like the following, because Person is juxtaposed with their Organization, and Ignite ensures that a complete result set is returned.

SELECT * FROM Organization as org JOIN Person as p ON org.id = p.orgId

The execution process of this query is as follows:

Query initiator nodes (mapper and reducer) send queries to all cached data nodes

All nodes that receive a query from reducer execute the query locally, using only local data to perform the association

These nodes feed back part of the result set to reducer

Reducer finally summarizes the result sets received from all remote nodes and then sends a final aggregate result to the initiating node.

Non-parallel distributed association

If the same query is executed on a non-relational juxtaposition of data, you will get an incomplete and inconsistent result, because versions prior to 1.7.0 of Apache Ignite will only execute the query on the local data (as described in step 2 of the above process). However, after Ignite 1.7.0, this is no longer the case, and it will support non-collocated distributed associations that no longer require juxtaposition of data. Now, you will use Person's real Id as the cache key instead of AffinityKey (id, orgId), and then add the orgId field inside the Person object to perform the association of the two caches, and even if these changes, you will still get a complete result, regardless of whether the Person data is actually juxtaposed with their Organization data, because the latest version of Ignite performs the same SQL query (mentioned above) in the following process:

Query initiator nodes (mapper and reducer) send queries to all cached data nodes

All nodes that receive a query from reducer execute the query locally, but use local data to associate with the remote node's data (because the data is fully clustered)

These nodes feed back part of the result set to reducer

Reducer finally summarizes the result sets received from all remote nodes and then sends a final aggregate result to the initiating node.

An important thing to note here is that due to the particularity of the query, a node will send a broadcast to the cluster to request the data lost in step 2. However, there is also a way to optimize it, that is, the SQL engine will switch the broadcast to unicast for a specific association type and a typical query, and the following changes will switch to unicast mode:

SELECT * FROM Organization as org JOIN Person as p ON org._key = p.orgId

In this query, if the SQL engine decides to execute the query on the Persons cache plus Organizations, then the engine uses org._key (s) to send a unicast request to the node storing the Organizations cache, where _ key is a special keyword used in the Ignite SQL query, which points to the cache key / primary key of an object. Basically, because the engine knows its cache key / primary key, it can easily find the node where the entry is stored, and the same is true of relational keys for multiple caches.

Thank you for your reading, the above is the content of "what are the new features of Apache Ignite1.7". After the study of this article, I believe you have a deeper understanding of the new features of Apache Ignite1.7, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.