Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to use Hibernate to update or insert database tables in Java in batches. This article is very detailed and has certain reference value. Interested friends must finish reading it.

Hibernate is the most popular ORM framework for interacting with databases in Java. In this article, we will explore the various methods of using batch selection and updates, as well as the most effective ways to use the Hibernate framework in Java.

I have tried three ways, which are as follows:

Use the Query.list () method of Hibernate.

Use ScrollableResults in FORWARD_ONLY scrolling mode.

Use ScrollableResults with FORWARD_ONLY scrolling mode in StatelessSession.

To determine which provides the best performance for our use cases, I did the following tests using the three methods listed above.

Select and update 1000 rows.

Let's apply each of the above three ways to the above operations and take a look at the code and the results.

Use the Query.list () method of Hibernate

Code executed:

Java:

List rows; Session session = getSession (); Transaction transaction = session.beginTransaction (); try {Query query = session.createQuery ("FROM PersonEntity WHERE id >: maxId ORDER BY id"). SetParameter ("maxId", MAX_ID_VALUE); query.setMaxResults (1000); rows = query.list (); int count = 0 For (Object row: rows) {PersonEntity personEntity = (PersonEntity) row; personEntity.setName (randomAlphaNumeric (30)); session.saveOrUpdate (personEntity); / / Always flush and clear the session after updating 50 (jdbc_batch_size specified in hibernate.properties) rows if (+ + count% 50 = = 0) {session.flush () Session.clear ();} finally {if (session! = null & & session.isOpen ()) {transaction.commit (); session.close ();}}

Test results:

Time: 360s to 400s

Heap mode:-gradually increased from 13m to 51m (from jconsole).

Use ScrollableResults in FORWARD_ONLY scrolling mode.

With this, we expect it to consume less memory than the first method. Let's look at the following results.

Code executed:

Java:

Session session = getSession (); Transaction transaction = session.beginTransaction (); ScrollableResults scrollableResults = session .createQuery ("FROM PersonEntity WHERE id >" + MAX_ID_VALUE + "ORDER BY id") .setMaxResults (1000) .requests (ScrollMode.FORWARD_ONLY); int count = 0 Try {while (scrollableResults.next ()) {PersonEntity personEntity = (PersonEntity) scrollableResults.get (0); personEntity.setName (randomAlphaNumeric (30)); session.saveOrUpdate (personEntity); if (+ + count% 50 = = 0) {session.flush (); session.clear () } finally {if (session! = null & & session.isOpen ()) {transaction.commit (); session.close ();}} Test result:

Time spent: 185 to 200 seconds.

Heap mode:-gradually increases from 13MB to 41MB (same as measured using jconsole).

Use ScrollableResults and FORWARD_ONLY scrolling modes in StatelessSession

Stateless sessions do not implement first-level caching, do not interact with any secondary caches, do not implement transactional post-write or automatic dirty checking, and do not cascade to associated instances. Stateless sessions ignore collections. Operations performed through stateless sessions bypass Hibernate's event model and interceptor.

This type of session is always recommended in the case of batch updates, because we really don't need the overhead of these dormant features in these types of use cases.

Code executed:

Java:

StatelessSession session = getStatelessSession (); Transaction transaction = session.beginTransaction (); ScrollableResults scrollableResults = session .createQuery ("FROM PersonEntity WHERE id >" + MAX_ID_VALUE + "ORDER BY id") .setMaxResults (TRANSACTION_BATCH_SIZE) .operations (ScrollMode.FORWARD_ONLY); try {while (scrollableResults.next ()) {PersonEntity personEntity = (PersonEntity) scrollableResults.get (0) PersonEntity.setName (randomAlphaNumeric (20)); session.update (personEntity);}} finally {if (session! = null & & session.isOpen ()) {transaction.commit (); session.close ();}}

Test results:

Time spent: 185 to 200 seconds.

Heap mode:-gradually increased from 13MB to 39MB.

I also did the same test on line 2000, and the results are as follows:

Results:

Using list ():-time spent: about 750s, heap mode: gradually increased from 13MB to 74 MB.

Using ScrollableResultSet: time spent: about 380s, heap mode: gradually increasing from 13MB to 46MB

Using Stateless: time spent: about 380 seconds, heap mode: gradually increasing from 13MB to 43MB

The interceptor problem of all the above methods

ScrollableResults and Stateless ScrollableResults provide almost the same performance, which is much better than Query.list (). But there is still a problem with all the above methods. Locked, all of the above methods select and update data in the same transaction, which means that as long as the transaction is running, rows that have performed updates will be locked, and any other operation must wait for the transaction to complete.

A way.

In order to solve the above problems, we should do two things here:

We need to select and update data in different transactions.

Moreover, these types of updates should be carried out in batches

So I performed the same test as above again, but this update was performed in a different transaction that commits in groups of 50.

Note: in the case of Scrollable and Stateless, we also need a different session because we need the original session and transaction to scroll the results.

Results of using batch processing

Using list (): time spent: about 400s, heap mode: gradually increasing from 13MB to 61MB.

Using ScrollableResultSet: time spent: about 380 seconds, heap mode: gradually increased from 13MB to 51MB.

Use stateless: time spent: about 190 seconds, heap mode: gradually increased from 13MB to 44MB.

Observation: the performance of ScrollableResults drops to almost equal to Query.list () at this time, but the performance of Stateless remains almost the same. Summary and conclusion

From all the above experiments, when we need to do batch selection and update, in terms of memory consumption and time, the best method is as follows:

Use ScrollableResults in stateless sessions.

Perform selections and updates in 20 to 50 batches (batches) in different transactions (note: batch size can vary on a case-by-case basis).

Sample code for best practices

Java:

StatelessSession session = getStatelessSession (); Transaction transaction = session.beginTransaction (); ScrollableResults scrollableResults = session .createQuery ("FROM PersonEntity WHERE id >" + MAX_ID_VALUE + "ORDER BY id") .setMaxResults (TRANSACTION_BATCH_SIZE) .operations (ScrollMode.FORWARD_ONLY); int count = 0; try {StatelessSession updateSession = getStatelessSession (); Transaction updateTransaction = updateSession.beginTransaction () While (scrollableResults.next ()) {PersonEntity personEntity = (PersonEntity) scrollableResults.get (0); personEntity.setName (randomAlphaNumeric (5)); updateSession.update (personEntity); if (+ + count% 50 = = 0) {updateTransaction.commit (); updateTransaction = updateSession.beginTransaction () }} updateSession.close ();} finally {if (session! = null & & session.isOpen ()) {transaction.commit (); session.close () }} the above is all the contents of the article "how to use Hibernate to update or insert database tables in batch in Java". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.