Shulou(Shulou.com)06/03 Report--

This article mainly introduces "what is Spring Batch". In daily operation, I believe many people have doubts about what is Spring Batch. The editor consulted all kinds of materials and sorted out simple and easy-to-use methods of operation. I hope it will be helpful for you to answer the doubts about "what is Spring Batch?" Next, please follow the editor to study!

How the 1.Spring Batch framework works

Before delving into the code, let's take a look at the Spring Batch framework. It contains the following main building blocks:

Don't miss the Spring family bucket study notes.

Spring Batch framework

A Batch (batch) process consists of a Job (job). This entity encapsulates the entire batch process.

A Job can consist of one or more Step (steps). In most cases, one step will read the data (through ItemReader), process the data (using ItemProcessor), and then write the data (through ItemWriter).

The JobLauncher process starts a Job (job).

Finally, JobRepository stores metadata about the Job (jobs) configured and executed.

To demonstrate how Spring Batch works, let's build a simple Hello World batch job.

In this example, we read a person's first and last name from the person.csv file. Generate a greeting from this data. Then write this greeting to the greeting .txt file.

two。 Sample overview

We will use the following tools / frameworks:

Spring Batch 4.1

Spring Boot 2.1

Maven 3.6

Our project directory structure is as follows:

Project directory structure

3. Maven configuration

We use Maven to build and run the example. If not, download and install Apache Maven.

Let's use Spring Initializr to generate the Maven project. Be sure to select Batch as the dependency.

Click Generate Project to build and download the Spring Boot project template. In the root directory of the project, you will find a pom.xml file, which is the XML configuration file for the Maven project.

To avoid having to manage version compatibility of different Spring dependencies, we will start from the

The spring-boot-starter-parent parent POM inherits the default configuration.

The generated project contains Spring Boo Starters that manages different Spring dependencies.

Spring-boot-starter-batch imports Spring Boot and Spring Batch dependencies.

Spring-boot-starter-test contains dependencies for testing Spring bootstrap applications. It imports libraries including JUnit, Hamcrest, and Mockito.

This is also dependent on spring-batch-test. This library contains some helper classes that will help test batch jobs.

In the plugins section, you will find the Spring Boot Maven plug-in: spring-boot-maven- plugin. It helps us build a single, runnable uber-jar. This is a convenient way to execute and distribute code. In addition, the plug-in allows you to start the example through the Maven command.

4.0.0 com.codenotfound spring-batch-hello-world 0.0.1-SNAPSHOT jar spring-batch-hello-world Spring Batch Hello World Example https://codenotfound.com/spring-batch-example.html org.springframework.boot spring-boot-starter-parent 2.1.5.RELEASE 11 org.springframework.boot spring-boot-starter-batch org.springframework.boot spring-boot-starter-test test org.springframework.batch spring-batch-test test org.springframework.boot spring-boot-maven-plugin 4. Spring Boot configuration

We use Spring Boot to make a Spring Batch application "run directly".

First create a SpringBatchApplication class. It contains the main () method, which starts the application using Spring Boot's SpringApplication.run ().

Note that @ SpringBootApplication is a convenient annotation that adds: @ Configuration, @ EnableAutoConfiguration, and @ ComponentScan.

For more information about Spring Boot, see the Spring Boot getting started Guide.

By default, Spring Batch uses a database to store metadata on configured batch jobs.

In this case, instead of using the database directly, we use Map based on memory mapping and run Spring Batch.

Spring-boot-starter-batch starter depends on spring-boot-starter-jdbc and will attempt to instantiate the data source. Add exclude = {

Add @ SpringBootApplication to the DataSourceAutoConfiguration.class} annotation. This prevents Spring Boot from automatically configuring DataSource for database connections.

Package com.codenotfound;import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;@SpringBootApplication (exclude = {DataSourceAutoConfiguration.class}) public class SpringBatchApplication {public static void main (String [] args) {SpringApplication.run (SpringBatchApplication.class, args);}} 5. Create a solid model

Before working with data, you usually want to map it to an entity object.

In my example, the input data is stored in the

In src/test/resources/csv/persons.csv file.

Each line in the file contains a comma-separated last name and first name.

John, DoeJane, Doe

We will map this data to the Person object. This is a simple POJO with a first and last name.

Package com.codenotfound.model;public class Person {private String firstName; private String lastName; public Person () {/ / default constructor} public String getFirstName () {return firstName;} public void setFirstName (String firstName) {this.firstName = firstName;} public String getLastName () {return lastName;} public void setLastName (String lastName) {this.lastName = lastName;} @ Override public String toString () {return firstName + "" + lastName;}} 6. Configure Spring Batch Job

Let's start by creating a BatchConfig class that will configure Spring Batch. The @ Configuration annotation at the top of the class indicates that Spring can use this class as the source of the bean definition.

We have added the @ EnableBatchProcessing annotation, which supports all the required Spring Batch features. It also provides a basic configuration for setting up batch jobs.

Adding this annotation will require a lot of action. Here is an overview of the creation of @ EnableBatchProcessing:

JobRepository (bean name "jobRepository")

JobLauncher (bean name "jobLauncher")

JobRegistry (bean name "jobRegistry")

JobExplorer (bean name "jobExplorer")

PlatformTransactionManager (bean name "transactionManager")

JobBuilderFactory (bean name "jobBuilders"), which conveniently prevents you from having to inject job repositories into each Job (job)

StepBuilderFactory (bean name "stepBuilders") to help you avoid injecting job repositories and transaction managers into each Step (step)

In order for Spring Batch to use Map-based JobRepository, we need to extend DefaultBatchConfigurer. Override the setDataSource () method to not set DataSource. This will result in automatic configuration of using Map-based JobRepository.

@ Configuration@EnableBatchProcessingpublic class BatchConfig extends DefaultBatchConfigurer {@ Override public void setDataSource (DataSource dataSource) {/ / initialize will use a Map based JobRepository (instead of database)}}

Now let's continue to configure the Hello World Spring Batch job.

Create a HelloWorldJobConfig configuration class and add the @ Configuration annotation.

In HelloWorldJobConfig Bean, we use JobBuilderFactory to create jobs. We pass the name of the Job (job) and the Step (step) that needs to be run.

Note that in helloWorlJob () Bean, Spring will automatically inject jobBuilders and stepBuilders Beans.

Different items for the execution of our steps are defined in HelloWorldStepBean. We use StepBuilderFactory to create the steps.

First, we pass in the name of the step. Using chunk (), we specify the number of items processed in each transaction. Chunk also specifies the input (Person) and output (String) types of the step. Then, we add ItemReader (reader), ItemProcessor (processor), and ItemWriter (writer) to the step.

We use FlatFileItemReader to read the person CSV file. This class provides the basic functions of reading and parsing CSV files.

There is a FlatFileItemReaderBuilder implementation that allows us to create a FlatFileItemReader. We first specify that the result of reading each line in the file is the Person object. Then we will use the name () method to add a name to the FlatFileItemReader and specify the resources that need to be read (in this case, the persons.csv file).

In order for FlatFileItemReader to process our files, we need to specify some additional information. First of all, the data in our definition file is delimited (comma is the delimiter by default).

We also specified how to map each field in a row to a Person object. This is done using names (), and you can make the Spring Batch map the fields by matching the name to the setter on the object.

In the example in this article, the first field of a line will be mapped using firstName setter. To achieve this, we also need to specify targetType, the Person object.

Note:

Names (new String [] {"firstName", "lastName"})

PersonItemProcessor processes data. It converts a Person into a greeting String. We will define it in a separate class below.

Once the data is processed, we will write it to a text file. We use FlatFileItemWriter to accomplish this task.

We use the FlatFileItemWriterBuilder implementation to create a FlatFileItemWriter. We add a name to the writer and specify the resource to which the data needs to be written (in this case, the greeting.txt file).

FlatFileItemWriter needs to know how to convert the generated output into a single string that can be written to a file. In this case, our output is already a string, and we can use PassThroughLineAggregator. This is the most basic implementation, which assumes that the object is already a string.

@ Configurationpublic class HelloWorldJobConfig {@ Bean public Job helloWorlJob (JobBuilderFactory jobBuilders, StepBuilderFactory stepBuilders) {return jobBuilders.get ("helloWorldJob") .start (helloWorldStep (stepBuilders)) .build ();} @ Bean public Step helloWorldStep (StepBuilderFactory stepBuilders) {return stepBuilders.get ("helloWorldStep") .chunk (10) .reader (reader ()) .processor (processor ()) .writer (writer ()) .build () } @ Bean public FlatFileItemReader reader () {return new FlatFileItemReaderBuilder () .name ("personItemReader") .resource (new ClassPathResource ("csv/persons.csv")) .delimited () .names (new String [] {"firstName", "lastName"}) .targetType (Person.class). Build ();} @ Bean public PersonItemProcessor processor () {return new PersonItemProcessor () } @ Bean public FlatFileItemWriter writer () {return new FlatFileItemWriterBuilder () .name ("greetingItemWriter") .resource (new FileSystemResource ("target/test-outputs/greetings.txt")) .lineAggregator (new PassThroughLineAggregator ()) .build ();}} 7. Processing data

In most cases, you will want to apply some data processing during a batch job. You can use ItemProcessor to operate.

In our example, we transform the Person object into a simple greeting, String

To do this, we create a PersonItemProcessor that implements the ItemProcessor interface. We implemented the process () method, which adds the first and last names to the string.

During debugging, we record the log results.

Public class PersonItemProcessor implements ItemProcessor {private static final Logger LOGGER = LoggerFactory.getLogger (PersonItemProcessor.class); @ Override public String process (Person person) throws Exception {String greeting = "Hello" + person.getFirstName () + "+ person.getLastName () +"! "; LOGGER.info (" converting'{} 'into' {}'", person, greeting); return greeting;}} 8. Test the Spring Batch sample

To test this example, we created a basic unit test case. It runs the batch job and checks to see if it completes successfully.

We use @ RunWith and @ SpringBootTest test annotations to tell JUnit to run using Spring's test support and to boot using SpringBoot's support.

Spring Batch comes with a JobLauncherTestUtils utility class for testing batch jobs.

Let's first create an internal BatchTestConfig class to add the helloWorld job to the JobLauncherTestUtils bean. Then run the batch job using the launchJob () method of this bean.

If there are no errors in the executed job, the value of ExitCode is COMPLETED.

@ RunWith (SpringRunner.class) @ SpringBootTest (classes = {SpringBatchApplicationTests.BatchTestConfig.class}) public class SpringBatchApplicationTests {@ Autowired private JobLauncherTestUtils jobLauncherTestUtils; @ Test public void testHelloWorldJob () throws Exception {JobExecution jobExecution = jobLauncherTestUtils.launchJob (); assertThat (jobExecution.getExitStatus (). GetExitCode ()) .isEqualTo ("COMPLETED");} @ Configuration @ Import ({BatchConfig.class, HelloWorldJobConfig.class}) static class BatchTestConfig {@ Autowired private Job helloWorlJob; @ Bean JobLauncherTestUtils jobLauncherTestUtils () throws NoSuchJobException {JobLauncherTestUtils jobLauncherTestUtils = new JobLauncherTestUtils (); jobLauncherTestUtils.setJob (helloWorlJob); return jobLauncherTestUtils }}}

To trigger the above test case, open a command prompt in the root folder of the project and execute the following Maven command:

Mvn test

The result is that the build is successful and the batch job is executed during this period.

. _ _ _ (() _ _ _ (() _ _ _ | |. _ | _ | | _ | _ | | _ | | / = | _ | = | _ _ / = / _ /:: Spring Boot:: (v2.1.5.RELEASE) 2019-05-30 19main 1112.784 INFO 14588-[main] c.c.SpringBatchApplicationTests: Starting SpringBatchApplicationTests on DESKTOP-2RB3C1U with PID 14588 (started by Codenotfound in C:\ Users\ Codenotfound\ repos\ spring-batch\ spring-batch-hello-world) 2019-05-30 19Swiss 1112.785 INFO 14588-[ Main] c.c.SpringBatchApplicationTests: No active profile set Falling back to default profiles: default2019-05-30 19 WARN 11 13. 305 WARN 14588-[main] o.s.b.c.c.a.DefaultBatchConfigurer: No datasource was provided...using a Map based JobRepository2019-05-30 19 WARN 11 13. 306 WARN 14588-[main] o.s.b.c.c.a.DefaultBatchConfigurer: No transaction manager was provided Using a ResourcelessTransactionManager2019-05-30 19 INFO 11 13. 328 INFO 14588-[main] o.s.b.c.l.support.SimpleJobLauncher: No TaskExecutor has been set Defaulting to synchronous executor.2019-05-30 19c.c.SpringBatchApplicationTests 11main 13.350 INFO 14588-[main] c.c.SpringBatchApplicationTests: Started SpringBatchApplicationTests in 0.894 seconds (JVM running for 1.777) 2019-05-30 19c.c.SpringBatchApplicationTests 11main 13.732 INFO 14588-[main] o.s.b.c.l.support.SimpleJobLauncher: Job: [SimpleJob: [name=helloWorldJob]] launched with the following parameters: [{random=459672}] 2019-05-30 19Switzerland 1113 INFO 14588-- -- [main] o.s.batch.core.job.SimpleStepHandler: Executing step: [helloWorldStep] 2019-05-30 19DA 11 Executing step 13.812 INFO 14588-[main] c.c.batch.PersonItemProcessor: converting 'John Doe' into' Hello John Doegoat 2019-05-30 19 Vista 13.822 INFO 14588-[main] c.c.batch.PersonItemProcessor: converting 'Jane Doe' into' Hello Jane Doegoat 2019-05-30 19 purge 13.842 INFO 14588-[main] o.s.b.c.l.support.SimpleJobLauncher: Job: [SimpleJob: [name=helloWorldJob]] completed with the following parameters: [{random=459672}] and the following status: [COMPLETED] [INFO] Tests run: 1 Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.953 s-in com.codenotfound.SpringBatchApplicationTests [INFO] [INFO] Results: [INFO] [INFO] Tests run: 1, Failures: 0, Errors: 0 Skipped: 0 [INFO] [INFO]-[INFO] BUILD SUCCESS [INFO]-- -[INFO] Total time: 6.852 s [INFO] Finished at: 2019-05-30T19:11:14+02:00 [INFO]- -

You can find the results in the target/test-output / greeting .txt file:

Hello John Doe!Hello Jane Doe! At this point, the study of "what is Spring Batch" is over. I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.