Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Spring-batch Groovy y Gradle
Similar a Spring-batch Groovy y Gradle (20)
Último
Último (20)
Spring-batch Groovy y Gradle
- 1. Spring Batch Groovy & Gradle integrationSpring Batch Groovy & Gradle integration Because we like it…Because we like it… because we have fun with itbecause we have fun with it
- 2. About me:About me: Born with C and Java Github: http://github.com/amr390 (mostly drops, and tests) Working with Java and Javascript, take any opportunity to use groovy and python (I love for both of them) Grown up with Java since 2000 Twitter: 390amr (not much into it)
- 3. Spring Batch. Groovy and GradleSpring Batch. Groovy and Gradle What is Spring BatchWhat is Spring Batch Why integrating spring-batch with Groovy and GradleWhy integrating spring-batch with Groovy and Gradle RequirementsRequirements ➢ DependenciesDependencies ➢ Gradle tasksGradle tasks What is Spring BatchWhat is Spring Batch ➢Configuring Jobs, Tasks, Steps and stufConfiguring Jobs, Tasks, Steps and stuf ➢Running and Handling JobsRunning and Handling Jobs Improving Spring Batch performanceImproving Spring Batch performance ➢Multi-threadingMulti-threading ➢Master-Slave approachMaster-Slave approach ➢Parallelizing Step executionsParallelizing Step executions
- 4. What is Spring Batch I Batch Framework Technical services Functionality to process large volumes of records ItemReader, ItemWriter, ItemProcessor Stateless or Stateful step executions High-volume High-performance High-scalability Trough optimization Partitioning techniques
- 5. What is Spring Batch IIWhat is Spring Batch II
- 6. What is Spring Batch IIIWhat is Spring Batch III
- 7. What is Spring Batch IVWhat is Spring Batch IV STEP: ➔ Custom tasklet ➔ Restartable ➔ Skippable ➔ Retry ➔ Rollback ➔ Listeners Before/After ✗ Reader ✗ Processor ✗ Writer ✗ Chunk ✗ Step ➔ Step flow (on status)
- 8. What is Spring Batch: exampleWhat is Spring Batch: example
- 9. Philae Lander dataPhilae Lander data Philae Lander is a spacecraft that sends analysis data from comet 67P/Churyumov– Gerasimenko (https://en.wikipedia.org/wiki/Philae_(spacecraft)) Sends tones data in files encrypted in Klingon or Murciano. Files are zipped, and organized with a naming rule of file1, file2 etc ...
- 10. It is always fun using groovy Can be nicely coupled to other platforms Grifon, Grails... Power of Gradle Homogeneous Architecture Why integrating Spring-batch withWhy integrating Spring-batch with groovy and Gradlegroovy and Gradle
- 14. Requirements IVRequirements IV Gradle tasks MainClassName: Sets the project main class (because it is a global definition) run: sets the required arguments for run task, in this case the job configuration class and the Job to be launched.
- 15. @Configuration @EnableBatchProcessing Injects a JobRepository a JobLauncher a JobRegistry a PlatformTransactionManager a JobBuilderFactory a StepBuilderFactory Spring Batch Groovy & GradleSpring Batch Groovy & Gradle integrationintegration Job definition configurations, steps, tasklets and stuf
- 16. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle integrationintegration Running and Handling Jobs Spring batch admin
- 17. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle integrationintegration Running and Handling Jobs Job Launchers org.springframework.batch.core.launch Interface JobLauncher All Known Implementing Classes: SimpleJobLauncher JobExecution run(Job job, JobParameters jobParameters) throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException, JobParametersInvalidException
- 18. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle integrationintegration Running and Handling Jobs Job Launchers org.springframework.batch.core.launch.support Class CommandLineJobRunner
- 19. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle integrationintegration Performance improvement There different approached in order to improve the job execution performance Multi-threading: Using a ThreadPoolTaskExecutor Remote chunking: Master executes read phase of step slaves are distributed in nodes and execute processing and writing from a chunkProvider. Partitioning: Clone Job execution step and run steps in parallel Remote Partitioning: Instead of running the steps clone locally steps are distributed Parallel steps: step execution is splitted to execute several steps in parallel flows.
- 20. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle integrationintegration Multi threading
- 21. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle integrationintegration Multi threading Using task namespace <task:executor id="taskExecutor" pool-size="10" /> By java configuration method: ThreadPoolTaskExecutor taskExecutor ... Steps must be thread safe
- 22. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle integrationintegration Master-Slave approach
- 23. PartitionHandler: Interface defining the responsabilities of controlling the execution of partitioned step Partitioner: Interface to determine the way of creating partitions (key-range, file-chunks...) Spring Batch Groovy & GradleSpring Batch Groovy & Gradle integrationintegration Master-Slave approach
- 24. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle integrationintegration Parallelizing Step executions
- 25. Spring Batch Groovy & GradleSpring Batch Groovy & Gradle integrationintegration Parallelizing Step executions @Bean public SimpleFlow mainFlow() { SimpleFlow splitFlow = new FlowBuilder<SimpleFlow>("Split Flow") .split(new SimpleAsyncTaskExecutor()) .add(flow2(), flow3()) .build(); return new FlowBuilder<SimpleFlow>("Main Flow") .start(flow1()) .next(splitFlow) .end(); } <batch:step id="step1" next="">...</batch:step> <batch:split id="splitSteps" next="step3"> <batch:flow> <batch:step … /></batch:flow> <batch:flow> <batch:step … /></batch:flow> </batch:split> <batch:step id="step3"> <batch:tasklet ref="tasklet" /> </batch:step>
- 26. Demo Github: https://github.com/amr390/groovy-springbatch-demo Spring batch reference page: http://docs.spring.io/spring-batch/trunk/reference/ Spring batch in action: http://www.manning.com/templier/ Spring batch horizontal-vertical scaling: http://www.ontheserverside.com//blog/2014/07/23/horizontal-and-vertical-scaling-strategies-for-batch-applications Spring batch admin: http://docs.spring.io/spring-batch-admin/ Spring Integration reference page: http://docs.spring.io/spring-integration/refer Groovy: http://www.groovy-lang.org/documentation.html Gradle: https://docs.gradle.org/current/release-notes Spring Batch Groovy & GradleSpring Batch Groovy & Gradle integrationintegration References