What should be strategy to deal with ultra eager Apache Camel route?

Question

In my Camel Context, I have created 5 routes. Each of these routes listen to a designated folder.

The Setup

Route-A listens to a folder /net for input files. This folder location used from a client over SFTP who drops file in this location. Once a file is received in this location the route will read the file name and as per the name pattern, the file is moved to a location /type_a/inbound.

Route-B listens to /type_a/inbound for input files. When file is received in this location it will process this file as per its business logic.

All other routes work as same.

Problem

In today feed there is a file with large size(1.4GB). When Route-A starts to copy this file at /type_a/inbound, before it is completed the Route-B starts to read from the file. This causes the error in the Route-B. This does not happen as the route configuration for Route-A has ?exclude=.*.filepart" added in the endpoint. This .filepart file is not being created by Route-A at /type_a/inbound.

What options I do have to block the Route-B till the copy process is completed?

Answer 1

Have a look at the readLock option for your route B file consumer endpoint - http://camel.apache.org/file2.html

Used by consumer, to only poll the files if it has exclusive read-lock on the file (i.e. the file is not in-progress or being written). Camel will wait until the file lock is granted.

Answer 2

For situations like this, you can also use a done file. This way your large file is not polled until the 'done' file appears in the directory. Here is an example:

file:bar?doneFileName=${file:name}.done