BizTalk Archiving Pipeline Component Consideration

Question

In my scenario, I have a Pipeline that (1) decrypts and then (2) disassembles a flat file on a receive port.

My requirement is to capture the file, and put it on a local fileshare, between (1) and (2).

My initial approach was to introduce an Archive component between these, but I have run into issues with this. The Archiving component uses direct access to storage to dump the file. This is essentially poor methodology, as per BizTalk principles, this is a function of a send port/send adapter. So, if for example the Archiving destination is an FTP host, the Archiving component is useless.

Hence two ideas come to mind:

A) Somehow configure the archiving component to use a Send Port(if that's even possible)

B) Abandon the idea of the archiving component and just use BizTalk's native functionality as follows:

-Receive the file using decrypt only pipeline

-Send the file to a temporary local storage using a Send Port

-Subscribe to the receive port to send the file to an archive

-Pick up the file form local storage using Disassemble pipeline (second receive port)

-Use orchestration to process the file from the second receive port.

Are there any issues with Option B)?

If NOT, then what's the point of even using an archive component?

Answer 1

Other options also include

C) Have an archive send port and a loop-back send port subscribe to the receive port, the loop-back send port would have the flat file dissembler on the receive.

D) Have an archive send port and an Orchestration that subscribe to the receive port. Call the dissemble pipeline in the Orchestration.

We've used used both these scenarios for different solutions.

Answer 2

If you are using Native Biztalk functionality setting up send ports subscribing to the message type for archive is sufficient.

If you are using the BizTalk ESB Toolkit it is very difficult to split message for archiving since you are executing in the pipeline context. Using an orchestration in your itinerary will allow you to split the message but that of course requires the itinerary to leave the pipeline and drop the message on the message box. Just doing simple message archiving may lend this solution to be over kill.

You can use a custom pipeline component such as the one below. It is a pipeline component that can be reused, works in a BizTalk ESB toolkit scenario (very handy if you want to original message because it is transformed), as a file archive or SQL archive and works on both inbound and outbound pipeline scenarios.

BizTalk Archiving - SQL and File

You will only be responsible for the maintenance of the old/unwanted messages to avoid bloat.