How to force the current message to be suspended and be retried later on from within a custom BizTalk **send** pipeline component?

Question

Here is my scenario. BizTalk needs to transfer a file from a shared/central document library. First BizTalk receives an incoming message with a reference/path to this document in the library. Then it simply needs to read it out from this library and send it (potentially through different adapters). This is in essence, a scenario not so remote from the ClaimCheck EAI pattern.

Some ways to implement a claim check have been documented, noticeably BizTalk ESB Toolkit Claim Check, and BizTalk 2009: Dealing with Extremely Large Messages, Part I & Part II. These implementations do however take the assumption that the send pipeline can immediately read the stream that has been “checked in.”

That is not my case: the document will take some time before it is available in the shared library, and I cannot delay the initial received message. That leaves me with 2 options: either introduce some delay via an orchestration or ensure the send port will later on retry if the document is not there yet.

(A delay can only be introduced via an orchestration, there is no time-based subscriptions in BizTalk. Right?)

Since this a message-only flow I’d figure I could skip the orchestration. I have seen ways on how to have "Custom Retry Logic in Message Only Solution Using Pipeline" but what I need is not only a way to control the retry behavior (as performed by the adapter) but also to enforce it right from within the pipeline…

Every attempt I made so far just ended up with a suspended message that won’t be automatically retried even though the send adapter had retry configured… If this is indeed possible, then where/what should I do?

Oh right… and there is queuing… but unfortunately neither on premises nor in the cloud ;)

OK I may be pushing the limits… but just out of curiosity…

Many thanks for your help and suggestions!

Answer 1

I'm puzzled as to how this could be done without an Orch. The only way I can think of would be along the lines of:

• The receive port for the initial messages just 'eats' the messages, e.g. subscribing these messages to a dummy Send port with the Null Adapter, ignoring them totally.
• You monitor the Shared document library with a receive port, looking for any ? any new? document there.
• Any located documents are subscribed by a send port and sent downstream.

An orchestration based approach would be along the lines of:

• Orch is triggered by a receive of the Initial notification of an 'upcoming' new file to the library. If your initial notification is request response (e.g. exposed web service, you can immediately and synchronously issue the response)
• Another receive port is used to do the monitoring of availability and retrieval of the file from shared library, correlating to the original notification message (e.g. by filename, or other key)
• A mechanism to handle the retry if the document isn't available, and potentially an eventual timeout, e.g. if the document never makes it to the shared library.
• And on success, a send port to then send the document downstream

Placing the delay shape in the Orch will offer more scalability than e.g. using Thread.Sleep() or similar in custom adapter or pipeline code, since BTS just calculates ad stamps the 'awaken' timestamp on the SQL record and can then dehydrate the orch, freeing up the thread.

The 'is the file there yet?' check can be done with a retry loop, delaying after each failed check, with a parallel branch with a timeout e.g. after an hour or so.

Answer 2

The polling interval can be controlled in the receive location, so I do not understand what you mean by there is no time based subscriptions in Biztalk. You also have a schedule window.

One way to introduce delay is to send that initial message to an internal webservice, which will simply post back the message to Biztalk after a specified time interval.

There are also loopback adapters, which simply post the message back into the messagebox. This can be ammended to add a delay.