How to not loose data between polled data avaible and post poll statement in WCF custom oracle

Question

note, not sure if the title is the best title

We have a BizTalk receive port that collects data with the standard polled data avaible, polling statement and post poll statement.

now we have the issue that data that becomes valid for the polled data avaible statement during a run, will also be updated by the post poll statement and so, will never be picked up.

this leaves 1 or 2 database lines per day ticked off as processed without being processed. we have turned off and on the ambient transaction parameter but this effect remains the same in our tests.

This seems a serious issue to me and it can't be we are the only ones that have this issue? is there a way to solve this? I have been thinking of writing a sendPort that updates the database and not using the postpoll statement but this would be quite a bit of extra logic and i can't see a way to do this without making Orchestrations for every single Project which is dis-advised by best practices (Try to use direct binding send to receive ports)

Answer 1

If you have control over the database table you are polling, add one more column of the type uniqueidentifier. When doing your PollingDataAvailable statement, set a variable to newid() and update the column with that ID + save it somewhere to read (preferably another table).

In your PollingStatement, you can read the last GUID from the separate table and only get what has been updated in your actual polling table + mark them as processed.

This way, no duplicates will be polled and you will get everything.

Also, if you have more than one BizTalk server in the group, consider separating a host, just for SQL polling and cluster that one (or put one active, the other passive). The way the polling works with multiple hosts sometimes makes it tricky to pick up records.

Hope this helps!

Answer 2

So, it sounds like the Post Poll statement is a little optimistic or just making an incorrect assumption, specifically that everything was processed by the Polling statement. This isn't really a problem with the client.

My recommendation would be to abandon Polled Data Available and Post Poll and Poll with a Stored Procedure. The SP should only returned 'processed' records so anything new would get picked up on the next interval.

If you are unable to use a Stored Procedure, then the Polling Statement should set a Flag then the Post Poll would update only records with that Flag.