Apologies in advance this might be quite a complex one! I should say at the outset that I don't want someone to write my solution for me, just am trying to understand a bit more about how to achieve transactionality in WF.

I'm currently trying to pilot an STP engine using Workflow Foundation to prove its suitability. Everything in the engine is driven using transactional MSMQ queues. It's essential that no information is lost, and that the workflows remain in a consistent state with the contents of the queue. These are the scenarios I'm trying to tackle at the moment:

1. A message arrives on a queue and triggers the creation of a new workflow.

2. A message arrives on a queue and is passed into a waiting workflow.

3. A workflow sends a message on a queue.

In all three the two actions (a queue action, and a workflow action) must happen as part of one atomic transactional unit, so in my mind the queue operation must be linked up with the persistence of the workflow.

The approach I've tried to follow for 2 and 3 is to tie into the IPendingWork and IWorkBatch capabilities of WF. I wrap the MSMQ transactions in a shim class that let it tie into System.Transaction, then add these as pending work items. When the IPendingWork implementing service is called to commit, I try to tie the MSMQ transactions into the transaction that gets passed into IPendingWork.Commit, using transaction.EnlistPromotableSinglePhase. The problem I face here is the the MSMQ transactions cannot be promoted.

I could take the simpler approach and when IPendingWork commits, just commit the MSMQ transactions at that moment in time. What I'm worried about is that if the computer crashes during the commit operation, some of the messages will be sent / or removed from the queue, and some will not. Thus my workflow and queue states will be inconsistent.

Is there something I'm missing? Am I scuppered by the fact the MSMQ does not support promotable transactions or am I doing something wrong on the WF front? If the former I suppose I can use SQL Server 2005 as an intermediary between MSMQ and WF, as SQL Server transactions are promotable.

On scenario 1, I'm lost as to what to do. As far as I can see there doesn't appear to be a similar IPendingWork mechanism for the creation of workflows. So my questions here are, when a workflow is created, is it persisted immediately, and is there a way of tying other transactions into this process?

One solution I can think of would be to pass in the MSMQ transactions as arguments to the workflow, then add a custom activiity with the [PersistOnClose] attribute which adds the transactions to the batch. This seems a little messy to me - is there a better way?

Any help will be much appreciated!

Terence - getting the receive of the message in a transaction is a little tricky since this version of WF doesn't support ambient transactions - but for the send - this works for me because System.Messaging supports ambient transactions:

public void Commit(System.Transactions.Transaction transaction, System.Collections.ICollection items)

using (TransactionScope ts = new TransactionScope(transaction))

MessageQueue mq = new MessageQueue(@".\Private$\wftestout");

foreach (MessageWorkItem mi in items)

mq.Send(mi.Data, MessageQueueTransactionType.Automatic);

public void Complete(bool succeeded, System.Collections.ICollection items)

For the code snip above the using(TransactionScope ts = new TransactionScope(transcation)) isn't necessary. The WF runtime creates a TransactionScope with the same transaction parameter transaction prior to calling IPendingWork.Commit.

Now, for the first case that you describe. I assume that what you want is the dequeue to be committed atomically with the workflow action (such that if you dequeue a "create" message the workflow is durably created and the two actions are atomic; likewise if you dequeue a "message delivery" message the message is delivered to the workflow in a durable manner and the two actions are atomic). Assuming this is correct the following may work for you (I'm not familiar with the sematics of the transaction that MSMQ creates so you may still get promotion issues with the following approach).

The WF runtime supports flowing in ambient (TransactionScope) transactions for the WorkflowInstance.Unload method. Due to time constraints in V1 this is the only place that we could support this behavior. This does, however, solve some key scenarios that are similar to yours. For example you can dequeue a create message, call WorkflowRuntime.CreateInstance and then call WorkflowInstance.Unload. The Unload logic will use the ambient transaction (if one is available) to commit the workflow. You can then commit the transaction and know that any work that is part of the transaction either failed or succeeded as a set. Follow this up with a WorkflowInstance.Start call if the transaction committed successfully. You can use a similar pattern when delivering a message to an instance. Just call WorkflowInstance.Enqueue (or EnqueueOnIdle or via OCS) followed by WorkflowInstance.Unload and then commit the transaction. Follow this up with a WorkflowInstance.Resume call if the transaction committed successfully.

One thing to note here is that the SqlWorkflowPersistenceService has an issue with the Create pattern if a crash occurs. You won't lose any data but the instance will not automatically be resumed when you restart the WorkflowRuntime. This is because the instance has been persisted in the Created state and the SqlWorkflowPersistenceService does not automatically start instances that are in the Created state. You'll need to either develop your own persistence service that automatically starts instances that are in the Created state during the WorkflowRuntimeService.OnStarted event or just be aware of the problem and have a process (manual or a script) that loads and starts any instances that are in the Created state after a system failure.

Thanks,

Joel West

MSFT - SDE in WF runtime and hosting

This posting is provided "AS IS" with no warranties, and confers no rights

I understand that Transaction.Current is there and available, as well as the Transaction parameter so that using TransactionScope isn't necessary.

I just think using it is the better programming style. ;-) It is explicit, and it makes it clear to the ambient transaction when I am happy and done. I just think it is more understandable than just "I know there is a transaction here and if I throw an exception the transaction will fail".

Since my TransactionScope will just use the passed in transaction - there isn't really any overhead either.

If a Transaction wasn't being passed into the Commit method - I'd still use TransactionScope - just the default constructor.

And as far as I can tell - this works fine with MessageQueue and its support for ambient transactions.

At a higher-level in the WF transaction discussion, I would like to use the following 2 patterns (see below) to insure that the database updates and WF updates are synchronized in a transaction. 

Does WF automatically enlist in the TransactionScope in each case (along with the database updates)? If not, any suggestions on how to implement this type of transactional behavior?

Pattern WFT1. Start Workflow In TransactionScope

WorkflowInstance oWorkflowInstance = null;

using (TransactionScope ts = new TransactionScope())

{

    UpdateDB1a();

    UpdateDB2a();

    oWorkflowInstance = oWorkflowRuntime.CreateWorkflow( typeof(OrderWorkflow), dictParameters );

    oWorkflowInstance.Start();

    UpdateDB3a();

    UpdateDB4a();

    ts.Complete();

}

Pattern WFT2. Signal Workflow in TransactionScope

using (TransactionScope ts = new TransactionScope())

{

    UpdateDB1b();

    UpdateDB2b();

    oMyWFLocalService.SendAction( new TaskActionSentEventArgs(oWorkflowInstance.InstanceId, dictParameters) );

    UpdateDB3b();

    UpdateDB4b();

    ts.Complete();

}

Neither scenario will work, the WF runtime in V1 only supports flowing in a transaction on WorkflowInstance.Unload. See my post above for more comments on this. There are various ways that you could try and hack this (with a custom persistence service or WorkflowCommitWorkBatchService) but if you do this it won't work correctly 100% of the time and the times when it fails (error conditions or failures causing the tx to rollback) will be exactly when you are expecting transactional consistency.

Bottom line - the only way to make this work is to call WorkflowInstance.Unload inside your transaction scope. This was the best that we could do in V1 to try and enable this pattern in some form. Not always ideal but it can be made to work for most scenarios that require usage of an external transaction.

Thanks,
Joel West
MSFTE - SDE in WF runtime and hosting

This posting is provided "AS IS" with no warranties, and confers no rights