BizTalk - It's Not Just For XML Data

When you work with BizTalk for a while, when you watch BizTalk training videos, when you read BizTalk blogs, you start to get the impression XML is at the heart of the product. You have schema files which describe your XML. You have maps which transform your XML. It's all about XML.

Well, while that's generally true, it's not compulsory. Here are two nice scenarios which allow you to use BizTalk without ever converting your messages to XML.

Scenario 1

A customer FTPs us an Excel file and we need to publish the Excel file to a number of locations.

The normal flow of nearly all message based integrations in BizTalk follows this pattern:-

1. A message is received at a Receive Location, of which there can be many grouped together in a Receive Port.
2. A Receive Location utilises a Receive Pipeline to process the message and store the contents in the BizTalk Message Box database.
3. One or more "subscribers" collect this message from the Message Box according to a set of Filter rules. These subscribers can be Orchestrations and/or Send Ports.
4. Eventually one or more Send Ports will utilise Send Pipelines to write the message to their destination.

The Pipelines are where some clever stuff goes on. They can read the message, validate it, modify it, extract and publish import information about the message and even work out where the message needs to go. You can build your own custom pipelines or you can use a number of out-of-the-box pipelines, such as the XML Receive pipeline.

There is a nice out-of-the-box pipeline called "Pass Through" (actually there are two, one for receiving and one for sending). The clever thing about the pass through pipeline is it actually does none of the tasks I mentioned above.

While this can have drawbacks - for example, you don't get to see any key information held inside the message - there is one big upside. Because it doesn't read the information, it doesn't validate it and it doesn't modify it, BizTalk will happily pass the message on without ever looking at the contents. BizTalk never discovers the message is not XML.

The thing is, and this is very important, BizTalk still knows just enough about the message to know such facts a which Receive Port picked it up and when. Use these "system" properties on your Send Port Filters to subscribe to messages processed by a particular Receive Location and you have a solution which receives files and forwards them to a range of locations without ever opening the file.

Scenario 2

A customer FTPs us several files, some of which may be XML, some may be Excel. We need to route these files to recipients based on the content and file type.

While the solution to scenario 1 is powerful, it is somewhat blind to the types of message it is moving around. To allow us to make complex decisions about the message we need to have an Orchestration. But Orchestrations require you to explicitly set the receiving and sending message types. Receive a message that does not conform and BizTalk is going to throw it's toys out of the pram!

Oddly there is a powerful quirk of BizTalk that is our friend here. It's not obvious at first glance but makes sense once you know what it's doing.

When I said BizTalk Orchestrations need to know the message type they will receive and send, there's actually a variety of choices you can make. The normal approach is to specify an XML Schema to use. However you can also use another set of datatypes - the .NET datatypes (well, a subset of). And one of those is System.Xml.XmlDocument.

Woah! How will that help? The issue we are trying to work around here is that our data may not be XML! Surely System.Xml.XmlDocument is the worst datatype to use?!

Well, that would be the case if System.Xml.XmlDocument only allowed you to store "Well Formed XML" that conforms to a specific schema. But it turns out that the System.Xml.XmlDocument data type can actually hold data that should theoretically be XML even if it is not well formed or does not match a specific schema. And it doesn't care how many problems you have with the data. One problem, one million problems, all the same to the .NET System.Xml.XmlDocument datatype.

The upshot of this quirk is that BizTalk will happily receive anything into a System.Xml.XmlDocument message from good XML, to text, to pictures, to video files. OK, so you need to be careful what your orchestration does next - you're not home and dry just yet - but with some carefully crafted .NET code you can examine the message and route it according to the contents you identify.

The summary of this post... What we do is hard, but sometimes it's easier than you may think.

(originally posted by Dave B)

Why Assign And Expression Shapes Are Different In BizTalk Orchestrations

Message Assignment shapes and Expression shapes are, at a glance pretty similar. Both allow you to run C# code to manipulate mesages. However there are fundamental differences.

Assignment Shape
The assignment shape lets you create new messages and assign values to its promoted properties. It can only exist inside a Message Construction shape and can only create the message specificed by the construction shape. To ensure that it creates that message and only that message, complex control flow such as if statements are not allowed - however the IDE isn't going to tell you that so if you include an if statement you won't see an error until build time.

Expression Shape
The expression shape lets you manipulate the properties of existing messages as long as they have already been constructed. However it won't let you construct new messages and won't let you manipulate promoted properties so you can't put this shape inside a Message Construction shape. The one upside of this shape is that you can put complex flow logic into the expression shape.

This type of behaviour leads to visually more complex orchestrations but actually ensures that you are processing different types of tasks in different shapes which is critical when debugging.

(originally posted by Dave B)

Maps For Every Journey

In general the job of transforming one message type to another requires a map.

Message A > MapA2B > MessageB

This is cool if you have a simple scenario that always happens, day in, day out. But what if you have mapping rules which change depending on properties not held in your source message?

Well, BizTalk has a handy feature to help. It is possible to invoke a transformation from two different shapes - the good old Transform shape and the Expression shape. And, as I mentioned in a previous blog article, expression shapes can have complex flow logic such as IF statements. That means you can invoke any map you like.

if (condition)
{
       MapType = Type.GetType("NameSpace1.Map1, AssemblyName1, Version=1.0.0.0,Culture=neutral, PublicKeyToken=111111111111");
}
else
{
       MapType = Type.GetType("NameSpace2.Map2, AssemblyName2, Version=1.0.0.0,Culture=neutral, PublicKeyToken=111111111111");
}
// use the map type in the transform
construct _responseMessage
{
       transform (responseMessage.x1, _responseMessage.x2) = MapType(requestMessage.s1, _requestMessage.s2);
}

// or you can use the fully-qualified map name in the transform
construct _responseMessage
{
       transform (_responseMessage.x1, _responseMessage.x2) = NameSpace.MapName (requestMessage.s1, _requestMessage.s2);
}

Fun eh?!

(originally posted by Dave B)