BizTalk - It's Not Just For XML Data

When you work with BizTalk for a while, when you watch BizTalk training videos, when you read BizTalk blogs, you start to get the impression XML is at the heart of the product. You have schema files which describe your XML. You have maps which transform your XML. It's all about XML.

Well, while that's generally true, it's not compulsory. Here are two nice scenarios which allow you to use BizTalk without ever converting your messages to XML.

Scenario 1

A customer FTPs us an Excel file and we need to publish the Excel file to a number of locations.

The normal flow of nearly all message based integrations in BizTalk follows this pattern:-

1. A message is received at a Receive Location, of which there can be many grouped together in a Receive Port.
2. A Receive Location utilises a Receive Pipeline to process the message and store the contents in the BizTalk Message Box database.
3. One or more "subscribers" collect this message from the Message Box according to a set of Filter rules. These subscribers can be Orchestrations and/or Send Ports.
4. Eventually one or more Send Ports will utilise Send Pipelines to write the message to their destination.

The Pipelines are where some clever stuff goes on. They can read the message, validate it, modify it, extract and publish import information about the message and even work out where the message needs to go. You can build your own custom pipelines or you can use a number of out-of-the-box pipelines, such as the XML Receive pipeline.

There is a nice out-of-the-box pipeline called "Pass Through" (actually there are two, one for receiving and one for sending). The clever thing about the pass through pipeline is it actually does none of the tasks I mentioned above.

While this can have drawbacks - for example, you don't get to see any key information held inside the message - there is one big upside. Because it doesn't read the information, it doesn't validate it and it doesn't modify it, BizTalk will happily pass the message on without ever looking at the contents. BizTalk never discovers the message is not XML.

The thing is, and this is very important, BizTalk still knows just enough about the message to know such facts a which Receive Port picked it up and when. Use these "system" properties on your Send Port Filters to subscribe to messages processed by a particular Receive Location and you have a solution which receives files and forwards them to a range of locations without ever opening the file.

Scenario 2

A customer FTPs us several files, some of which may be XML, some may be Excel. We need to route these files to recipients based on the content and file type.

While the solution to scenario 1 is powerful, it is somewhat blind to the types of message it is moving around. To allow us to make complex decisions about the message we need to have an Orchestration. But Orchestrations require you to explicitly set the receiving and sending message types. Receive a message that does not conform and BizTalk is going to throw it's toys out of the pram!

Oddly there is a powerful quirk of BizTalk that is our friend here. It's not obvious at first glance but makes sense once you know what it's doing.

When I said BizTalk Orchestrations need to know the message type they will receive and send, there's actually a variety of choices you can make. The normal approach is to specify an XML Schema to use. However you can also use another set of datatypes - the .NET datatypes (well, a subset of). And one of those is System.Xml.XmlDocument.

Woah! How will that help? The issue we are trying to work around here is that our data may not be XML! Surely System.Xml.XmlDocument is the worst datatype to use?!

Well, that would be the case if System.Xml.XmlDocument only allowed you to store "Well Formed XML" that conforms to a specific schema. But it turns out that the System.Xml.XmlDocument data type can actually hold data that should theoretically be XML even if it is not well formed or does not match a specific schema. And it doesn't care how many problems you have with the data. One problem, one million problems, all the same to the .NET System.Xml.XmlDocument datatype.

The upshot of this quirk is that BizTalk will happily receive anything into a System.Xml.XmlDocument message from good XML, to text, to pictures, to video files. OK, so you need to be careful what your orchestration does next - you're not home and dry just yet - but with some carefully crafted .NET code you can examine the message and route it according to the contents you identify.

The summary of this post... What we do is hard, but sometimes it's easier than you may think.

(originally posted by Dave B)