BizTalk Pipeline Dreams, Part I

Friday Jun 29th 2007 by Jeffrey Juday

Instead of baking the capabilities of a Pipeline into an assembly, wouldn't it be nice to configure a pipeline using an XML configuration file? Wouldn't it be great to augment a pipeline with a couple of managed classes? Your solutions start here.

I think I've been suffering from "configuration envy." Looking at products such as WCF and the easy way you can build much of a service with a simple XML file, one begins to wish some of the older Microsoft products worked the same way. Take, for example, the BizTalk 2006 Pipeline Component.

Instead of baking the capabilities of a Pipeline into an assembly, wouldn't it be nice to configure a pipeline using an XML configuration file? Wouldn't it be great to augment a pipeline with a couple of managed classes? Wouldn't it be heaven to allow a BizTalk administrator to add or remove functionality?

I'm going to show you a XML configurable Pipeline Component prototype called the Pipeline Channel Stack I built to solve the problems above. Pipeline Channel Stack is built using ideas garnered from WCF. First, I'm going to give you an overview of the Pipeline Channel Stack. Then, I'll explain how the idea germinated and grew to use WCF ideas rather than WCF classes. I'll conclude with a deep dive into class implementation details.

Pipeline Channel Stack Overview

The Pipeline Channel Stack components are depicted in the following figure.

Figure 1. The Pipeline Channel Stack Components

The core pieces of the Pipeline are stored in the Flex_CustomPipelineCore.dll assembly. Flex_CustomPipelineComponent.dll is a Custom Pipeline Component. If Custom Pipeline Component development is new to you, you may find "Build BizTalk 2004 Custom Pipeline Components to Process Non-XML Data" helpful. Flex_CustomPipelineComponent_BizTalk is a BizTalk Pipeline.

For the sake of simplicity, going forward I'll refer to the stack of classes as the Stack.

When the Pipeline is activated, activation trickles down to the Flex_CustomPipelineCore.dll, which builds a stack of classes to execute the functionality of the pipeline. PipelineBinding and PipelineBindingElements create the Stack and a message class is built from internal BizTalk data structures by the bottom-most class in the Stack and passed through to the topmost layers in the Stack. The class arrangements are depicted in the next figure.

Click here for a larger image.

Figure 2. Class arrangements within the Pipeline

The Pipeline Channel Stack gives a developer the following capabilities:

  • Creating "one-off" pipeline solutions will not require a completely new Custom Pipeline Component. Implementing a couple of classes and adding the new functionality to the stack are the only requirements. You'll see how to do this later in the article.
  • Creating a completely new Custom Pipeline also is not required. Simply create two new classes for the Stack and you'll effectively have a new Pipeline.

The Binding and BindingElement classes are dedicated to assembling the Stack. If you've used WCF, the arrangement will look familiar to you. In fact, I would like to share how WCF influenced the design, before diving into code.

From WCF Seeds an Idea Blooms

Building a Pipeline Component using WCF was my first notion. In fact, I built the first working prototype with WCF. The WCF prototype compiled with a lot of unimplemented code warnings, but it did work. The warnings were my first clue that something was wrong. My second clue was: I was working around WCF rather than working with WCF.

I realized that it wasn't WCF that I needed; rather, I needed to follow some of the WCF implementation and architectural patterns. If you're familiar with WCF, you'll recognize many of the class names and class functional philosophy.

I'm going to take you a little deeper starting where Pipeline Activation begins, in the Custom Pipeline Component code.

Custom Pipeline Component

For more detailed coverage of Custom Pipeline Components, review "Build BizTalk 2004 Custom Pipeline Components to Process Non-XML Data." I'm going to give an overview of Custom Pipeline Component development.

Here are the main Custom Pipeline Component development ideas.

  • Custom Pipelines, though managed, run inside an unmanaged environment. Thus, they must contain the appropriate attributes.
  • A Custom Pipeline must implement a number of .NET Interfaces and attributes.
  • Custom Pipelines are activated when the Execute function is invoked. Execute contains two parameters: IBaseMessage and IMessageContext. IBaseMessage contains, among other things, a byte Stream representing the data from the BizTalk Port of the Pipeline.
  • Custom Pipelines are used inside of Pipeline Components. To use a Custom Pipeline Component, you must copy the Pipeline Component and supporting classes to the Pipeline Components folder and add the Component to the Visual Studio toolbox.
  • Depending on Interfaces implemented and the attributes added by the Custom Component, a Custom Pipeline Component can be dropped into any number of stages in a Pipeline.

As stated earlier, the Execute function activates the Custom Pipeline Component. The body of the Execute function appears below:

public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg)

   BizTalkMessageIdentification idValues =
      new BizTalkMessageIdentification("MessageType",
   //this is where you add special handling for the message body
   BizTalkContext bizTalkContext = new BizTalkContext(pc, inmsg,
   PipelineExecutionContext exec;
   PipelineBinding binding = new PipelineBinding(bizTalkContext);
   StandardPipelineConfiguration config =
      new StandardPipelineConfiguration();

   exec = binding.GetExectutionContext();

   config.Init(bizTalkContext, "here");


   return exec.Run();

As you can see, Execute invokes a number of classes. I'll explain what each class does, starting with the BizTalkContext.


BizTalkContext is a wrapper for the IBaseMessage and IMessageContext parameters on the Execute function. BizTalkContext also handles PipelineMessage creation. Wrapping Execute parameters and handling PipelineMessage creation serves the following purposes.

IBaseMessage and IMessageContext will be used by the layers in the Stack. Wrapping the classes facilitates passing the classes throughout the Stack. Also, at some point in the future you may want to include auditing or restrict direct control to the Execute parameters.

PipelineMessage is passed throughout the Stack. I wanted to restrict PipelineMessage creation to the classes in the core assembly. So, I made the PipelineMessage constructor internal. Somewhere, though, I needed to provide a mechanism for other Stack classes to create a new PipelineMessage. PipelineMessage and the Execute parameters purpose are intertwined, so I housed their creation in BizTalkContext.

Now, turn to the role of PipelineExecutionContext.


PipelineExecutionContext is the "command and control" part of the Pipeline Channel Stack. PipelineExecutionContext works closely with the PipelineBinding class, invoking Stack building functions on the PipelineBinding and iterating through the Stack created by the PipelineBinding.

Run is the sole method in the class and to a great degree Run "is" the Pipeline. The body of Run appears below.

public IBaseMessage Run()
   PipelineMessage msg;
   PipelineChannelContext context;

       PipelineChannelTrace.WriteLine("Run Started..");

   _binding.Build();    //Assemble the channel stack

    //reach into bottom of the stack
   context = _binding.ChannelStack.Contents[0];

   msg = context.RetrieveMessage();    //Get the bottom message

   //Now pass the message through the stack
   for (int n = 1; n < _binding.ChannelStack.Contents.Count; ++n)
      context = _binding.ChannelStack.Contents[n];


      msg = context.RetrieveMessage();

   PipelineChannelTrace.WriteLine("Run Ended..");

   return msg.ApplyMessage();

As you can see in the loop, each member in the Stack consists of a PipelineChannelContext class. PipelineExecutionContext passes a PipelineMessage into the PipelineChannelContext invoking AcceptInnerMessage and then pulls a new PipelineMessage or transformed PipelineMessage out of the PipelineExecutionContext invoking RetrieveMessage.

PipelineChannelContext is really just a thin wrapper around a class implementing the IPipelineChannel interface. I wanted to include the ability to trace through Stack execution without relying on the developer, so I wrapped the IPipelineChannel in PipelineChannelContext.

I've mentioned PipelineBinding quite a bit. Now, I'm going to show you what it does.


The PipelineBinding class in the prototype serves a similar purpose as the Binding class in WCF. PipelineBinding houses a stack of PipelineBindingElements and the resulting Stack created by the stack of PipelineBindingElements. Build is the primary PipelineBinding function. The body of the Build function appears below.

internal void Build()
   Type currentType = null
   PipelineChannelContext channelContext = null
   PipelineBindingElement elm = null;

   for (int n = 0; n < _elementOrder.Count; ++n)
      currentType = _elementOrder[n];

      elm = _elements[currentType];

      channelContext = elm.BuildChannel();



The StandardPipelineConfiguration class is responsible for populating the PipelineBinding.

Binding is primarily the construction foreman and home for the Stack. All the real work of building the Stack happens in the PipelineBindingElement class.

Implementing a PipelineBindingElement

PipelineBindingElements build specific classes derived from the PipelineChannelContext or implementing the IPipelineChannel interface. PipelineBindingElement is an abstract class, so it cannot be instantiated directly.

As you will see in an upcoming article, adding an IPipelineChannel requires building a class implementing IPipelineChannel and building a PipelineBindingElement to build the IPipelineChannel class.

OnBuildChannel does the building work. PipelineBindingElement can pass the BizTalkContext to IPipelineChannels it creates if desired.

To Be Continued

There are still some open issues to cover, such as:

  • How do you build a PipelineBindingElement?
  • How do you set up the Configuration in XML?
  • How do you test and deploy a solution once you have built it?
  • What are some of the things you can do with a Pipeline Channel Stack?

I will answer these questions and more in the next article.


By applying the WCF ideas of BindingElements, Binding, and Channels, you can make a BizTalk 2006 Pipeline Component behave like a WCF Channel Stack. The included source code will show you what you need to do. In a future article, I will show you how to extend the solution.

Mobile Site | Full Site
Copyright 2017 © QuinStreet Inc. All Rights Reserved