Blogging BizTalk

Lets Discuss BizTalk

Category Archives: BizTalk Messaging Patterns

Is BizTalk server a T-Rex?


Before you press the panic button read this please..

Nino Crudele's Blog

Why do people start comparing BizTalk Server to a T-Rex?

Long time ago the Microsoft marketing team created this mousepad in occasion of the BizTalk 12th birthday.

 

And my dear friend Sandro produced the fantastic sticker below

 

Time later people start comparing BizTalk Server to a T-Rex, honestly, I don’t remember the exact reasons why but I like to provide my opinion about that.

Why a T-Rex?

In the last year, I engaged many missions in UK around BizTalk Server, assessments, mentoring, development, migration, optimization and more and I heard many theories about that, let go across some of them :

Because BizTalk is old, well, in that case I’d like to say mature which is a very good point for a product, since the 2004 version the product grew up until the 2013 R2 version and in these last 10 years the community build so…

View original post 1,058 more words

Advertisements

Find BizTalk Pipeline Component References


What if your BizTalk implementation has lot of pipeline components and custom pipeline used and at times the question might pop up: find where all a certain pipeline components has been referenced.

This might become more relevant if you are doing a deployment planning . One way is to go check the individual BizTalk solutions and find out where those have been referenced. There is a  high probability that something may be missed.
To overcome that we can use the BizTalk MdmtDb and query it to get the information on the go.
The table we need to refere in our query are
  • bts_component
  • bts_stage_config
  • bts_pipeline_stage
  • bts_pipeline_config
  • bts_Pipeline
Here is the query you will need to execute, you will be required to be the part of BizTalk operator group.
SELECT E.Name,E.FullyQualifiedName
FROM BizTalkMgmtDb.dbo.bts_component AS A,
BizTalkMgmtDb.dbo.bts_stage_config AS B,
BizTalkMgmtDb.dbo.bts_pipeline_stage AS C,
BizTalkMgmtDb.dbo.bts_pipeline_config AS D,
BizTalkMgmtDb.dbo.bts_Pipeline AS E
WHERE A.id=B.CompId
AND B.stageId=C.id
AND  C.id=D.StageId
AND D.PipeLineId=E.id
AND A.Name like  (‘%Your Component Name%’)
untitled

Decouple BizTalk Transformation logic From The BizTalk Solution.


Need a BizTalk solution which has no map and it still does the transformation, READ ON……

This article speaks of using an external XSLT file to achieve  transformation in BizTalk solution. You can visualize this as an equivalent of  BizTalk map which is only using a XSLT file for the transformation logic except you will not be required to use a actual BizTalk map with this approach. This also means no map or XSL file will be deployed. Advantage with this approach is- If there is a change in the transformation logic just modifying the XSLT placed on the file server location is enough. Which mean without any fresh deployment or change to your VS BizTalk solution you have achieved a new transformation logic in the BizTalk workflow.

 How do we do that:
 1>Modify the BTSNTSvc config file and add new appSetting keys for the physical path of XSLT file. This is path where you are going to keep the XSLT file(Or you can use any custom config approach for storing the values, for example SSO MMC).
ConfigChange
 COnfigManagerObject
1>Create a helper class which will return a XmlDocument after applying the XSLT on the input source xml message.
 TransformMethodWe will be  leveraging Transform method in our BizTalk helper class which Executes the transform using the input document specified by the XmlReader object and outputs the results to an XmlWriter. The XsltArgumentList provides additional run-time arguments.

public void Transform(XmlReader input, XsltArgumentList arguments, XmlWriter results);

TransformMethod
Now we can use an orchestration to call this method in a construct with MessageAssignment shape.
msgOP_Map = PKS_ExternalBizTalkTransformation.ApplyXSLT.ApplyXSLTToPayload(msgIpToMap, “YourXSLTFile.xsl”);
 Please let me know if you have any questions.
Note:For unit/local testing you will be required to use an app.config to define the XSL path since the BTSNTSvc app settings can be only accessed at BizTalk engine runtime .

Understanding Bussiness Rules Engine Via The Dynamic Endpoint Resolver Pattern


In this example I am going to demo how we can use Business rules engine in BizTalk to route messages dynamically.

Here is our scenario.

We receive a message having a node called PartyName.

Based on the value of PartyName the business rules policy named ‘ResolverRules’ will fetch the endpoint and adaptertype from from a SQL table named ‘RulesResolver’.This policy will be called from an orchestration which will use a dynamic send port based on the endpoint fetched by the rules.

Advantage: This will help us accomplish a decoupled messaging scenario. All we need to do is add the new PartyName and the corresponding endpoints, adapterUsed information in the SQL table and the orchestration dynamic send port will take care of the routing. Which essentially means a single send port can cater to multiple send end points.

DEMO:

STEP1: CREATING THE RULE

Create a DB and table in your SQL server. The table will have three columns for partyname adapter type and end point. Our orchestration will fetch all these information via the rules engine and assign it to the dynamic send port properties. Our table is going to look something like this.

DBTable

Open the Business Rules Composer.

Go to the facts explorer > Vocabularies tab> Select Vocabularies, right click and pick Add New Vocabularies.

Name your vocabulary as appropriate. Click on your newly created vocabulary and pick  Add new definition. We are going to define two definition one will fetch our endpoint and other will fetch the PartyName.

SetVocabulary

Once done right click on your vocabulary version and pick Publish. Without publishing the vocabulary we wont be able to use that in our rules policy. Also once the vocabulary is published it cannot be modified.If need arises we will have to publish a new version and use that instead.

VocabularyPublish

Now once vocabulary is ready we are going to create rules policy based on the DB facts and the XSD. To reference our xsd we will select the XML schemas tab and right click on schemas > Browse to our xsd file we have created in the BizTalk solution also holding our orchestration.

setxml

We are now ready to define our rules policy now. Click on policies and pick add new policy. Name it appropriately. After you are done configuring your policy the end result should look like this. The conditions we have defined has used the vocabularies we defined in the earlier step.

Please go ahead and deploy the policy.Without deploying the orchestration will not pick the rules.

policy

Now our Rules Policy is ready to be used.

STEP 2: CREATING THE ORCCHESTRATION TO CALL RULES.

Please create an orchestration looking ressembling following picture .

Orchestration

You will have to add System.data, System.transaction and Microsoft.RulesEngine reference  in the solution. Create three variable in the transaction scope of the call rule shape of the following types.

Microsoft.RuleEngine.DataConnection
System.Data.SqlClient.SqlConnection
System.Data.SqlClient.SqlTransaction

The expression shape in the orchestration are going to look like this.
orchexpressions

Use the call rule shape to point to our rules policy. Configure the input parameters, in this case two one message and other rules connection.

The last send port is dynamic and we will pass the port parameters in the last expression shape which in turn are derived from the message returned by rules policy. Notice the expression to assign the end point.

Now we can deploy this and our solution will be ready for testing.

Please leave me a comment if you have any questions.

Orchestration To Orchestration Call Via Direct Bound Ports For A Synchronous Process


This is a demo for calling one orchestration from another orchestration via direct bound port. Please note there is a strong binding between receiver and sender orchestrations hence if the receiver orchestration contracts or version changes at any point of time you will have to incorporate that in your sender orchestration as well.

Scenario: The main orchestration calls the sub orchestration and waits upon the reply to come back from sub orchestration to continue further.

Logic Behind: The main orchestration refers to the contract of the suborchestration via an assembly or directly (as I have done in this example). The main orchestration passes a unique promoted property to the sub orchestration(It can be a set of properties as well).Please ensure whatever property you are promoting has to be unique for each instance of the calling orchestration. The sub orchestration does its flow and before sending back the response it also returns the promoted property written in context of its response, back to the main orchestration. Now the main orchestration has a receive port listening on the sub orchestration response, it is following a correllationSet defined for the promoted property. Confused ? Ok, lets  walk through on how this has been implemented from coding perspective and I hope you will know what I am trying to explain.

1>Create a new BizTalk project and add two orchestration files there.

  • MainOrchestration.odx
  • SubOrchestration.odx

Define the request and response schemas for each of the orchestration. The body of the orchestration will look like this. I am not doing any complex logic here. The main orchestration creates the sub orchestration request>> sends it to the messegebox>> pick up the response from the message box>>creates its own response and sends it to a file send location. The main orchestration body will look like this.

Image

2>We need to assign a unique property to the sub orchestration request message. I have created a property schema and will be using it to define my correlation property.

New1

3>Since this unique property has to be available in the context of the sub orchestration request message it has to be promoted , we accomplish this by creating a new correlation set with the identified unique property.

4>Initialize this correlation set on the send shape for sending sub orchestration request and follow it on receive shape for the sub orchestration response.

write the remaining portion of Main Orchestration and you are done with it here.

5>The sub orchestration will look like this.

Image

6>Since the sub orchestration needs to return the property which was sent by the main orchestration here also you need to create another promoted property used to store the value sent by main orchestration.

Image

=Please note all the ports used to communicate with the sub orchestration is of direct binding type( Silly to mention but that is what I have designed this solution around)

Create a file receive port and trigger your orchestration. You should be able to see the following execution.This completes the demo. Please leave me a comment if you need some clarification or help. I will get back ASAP. Thanks.

OrchestrationExecution

Sequential Convoys / Singleton Orchestration In BizTalk Demo


A sequential convoy enables multiple single messages to join together to achieve a required result. A sequential convoy is a set of related messages that have a predefined order. Although the messages do not have to be exactly the same, BizTalk Server must receive them in a sequential order.

Few usage of them could be an aggregator orchestration or to introduce throttling. I will be covering those in later blogs.

In our example below all the messages arriving to one ReceivePort will trigger our orchestraion and all of them will be consumed within a SINGLE INSTANCE of the orchestration. In other words each incoming message is not triggering a new orchestration instance. I have tried to keep the orchestration simple but hopefully by the time you will finish reading this you will know what how the sequential convoys operate.

Using correlated Receive ports are key to sequential convoys. Here are the steps.

1>Create a new BizTalk solution and define your incoming and outgoing message schemas(You can refer to my previous posts in case you want to for doing that)

2>Add a new orchestration to your solution and design it similar to the Picture below.

SCOrchestration

3>Define correlation set and types and in the correlation properties we will use the “BTS.ReceivePortName” property since we want to have the convoy configured on all the messages arriving on a particular receive port.

correlation

Note:We can use a lot of other properties to define the “correlation ” for the sake of simplicity I have picked ReceivePortName.Let me know if you want to use some other properties and discuss it here.

4>On the first receive shape we will initialize this correlation set.

IntializeCorr

5>The second receive shape will follow this correlation set.

followcorr

6>Create a physical receive port and send port. Bind orchestration and start all the ports, orchestration

When the first message arrives in the message box the orchestration picks that up and starts while initializing the correlation, when the 2nd message arrives instead of being subscribed for a new orchestration instance it rather goes to the 2nd receive shape which is following the correlation initialized by the first receive shape hence this message gets consumed by the existing orchestration instance. This loop goes on unless we write a logic for when to break it.

Try dropping xml files to your receive location, the admin console will show you all the artifacts triggered notice there is only on orchestration instance triggered. During the time when there are no messages coming into the receive location or orchestration will be in dehydrated state and and will rehydrate in event of a new message arriving into the port. Also this orchestration is not designed to complete, you may want to add a branch with delay shape to exit the infinite loop in the code, anyways thats a simple thing to do and we may handle this in mutiple way. Hope the example comes handy.

SeqAdminConsole

Need source code, please drop a comment with your email id and I will revert back ASAP.

Dynamic Endpoint Resolver Using BRE and ESB Itineraries


In this example I will demo how we can use BRE engine in conjugation of ESB itinerary to achieve dynamic routing.

Create a receive port which is going to look like the below screen shot.
ReceivePort
The pipeline component being used is ItinerarySelectReceive. Or otherwise you can create you own custom pipeline which in a way mimics ItinerarySelectReceive, see below.
CustomPipeline.png
Create a business rule which will contain the information for routing.In this example we are toting a message based on the message type. For simplicity sake I have used FILE type adapter at both send and receive side.
Rules
Create a dynamic send port, picture below. Apply the filter as shown in the picture
SendPort
Lets get down to creating the itinerary which will be  attached from the receive port.
Drag an on ramp shape and under the properties set the receive port name to the one we just created.oNrAMP
Drag a messaging extender shape and set the service name to “Microsoft.Practices.ESB.Services.Routing”.
SetExtenderServiceType
SetOnRampINService
Add a new resolver in the messaging extender and set that to BRE Resolver Extention.
BREResolver
Pick the BRE policy we created for send port properties.
SetPolicy
At this stage we need to an off ramp and  extender for off ramp. Set the different propertied as below.
OffRampExtender
OffRamp
OffRamp_BTAPP
OffRamp_BTSendPort
ValidationErrorFix
ValidationErrorFix2
Once it is done set the Model Exporter to Database Itinerary exporter and right click >> Export Model.
PublishITIN.png
Start the RL and send port. Drop the message of  message type http://ItinRouting#Request, the RL will invoke receive pipeline which in turn will invoke the itinerary which further will call the BRE to get the routing information for dynamic send ports.
Please let me know if you have any questions.

Modify An Exiting BRE Policy and Vocabulary


Used Case :

This was an old project having a vocabulary which was used in multiple policies / multiple version of policies.

Need was to add a new definition to be used in an existing policy. One way to do this was to add a new vocabulary and use it. This option was ruled out because it would have added an additional vocabulary collection thus making it look cumbersome.
 
Reversion the vocabulary and add the new definition in it – This as well was ruled out since the current version of the vocabulary was used in many Policies – implying a need to update the reference of used vocabulary definition in each Policy.
Undeploy all the policies using the vocabulary in question , followed by undeploying the  vocabulary. After that modify the bindings and redeploy vocabulary followed by the policy. – This option was not opted out because of the complex nature of the BRE policies using the same vocabulary version. – Ideally speaking this is the option we should pick for any such needs.
 
Having said that there was an unconventional route taken to accomplish the need.
 
Modify a published Vocabulary
The vocabulary state is stored in nStatus filed under the table – re_vocabulary in – BizTalkRuleEngineDb. For each vocabulary there is a separate row.
 
It has two states 1 and 0. 1 indicates the Vocabulary is in Published state and 0 indicated it can be modified. Hence the corresponding vocabulary row –  nStatus field was edited to 0, making the vocabulary editable. Subsequent to it the new definitions were added using the Business Rules Composer.
 
1
 
 
Modify a published Policy
 
Policies state are stored in re_ruleset table column nStatus. It has three states.
 
0 Saved
1 Published
1 Deployed
 
Notice the Published and Deployed state value are the same. For a policy it is differentiated by  the entry in the table re_deployment_config. For any policy( nRuleSetID value) which is present in re_deployment_config and re_tracking_id table indicates that should be in deployed state.
 
For example I have to modify the Rules PKS_DemoPolicy version 1.0 without incrementing the version. First I need to get the nRuleSetID from re_ruleset table, this will be used  in deleting respective entries in re_deployment_config and re_tracking_id . 
 
GetUidPolicy
 
Once identified go ahead and undeploy the policy PKS_DemoPolicy .
Undeploy
 
Subsequent to that change the nStatus Column using following query.
 
 update [BizTalkRuleEngineDb].[dbo].[re_ruleset] set nStatus= 0 where nRuleSetID = ‘**Your nRuleSetID’
 
 
Next delete the entry for your PKS_DemoPolicy  in re_deployment_config
 
Delete from [BizTalkRuleEngineDb].[dbo].[re_deployment_config] where nRuleSetID=’**Your nRuleSetID’
 
Delete the entry from re_tracking_id table 
 
 Delete  FROM [BizTalkRuleEngineDb].[dbo].[re_tracking_id] where nRuleSetID=’**Your nRuleSetID’
 
Once done go back to Business Rule Composer and refresh. The policy state should be editable. Please leave a comment if you have any questions and I will get back as soon as I can.
Disclaimer – The approach taken here is not recommended by Microsoft and please use it at your own discretion.

Find Orchestration Details Using BizTalk Mgmt DB


Introduction

This topic is intended towards Microsoft BizTalk server users and I have tested it to be working with BizTalk 2006 R2/ BizTalk 2009/ BizTalk 2010/ BizTalk 2013/ BizTalk 2013 R2. The person inteding to use this need to be the BizTalk administrator.

What if you have hundrebs of orchestration deployed in your BizTalk enviroinments and you need to go look up specific information like binidings end point used, sunbscriptions, state etc. It can be a realy painful job to go look this up from BizTalk admin console. This article will provide you and alternate approach and obviously a quick one to get the required detail you need. I am leveraging the BizTalk Mgmt DB to find the information outlined below.

This SQL script is goign to provide you the all the bindings and end point details for BizTalk orchestrations, it can be utilized in SSRS report or web based UI to provide end uder with this information.IT can come realy handly while deployment planing in terms of understanding whihc orchestrationn is dependent on what endpointso

Building the Sample

Run the followinf SQL script on the BizTalkMgmtDB, the only prerequisite is you should be the part of BizTalk adminstrator group? 

924789.png

DECLARE @SendPortAddress TABLE 
(nID intnvcAddress varchar(256)) 
 
Insert @SendPortAddress (nIDnvcAddress) 
 
select nSendPortIDnvcAddress 
From BizTalkMgmtDB.dbo.bts_sendport_transport AS spt WITH (NOLOCK)  
where len(nvcAddress) > 0 
 
SELECT o.nvcName AS 'Orchestration Name' 
, ao.nVersionMajor 
, hst.Name AS 'Host' 
, op.nvcName AS 'Orchestration Port Name' 
, case when (sp.nvcName is null and rp.nvcName is Nullthen spg.nvcName 
  else sp.nvcName 
  end as 'Send Port' 
, spt.nvcAddress as 'Address' 
, rp.nvcName as 'Receive Port' 
, app.nvcName as 'Application Name' 
, o.nvcFullName 
, ao.nvcName as 'Assembly' 
FROM BizTalkMgmtDB.dbo.bts_orchestration_port_binding AS b WITH (NOLOCKINNER JOIN 
     BizTalkMgmtDB.dbo.bts_orchestration_port AS op WITH (NOLOCKON b.nOrcPortID = op.nID INNER JOIN 
     BizTalkMgmtDB.dbo.bts_orchestration AS o WITH (NOLOCKON o.nID = op.nOrchestrationID INNER JOIN 
     BizTalkMgmtDB.dbo.adm_Host AS hst WITH (NOLOCKON o.nAdminHostID = hst.Id INNER JOIN      
     BizTalkMgmtDB.dbo.bts_assembly AS ao WITH (NOLOCKON o.nAssemblyID = ao.nID LEFT OUTER JOIN 
     BizTalkMgmtDB.dbo.bts_application AS app WITH (NOLOCKON ao.nApplicationID = app.nID LEFT OUTER JOIN 
     BizTalkMgmtDB.dbo.bts_receiveport AS rp WITH (NOLOCKON b.nReceivePortID = rp.nID LEFT OUTER JOIN 
     BizTalkMgmtDB.dbo.bts_sendportgroup AS spg WITH (NOLOCKON b.nSpgID = spg.nID LEFT OUTER JOIN      
     BizTalkMgmtDB.dbo.bts_sendport AS sp WITH (NOLOCKON b.nSendPortID = sp.nID LEFT OUTER JOIN 
     @SendPortAddress AS spt ON sp.nID = spt.nID 
Order by app.nvcNameo.nvcNameao.nVersionMajorop.nID

Publishing Schemas As A Webservice With Exception Handling


Where you need to publish an schema or orchestration as service:
Scenario where the communication has to happen in real time for example someone wants to call an orchestration from an UI or wants to post a request to BizTalk.

How do you Do That:

Develop a schema project as usual. If required you can pick an orchestration as the subscriber or leave it. Since you are publishing the Schema it will be loosely coupled and you can have multiple subscribers to the same message type.

Once done please go ahead and deploy the schema/solution.(In the case of orchestration you will like to have your activate receive  shape directly bound to msgbox) Also please consider having a fault port in your orchestration. That way you will be able to return a custom exception in case of any issues from BizTalk.

Some Unique Errors You May Encounter are:
The Messaging Engine failed to register the adapter for “WCF-BasicHttp” for the receive location : Solution your app pool id has to be part of BizTalk Admin group.
must receive before sending a message whose message type corresponds to a request response operation on an implemented port.You need to move your second send shapes under a condition branch as shown in the picture.

Demo:
This scenario we are going to design a synchronous BizTalk service which will return a response for each request.
1> Define your request response contracts.For this example I am going to create an orchestration which will consume the request and create a response. Make sure the activate receive shape is directly bound to msgbox.
This is how your solution is going to look like.
Orchestration

2>Once done please go ahead and deploy it.Start the orchestration.

3>Go to BizTalk WCF publishing wizard and follow the following steps( BizTalk WCF publishing Wizard can be found in Visual studio for 2009 or in the Start>>program >> Microsoft BizTalk server >>). I am going to design a HTTP based 2 way communication here. Please follow the following steps as showed in the screenshot below.

note:We can also publish an orchestration as a web service but using schema gives you loosely coupled architecture. In the case of schema you can define multiple subscribers at your convenience.

WCFPublish

Once you are done with the Wizard it is going to create one receive port and location + one IIS application.

The app pool for the BizTalk application in IIS has to be a part of Admin group and you will also need to enable directory browsing.

BTIISApplication

WCFWiz_RcvPort

WCFWiz_RcvPort2

4>Once done this is going to create a two way receive location and port. Please go ahead and start that. The solution is ready to be tested.

How To Test:
1>Go to your IIS and take the WCF endpoint. You can use this endpoint in SOAP UI/Stylus.. to post a request and get the response.

2>Using A Windows Client:

WCFClientBasic

Once the project is created right click on references and pick add service reference

WCFClient2

This step will create a service reference for you. Now open the program.cs and make necessary changed to it.

WCFClient4

Run it and you will see the BizTalk process invoked.

InvokedBTAdminConsole

D Goins Insperience

Technological outformation for this day and age

[INACTIVE BLOG] Connected Thoughts - Thiago Almeida

Connected systems and the occasional picture

Uri Katsir's Blog

BizTalk , BizTalk RFID and .NET

prashantbiztalkblogs

My BizTalk Experiences

BizTalk Server Tutorial

BizTalk Server Concepts and Common Errors

Extremely Talented Monkeys

A Technical Blog by Ed Jones: Azure, .NET, BizTalk, WCF, and SQL Server

On All Things Web

Discussing web development without limits

Vikas Bhardwaj's Blog

Articles about BizTalk Server and .NET C#

Cloud develop

a blog about (cloud) development... because I'm a nerd

Hooking Stuffs Together

My learning logs from day to day work experience about Integration platform using Microsoft technologies.

Connected Pawns

Mainly BizTalk & Little Chess

jhelatissimo

a blogger in the process.

MS Innovations Blog

Tips, Tricks, and Workarounds for BizTalk and other Microsoft technologies

srirambiztalks

Katradhu Kaialavu,Kalladhadhu Ulagalavu!

Vijay Microsoft Technical

BizTalk, WCF, ESB ToolKit, Windows Azure

Mind Over Messaging

Musings on BizTalk, Azure, and Enterprise Integration