Santosh Benjamin’s Weblog

February 1, 2010

BizTalk Documenter v3.3

Filed under: Biztalk, Tools — Tags: — santoshbenjamin @ 10:57 pm

BizTalk Documenter has been updated to v3.3. This being a point release, there is still no links to other tools such as MapDocumenter, but thats on the list for the upcoming combined Documenter and Profiler.

The main changes are

  • The UI now allows selecting “Word” as an output option. This was always available in the command line but never surfaced in the WinForms.
  • Enhancements to the command line so along with specifying “/defaults” you can also specify “/apps: (comma separated list of target apps)” and “/provider:word” The default forced you into CHM earlier).
  • Bug fixes to the Word publisher (reported in the forums).

In the case of the Word output, there are no images of the orchestrations since we are actually writing Word 2003 XML directly and launching Winword.exe so there is no control over the transformations. I’ll take a look at including images but I’m not promising anything since I need to use the available time to concentrate on the new version where I plan to use the Open XML SDK.

Also note that the MMC SnapIn wont compile in BizTalk 2009 environments because the MMC API has now changed. The SnapIn is deprecated and will not be available in the next version. If you absolutely must have it, then let me know and I will keep it on the task list for upgrading but again, time is limited so no promises.

Hope you find the small enhancements useful. Check it out and let me know if you run into any issues.

AppFabric lands in VBUG (Bracknell)

Filed under: AppFabric, User Groups, WCF, WF, Workflow Services — Tags: , , , — santoshbenjamin @ 10:47 pm

Hot on the heels of my previous talk on AppFabric at SBUG, I will be reprising the talk, but with more of a Workflow Services slant at VBUG in Bracknell on Feb 15th 2010.

A big thanks to all who attended the SBUG talk and cool to see the large Avanade contingent comprising many familiar faces  :-) . It was an interesting conversation on BizTalk and AppFabric. I will be posting more content on those topics here soon. I’ve sent the slides to Mike Stephenson and they should be available on the SBUG site soon.

I hope to be able to do more of a hands on demo of Workflow Services at the VBUG event (but that depends on Windows Virtual PC behaving properly). Still, it should be a good evening. Again, lots of known folk at VBUG and hope to see a good crowd.

January 18, 2010

MockingBird – The Roadmap

Filed under: AppFabric, BizUnit, MockingBird, MockingBird Diary, Tools, Workflow Services — Tags: , , — santoshbenjamin @ 11:20 am

I’m really chuffed to have not one but 3 new team members on board the MockingBird project. Welcome to Shen Chauhan, my MCS colleague and WPF whizz-kid, Bram Veldhoen and Cormac O’Brien.

So, whats cooking for MockingBird in 2010 ? I have a veritable laundry list of features for the next release(s) but here’s the broad themes we are working towards.

(1) WCF : The first is the port to WCF which is long overdue and Bram has been hard at work in getting this off the ground so I expect to have something to announce soon. We will be implementing a message interceptor / router with a generic contract and from there leverage the simulation framework that is currently available with (hopefully) minimal changes. The challenge will be in the area of bindings since most teams customize bindings heavily so the service will need to support custom bindings.

(2) REST, XML/Http etc: Here we are looking at moving beyond just SOAP based services to plain XML/Http services and even REST based services. Cormac had come across some non SOAP services recently for which he had to manually code a mock, so we will be looking to incorporate such functionality here.  REST is an area that I haven’t paid much attention too so i see  a crash course in REST around the corner. If you have experience with REST and are keen to see support for this in MockingBird and can contribute in some way, do drop me a line.

(3) Generic ‘Action’ processing: For processing requests and building responses, there needs to be a more generic approach. Currently there is support for running an XPATH query on the request and returning a response based on the result of the XPath expression., But this is only one solution. What if we wanted to do a custom evaluation of the request where XPath is not enough? Further for the response, what if we wanted something more dynamic than just pointing to a specific response file? for example, Cormac suggested we dynamically execute an XSLT against the request. So, for this we need to look at the system executing a supplied “IAction” (IRequestAction, IResponseAction) and behaving accordingly. Unity and MEF should help here.

(4) Dynamic Mocking With BizUnit: This would be an API (+ a BizUnit step) which reduces the amount of upfront configuration needed. I envisage being able to say “For this endpoint, at the next request, return “X” response (and extend it to say, “do that for the next 10 calls or forever etc). We should then be able to “assert” that the system did indeed send the response. This would require surfacing the configuration file management code so that a config file could be written on the fly by the API/ Test step.

(5) .NET 4, AppFabric and Workflow Service: This is probably a longer term (MockingBird v3) where I am considering implementing the handler as a dynamic workflow service and using XAML. Now with this approach all the header parsing, IAction components etc would all  become custom workflow activities and it would be possible to dynamically compose specialized handlers on the fly. The message interceptor component would move to the official WCF Router.

So these are the big ticket items. For all of them I would appreciate your feedback as to what would give you the most value. We would also welcome more participation and contribution of ideas/code.  Looking forwad to your feedback.

Speaking on AppFabric @ SBUG

Filed under: AppFabric, General, User Groups, WCF, WF, Workflow Services — Tags: , , — santoshbenjamin @ 9:38 am

I’m delighted to be delivering my first presentation for the UK SOA/BPM User Group next week on the topic of Windows Server AppFabric (formerly code-named “Dublin”). The link to register is here and the page will be updated with the abstract shortly.

The tight integration between WCF and WF 4.0  and the hosting support in AppFabric is looking very promising now and will definitely provide a good platform for solutions which dont require BizTalk and, where BizTalk is already available, can work well alongside and integrate with it.   We’ll look at some scenarios where there may be overlap and how to approach the solution in such cases. Sure there will be some good discussions !

I know Yossi will be there :-) . Looking forward to catching up with some of the other UK / London gang as well.

December 9, 2009

BizTalk Documentor Update

Filed under: Biztalk, Tools — Tags: , , — santoshbenjamin @ 9:51 am

Darren has made available a new release (v.3.2.1) of the BizTalk Documentor. It fixes some long standing issues and also some issues that came up when used with BizTalk 2009. Check it out and let us know if you run into any problems.

We are planning some re-work of this tool as well as the venerable Orchestration Profiler  with a view to combining these tools and introducing new output types. We are also looking at fleshing out an API so that they could be used with other tools or in a programmatic way as well. For instance, in an earlier post,  I had voiced a wish to be able to link OrchProf with BizUnit so we could verify step execution etc directly from a test and make the test a bit more “white box”  (ok, so it wont be as whitebox as BizMonade but it could add more reach for BizUnit)   but at present there isnt an API that is surfaced to allow such a link. But this will change soon. 

I have some ideas for other features as well which I will write about soon and there are a couple of requests on the forum to be able to link the documentor with BizTalkMapDoc which I will look into.

So if you have any feature requests, feedback and other ideas on how these tools could work together in your projects do send them along, preferably on the CodePlex workspace forums or feel free to contact me directly through the blog and let me know.

November 28, 2009

Using INTERSECT with LINQ to XML

Filed under: Coding, General, LINQ — Tags: , , — santoshbenjamin @ 6:21 pm

In terms of hands-on coding (not general awareness) I’m a bit of a newbie to the world of LINQ actually, having only dabbled with a little LINQ to XML in MockingBird and even there I wasnt too impressed with it in the area of XPath queries. But I came across something yesterday that is a testimony to the power of LINQ.

My scenario was that I wanted to compare two XML documents that followed the same schema, but I wanted to do this in  a fairly generic way without writing code to explicitly pick up every element in the hierarchy. My requirement was to find all common elements between the two documents and also the elements in one and not the other.

Take the following example:

Document-1
<Authors>
  <Author ID=”1″ Name=”AuthorA” JoinDate=”3/1/2009″/>
  <Author ID=”2″ Name=”AuthorC” JoinDate=”3/1/2009″/>
</Authors>
Document-2
<Authors>
  <Author ID=”1″ Name=”AuthorA” JoinDate=”3/1/2009″/>
  <Author ID=”2″ Name=”AuthorB” JoinDate=”3/1/2009″/>
</Authors>

I quickly found that LINQ has this powerful INTERSECT function which would allow me to find the common elements and the EXCEPT function which will find the distinct elements.

My first attempt (at finding the common elements) was like this:

var commonFromA = aDoc.Descendants(“Authors”).Intersect(bDoc.Descendants(“Authors”));

But this did not work. After much more attempts and discussions with colleagues, it was beginning to look like i could only use INTERSECT with native types and I would either have to write a custom IEqualityComparer<T> or write more complex code involving anonymous types (which are , by the way, a brilliant feature of the framework).

But LINQ is supposed to be elegant, right? So I posted the question on the MSDN Forums and got an immediate reply from Martin Honnen   a MVP in this area, and yes, the solution was elegant and just in one line.

var commonFromA = aDoc.Descendants(“Author”).Cast<XNode>().Intersect(bDoc.Descendants(“Author”).Cast<XNode>(), new XNodeEqualityComparer());

As Martin explained, the set operators like INTERSECT and EXCEPT work on object identity not value comparisons and as I had distinct XElement objects in different documents my initial attempt would not work. However, the XNodeEqualityComparer comes to the rescue and casting the XElement to XNode was all that was required.

What’s even more interesting is that in .NET 4.0, we have something called “contravariance” which will allow the INTERSECT code above to work without the explicit cast. Martin explains this very well in this post on “Exploiting Contravariance with LINQ to XML”. I always wanted to understand what Covariance and Contravariance were all about and this is a great explanation.

Essentially, with Contravariance, you can pass in the base type XElement even though the comparison (with XNodeComparer) is expecting an XNode , (the derived type) and you dont need to mess with casting etc. With Contravariance you are also not mutating the object itself (actually, you cannot change the object) so this works.

On the same subject, also check out Eric Lipperts blog article.  I had come across that post earlier but didnt have any immediate need for that functionality so I didnt pay attention, but this time, I did.

So, there you have it. A one line solution for comparing XML documents. (The “EXCEPT” code was also one line). Of course if you want to find out specific attribute values and changes, then the code becomes more involved, but you’ve gotta admit that this is elegant. Can you imagine how much code this would need in the Xml DOM world!!

I’m starting to get hooked on LINQ!  :-)

BizTalk, WCF and the Bad Gateway

Filed under: Biztalk, Biztalk Adapters, WCF — Tags: , , — santoshbenjamin @ 5:52 pm

We ran into an interesting problem a couple of days ago and I decided it would be worth posting the solution in case anyone else runs into it.

The scenario involves a Windows Service communicating with a BizTalk WCF endpoint which is hosted using WCF-CustomIsolated and some fairly complex custom bindings, notably using X.509 Mutual Certificates. This is the second half of the communication, the first being where BizTalk sends a request to a vanilla WCF endpoint and gets an ACK, the service does some processing and the WinService then does a “call back” with the “real response” / outcome of the processing.  Both communication routes are secured via the same bindings. We didnt have any trouble with the transmission from BizTalk to the WCF service but the WinService initiated communication always failed with a HTTP 502 : Bad Gateway message. This had us stumped for a while as some articles we read seemed to suggest that there was a problem on the server. Now if you have looked into custom bindings and certs in BizTalk, you will know that there is some really heavy duty and scary stuff there so we wondered whether the way we had setup the certs and credentials info was playing havoc with the communication.

So I used the trusty WSCF.blue and mocked up another WCF service to mimic the BizTalk endpoint replete with the bindings (no, WSCF Blue doesnt support custom bindings yet, but will sometime soon) and swapped the URI in the WinService and it worked just fine. Still puzzled, I then created a console app to mimic the winservice and it was able to communicate with both my mock BizTalk service as well as the real endpoint. Curiouser and curiouser. At that time the only difference was in the App Pool between the BizTalk service and mock service, but i used the same app pool and still the problem only manifested between the Winservice and Biztalk.

Anyway, with some guidance from an expert colleague (who, unfortunately wasnt around when I was writing those mock clients and services) It turned out that the problem was with a setting “UseDefaultWebProxy”  in the Win Service. The difference between the WinService and the console client was that the console was running under my credentials and in the IE settings I had turned off the proxy for the specified URL so although I didnt specify the setting as “false” in the client it automatically picked up my settings. But the WinService was running under another user account, and so the system didnt care about my proxy settings and we had to explicitly set the value to “false”. 

That was it. A little setting buried deep in the bindings that caused us the grief. After sorting this out I stumbled across this post from Kenny Wolf  which points to the offending setting. Wish I had found that earlier, but then I wouldnt have realized that aspect of the console client where it picked those settings. So, hope this helps someone in a similar situation.

November 21, 2009

MockingBird v1 RTM

Filed under: MockingBird, Tools — Tags: , , , — santoshbenjamin @ 10:36 pm

MockingBird v1 is now formally released. With the squeeze in available time for personal projects, it’s taken me nearly 6 months since the beta and only recently did the site see any check-ins. There’s quite a few updates, and enhancements to the suite. Actually the beta was pretty stable with hardly any bugs that needed to be fixed but there were a few feature requests and to be honest, I started getting rather impatient to move to WCF but needed to formalize the v1 release first. Here’s a laundry list of all the features of the new release. The documentation, included with the package has also been substantially updated.

This release will be the last to support the ASP.NET HttpHandler. V2 will be completely revised to support WCF. The change in the engine should be transparent to the end user and of course with WCF we will get a lot more benefits including transport independent request interception. It will also be the last to support an independent “Studio” module as I have some plans for a brand new UI

Simulator functionality changes

· Now supports Enterprise Library logging as well as log4net

· Logging all request and response operations.

· Support for "mapped soap Actions" (when soapActions do not correspond one to one with operation names or when the default parsing of  soap actions cannot be done)

· Supports mapping of operations to request element root names (when soap actions are all the same, such as in Apache Axis web services)

Simulator Packaging changes

· Config for unity and services now externalized

· MockingBird system config now externalized

Simulator Internal changes

· System Configuration Manager abstraction layer for config data storage

· StrongTyped application configuration.

· New OperationNameInferenceEngine

· Support for tokens and token expansion in configuration settings.

Studio Fixes

· Disabling of context menu on all nodes except for message level nodes.

· WSDL Browser "Save As " dialog now supports proper file masks (also takes in the service name and suggests the name for the file)

· No unhandled exceptions in the studio.

· Auto resize of all controls on form size change.

XmlInstanceGenerator Functionality

· Additional entry points to the XIG.

A big thank you to all who sent comments and gave me feedback. I hope you find the updates in this release to be useful.

V2 is going to be really big, so watch this space. :-)

For v2, There are a number of work items published on the CodePlex space  so please weigh in on them and vote for the ones that will be most useful to you and also please feel free to send me suggestions for other features as well as bug reports.

November 6, 2009

To DAL or not to DAL

Filed under: Architecture, Biztalk, Coding — Tags: , , — santoshbenjamin @ 9:17 pm

Do BizTalk consultants need to care about Data Access Layers? Does a BizTalk solution really need a DAL?  These are the questions that I’ve been mulling over in the past few weeks. Let me explain.

There are a couple of places where a BizTalk solution encounters a DAL. The first is where the DAL acts as an integration enabler. Here the endpoint of the LOB application we are integrating with happens to be a database. The second is where the DAL acts as a process enabler. Here the DAL provides the underpinning of the business process (that is, as part of the business process, it is frequently necessary to update a database with the state of the business document being operated on).

In my current gig, we are using both BizTalk and SSIS. SSIS is great for the ETL and various data related actions. BizTalk then takes over and passes the data to an LOB application doing various business processes as part of that communication. The nature of the processes is such that there is a significant DAL. Early on in the project we went through the usual debate on whether a custom DAL was necessary or if we should just use the requisite database adapters. Isnt the database adapter an obvious choice?  Maybe, or maybe not. In an earlier post , i talked about just such a situation a few years ago where we had choose whether to link directly to the DB or wrap the system in a web-service first and as i explained, things didn’t turn out the way they were expected to.

So, what are the considerations?

  1. Firstly, (as I explained in the post and the follow up posts) one of the key issues is the level of abstraction you are given. Especially when dealing with the scenario of integration enablers, a database endpoint is very rarely coarse grained enough to support a service oriented approach. Its more likely that you will be provided with CRUD level interfaces. Even if you decide to direct all communication via an orchestration that wraps all this, how does the orchestration actually call the backend system? Go via the adapter or use a DAL?
  2. For the scenario of process enablers, abstraction comes into play again. You don’t want to be cluttering up your orchestrations with bits and pieces of database schema related stuff. You could choose to wrap the database calls in a coarser stored proc but this leads to the next key point which is
  3. Performance. If you have a number of send ports (for all these stored procs) in the middle of your orchestrations, there is a cost associated with all those persistence points. If your transaction handling requirements permit, you could think about wrapping some of those calls in atomic scopes, but you have to be  very careful with this. If you do encounter an issue and everything gets rolled back, are your processes really designed to start at the right place all over again without compromising data integrity?
  4. If your DAL is designed well, your orchestrations will benefit from having to call methods on business level entities and, just from a persistence point consideration, will, in my opinion, be better off.
  5. Transaction Bridging : There were a few situations where we had to bridge the transaction across the database and the segment of the business process. Fortunately, the DAL being of extremely high quality (courtesy of an expert colleague) made this very easy to do.

But, having said all this, a DAL doesn’t come free. You have to write code. Sometimes lots of it. The more code you write, the higher the probable bug density. If the functionality can be satisfied with a code-generator then that will reduce the code you have to write, but it DOES NOT reduce the amount of code you have to MAINTAIN. I think many developers forget about this last point. I’m all in favour of code-gen, but don’t forget the maintenance cost.  (Further, if the functionality in the middle of your processes can be satisfied with boiler plate code, perhaps it’s an opportunity to question what it’s doing there in the first place. Can it be pushed to a later stage and componentized? )

I must confess, at one point, when wading through a sea of DAL code early on in the project, I was quite tempted to throw it all away and go for the adapters, but the considerations above outweighed the pain at that point. Now much later, with everything having stabilized, we know just where to go to make any changes and the productivity is quite high.

But I’ve seen cases where BizTalk developers didn’t care about the SQL they wrote and they ended up in a mess with locking and poor performance. And it takes a really good developer to write a first class DAL and having interviewed and worked with a number of devs I can say that its hard to find good skills in this area. Pop quiz: Do you know how to use System.Transactions yet ?  :-)

There is always the option of using something like NHibernate. If you use some coarse grained stored procs and some business entities, you could kill all the “goo” in the middle by letting NH take care of the persistence. That, i wager would reduce the bug count in that area. But watch out for the maintenance times and the bug fixing. When there’s a component in the middle that you don’t know the internals of, it can make life very hard when trying to track down bugs.

That leads me on to the point of making choices based on knowledge and not ignorance. If you want to adopt “persistence ignorance”, don’t do it because you cant write proper DAL code yourself. Do it for the right reasons.

So I hope the points above have given some food for thought. Custom code is not always bad as long as it is approached and implemented correctly. Whether you choose to use a DAL or not, do it with careful thought on issues like the ones above. As always, your feedback is welcome.

Technorati Tags: ,,

October 3, 2009

WSCF.blue v1 RTW

Filed under: WSCF — santoshbenjamin @ 11:47 am

I’m a little late in announcing this. WSCF.blue is now formally released to web as v1. Although its only been out 3-4 days now we’ve crossed over a 100 downloads already. Whew. Looks like a lot of folk were waiting for this release :-)

Christian Weyer has blogged about the release. I would also like to call out Alex for taking so much initiative and coding till 3am on some days (and emailing the group at that time too !!). We definitely owe this release to him.

Now on to v2. My next post will be about the roadmap and our shopping list of features for the next few releases. Check out the final v1 release. Hope you like it. As always, your feedback and suggestions would be most welcome and appreciated.

Older Posts »

Blog at WordPress.com.