Look Ma ! An evil middleware product!

I couldn’t resist commenting on this issue. I was just doing some final prep for my VBUG talk tomorrow and came across Richard Hallgren’s oddly titled post – Does BizTalk have man-boobs?. Richard writes about a QCon webcast of a session done by Martin Fowler and Jim Webber and writes

“Their main point is that we use much to bloated middleware (BizTalk is mentioned as an example here) and that we should have a more agile approach to implement SOA and ESBs. They’ve used all the agile ideas (testing, isolated testable functionality, deliver small and often, continuous builds etc, etc) and applied them to integration..”.

Richard goes on to make some good points which I totally agree with. Check out the post for details. I commented on the post and decided I would make those points again here and add a couple more, which is also quite good timing (for me) considering I’m speaking on BizTalk tomorrow.

So, I agree that BizTalk is totally unsuitable for small situations and if it is naively (read – most often) used without any performance tuning whatsoever, then its latency can be bad. But if its tuned, it can totally rock (and I know this from MCS colleagues who’ve experienced it in very large BizTalk projects).

BizTalk is big. Yes, totally. But it has to be because it addresses a vast set of use cases. Same goes for its competitors like TIBCO, webMethods etc. If you dont want all the features, dont use them. They are in the box anyway and they wont slow you down if you dont use them. You want content based routing – its in the box; you want long running business processes – check, you want ‘aspect oriented’ interception of messages for tracking , you’ve got BAM. You want monitoring tools – check. You don’t want tracking – turn it off. You dont want business rules – ignore the BRE. The list goes on.

There’s also a case to be made for using something like the ESB toolkit to give you a decent jumpstart on your routing infrastructure.

If you only have a couple of systems to integrate (and you are very sure there wont be more) then go ahead and custom code it. Its not worth buying a product for that.

The problem I have is with some “agilists” who, it would seem, want to custom code every darn thing under the sun. As I commented on Richard’s post, the irony is that the very same folk will then stress the importance of having good, robust pre-built frameworks and good tools to help with the “agile” approach. But wait, those tools have to be on the “acceptable” short list. Take NHibernate for example. An excellent tool, no doubt. And for many folk, since its open source, so it must be sent from above, totally divine, but not BizTalk. That’s way too big. And besides, its from Microsoft. Gasp! horror! It must be evil !!

Some take the view that its about having confidence in your code. I can understand that. Having been exposed to a fair amount of TDD etc, I can attest to the feeling of security when your edge cases have all been tested and you see all those “green signals”. But in an integration scenario where use cases are similar , how many times will you write a test, write the code, refactor, refactor, blah.. till the code comes out of your ears. Pretty soon customers are going to wonder how many times they have to pay for something thats already been written.

If Ican have confidence in code I write, I can have the same amount and more in a product thats been tested out in hundreds of more scenarios than I could imagine. Do these products (such as Biztalk and its competitors – I’m not selling anything here ) have bugs? Well of course they do. But in many cases, you can be just as confident in commercial closed source as in open source. Besides, if pre-built tools werent a good idea, there wouldnt be any open or closed source tools would there?

I’m not partial. Go in for the Neuron ESB if you a want a pure WCF ESB, or go in for something thats gaining a good reputation like NServiceBus. Just don’t give me that story about having to write a 100 tests first and inherit a dozen interfaces before I can deliver anything of value. Bah! Humbug!

Ok, so enough of the rant.. Got to go and write some custom components now 

Speaking at VBUG (Bracknell)

I will be presenting an introductory session to Biztalk 2006 R2 (and spending some time on the WCF adapters) at the local chapter of VBUG in Bracknell. Of course, we will also spend a little while discussing the impact of Oslo and Dublin and the factors to keep in mind when choosing your integration technologies.

Despite the fact that it has been around for ages and has IMO a lot of visibility, it seems that BizTalk Server still remains a sort of niche product with a lot of developers totally unfamiliar with it and what it could give them. The last time I presented on Biztalk,WCF and WF at the VBUG conference  in 2008, we found that a significant number of the audience didn’t know anything at all about BizTalk and from discussing with contacts in other countries as well, it seems that its a familar situation . Well, anyway, I hope I can do something in my own small way towards rectifying that.

So if you’re in the area and Biztalk is unfamiliar territory why not check it out?

BSF meets Deployment Framework

This post is just to call out some work that Bram Veldhoen has done in linking the Biztalk Software Factory to the venerable Deployment Framework. One thing to note is that the integration was done to the NAnt version of the framework prior to the latest MSBuild version of the framework (5.x). I guess linking into the new MSBUILD version is next on the cards and I’ve chatted to Bram about the possibility of doing something jointly (as if i didnt have enough irons in the fire with MockingBird and BizUnitExtensions ). Check it out and post feedback on the BSF codeplex space.

Been there, done that, got the certificate!

Well, I finally overcame my continued procrastination in this area and sat the MCTS: BizTalk exam today. I always knew I was ‘certifiable’ and now I got the paper to prove it   I cleared it with a score of 899 out of 1000. I’m quite chuffed with it. It revealed a couple of areas I need to work on a bit more but it was a good experience overall.

I know a lot of people debate the merit of certifications in general (and there has been lots of debates on this particular one as well), but I’m not going to re-hash them here. My aim in writing this was to give me a spur to read through areas that I havent had the opportunity to work in and also in a depth that may not have been necessary in the past. As Saravana points out , if you seriously study for this, you get a good feel for the breadth and depth of the product.

Studying can never take the place of actual hands-on experience, but with any large framework or product its common to find yourself working in a ‘niche’ area and never having the chance to explore all that it offers. Biztalk being a huge product, it’s quite ‘normal’ to find developers who have never had to use things like TPM/RoleLinks or the BRE and even BAM (actually, i think BAM is criminally underused, but thats a topic for another post). The study process can open up areas for you to look into and improve your productivity which you may not even attempt otherwise.

I have been quite selective in my approach to certifications thus far, the only one I bothered to write was 70-320 (Building Web Services and  Server Components), that too 4 years ago, but at the end of that study process I was totally amazed at the breadth and depth of the .NET stack and was far more confident of my grasp on Web Services, Enterprise Services, Remoting etc than I had been till that point. I think that going forward that (selective approach) will continue. The R2 exam is now available so that’s next on the agenda and working on the WCF exam should be a useful process as well (assuming I ever get off my lazy backside and take them :-) )

I thought that Id’ just point out a few resources that were of great help.

• Saravana Kumar’s awesome 70-235 Exam Preparation Diary.
• Professional Biztalk - the best explanation of BAM, ever . Period.
• The BAM Training Kit - specifically the hands on exercises. The training material itself is pretty complex in some areas and I personally didnt get much out of those explanations, but the practice was good.
• For BRE, I honestly couldn’t find any single resource which allowed me to get to grips with it. There were some nuggets scattered here and there. I ended up writing my own study paper where I combined notes from all the books and whitepapers into something I could revise easily. The highlights of all the resources were the awesome BRE Tips and Tricks – MSDN Webcast – by Jeff Seagard and the MSDN BRE Walkthroughs . The MSDN exercises were exceptionally good. Saravana also points out (in his post) another techNet webcast on the BRE, but i didnt get a chance to watch that.
• Biztalk 2006 Recipes – I used this for a quick final review of BAM and BRE, Orchestrations etc.

So, if you are considering writing the exam, I’d say go for it. Working on the numerous areas involved can only make you a better developer (or at least a more ‘aware’ developer). And when you are done with it, look for areas that you can feed back the learning gained into your daily development tasks.IMO, the real payback is there. Good luck.!!

BizTalk Testing and Mocks

In an earlier article , I had briefly mentioned that some folk had used mocks with Biztalk, notably to test pipeline components. Since I didnt have the bookmarks at hand then I didn’t provide the links, but I have since found the links again so here they are (and I can also now use this as a note to self if I want to refer to them again or expand on any of the material they have written).

• Testing Pipeline Components – The Pipeline Testing Library – Nick Heppleston
• Unit Testing Pipeline components with Mocks - Rido
• BizTalk Testing Series – Testing Pipeline Components – Mike Stephenson
• TDD and BizTalk Orchestration Expression Code – Matt Podwysocki
• Rhino Mocks and custom BizTalk code revisited   – Matt Podwysocki
• Biztalk TDD – Pipeline Components - Arun Gopalam

While the  blog posts pointing to the Pipeline Testing Library are useful, if you want to go straight to the source, check out the WIKI page that Tomas has set up on GitHub. That page has more samples on how to use the API.

I’m going to have a play around with MoQ and pipeline components in the next couple of days . I think MoQ’s approach is a bit more elegant than Rhino (particularly, the absence of record and replay). I’m also going to link into Tomas’s excellent pipeline testing library from BizUnitExtensions. This has been a long overdue item on my roadmap.

UPDATE: Bram Veldhoen has already done some work on linking the Pipeline Testing Library into BizUnit and has very graciously contributed his code to be put into BizUnitExtensions so that will be released soon with Extensions 3.0.

Enjoy the links and if you find others of a similar ilk that are also useful feel free to put them in the feedback section here and I will update the post.

VS2008 – Generate XML Instances

It’s funny how we take things for granted. As Biztalk developers, we get used to the idea of being able to right click on a schema and generate an instance . In non Biztalk projects however, this couldnt be done. Till now.

I was playing around with writing some XML Instance Generation for MockingBird to finish off the next release and spent a lot of time poking around the Schema Object Model etc. While doing that, I quite accidentally opened the XML Schema Explorer tool window. Now I had seen that in the past and navigated through some types etc (and thinking it was just a simple add-on to the old VS i kind of took it for granted and didnt investigate further). 

What I did not realize is that for Elements, you can generate sample instances.  Check out the following screenshot.

Generate Sample XML

As you can see from the tree behind the popup window, elements are colored differently as well.

This is a great time-saver. I think this can be done only in VS2008 SP1.

Unfortunately, the downside is that there is no API into this tool-window (or rather, the library behind it), so we cannot programmatically generate instances in bulk. Also it will not open WSDL files, so you have to extract the XSD from the WSDL (if not already available separately) in order to work with this tool window. But i think its cool as we no longer have to depend on third party XML editors to get sample instances.

By the way, if you are looking for help in this area (of instance generation) , there is some sample code available in the MSDN article, Generating XML Documents from Schemas which is quite well written. While there are license restrictions on modification/ derivation (and then redistribution), plain redistribution without modification, I gather, is permitted, so the easiest thing for MockingBird would be to just redistribute the binaries of that sample with the GUI. No sense in reinventing the wheel.

In terms of the Biztalk Schema Editor and its instance generation, if any Biztalk folks know if there’s a programmatic way of doing that, please let me know  (Update: I mean specifically for 2006 and R2). I did a lot of digging around in the Developer Tools folder for an assembly that would allow it, but all the classes were internal. I did come across one public class finally (dont remember the assembly off the top of my head now), which had a public method but required some interface to be passed in but didnt work when i tried calling it from custom code. It would be useful to do this programmatically so we can generate instances in bulk for  a given set of schemas (useful when updating instances to correspond to schema changes etc). So, if you’ve managed to do this and are happy to share info then drop me a line.