Santosh Benjamin's Weblog

Adventures with AppFabric, BizTalk & Clouds

Look Ma ! An evil middleware product!

with 3 comments


I couldn’t resist commenting on this issue. I was just doing some final prep for my VBUG talk tomorrow and came across Richard Hallgren’s oddly titled post – Does BizTalk have man-boobs?. Richard writes about a QCon webcast of a session done by Martin Fowler and Jim Webber and writes

“Their main point is that we use much to bloated middleware (BizTalk is mentioned as an example here) and that we should have a more agile approach to implement SOA and ESBs. They’ve used all the agile ideas (testing, isolated testable functionality, deliver small and often, continuous builds etc, etc) and applied them to integration..”.

Richard goes on to make some good points which I totally agree with. Check out the post for details. I commented on the post and decided I would make those points again here and add a couple more, which is also quite good timing (for me) considering I’m speaking on BizTalk tomorrow.

So, I agree that BizTalk is totally unsuitable for small situations and if it is naively (read – most often) used without any performance tuning whatsoever, then its latency can be bad. But if its tuned, it can totally rock (and I know this from MCS colleagues who’ve experienced it in very large BizTalk projects).

BizTalk is big. Yes, totally. But it has to be because it addresses a vast set of use cases. Same goes for its competitors like TIBCO, webMethods etc. If you dont want all the features, dont use them. They are in the box anyway and they wont slow you down if you dont use them. You want content based routing – its in the box; you want long running business processes – check, you want ‘aspect oriented’ interception of messages for tracking , you’ve got BAM. You want monitoring tools – check. You don’t want tracking – turn it off. You dont want business rules – ignore the BRE. The list goes on.

There’s also a case to be made for using something like the ESB toolkit to give you a decent jumpstart on your routing infrastructure.

If you only have a couple of systems to integrate (and you are very sure there wont be more) then go ahead and custom code it. Its not worth buying a product for that.

The problem I have is with some “agilists” who, it would seem, want to custom code every darn thing under the sun. As I commented on Richard’s post, the irony is that the very same folk will then stress the importance of having good, robust pre-built frameworks and good tools to help with the “agile” approach. But wait, those tools have to be on the “acceptable” short list. Take NHibernate for example. An excellent tool, no doubt. And for many folk, since its open source, so it must be sent from above, totally divine, but not BizTalk. That’s way too big. And besides, its from Microsoft. Gasp! horror! It must be evil !!

Some take the view that its about having confidence in your code. I can understand that. Having been exposed to a fair amount of TDD etc, I can attest to the feeling of security when your edge cases have all been tested and you see all those “green signals”. But in an integration scenario where use cases are similar , how many times will you write a test, write the code, refactor, refactor, blah.. till the code comes out of your ears. Pretty soon customers are going to wonder how many times they have to pay for something thats already been written.

If Ican have confidence in code I write, I can have the same amount and more in a product thats been tested out in hundreds of more scenarios than I could imagine. Do these products (such as Biztalk and its competitors – I’m not selling anything here 🙂 ) have bugs? Well of course they do. But in many cases, you can be just as confident in commercial closed source as in open source. Besides, if pre-built tools werent a good idea, there wouldnt be any open or closed source tools would there?

I’m not partial. Go in for the Neuron ESB if you a want a pure WCF ESB, or go in for something thats gaining a good reputation like NServiceBus. Just don’t give me that story about having to write a 100 tests first and inherit a dozen interfaces before I can deliver anything of value. Bah! Humbug!

Ok, so enough of the rant.. Got to go and write some custom components now  🙂

Advertisements

Written by santoshbenjamin

February 23, 2009 at 11:14 PM

3 Responses

Subscribe to comments with RSS.

  1. […] BizTalk Linkflood, February 24, 2009 By Gerben Look Ma ! An evil middleware product! […]

  2. The problem is that BizTalk is really easy to abuse/misuse. A frequent scenario is an organization who doesn’t want any of BizTalk’s REAL benefits, but is seduced by the ambitious visions of executable-documentation in the form of graphical orchestrations. This then leads to complex solutions that suffer from BizTalk’s complexity while gaining none of the compensating advantages (“we don’t need long running persistent processes, BAM, BRE, tracking, we want nothing to do with asynchronous messaging nor message routing; but having drag-and-drop, that has serious value”). BizTalk offers many interesting tools, but its misuse can lead to instant legacy code if nobody stops an organization from taking its dreams of drag-and-drop too far (while neglecting the really useful tools it offers).

    About NHibernate, I don’t believe its main advantage is its open-sourceness. I’m not an expert in that tool, but I really like how its design allows quick TDD test/code/refactor cycles (when a “persistence ignorant” code style is used). Quick feedback cycles allow a developer to stay in context of the modification he was doing. This really helps maintaining “flow”, which brings more significant productivity enhancements than any wizard or drag-and-drop could provide. There’s no long compile/deploy/retest cycle between each modification, just do the modification and test it directly from the IDE to get almost instant feedback (no context-switching to other project tasks while waiting for the deployment to complete is required).

    A NHibernate based app can be designed in a way where dependencies (such as the database) are isolated as much as possible, so most tests can run without talking to the database. There are still integration tests to validate interaction with the database, but these tests are a minority so it is not as much a problem if they run a bit slower.

    BizTalk could be improved by learning from the design of these tools, for example by applying the “persistence ignorance”, or more generally, “infrastructure ignorance” principles in some of its artifacts, such as orchestrations. For example, an infrastructure-ignorant orchestration engine could substitute the persistence/dehydratation mechanisms with an in-memory implementation (within unit tests), to run unit tests quickly, without any interaction with the database or other external dependencies. Obviously, integration tests would still be needed to ensure the orchestration also works with the real BizTalk infrastructure, but again, they would be a minority of slower tests, which would be run only when the fast-running unit tests are all successful.

    In short, the agility doesn’t come from writing 100 tests before doing anything else (that would be waterfall-“TDD”/Big Testing Up Front), but more from TDD’s ability to enable flow/reduce context switching.

    Christos Karras

    February 25, 2009 at 5:35 AM

  3. Hi Christos,
    Thanks for the detailed feedback. I agree that Biztalk is misused and abused often and i suspect the way it was marketed orginally may have had a lot to do with that. But equally, I have seen situations where the developers/users simply do not understand that they are using a server product and expect it to perform at warp speed without tuning anything at all. But Biztalk is not bloated. The tools have a specific role to play. Very often integration solutions start by being a case of just do “A and B” and then “can we also do C, D ,E X, Y etc” and then you have all the code churn and reinventing a fairly complex wheel. This is where having all of those features already available in the platform means you can deliver even more rapidly , thus giving you a better chance to deliver business value and get customer feedback than by writing a ton of code.

    Re: NHibernate. I was actually using NHibernate as a “poster” for my comments. I do agree that even if it was closed source, its design is what makes it so successful.

    Re: Biztalk’s infrastructure: yes, it is coupled to the database tightly and it makes testing orchestrations difficult. But infrastructure ignorance alone wont solve the issue. (I have commented about that in another post – https://santoshbenjamin.wordpress.com/2009/01/14/on-the-opacity-of-orchestrations/). Besides the way the persistence mechanism is designed and the way the SQL engine is used i doubt the performance will be equal to what it is now with an additional layer of indirection. You also have to consider other things like BAM etc and the way they use SQL so, i’m not sure database independence would be a good thing technically. You would have to code to a subset of common features across popular databases.

    Re: TDD and context switching – I will actually follow this up with a post on the subject as this reply is getting rather long as it is 🙂

    Thanks again..
    cheers
    Benjy

    santoshbenjamin

    February 25, 2009 at 10:59 AM


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: