MockingBird@SBUG

The UK SOA BPM User Group (SBUG) is holding a series of mini-meetings over the next few months. These are short live meetings / webcasts. I’m delighted to be presenting my web-service mocking tool , MockingBird at the next mini-meeting to be held on the 29th July.

I will be discussing how MockingBird assists and simplifies integration testing with web services, how it is used in Biztalk and non Biztalk scenarios, how the system works, how it can be extended and the roadmap.

Please check the event link on the SBUG site for details.  All are welcome to join (no membership needed). Hope to see a good turnout

MockingBird – Beta

Whew! I’ve finally managed to get the first beta release of MockingBird out the door, almost 3 months after the alpha. The past 3 months have gone by in a blur with my first major project in MCS and the associated tight deadlines and long hours. Technically its been a good learning experience, but it doesnt leave any time at all for side projects like this one, but I’ve managed to steal the odd weekend here and there and get this done.

The key elements of the beta release are

• The branding and naming is now beginning to work out quite well. As I had written earlier, it started life as MockWeb, then became MockingBird and now I have decided that I like the name “MockingBird” so much its worth promoting to the ‘brand’ or the name of the toolset. The constituent applications now have their own names. The web application with the HttpHandler is now called the “Service Simulator” and the main WinForms application is now called the “Service Studio” which will hold many modules and plugins.
• The Service Simulator has been completely revised using Unity IoC. I feel a lot happier with the design now and i think its more loosely coupled and the DI /IoC has a good side effect in increasing the test surface. The Service Studio doesnt use the Unity configuration yet, but will do so by RTM.
• The little ‘Driver’ GUI of the alpha has now been rolled into the “Service Studio” and is officially called the “Service Invoker”. The layout has changed and now uses tabs so occupies less screen real estate.
• There is a new “WSDL Browser” module in the service studio that allows you to load up a WSDL and it will parse the WSDL for you and list all the operations and allow you to generate sample request and response messages.
• The WSDL  Browser in turn makes use of an Xml Instance Generator module that provides a more usable facade over a sample from the MSDN archives. The XIG module can stand on its own and will grow in future releases.

The design principle of the Service Studio is that every piece of functionality must be accessible independently and not depend on a UI. This is one of the reasons it took so long to get this done. I came across quite a few samples of WSDL handling where the authors were willing to share and/or the license permitted extension, but the code was simply too deeply hooked into the UI and did not have any “API” to speak of. In the end I wrote the whole thing myself. I’m still looking for ways to plug it into related tools such as WSCF which I also participate in developing.

There are some limitations in the beta especially with regard to WSDL Parsing.

• The first limitation is that it cannot handle WCF multi-part WSDLs. This will be done by around RTM time or along with the V2 release as V2 should be fully WCF based.
• While developing i tested it with different types of WSDLs but in the final push I focussed on testing it with a local monolithic WSDL and so it may fall over with complex WSDL documents. The WSDL spec allows for a wide variation in the actual WSDLs so its not possible for me to cater to all possible kinds of WSDLs. I hope that if you come across an issue, then the source is easy to fix or that you can share the WSDL with me and I will try and sort out the parsing.
• I also only finalised the testing with a “local” WSDL on the file system. The “SimpleWSDLRetriever” class that I have implemented does not use DiscoveryClientProtocol (although i shall plug in a new class soon that does this) so it cant handle connecting to a non local service and downloading the contract and associated metadata. (As you can see, I know how to address the limitation – just havent had the time to write it   ).

Anyway, those limitations aside, I’m happy with the code base so far and it really is quite stable and if you need to extend and use the source, i think the current structure is easy to understand. On the codeplex space I have written more documentation on how the internals of the system work so hopefully that will be useful to you. If you do end up extending it, I hope you will let me know what you are doing with it and maybe I can incorporate some of those extensions if it will make the product more generic.

As always, please send me feedback, especially if you run into any bugs. Hope you like the new release.

BizTalk Testing and Mocks

In an earlier article , I had briefly mentioned that some folk had used mocks with Biztalk, notably to test pipeline components. Since I didnt have the bookmarks at hand then I didn’t provide the links, but I have since found the links again so here they are (and I can also now use this as a note to self if I want to refer to them again or expand on any of the material they have written).

• Testing Pipeline Components – The Pipeline Testing Library – Nick Heppleston
• Unit Testing Pipeline components with Mocks - Rido
• BizTalk Testing Series – Testing Pipeline Components – Mike Stephenson
• TDD and BizTalk Orchestration Expression Code – Matt Podwysocki
• Rhino Mocks and custom BizTalk code revisited   – Matt Podwysocki
• Biztalk TDD – Pipeline Components - Arun Gopalam

While the  blog posts pointing to the Pipeline Testing Library are useful, if you want to go straight to the source, check out the WIKI page that Tomas has set up on GitHub. That page has more samples on how to use the API.

I’m going to have a play around with MoQ and pipeline components in the next couple of days . I think MoQ’s approach is a bit more elegant than Rhino (particularly, the absence of record and replay). I’m also going to link into Tomas’s excellent pipeline testing library from BizUnitExtensions. This has been a long overdue item on my roadmap.

UPDATE: Bram Veldhoen has already done some work on linking the Pipeline Testing Library into BizUnit and has very graciously contributed his code to be put into BizUnitExtensions so that will be released soon with Extensions 3.0.

Enjoy the links and if you find others of a similar ilk that are also useful feel free to put them in the feedback section here and I will update the post.

Waltzing with WSDL

One of my immediate goals for MockingBird is to have the “Configurator” UI done where you can just pick a WSDL on file or point to a URL and have the tool shred it and generate sample request and response messages into the correct folders (and generate the Handler Configuration) after which the user can then do any extra XPath based configurations to make the system more dynamic. Sounds easy enough, doesnt it? Well, thats what i thought too, until i delved into the intricacies of WSDL Parsing.

The class most people use is ServiceDescription and to be fair, it gives some information about the WSDL but navigating it is not easy at all and its not all that intuitive after you get to the point of listing all Operations inside the PortTypeCollection. There is also another big issue with trying to use ServiceDescription alone and that is the fact that WCF by default gives you multipart WSDLs. So getting hold of the WSDL from a WCF endpoint is not straightforward but one class that comes to the rescue is DiscoveryClientProtocol as Mike Hadlow points out in his article. This behavior can be changed as pointed out by Tomas Restrepo in hs post on “Inline XSD in WSDL with WCF” and Christian Weyer takes it up a notch in his article on Flat WSDL in WCF.

Anyway after pondering this a while, I decided it would be in my best interests to factor the code into a WSDLRetriever and a WSDLDescriptor. The Descriptor takes a dependency on WSDLRetriever (an interface of course, so its easy to mock) and expects  a single WSDL XDocument which it can work with. Now with this approach I dont care if the endpoint (if the WSDL is not on file) is a WCF endpoint or not, the Retriever does all that work. So the Descriptor can do its work and with that sorted i wrote up the first few tests for the Descriptor using a mock Retriever. So far so good.

Now, along with the requirement stated above, I also want to build a nice UI, where, you can not only do a single click to generate messages, but you can exercise finer control over the message content before storing them in the appropriate folders. So imagine if you will, a tree view which shows all the operations and the messages in them and immediately under the message name, the name of the complex type and additionally the body of the type (not in the tree node, but in an associated panel).

Of course, I’m not trying to write a completely generic WSDL parser, but something thats immediately usable by MockingBird and for other tools which can use a WSDL as input.

I had a look around at some blog posts and samples but none seemed to go as far as I wanted. I also took a look through the code for WSCF Blue  but couldnt make immediate use of that API. (I’ve since chatted with Buddhike about that so there may be something i could do with WSCF libraries in future). I looked at WebServiceStudio as the license allows reuse of the code there, but the code structure is simply awful. It works as is but everything is so deeply tied into the UI that it will take a feat of coding to extend it or pull out any reusable logic from it. I tried re-factoring a local copy but gave up pretty soon. So I had to work through the requirement myself.

Anyway, parsing WSDL is not easy at all. Once you get to the operation and message, you then have to wak through all schemas and their associated object tables to pick out the correct complex type corresponding to the message. This took me the best part of a few hours. Given that the user will not supply any namespaces, it gets very hard indeed. Thankfully the Schema Object Model is very decent. I used LINQ to XML for little bits such as extracting endpoint information (which also required me to resort to XPath extension methods because LINQ insisted i give it qualified names but I didnt know what the namespaces were !!) but for the schemas in WSDL, i think the SOM is better than LINQ. But I’m happy to be proved wrong!!

So I’ve got it to the point where it brings back the complex types. Now to generate sample instances from that for which I’ve got some resources. Once thats done and a simple Retriever implemented it wont be hard to put a UI on that because none of this is tied to the UI.

Thats it for now, a sort of ‘progress’ post for those waiting for the GUI for MockingBird and also to point out some of the dragons that await those who venture into the WSDL arena. Talk about ‘design by committee.. its a minefield!!

Couple of really good articles are Understanding WSDL by Aaron Skonnard (an old one, but an excellent read) and Walking the SOM by Stan Kitsis

Till next time

Introducing ‘Mockingbird’

In my previous post, I briefly mentioned a tool that helped with the testing of web-service consumers. I’m happy to announce that this is now publicly available on CodePlex under the moniker “MockingBird’. Here are some details about the tool – where it came from, how it helps and where it is going. (Most of this info is on the CodePlex space, but i’ll reproduce it anyway).

The Scenarios for MockingBird

(1) Imagine you are given the WSDL for a third-party web-service but no functioning system is available yet (it may be a brand new service or perhaps dev/site licenses are being negotiated). You need to get on with development right now. What do you code against? You maybe a ‘TDD & Mock Objects’ savvy dev which will help in many cases, but what if you ‘don’t do’ TDD & Mocks? Or it may be that you are maintaining/enhancing an existing system that wasn’t coded against interfaces etc. Or what if you are a BizTalk developer? You cannot mock/inject dependencies into your orchestrations and other components (well, except for pipeline components, but thats another story)!!

(2) Next, imagine that you are setting up a build server and multiple environments (DEV, TEST, UAT etc). But now the vendor says you can only have one license for their software. Now how do you run DEV, TEST, UAT in parallel with different data sets? Or you may have more than one license, but what if that service has maintenance schedules that clash with your build? Your build server is then completely exposed to something you don’t control.

I’ve expanded on these scenarios in the tool documentation here, but essentially these are the two main scenarios that MockingBird targets.

The Origins of MockingBird

MockingBird started life as MockWeb an internal tool that my former colleagues (Senthil Sai and Will Struthers) and I developed. All credit must go to them as Senthil first came up with the concept and Will then contributed a lot to the code-base.

It started when we got rather fed up with having to set up multiple instances of a third party service and build data-sets just to help with testing our BizTalk orchestrations reliably. It grew rapidly and organically. While we felt that this concept and tool would be useful to the .NET dev community at large (not just BizTalk teams), the structure of the codebase (at the time) would not lend itself to easy extension and needed to be refactored before making it publicly available. Its taken a while , but now I’ve completely rewritten MockWeb and as I have mentioned in the roadmap, I intend this to go beyond HTTP Web Services and hence the new name.

Whats with the name – ‘MockingBird’ ? Sounds daft !!

Ok, so its not a tool that sits there and laughs at you when you try to test !! . Thats not what the name is intended to convey. Wikipedia says that “They (mockingbirds) are best known for the habit of some species mimicking the songs of insect and amphibian sounds as well as other bird songs,[1] often loudly and in rapid succession” .

So the intent is to mimic web-services and I quite like the name ‘MockingBird anyway!!

The current implementation

There are two main elements to the current implementation

• An ASP.NET HttpHandler which is configured to return pre-set responses
• A WinForms GUI to setup the handler and associated configuration for a given WSDL.

The Roadmap

• The first thing I intend to do is revise the implementation to be fully WCF based as i want to be able to intercept calls irrespective of transport protocol. This is 0ne of the reasons i rewrote the code-base and i hope there wont be too much churn as i extend it
• The second thing is to make it a sort of platform to test webservices, generate BizUnit test steps, maybe C# test fixtures, possibly to code-gen Biztalk artifacts from given WSDLs and schemas.
• Thirdly to take it into the realm of mock Biztalk adapters. WCF is key to that. I envisage this as being a pair of custom WCF send and receive adapters that can dynamically be configured.
• Beyond that, lets see .

So, check it out. There’s an alpha release currently available and your feedback will be appreciated. If you have any thoughts on how the mock Biztalk adapters could be done, I will be glad to hear and of course, if you want to join the project, you will be welcome.