MockingBird- Beta 2

MockingBird – Beta 2 is now available.

The changes/ fixes are as follows
(1) The system can handle remote WSDL’s now (using DiscoveryClientProtocol). [But this will use the Default network credentials to get the WSDL. There are no facilities as yet for providing this information as settings for the tool).
(2) Fixed the WSDL Browser which crashed when loading WSDLs into the tree view where there were one way operations in the WSDL.
(3) Xml Instance Generator now allows an overloaded constructor where a GeneratorSettings instance can be passed in to control the output.
(4) Documentation has now been split into a User Guide and Technical Documentation and thoroughly revised.

As mentioned in previous posts, the system is very stable (at least for the WSDLs and services I have tested and used it with) , so the fact that it is still officially a beta-2 shouldnt prevent anyone from using it.

Things that need to be done for RTM are

• some additional entry points into the XmlInstanceGenerator,
• adding some “settings” forms where users can set things such as network credentials when dealing with remote WSDLs etc.
• Update the class library documentation
• Use Unity in the Service Studio
• Run the code-base through StyleCop and FxCop. (This last part has me worried . How much am I going to need to rewrite?))

Check it out and, as always, feel free to send feedback.

MockingBird – Beta

Whew! I’ve finally managed to get the first beta release of MockingBird out the door, almost 3 months after the alpha. The past 3 months have gone by in a blur with my first major project in MCS and the associated tight deadlines and long hours. Technically its been a good learning experience, but it doesnt leave any time at all for side projects like this one, but I’ve managed to steal the odd weekend here and there and get this done.

The key elements of the beta release are

• The branding and naming is now beginning to work out quite well. As I had written earlier, it started life as MockWeb, then became MockingBird and now I have decided that I like the name “MockingBird” so much its worth promoting to the ‘brand’ or the name of the toolset. The constituent applications now have their own names. The web application with the HttpHandler is now called the “Service Simulator” and the main WinForms application is now called the “Service Studio” which will hold many modules and plugins.
• The Service Simulator has been completely revised using Unity IoC. I feel a lot happier with the design now and i think its more loosely coupled and the DI /IoC has a good side effect in increasing the test surface. The Service Studio doesnt use the Unity configuration yet, but will do so by RTM.
• The little ‘Driver’ GUI of the alpha has now been rolled into the “Service Studio” and is officially called the “Service Invoker”. The layout has changed and now uses tabs so occupies less screen real estate.
• There is a new “WSDL Browser” module in the service studio that allows you to load up a WSDL and it will parse the WSDL for you and list all the operations and allow you to generate sample request and response messages.
• The WSDL  Browser in turn makes use of an Xml Instance Generator module that provides a more usable facade over a sample from the MSDN archives. The XIG module can stand on its own and will grow in future releases.

The design principle of the Service Studio is that every piece of functionality must be accessible independently and not depend on a UI. This is one of the reasons it took so long to get this done. I came across quite a few samples of WSDL handling where the authors were willing to share and/or the license permitted extension, but the code was simply too deeply hooked into the UI and did not have any “API” to speak of. In the end I wrote the whole thing myself. I’m still looking for ways to plug it into related tools such as WSCF which I also participate in developing.

There are some limitations in the beta especially with regard to WSDL Parsing.

• The first limitation is that it cannot handle WCF multi-part WSDLs. This will be done by around RTM time or along with the V2 release as V2 should be fully WCF based.
• While developing i tested it with different types of WSDLs but in the final push I focussed on testing it with a local monolithic WSDL and so it may fall over with complex WSDL documents. The WSDL spec allows for a wide variation in the actual WSDLs so its not possible for me to cater to all possible kinds of WSDLs. I hope that if you come across an issue, then the source is easy to fix or that you can share the WSDL with me and I will try and sort out the parsing.
• I also only finalised the testing with a “local” WSDL on the file system. The “SimpleWSDLRetriever” class that I have implemented does not use DiscoveryClientProtocol (although i shall plug in a new class soon that does this) so it cant handle connecting to a non local service and downloading the contract and associated metadata. (As you can see, I know how to address the limitation – just havent had the time to write it   ).

Anyway, those limitations aside, I’m happy with the code base so far and it really is quite stable and if you need to extend and use the source, i think the current structure is easy to understand. On the codeplex space I have written more documentation on how the internals of the system work so hopefully that will be useful to you. If you do end up extending it, I hope you will let me know what you are doing with it and maybe I can incorporate some of those extensions if it will make the product more generic.

As always, please send me feedback, especially if you run into any bugs. Hope you like the new release.

VS2008 – Generate XML Instances

It’s funny how we take things for granted. As Biztalk developers, we get used to the idea of being able to right click on a schema and generate an instance . In non Biztalk projects however, this couldnt be done. Till now.

I was playing around with writing some XML Instance Generation for MockingBird to finish off the next release and spent a lot of time poking around the Schema Object Model etc. While doing that, I quite accidentally opened the XML Schema Explorer tool window. Now I had seen that in the past and navigated through some types etc (and thinking it was just a simple add-on to the old VS i kind of took it for granted and didnt investigate further). 

What I did not realize is that for Elements, you can generate sample instances.  Check out the following screenshot.

Generate Sample XML

As you can see from the tree behind the popup window, elements are colored differently as well.

This is a great time-saver. I think this can be done only in VS2008 SP1.

Unfortunately, the downside is that there is no API into this tool-window (or rather, the library behind it), so we cannot programmatically generate instances in bulk. Also it will not open WSDL files, so you have to extract the XSD from the WSDL (if not already available separately) in order to work with this tool window. But i think its cool as we no longer have to depend on third party XML editors to get sample instances.

By the way, if you are looking for help in this area (of instance generation) , there is some sample code available in the MSDN article, Generating XML Documents from Schemas which is quite well written. While there are license restrictions on modification/ derivation (and then redistribution), plain redistribution without modification, I gather, is permitted, so the easiest thing for MockingBird would be to just redistribute the binaries of that sample with the GUI. No sense in reinventing the wheel.

In terms of the Biztalk Schema Editor and its instance generation, if any Biztalk folks know if there’s a programmatic way of doing that, please let me know  (Update: I mean specifically for 2006 and R2). I did a lot of digging around in the Developer Tools folder for an assembly that would allow it, but all the classes were internal. I did come across one public class finally (dont remember the assembly off the top of my head now), which had a public method but required some interface to be passed in but didnt work when i tried calling it from custom code. It would be useful to do this programmatically so we can generate instances in bulk for  a given set of schemas (useful when updating instances to correspond to schema changes etc). So, if you’ve managed to do this and are happy to share info then drop me a line.

Waltzing with WSDL

One of my immediate goals for MockingBird is to have the “Configurator” UI done where you can just pick a WSDL on file or point to a URL and have the tool shred it and generate sample request and response messages into the correct folders (and generate the Handler Configuration) after which the user can then do any extra XPath based configurations to make the system more dynamic. Sounds easy enough, doesnt it? Well, thats what i thought too, until i delved into the intricacies of WSDL Parsing.

The class most people use is ServiceDescription and to be fair, it gives some information about the WSDL but navigating it is not easy at all and its not all that intuitive after you get to the point of listing all Operations inside the PortTypeCollection. There is also another big issue with trying to use ServiceDescription alone and that is the fact that WCF by default gives you multipart WSDLs. So getting hold of the WSDL from a WCF endpoint is not straightforward but one class that comes to the rescue is DiscoveryClientProtocol as Mike Hadlow points out in his article. This behavior can be changed as pointed out by Tomas Restrepo in hs post on “Inline XSD in WSDL with WCF” and Christian Weyer takes it up a notch in his article on Flat WSDL in WCF.

Anyway after pondering this a while, I decided it would be in my best interests to factor the code into a WSDLRetriever and a WSDLDescriptor. The Descriptor takes a dependency on WSDLRetriever (an interface of course, so its easy to mock) and expects  a single WSDL XDocument which it can work with. Now with this approach I dont care if the endpoint (if the WSDL is not on file) is a WCF endpoint or not, the Retriever does all that work. So the Descriptor can do its work and with that sorted i wrote up the first few tests for the Descriptor using a mock Retriever. So far so good.

Now, along with the requirement stated above, I also want to build a nice UI, where, you can not only do a single click to generate messages, but you can exercise finer control over the message content before storing them in the appropriate folders. So imagine if you will, a tree view which shows all the operations and the messages in them and immediately under the message name, the name of the complex type and additionally the body of the type (not in the tree node, but in an associated panel).

Of course, I’m not trying to write a completely generic WSDL parser, but something thats immediately usable by MockingBird and for other tools which can use a WSDL as input.

I had a look around at some blog posts and samples but none seemed to go as far as I wanted. I also took a look through the code for WSCF Blue  but couldnt make immediate use of that API. (I’ve since chatted with Buddhike about that so there may be something i could do with WSCF libraries in future). I looked at WebServiceStudio as the license allows reuse of the code there, but the code structure is simply awful. It works as is but everything is so deeply tied into the UI that it will take a feat of coding to extend it or pull out any reusable logic from it. I tried re-factoring a local copy but gave up pretty soon. So I had to work through the requirement myself.

Anyway, parsing WSDL is not easy at all. Once you get to the operation and message, you then have to wak through all schemas and their associated object tables to pick out the correct complex type corresponding to the message. This took me the best part of a few hours. Given that the user will not supply any namespaces, it gets very hard indeed. Thankfully the Schema Object Model is very decent. I used LINQ to XML for little bits such as extracting endpoint information (which also required me to resort to XPath extension methods because LINQ insisted i give it qualified names but I didnt know what the namespaces were !!) but for the schemas in WSDL, i think the SOM is better than LINQ. But I’m happy to be proved wrong!!

So I’ve got it to the point where it brings back the complex types. Now to generate sample instances from that for which I’ve got some resources. Once thats done and a simple Retriever implemented it wont be hard to put a UI on that because none of this is tied to the UI.

Thats it for now, a sort of ‘progress’ post for those waiting for the GUI for MockingBird and also to point out some of the dragons that await those who venture into the WSDL arena. Talk about ‘design by committee.. its a minefield!!

Couple of really good articles are Understanding WSDL by Aaron Skonnard (an old one, but an excellent read) and Walking the SOM by Stan Kitsis

Till next time

Introducing ‘Mockingbird’

In my previous post, I briefly mentioned a tool that helped with the testing of web-service consumers. I’m happy to announce that this is now publicly available on CodePlex under the moniker “MockingBird’. Here are some details about the tool – where it came from, how it helps and where it is going. (Most of this info is on the CodePlex space, but i’ll reproduce it anyway).

The Scenarios for MockingBird

(1) Imagine you are given the WSDL for a third-party web-service but no functioning system is available yet (it may be a brand new service or perhaps dev/site licenses are being negotiated). You need to get on with development right now. What do you code against? You maybe a ‘TDD & Mock Objects’ savvy dev which will help in many cases, but what if you ‘don’t do’ TDD & Mocks? Or it may be that you are maintaining/enhancing an existing system that wasn’t coded against interfaces etc. Or what if you are a BizTalk developer? You cannot mock/inject dependencies into your orchestrations and other components (well, except for pipeline components, but thats another story)!!

(2) Next, imagine that you are setting up a build server and multiple environments (DEV, TEST, UAT etc). But now the vendor says you can only have one license for their software. Now how do you run DEV, TEST, UAT in parallel with different data sets? Or you may have more than one license, but what if that service has maintenance schedules that clash with your build? Your build server is then completely exposed to something you don’t control.

I’ve expanded on these scenarios in the tool documentation here, but essentially these are the two main scenarios that MockingBird targets.

The Origins of MockingBird

MockingBird started life as MockWeb an internal tool that my former colleagues (Senthil Sai and Will Struthers) and I developed. All credit must go to them as Senthil first came up with the concept and Will then contributed a lot to the code-base.

It started when we got rather fed up with having to set up multiple instances of a third party service and build data-sets just to help with testing our BizTalk orchestrations reliably. It grew rapidly and organically. While we felt that this concept and tool would be useful to the .NET dev community at large (not just BizTalk teams), the structure of the codebase (at the time) would not lend itself to easy extension and needed to be refactored before making it publicly available. Its taken a while , but now I’ve completely rewritten MockWeb and as I have mentioned in the roadmap, I intend this to go beyond HTTP Web Services and hence the new name.

Whats with the name – ‘MockingBird’ ? Sounds daft !!

Ok, so its not a tool that sits there and laughs at you when you try to test !! . Thats not what the name is intended to convey. Wikipedia says that “They (mockingbirds) are best known for the habit of some species mimicking the songs of insect and amphibian sounds as well as other bird songs,[1] often loudly and in rapid succession” .

So the intent is to mimic web-services and I quite like the name ‘MockingBird anyway!!

The current implementation

There are two main elements to the current implementation

• An ASP.NET HttpHandler which is configured to return pre-set responses
• A WinForms GUI to setup the handler and associated configuration for a given WSDL.

The Roadmap

• The first thing I intend to do is revise the implementation to be fully WCF based as i want to be able to intercept calls irrespective of transport protocol. This is 0ne of the reasons i rewrote the code-base and i hope there wont be too much churn as i extend it
• The second thing is to make it a sort of platform to test webservices, generate BizUnit test steps, maybe C# test fixtures, possibly to code-gen Biztalk artifacts from given WSDLs and schemas.
• Thirdly to take it into the realm of mock Biztalk adapters. WCF is key to that. I envisage this as being a pair of custom WCF send and receive adapters that can dynamically be configured.
• Beyond that, lets see .

So, check it out. There’s an alpha release currently available and your feedback will be appreciated. If you have any thoughts on how the mock Biztalk adapters could be done, I will be glad to hear and of course, if you want to join the project, you will be welcome.