Archive for the ‘MockingBird’ Category
I’ve taken my first foray into the world of producing videos with a set for MockingBird.
- Introduction : Getting up and running with MockingBird V2, using the Web Application and the Configurator Module.
- WCF Support : How the Console Host works to provide the WCF Support.
- Understanding MB Configuration: Digging a little deeper into the configuration files.
Take a look and let me know what you think of the quality and content (and also if these links all work for you. SkyDrive has been giving me some problems in sending links around). I will work on doing more videos if there is a demand for them. Looking forward to your feedback.
I’m delighted to announce that MockingBird v2 has finally hit the RTM milestone. Whew! That was a long RC !
So, what’s new in this release ?
- First off, there’s a brand new Service Configurator module which makes it dead simple to create a mock service. Just pick a WSDL, click through a couple of menu options and it generates a complete service with all the default settings. You can then edit the config (manually) if you want to tweak the behavior further. Watch this space for enhancements to the tooling.
- The Service Studio app (of which the Configurator module is a part) has been completely rewritten and the GUI now employs the (free) Krypton Windows Forms Control Toolkit. That is simply an awesome piece of software and the fact that its free is mindblowing. I’m no GUI expert so any help in this area is a plus. I think you’ll agree that the new UI is way better than the old. (What’s that? you dont like it ? Hmm, if you know WPF/Silverlight, how about helping me write a better one? 🙂 ) . Studio now is a full fledged management application for the Simulator. Tr
- Configuration has been given a complete overhaul. The config system no longer expects all config files to be siblings of the system config, which frees up the app to have centralized configuration (shared by the console host and the web application). I have even got rid of the EntLib config gunk from the app.config and kept it totally separate. I’ll write about all that separately. The next step in this is to externalize the WCF service and client config (which I know how to do but just havent had time) so again, watch this space.
- Tracing has now been added in so you can launch DebugView and happily monitor whats going on in the system. So you dont really need the log files (although you can always choose to use Log4net or EntLib additionally).
- Multi-Part WSDL support in the Configurator and the Message Instance Builder (formerly called WSDL Browser). Thanks to some cool metadata parsing code that Alex put into WSCFBlue and kindly allowed me to copy out, the system can now handle WSDLs on file and from any live service endpoint.
- Installation And Configuration is now made pretty much single click (3 clicks – 1 for each host and 1 for the studio) with a Powershell script to give permissions.
- RollUp of some functionality that the team put into the code-base post RC such as XsltResponse (to dynamically generate a response by applying XSLT to the request), a Soap Header aware message handler and various other fixes. The handlers now come with a proper MessageHandlerBase class so if you want to extend MockingBird with your own handlers, you can simply inherit from that and write your specific code.
Things to watch out for
- WSDLs are generally a minefield, so i wont claim that MB can handle any metadata you throw at it. It will choke on some WSDLs (notably Amazon web services) but under the covers I’m using stuff like MetadataExchangeClient and WsdlImporter from the core .NET framework so there’s not a lot I can do to handle weird WSDL. Anyway, if this is an area that gives you grief, let me know and we’ll see if we can start to support custom ‘import extensions’ like svcutil.
- Windows 7, 2008+ and IIS7.x : Limited time = limited test surface. I extensively tested on Windows Server 2003 R2 but not on anything else although the team does use Win 7 and Windows 2008 when they code MB features. I have included some notes on troubleshooting in the guide. The number 1 area that you are likely to fall a victim to (in IIS 7.x) is that the AppPool running MockingBird should be in Classic mode.
So, what’s next ?
- Some time to sleep I hope 🙂 (although with baby no. 2 due shortly, that’s not going to happen is it?
- REST support
- Azure anyone?
- More in the configurator GUI
Sound appealing ? think you can help (except with the sleep !!) ? Let me know.
MockingBird v2 has almost reached RTM. Just a few more days to make some final touches and it will be ready. The team has been discussing how to take it further and one thing that’s been on my mind for a long time is the topic of mock BizTalk adapters. What I would like to do is write some adapters with the WCF LOB Adapter SDK that link up with MockingBird’s simulation engine so we can then simulate various protocols such as SQL, Oracle, MSMQ, MQSeries etc.
The first question that anyone would ask me is “Why? “. Currently MockingBird works decently as a receiver from BizTalk in one way and solicit response mode. (It also supports duplex channels). If you have a one way message from BizTalk or a solicit response, then all you need is to replace the URL of the endpoint with the MB url and with the right config, the system will send back the messages you were expecting. So, why do we need to mock an adapter when we can simply avail of the WCF-Custom adapter?
While the first thing that comes to mind is that MB only handles the send side and does nothing on the receive side, i do think that there are many other reasons to go down the adapter route (on both the send and receive sides)
For the send side
- Adapters will allow us to manipulate & use context properties. For situations where transport level correlation is being used (MQSeries, MSMQ etc), the adapter will let us force copy correlation identifiers to the response. Without this, we would need to have a queue listener that understood the MQ/MSMQ protocols and set properties accordingly.
- Adapters will allow us to simulate transactional behaviour.
- Minimizes the ‘intrusion footprint’ of the tool. Currently even if you were targeting SQL you would need to switch to WCF Custom and make a lot more binding changes. If we could adhere to the SQL adapter properties, it would reduce the amount of work. In fact all we should require is a change of url (eg: mbsql:// )
- If an application interface doesn’t support web services, then piggy-backing on WCF LOB adapters will allow us to mock that system (assuming we are wrapping that system with the WCF LOB Adapter, we could potentially leverage that adapter to direct messages into mockingbird). So if there is a SQL/Oracle database, because we can connect to that via WCF-SQL , WCF-Oracle etc, we can just add the mock flavors of the same otherwise we would have to mock the entire SQL/Oracle protocols which is not feasible.
For the receive side
- Although we can always create a test receive location with the file adapter, this is limited to ‘unit tests’. We cannot simulate other transport protocols (for example, polling sql. Also with queues (MQ/MSMQ) we would need an app that could send to the queue to activate the receive location.
- We could extend an adapter to use the ‘Submit Direct’SDK sample to send messages into BizTalk directly and configure this.
- The context properties argument also applies here. A lot can be set artificially on the message. Similarly transactional behaviour could also be applied.
So, what about other current tools in this general area ?
- Some of this kind of testing can be done with BizUnit (example : sending messages to a queue or reading from a queue). BizUnit could also be extended (example : a correlation aware queue listener) but BizUnit’s focus is on distinct steps that verify behaviour rather than on causing/forcing some behaviour which we can do via the adapters. The adapter approach also keeps this testing ‘within the boundary of Biztalk’ . while BizUnit will complement this nicely from the outside.
- LoadGen can also help on the receive side. Again, LoadGen’s focus is on load/stress testing rather than functional behaviour. I think it may be possible to use LoadGen with Mockingbird (maybe as part of these adapters).
- BizMock is another tool that is in similar territory. But having discussed MockingBird with Pierre Milet Llobet, the author of BizMock, I don’t see any overlap here. BizMock is focussed on providing a fluent interface to help with a TDD approach to BizTalk testing and on providing a mock adapter that is used within that test scope, but MockingBird’s adapters will be ‘proper’ infrastructure and integration test faciliators rather than a unit test tool.
Okay, so these are my thoughts on extending MockingBird into the arena of mock BizTalk adapters and I would really like to get feedback from the BizTalk community on this. What do you think of the idea? If you have some experience with the WCF LOB Adapter SDK (which I’ve only played with briefly) perhaps there’s some suggestions you can make or tips/gotchas you can make us aware of ? Is this an area you could get involved with (not necessarily have to contribute code, there are bandwidth constraints on all of us, but potentially send us suggestions, design ideas , maybe be an early tester etc?) .
Let me know. All feedback would be greatly appreciated.
I owe this release pretty much entirely to the stellar contributions of Bram Veldhoen who wrote the entire WCF interceptor and dived into the Unity based dependency resolution and fixed a number of issues with threading and concurrency and even introduced me to the awesome (and scary) Microsoft CHESS concurrency testing tool which I’m still trying to get my head around.
Basically we have now added 2 things to the mix namely a “SimulatorService” (.svc) endpoint that implements a generic contract. The message exchange patterns are modeled after the BizTalk WCF adapters and the current set is : IOneWay, IOneWayDuplex, IOneWaySession, ITwoWay, ITwoWayVoid and the corresponding Async interfaces (for example ITwoWayAsync ). The SimulatorService.svc can be hosted in IIS or in a ‘SimulatorHost’ console application (which also supports http and https). Although both are installed, it is not possible to run them simultaneously because of port sharing issues, so if you want to run the SimulatorHost ensure that the website that contains the ASMX and SVC is stopped.
The high level design of the system is shown in the following diagram
(If you are not familiar with it from previous versions, essentially it consists of a generic HttpHandler and a generic WCF message interceptor that can pick up any message sent to the endpoint and to respond to this message it looks at a model XML file that instructs it how to respond. All it needs to function straight out of the box is the model file which is very simple to setup)
In terms of the physical layout MockingBird is manifested as follows
- A virtual directory : This contains the SimulatorService.svc file (the WCF message interceptor) and in the bin folder we have the Interceptor.dll which contains HttpHandler (implemented as a class that inherits the IHttpHandler interface rather than an .ashx) . The bin folder also contains the WCF Simulator Host application and a simulator host tester application. The .svc and the httpHandler both share the web.config and as explained in the technical documentation, there are a number of other .config files
- A folder (usually in Program Files) containing the Service Studio GUI application.
Please note that in the RC, due to the single package and co-location of the WCF SimulatorHost with the web application, the 4 config files are duplicated in the root of the web app as well as in the bin folder. This is intentional because the system looks for ‘sibling’ configuration files and for web apps they need to be in the root whereas for the Simulator Host they need to be co-located with the .exe.config which happens to be in the bin folder of the web app.
In v2 RTM we plan to split this up so that the simulator host is installed in a separate folder (under Program Files). This will avoid the confusion that may appear when the user sees the config files in 2 adjacent folders, however, the apps still need their own copies of the configuration files. Perhaps in a future version, we will build some advanced centralized configuration so that irrespective of the simulator chosen, the configuration is always taken from a single central location.
I hope you find this new version useful. Looking forward to the bug reports and feedback. I hope to be able to get the RTM out in a month or so (approx 20 April 2o10). Do check it out and let us know what you think of it.
So, whats cooking for MockingBird in 2010 ? I have a veritable laundry list of features for the next release(s) but here’s the broad themes we are working towards.
(1) WCF : The first is the port to WCF which is long overdue and Bram has been hard at work in getting this off the ground so I expect to have something to announce soon. We will be implementing a message interceptor / router with a generic contract and from there leverage the simulation framework that is currently available with (hopefully) minimal changes. The challenge will be in the area of bindings since most teams customize bindings heavily so the service will need to support custom bindings.
(2) REST, XML/Http etc: Here we are looking at moving beyond just SOAP based services to plain XML/Http services and even REST based services. Cormac had come across some non SOAP services recently for which he had to manually code a mock, so we will be looking to incorporate such functionality here. REST is an area that I haven’t paid much attention too so i see a crash course in REST around the corner. If you have experience with REST and are keen to see support for this in MockingBird and can contribute in some way, do drop me a line.
(3) Generic ‘Action’ processing: For processing requests and building responses, there needs to be a more generic approach. Currently there is support for running an XPATH query on the request and returning a response based on the result of the XPath expression., But this is only one solution. What if we wanted to do a custom evaluation of the request where XPath is not enough? Further for the response, what if we wanted something more dynamic than just pointing to a specific response file? for example, Cormac suggested we dynamically execute an XSLT against the request. So, for this we need to look at the system executing a supplied “IAction” (IRequestAction, IResponseAction) and behaving accordingly. Unity and MEF should help here.
(4) Dynamic Mocking With BizUnit: This would be an API (+ a BizUnit step) which reduces the amount of upfront configuration needed. I envisage being able to say “For this endpoint, at the next request, return “X” response (and extend it to say, “do that for the next 10 calls or forever etc). We should then be able to “assert” that the system did indeed send the response. This would require surfacing the configuration file management code so that a config file could be written on the fly by the API/ Test step.
(5) .NET 4, AppFabric and Workflow Service: This is probably a longer term (MockingBird v3) where I am considering implementing the handler as a dynamic workflow service and using XAML. Now with this approach all the header parsing, IAction components etc would all become custom workflow activities and it would be possible to dynamically compose specialized handlers on the fly. The message interceptor component would move to the official WCF Router.
So these are the big ticket items. For all of them I would appreciate your feedback as to what would give you the most value. We would also welcome more participation and contribution of ideas/code. Looking forwad to your feedback.
MockingBird v1 is now formally released. With the squeeze in available time for personal projects, it’s taken me nearly 6 months since the beta and only recently did the site see any check-ins. There’s quite a few updates, and enhancements to the suite. Actually the beta was pretty stable with hardly any bugs that needed to be fixed but there were a few feature requests and to be honest, I started getting rather impatient to move to WCF but needed to formalize the v1 release first. Here’s a laundry list of all the features of the new release. The documentation, included with the package has also been substantially updated.
This release will be the last to support the ASP.NET HttpHandler. V2 will be completely revised to support WCF. The change in the engine should be transparent to the end user and of course with WCF we will get a lot more benefits including transport independent request interception. It will also be the last to support an independent “Studio” module as I have some plans for a brand new UI
Simulator functionality changes
· Now supports Enterprise Library logging as well as log4net
· Logging all request and response operations.
· Support for "mapped soap Actions" (when soapActions do not correspond one to one with operation names or when the default parsing of soap actions cannot be done)
· Supports mapping of operations to request element root names (when soap actions are all the same, such as in Apache Axis web services)
Simulator Packaging changes
· Config for unity and services now externalized
· MockingBird system config now externalized
Simulator Internal changes
· System Configuration Manager abstraction layer for config data storage
· StrongTyped application configuration.
· New OperationNameInferenceEngine
· Support for tokens and token expansion in configuration settings.
· Disabling of context menu on all nodes except for message level nodes.
· WSDL Browser "Save As " dialog now supports proper file masks (also takes in the service name and suggests the name for the file)
· No unhandled exceptions in the studio.
· Auto resize of all controls on form size change.
· Additional entry points to the XIG.
A big thank you to all who sent comments and gave me feedback. I hope you find the updates in this release to be useful.
V2 is going to be really big, so watch this space. 🙂
For v2, There are a number of work items published on the CodePlex space so please weigh in on them and vote for the ones that will be most useful to you and also please feel free to send me suggestions for other features as well as bug reports.
MockingBird – Beta 2 is now available.
The changes/ fixes are as follows
(1) The system can handle remote WSDL’s now (using DiscoveryClientProtocol). [But this will use the Default network credentials to get the WSDL. There are no facilities as yet for providing this information as settings for the tool).
(2) Fixed the WSDL Browser which crashed when loading WSDLs into the tree view where there were one way operations in the WSDL.
(3) Xml Instance Generator now allows an overloaded constructor where a GeneratorSettings instance can be passed in to control the output.
(4) Documentation has now been split into a User Guide and Technical Documentation and thoroughly revised.
As mentioned in previous posts, the system is very stable (at least for the WSDLs and services I have tested and used it with) , so the fact that it is still officially a beta-2 shouldnt prevent anyone from using it.
Things that need to be done for RTM are
- some additional entry points into the XmlInstanceGenerator,
- adding some “settings” forms where users can set things such as network credentials when dealing with remote WSDLs etc.
- Update the class library documentation
- Use Unity in the Service Studio
- Run the code-base through StyleCop and FxCop. (This last part has me worried :-). How much am I going to need to rewrite?))
Check it out and, as always, feel free to send feedback.