Archive for the ‘BizUnit’ Category
I’m back in the NUnit world after a long time. We are looking at packaging up the NUnit GUI along with some BizUnit tests for my current gig so that we have a set of integration tests to run on the rigs. Ironically, the project isnt even a BizTalk one, but BizUnit is still a useful framework, nonetheless. More about that in a later post and back to the main subject here.
I created a separate NUnit project file and specified assemblies etc. I intend the project file to be deployed on the rigs as well so we can just open the GUI, select it and run the tests. The first hitch I ran into was that the NUnit GUI didnt recognize my app config file. After some digging around I found this post by Charlie Pool which shed some light on how NUnit looks for config files however the key point missing in Charlie’s post (or maybe its obvious and i’m just dense :-) )there is the exact location of the file. The file should be a sibling of the NUnit project file and not in the bin\debug folder (unless of course your nunit proj file is there)
To summarise Charlie’s point with my extra clarification: If you load an NUnit project, such as mytests.nunit, the config file must be named mytests.config and should be co-located with the .nunit file.
I also found a gem in the comments that helped me sort the issue. The commenter indicated that we can examine AppDomain.CurrentDomain.SetupInformation.ConfigurationFile in debug mode. I didnt know this could be done and found a ton of interesting stuff in the SetupInformation structure too which i should look into some more.
Hope this helps anyone running into this issue.
BizUnit meets XAML
So, Kevin has got round to making an alpha release of BizUnit v4. This time the steps are all in XAML which i think should now faciliate new tooling. I’m getting my grubby hands on it now.
BizUnit (3.x) Snippets
This is sooo like me. Always late to the party. . Just this evening, before i found out that v4 had been released (although i knew it was in the works for sometime) , in the latest code drop for BizUnitExtensions (22.214.171.124) , I’ve included some snippets that i wrote quite a while ago and just never got round to uploading. Take a look. I know its rather late and probably completely out of date now that v4 is released but hey, v4 is still alpha so i have some time to catch up.
So what of the BizUnitExtensions project itself? As you may have seen, there hasnt been any updates or releases for many months now. The reason is that although for the past 18 months I have been involved in a major BizTalk gig, we have been required to use a custom test framework which has left no scope to use BizUnit at all. Consequently, I let the codebase languish with only one or two minor updates.
Although project-wise nothing has changed, just today I decided it was time to give BizUnitExtensions some attention. I have recompiled the sources (after the codeplex TFS upgrade) and added the snippets in. I have also started to do some major restructuring of the code-base and recompiling against .NET 3.5, EntLib 4.1 etc which I will make available shortly and possibly include some fixes that folks have described in the forums.
Beyond a minor release, I dont think I will continue to maintain BizUnitExtensions as a separate project. I’ve had discussions with Kevin Smith about merging the steps from this project into the main BizUnit distribution and v4 is where this will happen as soon as i get the go-ahead from him. With MockingBird and WSCFBlue taking up the majority of my limited spare time, I have to let go of some projects. But i think merging the code into BizUnit (core) will be good for the community so it becomes a one stop shop. Most of the steps complement the core nicely by supplementing it with missing features and with the XAML foundation, efforts can now be directed towards tooling.
Check out the snippets and let me know if they add any value to you. If this is of some value, then I can try and do the same for v4 so when we merge we have a full set of new snippets. If you are an early adopter of BizUnit v4 and want to use Extensions functionality and can help with porting to XAML and/or writing snippets, do let me know.
So, whats cooking for MockingBird in 2010 ? I have a veritable laundry list of features for the next release(s) but here’s the broad themes we are working towards.
(1) WCF : The first is the port to WCF which is long overdue and Bram has been hard at work in getting this off the ground so I expect to have something to announce soon. We will be implementing a message interceptor / router with a generic contract and from there leverage the simulation framework that is currently available with (hopefully) minimal changes. The challenge will be in the area of bindings since most teams customize bindings heavily so the service will need to support custom bindings.
(2) REST, XML/Http etc: Here we are looking at moving beyond just SOAP based services to plain XML/Http services and even REST based services. Cormac had come across some non SOAP services recently for which he had to manually code a mock, so we will be looking to incorporate such functionality here. REST is an area that I haven’t paid much attention too so i see a crash course in REST around the corner. If you have experience with REST and are keen to see support for this in MockingBird and can contribute in some way, do drop me a line.
(3) Generic ‘Action’ processing: For processing requests and building responses, there needs to be a more generic approach. Currently there is support for running an XPATH query on the request and returning a response based on the result of the XPath expression., But this is only one solution. What if we wanted to do a custom evaluation of the request where XPath is not enough? Further for the response, what if we wanted something more dynamic than just pointing to a specific response file? for example, Cormac suggested we dynamically execute an XSLT against the request. So, for this we need to look at the system executing a supplied “IAction” (IRequestAction, IResponseAction) and behaving accordingly. Unity and MEF should help here.
(4) Dynamic Mocking With BizUnit: This would be an API (+ a BizUnit step) which reduces the amount of upfront configuration needed. I envisage being able to say “For this endpoint, at the next request, return “X” response (and extend it to say, “do that for the next 10 calls or forever etc). We should then be able to “assert” that the system did indeed send the response. This would require surfacing the configuration file management code so that a config file could be written on the fly by the API/ Test step.
(5) .NET 4, AppFabric and Workflow Service: This is probably a longer term (MockingBird v3) where I am considering implementing the handler as a dynamic workflow service and using XAML. Now with this approach all the header parsing, IAction components etc would all become custom workflow activities and it would be possible to dynamically compose specialized handlers on the fly. The message interceptor component would move to the official WCF Router.
So these are the big ticket items. For all of them I would appreciate your feedback as to what would give you the most value. We would also welcome more participation and contribution of ideas/code. Looking forwad to your feedback.
The Pipeline steps use Tomas Restrepo’s Pipeline Testing Library.
I had originally planned on incorporating Bram’s steps into Extensions as I wrote previously. However after discussing with Kevin, those plans changed so its now in the core BizUnit.
There is a similar MapExecutingStep in Extensions that has been around for a while now and when I get some time I’ll take a closer look at Bram’s step and see if there are any differences that are worth keeping or retire the Extensions step if not required anymore. I have had a couple of bugs reported on Extensions beta-1 which I shall fix soon and get the RTM released which will be compiled against BizUnit 3.1.
Finally made some progress on BizUnitExtensions and have released Beta-1 for the new version which now complies with BizUnit 3.0′s object model. This release specifically centres around the main Extensions and the Extensions.BizTalk libraries.
- The Altova library has now been deprecated as there is much better xml validation support in the .NET Framework so the altova library is redundant.
- The steps for manipulating applications, receive locations and send ports have now been completely overhauled, so there is a separate ApplicationConductorStep, ApplicationGetStatusStep and ApplicationVerifyStatus step and likewise for ReceiveLocations and Send Ports. The GetStatus steps will load the current status into the context which can be useful when we need to use that info in subsequent steps. VerfyStatus will check the status against a given value.
- The ReceivePortConductorEx step in the old code-base has been transitioned to the above mentioned set of ReceiveLocation steps as it really was dealing with locations rather than the ports. There is a new ReceivePortConductor step which allows the bulk enabling and disabling of all the locations in that port.
- Updates to the Orchestration Conductor allowing a STOP action on an orchestration.
- The namespaces and assembly names have all been revised (for example, using the Biztalk steps requires the assembly name to be BizUnit.Extensions.BizTalk.dll) so your existing test cases will need to be updated.
Forthcoming Updates and Releases
- Bram Veldhoen has contributed a fair amount of code including steps to link in with the Pipeline Testing Library. I will incorporate that into Beta-2 as I need to work through all the steps and the associated helpers and utility classes.
- I have updated the Oracle and EntLib libraries as well, but haven’t had any time to test them. As I now have Oracle XE, I should be able to make some progress on that pretty soon. With EntLib, its a little more complicated. As I didnt write those steps myself, I am not too familiar with the code, so it will take a while to update. So for the present, its still using EntLib 3.1.
- The codeplex workspace has also been cleaned up and slimmed down drastically. A lot of the pointers to tutorials etc are all largely redundant anyway.
- The Roadmap page has been completely rewritten. The framework has come a long way since that material was written. The only stuff left there is a list of probable new steps and some ideas for tooling around BizUnit which may make it into some future version of the framework.
Hope you find the updates useful. Let me know if you encounter any bugs or if you have any suggestions.
In an earlier article , I had briefly mentioned that some folk had used mocks with Biztalk, notably to test pipeline components. Since I didnt have the bookmarks at hand then I didn’t provide the links, but I have since found the links again so here they are (and I can also now use this as a note to self if I want to refer to them again or expand on any of the material they have written).
Testing Pipeline Components – The Pipeline Testing Library – Nick Heppleston
BizTalk Testing Series – Testing Pipeline Components – Mike Stephenson
TDD and BizTalk Orchestration Expression Code – Matt Podwysocki
Rhino Mocks and custom BizTalk code revisited – Matt Podwysocki
Biztalk TDD – Pipeline Components - Arun Gopalam
While the blog posts pointing to the Pipeline Testing Library are useful, if you want to go straight to the source, check out the WIKI page that Tomas has set up on GitHub. That page has more samples on how to use the API.
I’m going to have a play around with MoQ and pipeline components in the next couple of days . I think MoQ’s approach is a bit more elegant than Rhino (particularly, the absence of record and replay). I’m also going to link into Tomas’s excellent pipeline testing library from BizUnitExtensions. This has been a long overdue item on my roadmap.
UPDATE: Bram Veldhoen has already done some work on linking the Pipeline Testing Library into BizUnit and has very graciously contributed his code to be put into BizUnitExtensions so that will be released soon with Extensions 3.0.
Enjoy the links and if you find others of a similar ilk that are also useful feel free to put them in the feedback section here and I will update the post.
Richard Hallgren recently sent me a link to a question that was posted on StackOverflow dealing with “Mocking Web Services consumed by Biztalk Request Response ports” and asked for my thoughts on the subject as BizUnit and BizUnitExtensions were both mentioned. There were a couple of posts there at the time, one questioning why bother, another directing him to use SoapUI and another really good reply by David Hall on the topic of mock BizTalk adapters (which I want to talk about in some more depth later). David’s points helped me crystallize some of my thoughts on the subject of testing orchestrations and I replied there and figured that it was a good subject to blog about in more detail.
David raised the point about there not being real ‘unit tests’ in BizTalk (at least not unit tests in the purist sense of the word) and that the ‘opaque’ nature of orchestrations made it very difficult to actually ‘unit’ test them. He did also raise the point about BizUnit being a bit of a misnomer (the ‘Unit’ part being somewhat misleading to someone new to BizTalk). I do agree with the points he raises there. Here are some of the things I wrote there and which I will expand upon to some extent in this post and some extra points.
One approach which I have recently got sold on is the use of mock object frameworks (such as MoQ, RhinoMocks, TypeMock and NMock2). I used NMock2 in my last ASP.NET project and have recently started with MoQ which i think is much better and i’m really getting into the lambdas.. they make expressing intent so much more concise. I’ll write a bit more about how I’m using them in another post, but suffice to say here that, although I havent tried it myself, several Biztalk folk have successfully used mock objects to test pipeline components.
Orchestrations unfortunately, in this respect (of being able to completely control what their dependencies are and ‘faking’ the behavior of the dependency) can be considered opaque. But there are good reasons for that.
a) Because of the huge subscription system in the message box – that orchestrations use when being activated etc, it is not possible to fire up some “virtual” process to host the orchestration (I think Tomas’s Pipeline Testing Library does something along these lines for pipeline components, but I havent had the opportunity to use that so far although i have heard good things about it).
(b) Now for arguments sake, lets assume you could write such a virtual host. Now how would this virtual process handle persistence and dehydration?. I’d wager that people using WF would have the same problem in trying to test the workflow fully. (Soon after writing this I came across Matt Milners MSDN article on unit testing workflows, but workflows are just CLR code)
(c) Although you can see the C# equivalent of the ODX, we dont develop with the C# equivalent directly, so there is no way we can “inject” a mock interface into the orchestration code. Orchestrations are at a higher level of abstraction than ‘normal’ code so while you may declare variables and use them in expression shapes etc, its not the same as writing getters and setters in regular C#.
(d) An orchestration is not really a “unit”. It is a composite element. The units are the messages going to and from the message box and the external components called through expression shapes.So even if you could inject a mock webservice interface you cannot inject mock message boxes and correlation sets and other things.
Of course, it may be possible to get more testability for orchestrations in future versions of the product (beyond v.2009), but this is the situation now. If there are other approaches you use to dealing with the innards of orchestrations then do let me know.
In terms of getting more info on the orchestration itself, one thing that can be done (and i’ve been considering an addition to the BizUnitExtensions library to do this) is to link in with the OrchestrationProfiler tool a(s that tool gives a pretty detailed report of all the shapes ). Now from the BizUnit test we can verify that individual steps were executed and perhaps also check the time it took for execution. This is a bit like setting expectations for mock objects. This could go quite far in making the orchestration a bit more of a white box.Also considering that the orchestration debugger shows a lot of the variable values, surely it must be possible to get that info via an API (one hopes!!) to show what the values of variables were at a given point for a given instance.
One thing which a lot of people get hung up on is terminology (unit tests vs. functional tests). I dont know why its hard for some folk to accept that the granularity of a unit is entirely subjective and depends a lot on the system. For a system like Biztalk, especially with orchestrations, in my opinion, the unit of testing includes the pickup of the message from the source, the processing and its dispatch. A functional test will expand on that to include things like getting the data into the source and an integration test will span the entire system, and of course, since the solutions built with Biztalk are quite different this definition of whats involved in each category of testing needs to be ‘adapted’ for each project.
Back to Richard’s question though, my previous dev team had a solution. Basically what we did was to write a generic configurable HttpHandler that parsed incoming service requests and returned pre-set responses. The response sent back was configurable based on conditions such as XPath. In the BUILD and DEV binding files, the webservice end point was the mock. This worked brilliantly in isolating the BUILD and DEV environments from the actual third party webservices. This also helped in a “contract first” approach where we built the mock endpoint and the orch developer used it while the webservice author went ahead and built the actual service behind the endpoint. In my next post, I’ll describe this in more detail as I have now put it on CodePlex.
As my previous team wasnt into traditional mock objects, we used this quite successfully for both Biztalk consumers and non Biztalk consumers. If you are not dealing with Biztalk , and if you have control over the consumer (especially if you are writing it from scratch), then i would recommend a mock object framework.
BizUnitExtensions crossed the 400 downloads mark recently and im really chuffed about that. One of the things I’ve been meaning to do for a very long time is to release some work i did with my previous project team in restructuring the codebase for Extensions. Basically, in alignment with BizUnit 2.3 we moved all non core steps into their own assemblies so now we have separate assemblies for Extensions.Biztalk, Extensions. EntLib and so on. This release is now on CodePlex. (Its not a default release so the homepage still shows the old one so use this link to get it).
Before you download this however please note the following
- There is no new functionality as such (as far as i remember) and this is only an interim release to set the stage for v3.0 which has a completely new object model, strong typing of test step properties and so on. So you probably dont need this if you are happy with the previous release.
- As the steps have been moved into different assemblies, the namespaces have also been changed to correspond with them and this will cause existing XML steps to break unless updated.
- Since we are no longer redistributing the core BizUnit engine with Extensions, the extra entlib logging we had added to the core engine has now been dropped. I hope we can revisit this sometime in future. (The EntLIb database steps are all safe in their own assembly.. just the core logging no longer uses entlib).
- The assembly version numbers have at last been updated and they are all fixed at 126.96.36.199. (This was doing my head in for a long time and the procrastination finally got to me).
The wiki had a roadmap suggestions page for a long time and the points there had been converted to work items and for a few i actually got votes on what features would be useful. I’ve now been through the list with a fine tooth comb and extracted the ones to work on immediately.
I have been corresponding closely with Jean Paul Smit and looking through his excellent work on the Biztalk Solution Factory and eventually Extensions will make it into a full fledged Guidance Package.
The 4 main areas to work on are as follows (outside of the upgrade to BizUnit 3.0). For notes on the areas listed check out the old roadmap suggestions page on codeplex.
- Pipeline Tests
- Link into OrchProfiler
- Link into Quick Counters
- Code Snippets
- TreeView GUI / WPF GUI
- WF Test Script Flow GUI
- Generate BizUnit Test script (now in BSF)
- Test Guidance Packages (also BSF)
- Custom DSL
So thats my priority list. Unfortunately i cant promise any dates. If anyone has written steps in the areas listed below (I’ve chatted to Mike Stephenson about WSE steps), or even other areas that i’ve not listed here then please let me know if you would like to contribute your steps. It will all go towards making this a more comprehensive library.
Jean-Paul Smit has released a new version of Biztalk Solution Factory now with support for generating unit tests. The unit test support is based on BizUnit 2.2 and NUnit 2.4.7. There are actions for generating schema tests, map tests and orchestration tests. Check it out…
I just came across this excellent material on Technet – the Biztalk Server Performance Optimization Guide. Whats even cooler about this paper is that it goes into some pretty good depth on how to use BizUnit in conjunction with LoadGen to do performance testing. This is really good stuff. I was just thinking how good it would be to see a step by step guide to setting up performance testing, including how to get your solution fully isolated and ready etc and along comes this beauty. Its got a complete configuration file which we can customize and use for our projects.
It gets even better wherein there is an entire section on how BizUnit can be used with the end to end BPM scenario. This is an absolute gem. I remember talking with Gar when we first setup BizUnitExtensions on how good it would be to get hold of the end to end scenarios and setup BizUnit for them and this article provides a lot of that. This, IMO, is probably one of the most valuable articles around not only for BizUnit but also because it describes an approach to testing a complete solution.
Check out the articles and happy Biztalking and testing Enjoy.