Parsing Multi-Part WSDLs

In the ASMX world, handling WSDL (for the purposes of say, generating code or custom documentation etc) is pretty straightforward. If you are given a .asmx?wsdl  endpoint (or even  a local WSDL file), all you need to do is instantiate a System.Web.Services.Description.ServiceDescription over it and that will give you access into all the objects that make up the WSDL (endpoints, bindings and so on). You can also  construct this class manually (if you need to) and generate a WSDL from that (which is similar to the approach we take in WSCFblue). 

However, with WCF you do not get a single WSDL with all the info, rather its split up into multiple-parts. (Also note that although you can enable metadata publishing on special endpoints or on different endpoints in a service, the WSDL is still essentially the same because the metadata is at the service level not the endpoint level, which is why the class is , rather obviously named ServiceDescription. At first glance, one may think that the ServiceModel equivalent (System.ServiceModel.Description.ServiceDescription) will get you everything about the multi-part WSDL, but that’s not entirely true. 

In a discussion with Youssef Moussaioui on an internal forum, I gleaned the following vital nugget of info. 

“There are two different ServiceDescription classes that have *very* different meanings: 

• System.Web.Services.Description.ServiceDescription is an object model for WSDL documents
• System.ServiceModel.Description.ServiceDescription is an object model representing a WCF service”

As with most things, in retrospect it makes sense (and the object model is very rich indeed) but initially it was confusing to see that the two classes returned different properties and details about the ‘service’. 

So, if you are given a MEX endpoint (/mex or .svc?wsdl or .asmx?wsdl), how do you go about getting the metadata and using that?. The first thing to bear in mind is that the consequence of a rich object model is that the user then has to navigate a pretty deep tree to get at the data that is needed. But I must say, you gotta love the way WCF works sometimes. With the aid of 3 nifty classes , the MetadataExchangeClient, the MetadataSet (which is returned by the MexClient) and the WsdlImporter (which works on the MetadataSet) you can get pretty much everything you need. 

Here’s some sample code that can help. I chose to dump the results into the Console, but of course you could do a whole lot more (for example, in WSCFblue, we are building a ‘flat container’ that represents a WSDL, which we are going to use for some new functionality). 


// Define target endpoint
//Uri endpointUri = new Uri("http://localhost:45173/RestaurantServiceWcf/RestaurantService.svc?wsdl");

// Setup the mexClient and importer to get the metadata
MetadataExchangeClient mexClient = new MetadataExchangeClient(endpointUri, MetadataExchangeClientMode.HttpGet);
mexClient.ResolveMetadataReferences = true;
MetadataSet metadataSet = mexClient.GetMetadata();
WsdlImporter importer = new WsdlImporter(metadataSet);

 

// Now get all the metadata from the importer
Collection cds = importer.ImportAllContracts();
ServiceEndpointCollection endpointCollection = importer.ImportAllEndpoints();
Collection bindings = importer.ImportAllBindings();
XmlSchemaSet schemaSet = importer.XmlSchemas;
ICollection schemata = schemaSet.Schemas(); 


Console.WriteLine("Schemas");
foreach(XmlSchema schema in schemata)
{
Console.WriteLine(schema.TargetNamespace);
}


Console.WriteLine("Endpoints");
foreach(ServiceEndpoint endpoint in endpointCollection)
{
Console.WriteLine(endpoint.Address);
}


Console.WriteLine("Operations");
foreach(ServiceEndpoint endpoint in endpointCollection)
{
foreach(OperationDescription operation in endpoint.Contract.Operations)
{
Console.WriteLine(operation.Name);
}
}


Now, there is one limitation with the MetadataExchangeClient namely, that it cannot handle a local WSDL (makes sense, because the file system is passive and cannot obey a protocol to provide metadata). So, my WSCFblue colleague, Alex came up with a interesting approach where he downloaded all the metadata using DiscoveryClientProtocol and then picked up all the metadata documents (whether they were WSDL or XSD or Policy) and attempted to export the metadata using methods such as MetadataSection.CreateFromServiceDescription() and MetadataSection.CreateFromXsd(). These sections are then added to a MetadataSet from which we can then use the WsdlImporter to get the rest of the juicy bits. (This summarises what he did but there’s a bit more to it that I will write about a little later when we get to ship those bits).

So, there you have it. Some utility code that can get you started discovering the treasure trove that is the ServiceModel.Description.

MockingBird meets WCF in v2.0 RC

I’m delighted to announce that MockingBird v2.0 is now available as a Release Candidate.   This is the long awaited ( ) version that adds a WCF Message Interceptor that now sits alongside the HttpHandler and uses the same generic simulation engine.

I owe this release pretty much entirely to the stellar contributions of Bram Veldhoen who wrote the entire WCF interceptor and dived into the Unity based dependency resolution and fixed a number of issues with threading and concurrency and even introduced me to the awesome (and scary) Microsoft CHESS concurrency testing tool which I’m still trying to get my head around.

Basically we have now added 2 things to the mix namely a “SimulatorService” (.svc) endpoint that implements a generic contract. The message exchange patterns are modeled after the BizTalk WCF adapters and the current set is : IOneWay, IOneWayDuplex,  IOneWaySession, ITwoWay, ITwoWayVoid and the corresponding Async interfaces (for example ITwoWayAsync ). The SimulatorService.svc can be hosted in IIS or in a ‘SimulatorHost’ console application (which also supports http and https). Although both are installed, it is not possible to run them simultaneously because of port sharing issues, so if you want to run the SimulatorHost ensure that the website that contains the ASMX and SVC is stopped.

The high level design of the system is shown in the following diagram

(If you are not familiar with it from previous versions, essentially it consists of a generic HttpHandler and a generic WCF message interceptor that can pick up any message sent to the endpoint and to respond to this message it looks at a model XML file that instructs it how to respond. All it needs to function straight out of the box is the model file which is very simple to setup)

In terms of the physical layout MockingBird is manifested as follows

• A virtual directory : This contains the SimulatorService.svc file (the WCF message interceptor)  and in the bin folder we have the Interceptor.dll which contains HttpHandler (implemented as a class that inherits the IHttpHandler interface rather than an .ashx) . The bin folder also contains the WCF Simulator Host application and a simulator host tester application. The .svc and the httpHandler both share the web.config and as explained in the technical documentation, there are a number of other .config files
• A folder (usually in Program Files) containing the Service Studio GUI application.

Please note that in the RC, due to the single package and co-location of the WCF SimulatorHost with the web application, the 4 config files are duplicated in the root of the web app as well as in the bin folder. This is intentional because the system looks for ‘sibling’ configuration files and for web apps they need to be in the root whereas for the Simulator Host they need to be co-located with the .exe.config which happens to be in the bin folder of the web app.

In v2 RTM we plan to split this up so that the simulator host is installed in a separate folder (under Program Files). This will avoid the confusion that may appear when the user sees the config files in 2 adjacent folders, however, the apps still need their own copies of the configuration files. Perhaps in a future version, we will build some advanced centralized configuration so that irrespective of the simulator chosen, the configuration is always taken from a single central location.

I hope you find this new version useful. Looking forward to the bug reports and feedback. I hope to be able to get the RTM out in a month or so (approx 20 April 2o10). Do check it out and let us know what you think of it.

MockingBird – The Roadmap

I’m really chuffed to have not one but 3 new team members on board the MockingBird project. Welcome to Shen Chauhan, my MCS colleague and WPF whizz-kid, Bram Veldhoen and Cormac O’Brien.

So, whats cooking for MockingBird in 2010 ? I have a veritable laundry list of features for the next release(s) but here’s the broad themes we are working towards.

(1) WCF : The first is the port to WCF which is long overdue and Bram has been hard at work in getting this off the ground so I expect to have something to announce soon. We will be implementing a message interceptor / router with a generic contract and from there leverage the simulation framework that is currently available with (hopefully) minimal changes. The challenge will be in the area of bindings since most teams customize bindings heavily so the service will need to support custom bindings.

(2) REST, XML/Http etc: Here we are looking at moving beyond just SOAP based services to plain XML/Http services and even REST based services. Cormac had come across some non SOAP services recently for which he had to manually code a mock, so we will be looking to incorporate such functionality here.  REST is an area that I haven’t paid much attention too so i see  a crash course in REST around the corner. If you have experience with REST and are keen to see support for this in MockingBird and can contribute in some way, do drop me a line.

(3) Generic ‘Action’ processing: For processing requests and building responses, there needs to be a more generic approach. Currently there is support for running an XPATH query on the request and returning a response based on the result of the XPath expression., But this is only one solution. What if we wanted to do a custom evaluation of the request where XPath is not enough? Further for the response, what if we wanted something more dynamic than just pointing to a specific response file? for example, Cormac suggested we dynamically execute an XSLT against the request. So, for this we need to look at the system executing a supplied “IAction” (IRequestAction, IResponseAction) and behaving accordingly. Unity and MEF should help here.

(4) Dynamic Mocking With BizUnit: This would be an API (+ a BizUnit step) which reduces the amount of upfront configuration needed. I envisage being able to say “For this endpoint, at the next request, return “X” response (and extend it to say, “do that for the next 10 calls or forever etc). We should then be able to “assert” that the system did indeed send the response. This would require surfacing the configuration file management code so that a config file could be written on the fly by the API/ Test step.

(5) .NET 4, AppFabric and Workflow Service: This is probably a longer term (MockingBird v3) where I am considering implementing the handler as a dynamic workflow service and using XAML. Now with this approach all the header parsing, IAction components etc would all  become custom workflow activities and it would be possible to dynamically compose specialized handlers on the fly. The message interceptor component would move to the official WCF Router.

So these are the big ticket items. For all of them I would appreciate your feedback as to what would give you the most value. We would also welcome more participation and contribution of ideas/code.  Looking forwad to your feedback.

BizTalk, WCF and the Bad Gateway

We ran into an interesting problem a couple of days ago and I decided it would be worth posting the solution in case anyone else runs into it.

The scenario involves a Windows Service communicating with a BizTalk WCF endpoint which is hosted using WCF-CustomIsolated and some fairly complex custom bindings, notably using X.509 Mutual Certificates. This is the second half of the communication, the first being where BizTalk sends a request to a vanilla WCF endpoint and gets an ACK, the service does some processing and the WinService then does a “call back” with the “real response” / outcome of the processing.  Both communication routes are secured via the same bindings. We didnt have any trouble with the transmission from BizTalk to the WCF service but the WinService initiated communication always failed with a HTTP 502 : Bad Gateway message. This had us stumped for a while as some articles we read seemed to suggest that there was a problem on the server. Now if you have looked into custom bindings and certs in BizTalk, you will know that there is some really heavy duty and scary stuff there so we wondered whether the way we had setup the certs and credentials info was playing havoc with the communication.

So I used the trusty WSCF.blue and mocked up another WCF service to mimic the BizTalk endpoint replete with the bindings (no, WSCF Blue doesnt support custom bindings yet, but will sometime soon) and swapped the URI in the WinService and it worked just fine. Still puzzled, I then created a console app to mimic the winservice and it was able to communicate with both my mock BizTalk service as well as the real endpoint. Curiouser and curiouser. At that time the only difference was in the App Pool between the BizTalk service and mock service, but i used the same app pool and still the problem only manifested between the Winservice and Biztalk.

Anyway, with some guidance from an expert colleague (who, unfortunately wasnt around when I was writing those mock clients and services) It turned out that the problem was with a setting “UseDefaultWebProxy”  in the Win Service. The difference between the WinService and the console client was that the console was running under my credentials and in the IE settings I had turned off the proxy for the specified URL so although I didnt specify the setting as “false” in the client it automatically picked up my settings. But the WinService was running under another user account, and so the system didnt care about my proxy settings and we had to explicitly set the value to “false”. 

That was it. A little setting buried deep in the bindings that caused us the grief. After sorting this out I stumbled across this post from Kenny Wolf  which points to the offending setting. Wish I had found that earlier, but then I wouldnt have realized that aspect of the console client where it picked those settings. So, hope this helps someone in a similar situation.

WSCF blue: Beta-1

WSCF-blue, the WCF successor to the popular WSCF (Web Services Contract First) tool is now available in its first beta release at CodePlex. I’m delighted to be a part of the team working alongside guys like Christian Weyer, Buddhike De Silva, Edward Bakker and Alex Meyer-Gleaves.

If you aren’t familar with the tool, basically WSCF provides tooling to approach web service development in a contract first manner, specifically WSDL/XSD first rather than code-first. WSDL is often the favored ‘neutral’ ground for specifications to be agreed on in large projects where interoperability is key between heterogenous technologies. It is also common to find that the WSDL for web services is the first thing to be discussed between an application development team and a service provider (in any development / integration project). So , having a tool that allows you to work with WSDL and XSD in a consistent, reliable manner without getting bogged down in the intricacies of WSDL and associated specifications is a must. WSCF aims to address the lack of tooling in this space (especially in the .NET world).

WSCF is a Visual Studio 2008 add-in that provides the following features

• A WSDL Wizard that allows the developer to step through the creation of a WSDL from one or more XSDs
• A DataContract Generator (similar to XSD.exe, XSDObjectGen.exe and Svcutil.exe) that generates the . NET equivalent of the XSD types
• A Service/endpoint stub (SVC) generator and
• A client proxy generator.

In the current beta release, the stand-alone Data Contract generation option is NOT available, but that is something we will be addressing as a priority and should be available in the RTM release. However,  data contracts are generated as part of the service side stubs and client side stubs.

You can either install the MSI which makes the add-in available for an open solution (as the context menu for an XSD or WSDL) or if you are using the source version, you can debug the add-in directly using F5 which launches VS in the experimental hive as is normal for all add-ins.

I have been an regular user of WSCF since its ASMX days and found the quality of the code generated to be very high. We do hope you find that this tool goes some way toward addressing your needs with WCF development and are all keen to get feedback, bug reports and feature suggestions. With the number of project templates that are available for WCF (Web Site, Web Site -Remote IIS, WCF Class Library, WCF Service Application) and the rules that some of them have around structure, it has been quite a bit of work to get the features such as SVC generation sorted and I’m pretty sure that there will be other scenarios you are involved with where the tool needs improvement or enhancements.

Here are some other posts on the beta release by the other members of the team

• Next generation of a success story: WSCF.blue for WCF goes Beta 1
• WSCF.blue Beta 1 is out!
• WSCF blue – Are you doing contract first?

Keep an eye on the CodePlex space for information relating to the roadmap etc. Do check it out and let us know what you think of it.

Workflow Services & Application Protocols

Its taken some time (and a lot of procrastination) but I’ve finally decided to properly get into WF and WCF and as a Biztalk guy, one of the things that most interests me is Workflow Services. I’ve known the basics of all of this for a long time but never dived into it, so i started sort of working backwards and chose Workflow Services as my starting point (no use in trying to learn the old Data Exchange mechanisms of WF V1 now really). I picked up some nice videos from Channel 9 and some from the VS2008 Training Kit.

I soon got into the video titled Building WCF Services with WF and within 2 minutes ran into something that disturbed me so much I had to write this immediately. When talking about the advantage of writing WCF services with WF, Pravin says that one of the pros is that “Application Protocol is enforced” and goes on to give an example where if we had a service where there was an AddPurchaseOrder and also a CreateCustomer, we (as service implementors) might expect that the Customer would be created first before the Purchase Order. But the client would not know this by looking at the WSDL. So we set up a flow in the WF and expose these methods at some point and if the client calls them out of sequence we can throw exceptions etc.

I dont know about you, but IMO, this would not be good service design. Let me try to explain. First of all I wouldnt mix two widely different documents in the same service unless this was merely a ‘composite’ which internally invoked the CustomerService and the PurchaseOrderManagement service. Even if it was a composite, the contract specified by this composite should allow the client to give the customer info (or a pre-existing ID) along with the other PO info all together and then communicate with the Customer and POMgmt to create the customer or update an existing record and then pass stuff between them all through well defined messages.  Those backend services are still valid on their own and can accept messages being sent directly to them from other clients but the composite is providing some extra functionality. Now if the whole process was rather long running and there was state to be maintained in between calls to those backend services then we can make use of low level persistence services in the framework so we could dehydrate the composite while waiting for responses. But I would not make the composite service so ‘stateful’ that it exposed two or more methods as entry points and insisted that they be called in a sequence. (its a completely different matter if you are implementing a convoy in Biztalk where you may have multiple receive shapes which are linked together by correlation. Here we are talking about different ‘method calls’). Theres no service provided here just a wrapped collection of methods.

Just creating a few classes (whether declaratively implemented or handcoded) and exposing them over the wire doesnt make them business services. It worries me that devs are going to be throwing together some simple classes , fitting WCF endpoints on top of them and saying “Look, we have SOA” (and worse still, with all the designers, the claim would be “we have model driven SOA”.)  This is like going back to the ‘bad’ old days where people thought they had webservices just because they could stick [WebMethod] attributes on function calls. Let me re-iterate my point, a collection of visually designed classes that can listen on the wire is not model driven SOA. Its just that – a collection of classes, nothing more, nothing less.

So, back to the ‘protocol’ business. Sometime ago there was a lot of talk of web service choreography and a choreography description language to specify a sequence of ordered message exchanges. If the ‘composite’ service (discussed above) was implementing something like this, it would make sense. It shouldnt expose two random entry points and force a sequence of invocation. And if the composite was not providing any other  value other than just stringing together two calls why not just make the contract of the ‘second’ service explicit (for instance , require a CustomerNumber or something) that makes it apparent that Customers should exist first and then we could do away with the composite altogether.

But theres nothing wrong with the Workflow Services as such (at least nothing thats immediately apparent considering im just getting into them.. flaws may surface in a few days). It seems like WF and WCF were made for each other and this is a good way of linking the two. I just dont buy the ‘application protocol enforcement’ claim.

What do you think? What ways would you choose to implement sequencing? Does anyone else use the workflow services to enforce sequence?