.NET My Services is Microsoft's first attempt at creating a professional, commercial and widely available Web Services platform.

The .NET My Services umbrella hosts a number of different Web services, such as a Calendar service, a Contacts repository, and much, much more. These services are major building blocks for the "Everywhere, Anytime" vision, but best of all, they are relatively easy to implement and use in your own applications and Web sites!

When I demonstrate .NET My Services (NMS), people frequently ask why they should care about this new technology. After all, many of the available (or planned) services seem to be pretty simple. An example is the .NET Calendar service. Why do we need another calendar, since Exchange, Outlook and other PIM applications provide great calendaring capabilities? While these questions are valid, .NET My Services represents a significant paradigm shift in managing this type of information. Calendar information is traditionally stored in a local database, such as an Outlook data file. More sophisticated applications store this kind of information in more centralized data store, thus allowing co-workers to share their calendar information (this was one of the main reasons I moved to Exchange Server). However, those scenarios are still very limited. Although co-workers can have access to calendar information, this information may not be available to the wide range of applications such as PDA's, cell phones, pagers and even other applications or Web services.

For security reasons, all .NET My Service interaction requires user consent on a very granular level.

Imagine a scenario where you book concert tickets online. A Web site may show you several options for a certain event and, at the same time, present your calendar, your wife's calendar and a friend's calendar. The Web site can then automatically suggest a concert date that coincides with openings in each schedule (this assumes your wife and friend give you access to their calendars). Once you book those tickets, a calendar entry is automatically added to your calendar, your wife's calendar and your friend's calendar. In addition, .NET My Services (NMS) sends an alert to your wife and friend to make sure they are aware of the event and approve of the new calendar item. The .NET Alerts service knows how to contact your friend and your wife, based on data contained in the .NET My Profile service, which contains information such as email addresses, cell phone numbers, pager numbers and instant messaging information. Of course, .NET Alerts are highly configurable to make sure everyone receives alerts when and where they want them.

Once the tickets are booked, the service can send those tickets to you or to all involved parties individually. To do so, the Web site keeps a list of customers in .NET My Contacts. One classic problem with contact information is its likelihood to become outdated quickly. To avoid this problem, contacts stored in .NET My Contacts may be linked to profiles, instead of being stored as "stand-alone." This way, rather than storing your address in their database, a "subscription" to your .NET My Profile information is stored in the Web site's My Contacts repository. When querying contact information, the data looks the same as a stand-alone contact; however, it is automatically synchronized with your profile. Assuming you keep your own profile up to date (and so do your wife and friend), the Web site will be able to send the tickets to the right addresses, no matter how often you move or how many residences you have. The same is true for your magazine subscriptions, bills and birthday cards you receive.

Of course, all of this has security implications. First of all, do you really want other people, organizations and businesses having access to your address information? Well, if you subscribe to a magazine or order tickets, you probably do. However, if "Victoria's Great Web Cam" wants to send you unsolicited information, you are probably less interested. For this reason, NMS requires the user's consent for all interaction with NMS data and services. In other words, if you want the event Web site to access your wife's calendar or profile, your wife must first grant permission to that Web site. All NMS interaction is opt-in only, meaning there is no access granted by default. This effectively puts an end to the possibility of spam via .NET Alerts or other NMS services.

NMS requires all interaction to be authenticated and authorized through Kerberos security tickets and tokens. Those tokens are acquired through .NET Passport authentication (yes, .NET Passport is now Kerberos based, or at least will be when all of this becomes available). This means that virtually all NMS services rely on .NET Passport.

Can You Use This Today?

At this point, you may think that some of this sounds like science fiction. And for most people it is. The production version of .NET My Services is not yet available (with the exception of .NET Passport and .NET Alerts). However, you can download a technology preview version (pre-beta) from Microsoft at http://www.microsoft.com/MyServices.

The technology preview includes what's referred to as ".NET My Services In-A-Box." This preview installs all of its services on your local computer. This effectively takes care of one of the biggest problems with Microsoft's old Passport SDK (arguably Microsoft's first real Web service), which required the developer to sign up for a test account on the Passport test server. This was a cumbersome and manual task. The majority of development time went into configuring your services correctly, involving email communication with Microsoft, which can take a considerable amount of time. This process was discussed in Issue 3/2001 of Component Developer Magazine. With NMS, these problems are a thing of the past. Everything can be developed and tested locally, even if you don't even have a connection to the Internet (which is nice, since you can work on your application on a plane).

For more information on setup and configuration, see the sidebar "Setup Woes."

Basic Interaction with .NET My Services

Interaction with NMS is SOAP based. All requests to NMS are based on XML messages wrapped in SOAP Envelopes. What makes things a little tricky is that the SOAP envelope has to contain security information. In a real-world environment, those security tokens are provided by .NET Passport. For now however, these tokens are based on local system accounts.

All .NET My Services are opt-in by nature, putting an end to unsolicited information or unwanted data access by third parties.

The XML messages embedded in the SOAP package are based on a standard called "HSDL." HSDL is an XML-based language of its own. However, it is based on standards such as XPath, making it easy for people familiar with XML to interact with .NET My Services. The sum of all those XML-based standards is referred to as the XML Messaging Interface or XMI.

The core NSM services are based on the data stored in the NSM respository, as well as the logic and services surrounding that data. The combination of NMS data and logic is referred to as the NMS Service Fabric. Figure 1 provides an overview of the NSM architecture.

Figure 1: Basic interaction with the .NET My Services service is based on SOAP messages that use .NET Passport-based Kerberos security tickets.

Once we are past the details of SOAP interaction (which, can utilize HTTP as well as several other protocols), there are two main building blocks the developer has to be concerned with: the XML schema used by each individual service and the HSDL used to interact with the service. You can imagine each service as a humongous XML document stored on the Web against which you can run queries and commands. For instance, the .NET MyFavoriteWebSites service presents information in the following format:

<myFavoriteWebSites>
  <favoriteWebSite 
    id="C30C45FB-11DA-4CC0-8EE6-70FDD12858D2">
    <title xml:lang="en">Microsoft<title>
    <url>http://www.microsoft.com</url>
  </favoriteWebSite>
  <favoriteWebSite 
    id="137E8DD0-DB35-4FDC-8C40-238C41A1CF92">
    <title xml:lang="en">EPS Software</title>
    <url>http://www.eps-software.com/<;/url>
  </favoriteWebSite>
</myFavoriteWebSites>

This represents the minimum amount of information stored in the repository. Even simple services, such as .NET MyFavoriteWebSites, are capable of storing much more information, but most of it is optional.

This article will demonstrate using the .NET MyFavoriteWebSites service. Future issues will explore several of the more complex services in detail (see Table 1 for a list of all available services).

Using HSDL, you can run a simple query against the MyFavoriteWebSites service. The following query returns all favorites stored in a user's MyFavoriteWebSites repository:

<queryRequest>
   <xpQuery select="//myFavoriteWebSites" /> </queryRequest>

The root element indicates to NMS that this is a query request. The xpQuery node ("xp" stands for "XPath") defines the query in the select attribute. Note that whatever is specified here can be any XPath expression (provided it conforms to the MyFavoriteWebSites schema).

This is the basic query structure. One item to note is that the query must also contain information about the appropriate .NET My Services namespaces. NMS uses this information to verify your requests against the HSDL standards and the MyFavoriteWebSites schema. For this reason, a real-world query would look like:

<hs:queryRequest xmlns:hs = "http://schemas.microsoft.com/hs/2001/10/core" 
xmlns:m = "http://schemas.microsoft.com/hs/2001/10/myFavoriteWebSites">
  <hs:xpQuery 
    select="//m:myFavoriteWebSites" /> </hs:queryRequest>

The main difference with this syntax is the inclusion of the "hs" namespace in the query commands, and an "m" namespace for all information that's directly related to the favorite's data. Note that the "m" namespace is included in the XPath expression stored in the select attribute.

To execute this query, you need to send it to NMS. This can be done a number of different ways. For instance, you could make a SOAP call using the Microsoft SOAP Toolkit, from within a COM-based environment. The easiest way is to use the hspost.exe command line utility provided by the NMS SDK. Before you use that utility (or any other communication mechanism), you have to make sure your server is provisioned appropriately. Provisioning is the process of granting permission to a service. The following code demonstrates granting access to the MyFavoriteWebSites service. This command is run from a command window.

hsprov ?l http://localhost ?o markus ?s myFavoriteWebSites

This is a one-time event. For more information on server provisioning, see the sidebar.

Now, you are ready to post your query. First, store the query in a file called "request.xml." Then, execute the following command from the Command Line interface (DOS):

hspost ?f query.xml ?u markus ?s myFavoriteWebSites

Make sure that you replace the user name ("markus") with the user you provisioned. This goes for all the examples in this article.

When you run this command, you will be shown a SOAP Response message. Within the body of that message, you will see the NMS response, similar to the response shown in Listing 1. In this example, a single request was sent to the service (each HSDL request can contain multiple queries). The service then returns a queryResponse with a single xpQueryResponse node embedded. Within that response you will see the MyFavoriteWebSites information as presented above, except that this time the namespace information has been added.

Note that the response status is "success." If NMS encountered an error, the status would have been "failure" and the response would contain information about the error encountered. If something goes wrong at this point, you most likely forgot to provision the server (or provisioned it for the wrong user), or you didn't provide the correct namespace information. If you are confident that the problem must be of a different nature, make sure your setup was successful (see sidebar "Setup Woes"). Note that your response might simply be blank, yet successful. By default, there is no data stored in a user's favorites.

Understanding HSDL

All NMS services support six basic commands: insert, update, query, replace, delete and subscriptionResponse. Individual services may support commands beyond these basic commands. For instance, the My Calendar service supports a getFreeBusyData request, among others.

.NET My Services replaces rather common functionality. However, the way it is done represents a major paradigm shift that results in quite a revolutionary user experience.

To utilize the full power of HSDL, you need to understand the schema of each service. In addition to plain schema information, certain nodes within a schema are color coded. There are blue, red and black (or plain) nodes. Blue nodes are main nodes, typically the root node of a schema, or the first level sub-nodes. In the MyFavoriteWebSites example, the myFavoriteWebSites as well as each favoriteWebSite node are considered "blue" nodes. Each blue node represents an entity that can be directly queried, inserted, and deleted.

Red nodes can be used for queries, but they are not stand-alone nodes. For instance, you can query the title of a favorite, but you could not add or delete a title node, without adding or deleting a whole favorite Web site (one of the blue nodes).

Black (plain) nodes are opaque to HSDL queries, meaning that they will be returned, but they cannot be queried directly. This is fine, because most nodes are either red or blue. An example of a black node is the xml:lang node/attribute on a favorite Web site's title. Although this node is part of every result set, and it in fact is a required node when adding a new favorite, you cannot directly query the language information. If you want to retrieve information about Microsoft's German Web site, you'd have to first query all of the Microsoft sites and then pick the appropriate one from the result set.

Now that you understand these basic concepts, you can insert a new favorite in our repository. To do so, create the following HSDL in a file named insert.xml:

<hs:insertRequest 
  select="/m:myFavoriteWebSites"
  xmlns:hs = "http://schemas.microsoft.com/hs/2001/10/core" 
  xmlns:m = "http://schemas.microsoft.com/hs/2001/10/myFavoriteWebSites">
  <m:favoriteWebSite>
    <m:title xml:lang="en">Component Developer Magazine (CoDe)</m:title>
    <m:url>http://www.code-magazine.com/<;/m:url>
  </m:favoriteWebSite>
</hs:insertRequest>

You can send this request to the server using the following command:

hspost ?f insert.xml ?u markus ?s myFavoriteWebSites

Note the significance of the blue element here. myFavoriteWebSite is a blue element and can therefore be inserted as a sub-element of myFavoriteWebSites (as specified in the select attribute). The title element on the other hand is a red element. We cannot add a title element to the repository by itself.

If the response is a success, you can re-run the previous query (see above) to retrieve the full list of favorites, which should now include a link to CoDe Magazine's Web site. You may notice that the title is a little long. You can replace the current (long) title with a shorter version like so:

<hs:replaceRequest
  xmlns:hs = "http://schemas.microsoft.com/hs/2001/10/core" 
  xmlns:m = "http://schemas.microsoft.com/hs/2001/10/myFavoriteWebSites"
  select="//m:favoriteWebSite[@id='E1016A65-BE06-4962-B40B-31393BE6EAC4']">
  <m:favoriteWebSite>
    <m:title xml:lang="en">CoDe</m:title>
    <m:url>http://www.Code-Magazine.com</m:url>
  </m:favoriteWebSite></hs:replaceRequest>

Note the select attribute on the replaceRequest node. It uses an XPath expression to select a specific Web site based on its id attribute (all blue tags have id attributes). The id is a GUID (globally unique id). Whenever a new item gets added to the NMS repository, it is assigned such an id. You will have to adjust that id based on the id generated when you inserted the favorite (above).

This brings us to more complex query operations. As mentioned before, the select attribute can contain a full-featured XPath query. For instance, you could query Microsoft's Web site in the following fashion from our repository of favorites:

<hs:queryRequest 
  xmlns:hs = "http://schemas.microsoft.com/hs/2001/10/core" 
  xmlns:m = "http://schemas.microsoft.com/hs/2001/10/myFavoriteWebSites">
  <hs:xpQuery select = "//m:favoriteWebSite[m:title='Microsoft Corp.']" /> </hs:queryRequest>

This would generate the following result set (namespaces omitted for readability):

<queryResponse>
  <xpQueryResponse status="success">
    <favoriteWebSite 
    id="C30C45FB-11DA-4CC0-8EE6-70FDD12858D2">
      <title 
      xml:lang="en">Microsoft Corp.</title>
      <url>http://www.microsoft.com/<;/url>
    </favoriteWebSite>
  </xpQueryResponse>
</queryResponse>

Note that in this example, no myFavoriteWebSites node was generated. That's because the XPath query specifically requested the favoriteWebSite node to be the top-level node (and there could have been several of those). This is very useful. For instance, you could query a list of all URLs stored in your favorites using the following query:

<hs:queryRequest>
   <hs:xpQuery select="//m:myFavoriteWebSites/m:favoriteWebSite/m:url" /> </hs:queryRequest>

The result would be as follows:

<queryResponse>
   <xpQueryResponse status="success">
      <url>http://www.microsoft.com/<;/url>
      <url>http://www.eps-software.com/<;/url>
   </xpQueryResponse>
</queryResponse>

You should now understand the basics of the HSDL. If you are confused by this, I recommend reading up on XPath. If you would like additional information on other HSDL commands, refer to the SDK documentation.

So far, all we have done is post requests to NMS using hspost.exe. Of course, this is not how things are done in the real world. Instead, you would directly interact with NMS from within your application or Web site. So let's take a look at how this is done.

Interacting with NMS through SOAP

For the sake of this article, I'm assuming that most Visual Basic developers have moved on to Visual Studio .NET. For this reason, I create my SOAP examples using Visual FoxPro and Visual Basic.

Using Visual FoxPro 7.0, create a new form and drop a Microsoft Web Browser ActiveX control on your form. The browser control is notorious for trying to refresh itself before the VFP form is ready. For this reason, you will have to put the following code in the Refresh event of the browser control:

NODEFAULT

Next, drop a ComboBox and a CommandButton on the same form and arrange them as shown in Figure 2. Add the following code to the Valid() event of the ComboBox:

Figure 2: A simple .NET My Favorites aware web browser written in Visual FoxPro.
THISFORM.OleControl1.Navigate2(;
   THIS.DisplayValue)

Note that "OleControl1" is the name of the Web Browser control.

The idea is to query all the favorites from the repository when the form initializes. The favorites are simply added as list items to the ComboBox. Listing 2 shows the code that goes into the form's Init() event. The code is quite lengthy, but not very complicated. A SOAP Connector is utilized to connect to the NMS server (on the local machine). Although the SOAP message could be created as a regular string, I generate it using the SOAP Serializer, which greatly simplifies the task. Finally, the SOAP message is sent off to the server using the SOAP Reader object. If you are not familiar with this task, refer to the SOAP documentation.

Note that part of the SOAP header is a Passport User ID (PUID). In the real world, the Passport Manager object would provide this. However, since there is no real Passport interaction, NMS In-A-Box generates a temporary, local PUID. You can query that PUID using the hspost.exe command line utility:

hspost.exe ?p

This returns the temporary PUID for the current user name. Remember to replace that PUID in the code in Listing 2 and Listing 3 (stored in the lcUserID variable). Otherwise, NMS will not grant you access.

The real interesting part of the SOAP message starts when adding the queryRequest element. This section adds XML identical to the contents of the query.xml file created above. All the other SOAP code concerns the SOAP envelope, a task that is handled by the hspost.exe utility when using the command line utility.

Towards the end of the listing, the data requested is extracted from the Body element of the returned SOAP message. Now you can use the XMLDOM to parse the result, retrieve all the URLs and add it to the ComboBox object where the user can pick a URL to navigate.

Most of this code remains the same, no matter what action you desire or what service you interact with, with the exception of the utilized namespaces and the actual HSDL command you send. Also, if you intend to insert information into the repository, you need to change the method from "query" to "insert." Listing 3 demonstrates that. The code in Listing 3 goes into the Click() event of the button. It is mostly the same as Listing 2, with the exception of the actual body (HSDL) and the query method ("insert"). Listing 3 is the equivalent of posting insert.xml to NMS using hspost.exe, with the exception that the URL and title get generated based on the selected value of the ComboBox. This gives you the ability to add new items to the favorites. Voila! Your simple .NET MyFavoriteWebSites-aware browser is complete.

Note that Visual FoxPro 7.0 has foundation classes that make it easier to deal with SOAP messages. Also, the IntelliSense Manager can automatically generate SOAP-specific code to access Web services. I didn't use any of those mechanisms for two reasons: first, I wanted to keep my example generic so the example would make sense for non-VFP SOAP programmers. Second, I couldn't get Visual FoxPro to recognize the WSDL (see sidebar) associated with the individual NMS services. I assume this to be a problem with the current beta of NMS and will be fixed before release.

Listing 4 demonstrates how to access .NET MyFavoriteWebSites via SOAP and Visual Basic 6. It is a little simpler than the Visual FoxPro versions, because it doesn't do anything with the returned information, other than displaying it in a message box. However, it should be sufficient to get you started with NMS using VB6 (although I'm sure you are much more eager to try this with Visual Studio .NET).

Using NMS with Managed Code (VS.NET)

When using .NET My Services with managed code (either through C#, Visual Basic .NET, or any other .NET language), the basic principle of accessing the services remains the same. However, Microsoft provides a number of proxy classes that make it easier to use NMS and require very little code. Basically, most of the SOAP and HSDL tasks are reduced to a minimum. However, it is still important to understand these basic building blocks because many of the abstract classes map directly to HSDL request and respond messages.

In the early beta of the .NET My Services SDK, Microsoft provided two helper projects/assemblies to work with NMS: HsSoapExtension and ServiceLocator. The SOAP extensions assembly deals with NMS-specific SOAP tasks, such as encrypting and decrypting SOAP calls. You won't directly deal with this assembly, but it is used by the ServiceLocator assembly, which provides the proxy classes that map to the individual services.

As an example, I'm going to create a simple form that shows all favorites in a Treeview control. To follow along the example, create a new C# Windows application. The first step is to import both the HsSoapExcension and ServiceLocator projects into your solution. Also, add references to both assemblies in your new project. You should now have one solution with three projects and your Windows Application as the startup project, as shown in Figure 3.

Figure 3: A Visual Studio .NET test solution.

Now, add a Web Reference to your project, to reference the .NET MyFavoriteWebSites service. To do so, right-click on your project and click "Add Web Reference...". Browse to http://localhost/wsdl/NETMyServices.disco, pick the favorites service, and click the "Add Reference" button (see Figure 4). This adds a Web reference named "localhost." You can rename that service if you wish, however, we are going to stick with that name for this example.

Figure 4: Adding a Web reference to .NET My Services.

Now, open your main Web form for editing. Add a Treeview control, which we will use to display the retrieved favorites. Then, open the code view and add the following references to the very top of the source:

using System;
using System.Drawing;
using System.Collections;
using System.ComponentModel;
using System.Windows.Forms;
using System.Data;
using Microsoft.Hs.ServiceLocator;
using Microsoft.Hs;
using MyServicesTest.localhost;

Remember to replace "MyServicesTest" with the name of your project.

Now add the code from Listing 5 to the form's load event. The code is pretty straightforward. First, the ServiceLocator object instantiates a reference to NMS in general and subsequently to the MyFavoriteWebSites service. The myFav object is the provided wrapper object specific to the MyFavoriteWebSites service.

Then, we create a new queryRequest object as well as an xpQueryType object. We set the select property of the xpQuery object to retrieve favorite Web sites based on an XPath query. Note that this construct maps 1:1 to the HSDL XML structure we would pass through SOAP (which is what happens behind the scenes). We attach the xpQuery object to the queryRequest object (this is a collection and one could attach as many queries as we desired), and fire the query through the myFav proxy object. This generates a queryResponse object. This object has a collection of query responses and each of those has a collection of favorite Web sites. We can now simply access properties on those objects and add corresponding items to the Treeview on your form. The result is shown in Figure 5.

Figure 5: Adding favorites to a Treeview in a .NET-based WinForm.

As you can see, the facilities provided in Visual Studio .NET make accessing NMS fairly simple. Developers do not have to worry about any underlying SOAP details. As mentioned above, the proxy objects used in this example are preliminary objects. Microsoft provides very little documentation on these objects and I wouldn't be surprised if these change significantly before release.

Conclusion

.NET My Services is a fascinating new platform. While the services may seem common and ordinary, this is not the real power. The real power comes from the interaction and cooperation of the different services.

This article examined the very basics of .NET My Services, with examples limited to .NET MyFavoriteWebSites. In future issues of Component Developer Magazine, I will examine several of the individual .NET My Services in more detail. Make sure you keep this issue as a reference!

Markus Egger