Though the XML Editor in Visual Studio 2005 has many improvements, it still lacks support for writing and testing XPath queries.

In this article, I’ll show you how to leverage the Visual Studio SDK to extend the XML Editor to allow you to write and text XPath queries in Visual Studio 2005.

The XML Editor has many enhancements in Visual Studio 2005:

Design time well-formedness and validation errors.
Validation support for Schema, DTD, and XDR.
Inferring an XSD Schema from an XML instance.
Converting a DTD or XDR to XSD Schema.
Context-sensitive IntelliSense.
XSLT editing, viewing the results of the transform.
Standard Visual Studio code editing, such as outlining and commenting or un-commenting.

To add support for testing and writing XPath statements, I decided to keep things pretty simple by just creating a Visual Studio tool window where a developer can write an XPath statement and test it against the current document in the XML Editor. After running an XPath query it will list and highlight the nodes returned from the query. I called this project, XPathmania, and it is part of the Mvp.Xml open source project on CodePlex. The Web address for the project is http://www.mvpxml.org and besides XPathmania, the site contains .NET implementations of EXSLT, XML Base, XInclude, and XPointer, as well as a unique set of utility classes and tools making XML programming in .NET platform easier, more productive, and effective.

One of the biggest stumbling blocks when developers first learn XPath is the concept of namespaces and especially the topic of default namespaces. So, the Tool window will have to support entry of namespaces and their associated prefixes, along with some guidance for the case when an XPath query doesn’t seem to find nodes and there is a default namespace. For more information, please read the XPath and Namespaces sidebar.

Creating a Tool Window Project

To create this project, I first downloaded and installed the most current Visual Studio 2005 SDK. (I used the Feb 2007 release also known as VS SDK 4.0.) With Visual Studio 2005 loaded, under the Other Project Types folder you will find a new folder named Extensibility along with a number of new project types. I created a new project and selected the Visual Studio Integration Package project type. Visual Studio will walk you thru a wizard where there are a number of different options. Make sure you choose to create a C# project and to create a tool window (see the Visual Studio SDK documentation for more details on this step). The wizard will generate a new project with a number of classes and resource files, but the ones that you are interested in are:

    VsPkg.cs  The class that implements the managed package which will extend Visual Studio.

    MyToolWindow.cs  The tool window that is exposed by the package.

    MyControl.cs  The control used by the XPathToolWindow, and is the heart of this project.

The wizard also added these Visual Studio SDK assembly references to the project:

Microsoft.VisualStudio.OLE.Interop

Microsoft.VisualStudio.Shell

Microsoft.VisualStudio.Shell.Interop

Microsoft.VisualStudio.Shell.Interop.8.0

Microsoft.VisualStudio.TextManager.Interop

I then added some additional references that I will need later:

Microsoft.VisualStudio.Package.LanguageService

Microsoft.VisualStudio.TextManager.Interop.8.0

I also renamed the MyToolWindow.cs file to XPathToolWindow.cs and MyControl.cs to XPathControl.cs. The VsPkg and XPathToolWindow classes are important to hosting your customizations within Visual Studio, but are not the focus of this article. I will concentrate on the implementation details and not on the generated classes.

Building the User Interface

Table 1 lists the user interface of this control (Figure 1) I created.

Steps to Execute an XPath Query

When an XPath query is requested to run (either by clicking the Query button or pressing Enter) the following should happen:

The first step is relatively easy-just clear out the ArrayList<T> variable, errors, set the DataSource of the resultsGridView to null, and set the BindingList<XmlNodeInfo> results class level variable to null. But before you set the results field to null, you need to loop thru the items, remove the event handlers for the OnMarkerChanged and OnMarkerDeleted events and then call Dispose on the instance of the item. Those event handlers are wired up in the “Build Results List” section of the code, and because you’re dealing with COM interop, if you don’t do this house cleaning, it will cause issues later.

Determining the Currently Active Visual Studio Document

Since Microsoft had to build the core of Visual Studio while they were still developing .NET, it is COM-based, which means that if you have to develop managed package extensions, there is a lot of COM interop going on. If you never wrote COM in C++, some of this can be a bit confusing. But once you get past that, then you will be ready to tackle the Visual Studio object model.

The XPath Tool is really only concerned with Visual Studio documents that are valid XML. But before you can get to the step of determining that, you have to go thru the process of getting access to the text editor of the currently active document and various parts of its user interface. As part of this process you’ll have to keep a few different variables grouped together. You could just declare them as private class-level fields, but that could get a little confusing, and there are a few methods and events that you may need, so I’ll show you how to create the internal class VisualStudioDocument that can encapsulate most of the data and behaviors associated with a Visual Studio document. I could have tried to extend a class from the Visual Studio object model, but I couldn’t really find one that fit the same definition as the one I had created, plus, any class would probably have been a managed wrapper over a COM object. Even after determining that I needed a VisualStudioDocument class, it still takes some steps to get the active document that really doesn’t seem to fit in the new class, so I put it in the GetCurrentSource method in the XPathTool control, which is called by the XPathQueryButton click event handler.

To create an instance of a VisualStudioDocument, you need three things: the Window Frame (IVsWindowFrame) for the currently active document, the Text Buffer (IVsTextLines) for the Window Frame, and the View Service (IOleServiceProvider) for the document.

First you’ll call Package.GetGlobalService and pass in the serviceType of SVsShellMonitorSelection.

IVsMonitorSelection selection = 
(IVsMonitorSelection)Package.GetGlobalService(
    typeof(SVsShellMonitorSelection)
);

Use the ShellMonitorSelection to get the IVsWindowFrame interface of the DocumentFrame.

object pvar = null;
if (!ErrorHandler.Succeeded(
  selection.GetCurrentElementValue(
 (uint)VSConstants.VSSELELEMID.SEID_DocumentFrame,
  out pvar)))
{
    this.currentDocument = null;
    return;
}
IVsWindowFrame frame = pvar as IVsWindowFrame;
if (frame == null)
{
    this.currentDocument = null;
    return;
}

Then use the IVsWindowFrame interface of the DocumentFrame to get the DocData object.

object docData = null;
if (!ErrorHandler.Succeeded(
  frame.GetProperty(
    (int)__VSFPROPID.VSFPROPID_DocData,
    out docData)))
{
    this.currentDocument = null;
    return;
}

The returned DocData could either be a pointer to an IVsTextLines interface, or if the document has an alternate editor, it could be a pointer to an IVsTextBufferProvider interface and you’ll need to use it to get a pointer to the current IVsTextLines.

IVsTextLines buffer = docData as IVsTextLines;
if (buffer == null)
{
    IVsTextBufferProvider tb = docData as
      IVsTextBufferProvider;
    if (tb != null)
    {
        tb.GetTextBuffer(out buffer);
    }
}
if (buffer == null)
{
    this.currentDocument = null;
    return;
}

The last bit of information needed is a pointer to the IServiceProvider for the current document’s view object, which can be requested using the IVsWindowFrame of the DocumentFrame.

object docViewServiceObject;
if (!ErrorHandler.Succeeded(
    frame.GetProperty(
(int)Microsoft.VisualStudio.Shell.
     Interop.__VSFPROPID.VSFPROPID_SPFrame,
out docViewServiceObject)))
{
    this.currentDocument = null;
    return;
}

Now you have all the information needed to create a VisualStudioDocument. If the currentDocument is null or, if it isn’t null and the instance of IVsTextLines just retrieved from the above step isn’t the same as the one in the currentDocument (meaning it is not the same document as represented by the current VisualStudioDocument), you should construct a new instance of VisualStudioDocument, passing in IVsWindowFrame, IVsTextLines and IServiceProvider from the early steps and setting it to the class-level field currentDocument. If you didn’t have to create a new instance, you have to check and see if there were any changes made to the source since the last time you executed a query. I accomplished this by using the Source property of my VisualStudioDocument class. I’ll cover the Source property, an instance of the Visual Studio SDK’s Package.Source class, in the next section of this article.

if (this.currentDocument == null || buffer != 
this.currentDocument.TextEditorBuffer)
{
    this.currentDocument = new
    VisualStudioDocument(frame, 
    buffer, docViewService);
    this.changeCount =
    this.currentDocument.Source.ChangeCount;
}
else
{
    if (this.changeCount !=
    this.currentDocument.Source.ChangeCount)
    {
      this.currentDocument.Reload();
      this.changeCount =
        this.currentDocument.Source.ChangeCount;
    }
}

Completing the Visual Studio Document

Using the frame, buffer, and document view service passed in via the constructor, you can go out and get the rest of information needed to complete this instance of the VisualStudioDocument. You need to stash the frame and buffer in the class-level variables, currentWindowFrame and textEditorBuffer. Then you can go out and get the instance of the Package.Source for this Visual Studio document and use that to parse and load the XmlDocument (Listing 1).. Since the Package.Source is maintained by the Language Service for this document, and you’re really only interested in Visual Studio documents that are using the XmlEditor, this is also a good time to check to see if the Language Service used for this textEditorBuffer is the XML Editor Language Service. To do this you retrieve the GUID for the textEditorBuffer’s Language Service and compare it to the GUID for the XmlEditor’s Language Service. Instead of having to add a reference to the XmlEditor and get the GUID at run time, I used Lutz Roeder’s Reflector on the XmlEditor dll, got the GUID, and placed it in a static private read-only field, xmlLanguageServiceGuid. If the GUIDs don’t match, this Visual Studio Document is not being edited by an editor you’re interested in, so don’t bother trying to load the Xml Document.

Loading the Xml Document with the DomLoader

It may seem sort of odd that this solution requires that you create and load an instance of XmlDocument since the XmlEditor should already have a parsed version of Xml in the XmlEditor, but two problems force you to load the XML a second time. First, only the objects and interfaces exposed via the Visual Studio SDK are supported, and the XmlDocument exposed from the XmlEditor is not part of the SDK. Second, the XmlDocument in the XmlEditor is not the System.Xml.XmlDocument, but a slimmed down version that does not include any XPath functionality. Microsoft will address both of these issues in Visual Studio 2008.

The System.Xml.XmlDocument is missing a very important feature that you’ll need if you are to map the nodes returned from an XPath query to the original source, the line and column that generated the instance of the node (this is another reason the XML editor has its own internal parse tree). To add this new functionality, I could have either created a new XmlDocument class that inherits from System.Xml.XmlDocument or just create a utility class that handles the parsing and mapping of the XML. Since I already had a version of the utility class (donated by Chris Lovett from Microsoft) named DomLoader, I decided to adopt this, and maybe at some future time, refactor to use a class inheriting from XmlDocument.

DomLoader

The DomLoader is an internal utility class that manually loads an XmlDocument. Since this article is about extending the XML Editor and not about manually parsing XML, I’ll spare you the details and offer a quick summary. Most of the magic is done in the LoadDocument method by looping thru the passed in XmlReader and creating a node matching the one in the XmlReader, and then adding it to the new XmlDocument. After adding the new node to the XmlDocument, it then adds the node and its location in the original source code (represented by the internal class LineInfo) into the Generic Dictionary<XmlNode, LineInfo> lineTable. Once the DomLoader has parsed the XML you can then use the GetLineInfo method to get the LineInfo for any XmlNode found in the XmlDocument created by the Load method.

Test for Valid XML Document

Now that you’ve loaded the active Visual Studio Document you can test it to see if it is a valid XML document, and if it isn’t, report back a warning that you have an invalid XML document and cannot run a query against it.

if (!this.currentDocument.IsValidXmlDocument())
{
   ReportError(
   new ErrorInfoLine(
      "Invalid XML Document - see Error List “+
      “Window for details",
      errors.Count + 1,
      ErrorInfoLine.ErrorType.Warning));
   return;
}

In the IsValidXmlDocument method, the code just checks to see if the currentXmlDocument of the VisualStudioDocument is null. If it is, then the VisualStudioDocument is not a valid XmlDocument.

Add the Prefixes and Namespaces

For each of the rows in the namespaceGridView provided by the user, you need to add a namespace into the currentDocument’s XmlNamespaceManager (so the XPath query can resolve the prefixes to namespaces). The first column will contain the prefix and the second column contains the actual namespace. The prefixes loaded here should be the same prefixes that the user utilized in their XPath query. If they are not the same, the XPath query will not return the desired results. If they did not use any prefixes in their XPath statement, then there will be no rows.

You need to skip any empty rows (rows that have both the prefix and the namespace cells equal to null), and you need to report an error if there is a row with a namespace without a prefix.

The VisualStudioDocument has a XmlNamespaceManager property (exposing the XmlNamespaceManager for the XmlDocument it contains). Use the AddNamespace method of the XmlNamespaceManager to add the prefix and namespace to the document.

Execute the XPath Query

Utilizing the VisualStudioDocument.Query method you can now execute the XPath statement entered by the user. If there are any errors you should also report them to the user.

XmlNodeList XPathResultNodeList = null;
try
{
   XPathResultNodeList =
    this.currentDocument.Query(xpathTextBox.Text);
}
catch (XPathException ex)
{
   ReportError(ex);
   return;
}

The VisualStudioDocument.Query method just calls the overloaded SelectNodes method of the currentXmlDocument with the XPath statement and the currentNamespaceManager, and returns the XmlNodeList.

internal XmlNodeList Query(string xpath)
{
   return this.currentXmlDocument.SelectNodes(
      xpath,this.currentNamespaceManager);
}

Display Results

The final solution will display the results in two ways: by listing the text of the node as a row in the resultsGridView, and by highlighting the text that represents that node in the XmlEditor. I’ll also add the ability to double-click on a row in the resultGridView and select the corresponding text in the XmlEditor. To accomplish both of these goals, I needed to create a new internal class XmlNodeInfo that encapsulated all the goop needed to highlight the text and convert the XmlNode to a string format. By encapsulating all the gory details into XmlNodeInfo, the actual code to display the results boils down to the lines of code in Listing 2.

Note that a nice side effect of using LineMarkers to display the results is that they track subsequent edits in the buffer and the line numbers in the Results grid will automatically update as a result.

Listing 3 shows the last bit of code needed to display the result grid and help supply some guidance around Xml documents that contain a default namespace (other than an empty namespace). If there were no results found for the query and the document element (aka root element) has a namespace without a prefix, then use the ReportError method to add a warning. This technique solves the number one question on newsgroups around getting XPath queries to work and was a major motivation for building this tool.

Conclusion

I developed XPathmania to help bring some XPath functionality to Visual Studio 2005. XPathmania is part of the open source Mvp.Xml project (www.xmlmvp.org) and the source code is hosted on CodePlex (www.codeplex.com). The Mvp.Xml projects are administered and (mostly) developed by Microsoft MVPs in XML technologies and Connected Systems worldwide. Its aim is to supplement the .NET Framework XML processing functionality available through the System.Xml namespace and related namespaces such as System.Web.Services.

XPath and Namespaces

One of the biggest issues developers new to XPath have with the language is how namespaces (and especially default namespaces) work. Most developers learn XML and XPath by way of XML documents without namespaces, and everything seems to work fine. But once you start adding namespaces to the mix, things get confusing. Take the following XML snippet: <music> <cd></cd> </music> If you want to find all the cd elements in the document, the XPath query would be //cd. That seems to make sense. But the confusion comes in when you make one seemingly small change-adding a default namespace to the music element. <music xmlns="urn:mvpxml.org/music"> <cd></cd> </music> Semantically, you just stated that the element music, and any child elements without a prefix, until overridden, is of the namespace urn:mvpxml.org/music. So when you execute the query //cd again, you would expect that you would get the same results as the first time, but that will not be the case. By not declaring a namespace you have successfully hidden from a little back corner of the XPath spec. You see, XPath doesn’t work on just the element name (or the literal string of the prefix plus the element name) but it uses something called a qualified name (QName) EXCEPT for the case when there is no namespace declared for the element (as in the above case). Once you add namespaces, the qualified name becomes the full namespace (replacing the prefix, if it exists) plus the element name (actually it looks more like this {urn:mvpxml.org/music}cd but you get the idea. The problem with the XPath statement is that you did not declare a namespace, as the XPath statement is not part of the XML document and thus does not use the same namespace-prefix declarations used in the document. So you have no choice but to add a namespace to the XPath query, which you do using the XmlNamespaceManager. But there is one last twist. You can’t add a namespace to the namespace manager without a prefix (i.e., no default namespace). In this case I’m giving the namespace urn:mvp.xml.org/music the prefix of “a” and adding this association to the namespace manager. With the mapping of the prefix “a” to urn:mvp.xml.org/music, the XPath query of //a:cd resolves to the query: find all the {urn:mvpxml.org/music}cd elements in the document, and now you will get the desired results. In theory the developer writing the XPath statement has control over the namespace-prefix mapping used in the XPath statement but not the namespace-prefix mapping used in the XML document. The following snippet is syntactically identical to the previous XML snippet, from the point of view of the XPath model: <b:music xmlns:b="urn:mvp.xml.org/music"> <b:cd></b:cd> </b:music> The difference is only in the prefix used in the documents (none in the first and “b” in the second), and the XPath model resolves both forms of the element cd to the QName {urn:mvpxml.org/music}cd. The XPath statement //a:cd (using the previous namespace manager) will work on both XML documents, since the query uses the QName not the element name or the prefix plus element name. This may seem a little odd at first, but once you realize that the XPath statement isn’t built within the context of the XML document being queried, but as a separate entity, it (hopefully) makes a little more sense.

Listing 1: VisualStudioDocument constructor

internal VisualStudioDocument(IVsWindowFrame frame, 
    IVsTextLines buffer, IOleServiceProvider docViewService)
{
   this.textEditorBuffer = buffer;
   this.currentWindowFrame = frame;
   if (GetSource(docViewService))
   {
      LoadDocument();
   }
}
private bool GetSource(IOleServiceProvider docViewService)
{
   //get language service id from TextBuffer
   Guid languageServiceGuid;
   this.textEditorBuffer.GetLanguageServiceID(
      out languageServiceGuid);
   // check to see if it the Xml Editor's Service Id
   if (languageServiceGuid !=
     VisualStudioDocument.xmlLanguageServiceGuid)
   {
      //if not Xml Editor Service Id, return false (not valid XML
      // source)
      return false;
   }
   // get the Language Info for the XML Editor (for colorizers)
   IntPtr ptr;
   Guid guid = VisualStudioDocument.xmlLanguageServiceGuid;
   Guid iid = typeof(IVsLanguageInfo).GUID;
   if (!ErrorHandler.Succeeded(
      docViewService.QueryService(ref guid, ref iid, out ptr)))
   {
      return false;
   }
   this.currentDocumentLanguageInfo =
     (IVsLanguageInfo)Marshal.GetObjectForIUnknown(ptr);
   Marshal.Release(ptr);
   // upcast the Language Info Interface to the full 
   // Language Service
   LanguageService langsvc = this.currentDocumentLanguageInfo 
      as LanguageService;
   // get the Source for the current text buffer
   this.currentSource = langsvc.GetSource(this.textEditorBuffer);
   return true;
}

Listing 2: Build results list

results = new BindingList&lt;XmlNodeInfo&gt;();
if (XPathResultNodeList != null)
{
  foreach (XmlNode Node in XPathResultNodeList)
  {
    LineInfo CurrentLineInfo =
    this.currentDocument.GetLineInfo(Node);
    if (CurrentLineInfo != null)
    {
      XmlNodeInfo info = new XmlNodeInfo(Node,
        CurrentLineInfo.LineNumber, 
        CurrentLineInfo.LinePosition);
      results.Add(info);
      TextSpan span = GetNodeSpan(info);
      IVsTextLineMarker[] amark = new IVsTextLineMarker[1];
      int hr =
        this.currentDocument.TextEditorBuffer.CreateLineMarker(
          (int)MARKERTYPE2.MARKER_EXSTENCIL,
          span.iStartLine, 
          span.iStartIndex, 
          span.iEndLine, 
          span.iEndIndex,
          info, amark);
      info.Marker = amark[0];
      info.MarkerChanged += new EventHandler(OnMarkerChanged);
      info.MarkerDeleted += new EventHandler(OnMarkerDeleted);
    }
  }
}

Listing 3: Bind grid to results

this.resultsGridView.AutoGenerateColumns = false;
this.resultsGridView.DataSource = results;
if (this.results.Count &gt; 0)
{
   this.resultsGridView.Focus();
   this.queryTabControl.SelectedTab = this.resultsTabPage;
}
else
{
   if (!String.IsNullOrEmpty(
      this.currentDocument.DocumentElementDefaultNamespace) &amp;&amp;
      !(this.currentDocument.DocumentElementDefaultNamespace
        == "")
      )
   {
      this.ReportError(
         new ErrorInfoLine("Document has a default namespace.  “+
            “Did you make sure to add it to the Namespace Table “ +
            “and use its prefix in your XPath query?", 
            errors.Count + 1, ErrorInfoLine.ErrorType.Warning));
      return;
   }
}

Control	Tab	View	Description
xpathTextBox			a text box for the input of the XPath query
xpathQueryButton			a button to execute the XPath query
queryTabControl			a tab control with 3 tabs
	Results	resultsGridView	displays the results of the query
	Namespace Table	namespaceGridView	to input any required namespaces and their associated prefixes
	Error List	errorListGridView	to display any errors, warnings or guidance information

XPathmania: Extending the XML Editor in Visual Studio 2005

Published in:

Filed under: