I have a funny story about how Visual Studio extensibility works. I work at Microsoft in the SQL Server division focusing on XML technologies and I went to the Visual Studio team a few years ago and asked them if they could build some better XML tools. They said they’d put it on their list. You know how that goes. The bottom line is that the Visual Studio team cannot possibly build every tool on the planet. So I installed the Visual Studio SDK and started hacking away in C#. I wanted to build an XML editor that understood namespaces and XSD built on the new System.Xml platform provided by our team. This was back in 2003, and so there was no managed language service framework, so I built one by porting the native Babel C++ framework to managed code-this is now the Microsoft.VisualStudio.Package.LanguageService framework in the Visual Studio SDK. Once that was done I was able to get an XML editor demo up and running in a few weeks and I took it over to the Visual Studio team and they liked it so much they asked to ship it in Visual Studio 2005. I’m happy to see many other languages adopt it, from IronPython to the new XAML editor and a new T/SQL language service being developed in our building.

I’ve really enjoyed working with the Visual Studio Developer Tools Platform team and I guess I’m now considered an internal partner. In fact, we now have a whole team in our building called “Data Programmability Tools” (DP Tools) dedicated to extending Visual Studio to provide all sorts of “data-centric” tools from XML editing, XSLT debugging in Visual Studio 2008, and now we are building graphical design tools for editing XSD and EDM schemas. So it’s been a fun ride!

The Visual Studio SDK contains a lot of functionality! Though it might seem a bit daunting at first, I’ve generally found it’s pretty cleanly componentized so I don’t need to know the stuff I don’t use. Microsoft built the XML Editor (Figure 1) on the managed package framework (MPF) and the managed language service framework (MLSF), it integrates with the XSLT debugger, and it uses quite a few Visual Studio shell features.

Figure 1: Microsoft built the XML Editor on the managed package framework (MPF) and the managed language service framework (MLSF).

For example, it uses IVsWebBrowserUser to render the XSLT output, it overrides various MPF methods to plug in custom tools/options and user profile persistence, and so on. It also uses the IVsSolution and IVsProject interfaces and project tracking events and so on so it can track location of XSD schemas for IntelliSense. It also implements custom EditorFactory and IVsEditorFactoryNotify interfaces so it can intercept XML Editor creation and redirect to registered XML designers when appropriate.

After shipping Visual Studio 2005 we started work on our new XSD Designer. Figure 2 shows our new “XML Schema Explorer” which shows all the XML schemas in a given “Set” and all the global types defined in the set. We knew we needed a better way to integrate with the XML Editor. We wanted to support “View Code” functionality so you can edit the raw XML using the XML Editor, and “View Designer” functionality to switch back to the graphical view all with smooth integration. But the only story we had for folks who are building on Visual Studio 2005 is to copy the Visual Studio SDK sample called “SynchronousXmlDesigner”. The sample XML designer does what the Data Set Designer and ResX Designer do, which is to synchronize changes at the “whole buffer” level. This means that when the user edits the raw XML, the designer sucks in the whole IVsTextLines buffer again and re-parses the whole thing and rebuilds the whole UI over again-hopefully with some smart UI context preservation. Conversely, when the user edits the graphical view, the designer re-serializes the entire thing back to XML and blasts that into the buffer (losing whatever user formatting was done, and sometimes even losing things like XML comments and other XML stuff not modeled by the designer).

Figure 2: XML Schema Explorer.

Figure 3 illustrates this problem.

Figure 3: No shared “parse tree” can be a problem.

The core of the problem here was that we had no shared “parse tree” that we could give folks. So we set about fixing that in Visual Studio 2008. Those of you who’ve built language services will appreciate how much work it is to support “incremental parsing” where you can take any set of user edits and figure out the matching update required in your parse tree to “synchronize” with those edits, including handling every error condition that can happen with any random gibberish in the text buffer. Fortunately we had a really good developer named Yana Kadiyska working on it full time for most of the development of Visual Studio 2008 to make sure we got it right. As a result, we can now tell developers to work against our XML parse tree instead of directly against the IVsTextLines buffer. Figure 4 illustrates the new architecture.

Figure 4: Architecture of the new XML parse tree.

You could think of this as a “code model” for XML. In fact, we also solved the problem of managing multiple XML Editor buffers and provided a higher level XmlStore concept where a pseudo-transaction (called “XmlEditingScope”) manages the parse tree and buffer synchronization across multiple buffers at once. You’ll find all this in the new Microsoft.VisualStudio.XmlEditor assembly.

The really cool thing is that this public parse tree is built using the new .NET Framework 3.5 System.Xml.Linq API. So it’s now possible to build an XML designer that binds directly to XElement nodes and responds to events whenever the user edits the buffer. Updating the buffer is also super easy-all you need to do is edit the XElement nodes inside an XmlEditingScope as the following code shows:

using (XmlEditingScope es =
this.store.BeginEditingScope("Add Node",
                              null)){
   XElement newXmlNode = new
   XElement("foo", GetNamespace(parent)));
    parent.Add(newXmlNode);
    es.Complete();
}

This preserves the user formatting in the XML buffer. Microsoft had to actually fix some things in the Managed Language Service framework to support all this, and we put those fixes in Microsoft.VisualStudio.Package.LanguageService.9.0.dll. As a side effect of all this work, as you might imagine, the XML Editor performs better when editing large XML documents because it doesn’t keep rebuilding the whole parse tree over and over.

The other thing we found is that tracking the location of XML schemas in the solution, and in the %vsinstalldir%\xml\schemas directory, is a bit of work to get right. You have to be very careful how you interact with the various different project systems (C++, C#, Visual Basic, Web, etc.) so you don’t cause performance problems for those projects, and so we figured this would also be useful to expose as a new service for other XML tools, and we called it the XmlSchemaService (Figure 5).

Figure 5: The XmlSchemaService.

The Visual Basic 9.0 editor uses these new API’s to provide XSD IntelliSense as does our new XSD designer and we hope lots of others start to use them also.

Lastly, we have some screencasts posted on the XML developer center you can look at to checkout some of our new Visual Studio 2008 work.

So all this new XML tool extensibility will be available in Visual Studio 2008 and it should make it super easy for any VSX developer to build all kinds of cool new XML tools on top.

Chris Lovett is an architect on the Data Programmability Tools team in SQL Server. Chris has worked on pretty much every XML core technology from Microsoft starting with MSXML in 1997, then to System.Xml in the .NET Frameworks and is now focusing on building XML tools in Visual Studio. Before working at Microsoft, Chris started his own company, he also worked at the Apple/IBM joint venture called Taligent, and before that he was a consultant at IBM working on OS/2 applications. See http://msdn.microsoft.com/xml.