In show #401 we talked to Chris Sells about Oslo, to really try and figure out what it is, and what we should expect from it.
Chris Sells: Oslo is the name, the new name for this thing I’ve been literally doing for the last almost four years with Microsoft, which has been the idea that we can gain considerable productivity by formalizing an idea and generalizing an idea for something that we do all the time already, which is writing down what we know about our system and using interpreters to runtime frameworks, whatever we want to call it, to interpret that data and to do the thing that’s usually interesting for us. For example, let me give you something that I was working on the last week in fact. I don’t know if you guys have ever built a setup.
Carl Franklin: A setup?
Chris Sells: Yeah, like an MSI file.
Carl Franklin: Sure, yeah.
Chris Sells: So an MSI file is a combination of declarative metadata that’s says, “Here is how I want things to be when I’m done with the setup, when we’ve executed the set up and the user had been shown the options. Here is how I would like it to appear in the environment wherever the environment is that I’m going to install when I’m done. And here are the files and the resources and the bitmaps and whatever to make all that happen.” Then, you pack all that up into an MSI file and you stick that on someone’s machine and they execute it with the MSI interpreter that sits on modern copies of Windows. And that interpreter reads that data and then does whatever it needs to do to make the environment that you described happen. Now, of course, to define it declaratively, here is how I’d prefer my environment when I’m done.
The interpreter has to translate that into step-by-step. We don’t care what it does to make that happen; all we care is here is what we want when we’re done. So that kind of pattern happens over and over in software, and more and more lately. For example, Don [Box] likes to tell the story of the console bit. If you flip a bit on a Win32 executable that says I want a console app, the shell gives you a console window when you launch it, and hooks it up to your app and you do Console.WriteLine or printf or whatever it is you’re using, and it appears on that console window without you having to do anything else. That’s a very early form of declarative programming where you just say, “I want this environment to happen and I’ll assume it is and it will be provided by the environment around me.”
Now that one is very small but we’ve been building on that idea again and again, over and over again until we have the MSI thing I just talked about. We’ve got WPF, which uses XAML to say here is what I want my UI to look like and here’s how I want my animations to look. If I hover over this button, here’s how I want the styles to change... and WPF does the work.
Or Workflow. In XAML, here is how I want to transition from activity to activity and by the way, I’m describing it in this form so that you can do things like put me on hold serializing me to the disk, go away for a year, come back, and continue executing as if I’ve always been running in memory. We have the WF framework that interprets that data and provides that environment.
[How about] COM+ or Transactional COM? I mean, we have been doing this kind of declaratively saying to the various runtimes that we’re building, “Here’s what I want, and then we just trust you to make it happen for a long time.”
The idea of Oslo is that we can take that idea and build a set of very powerful general purpose site tools around it and languages, and let us do it in a general purpose way so that we can have our developers understand how to do that in one way and we get the languages and the tools and the experience and the education that we can then take from project to project even when we’re doing something as different as console apps or workflows or rich client GUIs or web GUIs or whatever we’re doing. We have that same set of tools and experience and knowledge to build our applications.
Carl Franklin: So essentially how this relates to Oslo is you’re allowing the developer to build a runtime?
Chris Sells: We allow the developer to do a lot of things out of Oslo. There’s actually a lot in the package. Oslo is three things primarily. It’s a language, or really a family of languages which we call M and that breaks that into three pieces: MGraph, MSchema, and MGrammar.
Carl Franklin: Yeah.
Chris Sells: It’s a tool, which is Quadrant, for creating instances of data that we describe with M or editing viewing data. Think of it as a really powerful data manipulation environment with all kinds of different views and it is extensible so you can add your own views. Then the thing that holds it together, the lynch pin, is this thing called the repository, which is a set of services and database schema built on top of SQL Server 2008. So, you know, when you install a repository, it expects SQL Server to already be running and it creates an instance of the database and it installs the triggers and the functions and the tables, and that instance data that is assumed to be there when you start describing data with M and pushing it into the SQL database. That bunch of features that you get when you get SQL out of M are built on the repository. There’s a lot.
Carl Franklin: And just fundamentally here, just for trying to grok this: you’re really allowing developers to build their own DSLs, external DSLs. Is that right?
Chris Sells: Yes. So here’s the thing. One of the features of M is this idea of a general purpose data definition language. So I can walk up to M and I say I’ve got a person who has a name and an age, and the age is an integer of value less than a hundred, and the name is characters of length, whatever, and person has a relationship to other people, etc., I can model (or create) whatever domain or universe I want to. I can even describe it.
Carl Franklin: And this is MSchema, right?
Chris Sells: MSchema.
Carl Franklin: Right.
Chris Sells: Now, let’s suppose I wanted to track instances of a Podcast like the .NET Rocks! show. So when you’re posting stuff about the .NET show like you got a title and you’ve got a description, I know you’ve got a set of related links and I don’t know, I mean, what else do you use to describe...
Carl Franklin: No, no, that’s it.
Chris Sells: Yeah, okay.
Carl Franklin: We have guests and we have sponsors and shows and they all go together.
Chris Sells: Okay, yeah, so you got data that you’re keeping track of and so you could walk up to M and use M the language and say “Type .NET Show” -and then it’s got a title and description and a set of sponsors and then what is a sponsor? Well, sponsor is a name and a contact piece of information, whatever contact information, etc., you could model as much of that universe that you wanted in M, its MSchema.
Carl Franklin: And you could think of this as sort of XML-like Schema. I think it’s essentially...
Chris Sells: It’s Schema in a sense of, yeah, you could think of it as XML structural. Meaning we care about the data and the forms the data takes. This is not an object-oriented type of system. It’s a structural type of system.
Carl Franklin: Yup.
Chris Sells: We use MSchema to define the various types in the universe and the storage, where we keep things, and then we can use MGraph to say, oh, now I’ve got an instance of the .NET show. Chris Sells was on this show and it is show number whatever it is, and here are the sponsors and here is the description and Scott Hanselman was on this other show. We can use the MInstance language to be able to just say these are instances of data that go with these types.
Carl Franklin: During one of Doug Purdy’s demos, I think this is where he did something with like CDs and albums and artists and that kind of stuff, and he said something like "CD [name] is by [artist]", and then he translated a sentence into data. Is that what MGraph does?
Chris Sells: So MGraph is the built-in generic instance syntax that works with any set of types you use to build in M, the generic kind. You can think of it as the C# initialization syntax. It’s kind of like that. Its general purpose; it works with any M types. Now the thing that Doug was talking about is something that you can create with something called MGrammar and MGrammar allows you to say, for a set of related types, I want to provide a rich text experience for them to program against this set of types to create instances in the database for these set of types. I want normal people to be able to think about what they think about, like I have a CD by whomever, titled whatever, and I like it.
Carl Franklin: Yeah. Show 378 is with Miguel Castro and he talked about blah, blah, blah.
Chris Sells: Yeah, exactly right. So that you could then build a grammar around it. You know, I’ve got this keyword ‘show’and I’ve got this keyword ‘is’and I’ve got this keyword ‘this other thing’and then I can pull out the data and I can build the grammar that parses that data and detects errors so that we can report the errors so I can use it with the data and format the data properly. Even though we’re building a language specifically to make it easy for them, that doesn’t mean they’re going to do it perfectly every time.
Carl Franklin: Sure, sure.
Chris Sells: So based on that grammar, you provide a grammar-aware toolset. So for example, you can load the grammar into a program we call IntelliPad, which is a free component-based editor that we shipped in the Oslo SDK, and you could load the grammar into IntelliPad and you can say now I want this document to read instances of this grammar and IntelliPad will actually not only give us on-the-fly error reporting when it did it, but it will do it like IntelliSense. Every custom grammar can have IntelliSense in IntelliPad. So now as I type along, I get statement completion. You know, from a developer’s point of view, it’s very much like typing in code to Visual Studio but it works for any language that you define. From a business analyst point of view, it’s like a really smart word processor that understands the specific data that you’re giving it and gives you red squigglies when you get it wrong.
Carl Franklin: That’s interesting. This is really for inputting data just like a data entry thing?
Chris Sells: Yeah. So you could tell MGrammar is kind of general purpose. If you have ever built a lexer or a parser or a custom language yourself, MGrammar is general purpose, meaning you can parse whatever stream of text you want and you’ll get back out an abstract symbol tree, which is really just a fancy name for the object model that describes all of the parsed data that we pulled in from the text file.
Carl Franklin: Okay. What you do with that is up to you is what you’re saying.
Chris Sells: Well, what I’m saying is step 1 is out of the box. Its, you know, if you’ve written compilers before it’s a lexer and a parser built into one language so if you’re familiar with those tools, MGrammar does all of that. However, because it’s part of the M family of languages, one of the things that you can use it to do, and this is a much simpler thing because this is what you want to do, instead of writing C# code, where you could parse (analyze) an abstract syntax tree and then put records in the database or execute it on a runtime or whatever you’re going to do like whatever a classic compiler would do. You can actually, in the grammar file say, this element in MGrammar actually produces this set of MGraph instances but instead of making people type in MGraph instances to get data into the database, you can actually give them a little language. But it’s also easier for the guy building the language because he can just translate the little languages into MGraph instances and then have those going into the database. So you don’t actually have to write any C# code to parse a set of text or random set of text provided by the user into a set of instances in the database. You can do that all declaratively with the M family of languages.
Richard Campbell: So would we be deploying that capability out to the user that they would just be routinely adding data through this mechanism?
Chris Sells: So imagine the lifecycle here where I can say, okay, I’ve got this set of types that I define in M and I’ve used that M to create a database in SQL Server that will hold the instances of those types, and then once I have that I can fire up Quadrant and I can go to those tables and I can have the rich set of default views that Quadrant comes with, and I can add instances to it and edit instances and delete instances of that data in those tables. So think of it as a really friendly SQL Server Management Studio if you want.
Carl Franklin: Right, right.
Chris Sells: You don’t have to type INSERT, you can just point, click, type and choose from drop-downs and check boxes and that kind of thing. And you can flip between various views. I want to see my data in the treeview; I want to see my data in the table view; I want to see my data in a property browser view; I want to see my data in graph view. You can flip between textual and graphical views as well, so I can walk up to my data and say, “Well, let me see how that looks in text. Oh, well, this is a set of text that is my set of .NET shows. Well, let me just edit that one on line 3 and let me go add another one in the bottom and let me have that rich text experience just like I was a programmer or just like I was an advanced IT person or advance business analyst.” I could do all of the ideas and I can do all of that in Quadrant and I can see it as my M instances. I could see it as my DSL language; I could see it as maybe XAML or some kind of XML format, but I can imagine a bunch of rich text views in Quadrant in the exact same way that I can imagine a bunch of rich graphical views. Now, in the PDC Bits, we don’t have that but we might-I’m not making any promises for dates of releases-but right now you can already do that kind of thing in IntelliPad in the shipping disk.
Carl Franklin: Okay, so the MGrammar is what you used to create your DSL essentially.
Chris Sells: Yes.
Carl Franklin: Yeah.
Chris Sells: And the language services that go along with the DSL as well.
Carl Franklin: In Quadrant, we should also say, I don’t know if we emphasize this, but it is a GUI, right?
Chris Sells: Quadrant isn’t a GUI, it’s a metaphor. I point here, I click there, I see my list of types in the database, and I can see the instances in the table. I mean, we’re working live against the copy of the database and I can edit and add and delete instances just like I expect and I can switch between views to see my data in whatever ways it can mean for me. Also Quadrant is extensible in the exact same way that I can extend the M family of languages with MGrammar and build my own textual DSL that lets me type in data, all of those users type in data in whatever ways convenient for them. The idea is with Quadrant I can configure together a bunch of the basic views and build more complicated views and associate those richer views with specific kinds of data so that I can tailor the data editing experience inside of Quadrant anytime a user goes and edits those types.
Carl Franklin: Would you also use the IntelliPad? I guess IntelliPad is sort of this interactive thing that you’re talking about. Would you also use that to query the system, to just ask like a regular English language question and get some sort of detailed data out of that?
Chris Sells: So right now, the query language of the data repository is, well, with any kind of language-if you want to program against the repository-it is all query based; so if you do it inside of Quadrant when you’re configuring things, you’re absolutely writing a query as to be able to pull up bits and pieces if you’re customizing Quadrant. If you are writing C# programs against it, you’re writing SQL statements because the data is kept in a SQL database and you can use whatever SQL data access technology happens to make you happy, that will be fine, just make sure it goes to SQL. We try to map the types from M to SQL in the most natural way possible.
Carl Franklin: So in other words, you translate the more natural English kind of language that they’ve typed as a query which becomes an object graph which you can then query SQL Server from there. Is that what you’re saying?
Chris Sells: What I’m saying is MSchema is the language for defining types and we have a mapping from that to SQL so when you type the word type and define a type in M, that translates into a CREATE TABLE statement in SQL, and we have tools for doing that translation and deploying those types into the repository into the SQL database. When you type M instance, when you create instances of data in M, in the MGraph syntax, those translate into SQL INSERT statements. When you build a calculated value in M, those translate mostly into cuts and views in SQL. You can write a program that gets the data out by accessing the views. When you do something with MGrammar and you create instances, ultimately those also translate into SQL INSERT statements.
Carl Franklin: Well, I’m looking more for the SELECT statements like could I walk up to it and say, “Who is the guest in show number 29?”
Chris Sells: So we provide no natural language processing that turns English into SQL SELECT statements.
Carl Franklin: But I guess if you had, yeah, and this is what I was getting at, if you had the MGrammar elements, you know, who and guest...
Chris Sells: You could totally use MGrammar to build a natural language processor that creates SQL statements. Absolutely.
Carl Franklin: Yeah.
Chris Sells: And in fact, I think that would be an excellent way for some enterprising third-party developer to stretch the platform and tell us what they think.
Carl Franklin: I remember, I don’t know Richard, do you remember this computer database inventory program called Q and A?
Richard Campbell: Yeah.
Carl Franklin: On DOS.
Richard Campbell: And I think SQL 2000 had a natural language processor, like there have been a number of efforts in this area over time and so far though a lot of the stuff you’re saying Chris, it sounds very SQL Server centric.
Carl Franklin: Yeah.
Chris Sells: Oh, absolutely. I mean, we absolutely love SQL Server; it’s a fabulous environment for storing and manipulating data and it has all kinds of wonderful scalability, versioning, security, globalization-I mean, we got just a ton of these great features and we have been tuning it forever and it just automatically tunes itself over time. SQL Server is a wonderful data environment and so we wanted to build a set of tools that allow people to take in more abstract terms, so we built M instead of tools but our set of tools are around creating M types, which translates into SQL tables creating M values, which translated the SQL data creating DSL so that you can more usually create M values that translates into SQL data and providing a set of additional features that calls the repository on top of SQL Server and then building Quadrant to operate a visual view on top of that data, all in SQL Server. So yes, we love SQL Server and we have built a repository and Quadrant and M around SQL Server.
Carl Franklin: What I’m getting at here, also, is for things that aren’t necessarily dealing with data but dealing with objects. When you’re using IntelliPad, your code is getting called from the parser and I guess like you said, it passes an object graph into your code so that you know exactly what this person is trying to do.
Chris Sells: You can absolutely extend IntelliPad with custom code and do all kinds of wonderful things. IntelliPad is extensible but one of the big points of MGrammar is to be able to do it all declaratively and get yourself a set of M since we’re not writing any C# code at all. Now that is not to say that MGrammar couldn’t also be used to build full-featured language compilers, and in fact we’ve done some of that internally and we think it’s a fabulous tool. In fact, it is a way to test. We built some real compilers that we were using for a while. We worked hard to make sure that MGrammar is a real tool for building compilers, but we also made sure that it works really well for generating M, which is one of the things we think it will be used for, and also M because it is a general purpose data definition language. We have provided an out-of-the-box SQL Server project code generator to translate M into SQL, but all of the object model that, I mean the code compiler, the bit of our code that parses M and hands it up as an object model right before we generate to SQL, you can have access to it, so if you want to just take the M and interpret it at runtime and do whatever you want with it or use it to generate code or use it to generate XML and use it to generate whatever you want to, all of that is available to you as well so you don’t have to just generate SQL with it.
Carl Franklin: And your users never have to go out to the command line and compile anything.
Chris Sells: Also, we definitely do in the SDK, there’s definitely a 1.0 SDK like a .NET 1.0 SDK. We have a full set of command line tools for compiling M at the command line, compiling MGrammar to the command line, generating SQL script, generating something we call MImages, deploying SQL or deploying MImages into the repository; all of that is available from the command line. We’ve also built a set of language services and build tools and projects, project templates, that plug into Visual Studio 2008 and Dev 10 so you can do it all inside of Visual Studio. We’ve also built a set of extensive commands to IntelliPad so you can have that whole lifecycle of build deploy inside of IntelliPad as well. So the idea is that you can do it all from the command line if you want to but with our two main tools that we’re targeting, you don’t have to do it from the command line.
Carl Franklin: So IntelliPad and Quadrant really are what the end-users are going to use and they don’t have to go down there.
Chris Sells: I see most of our end users spending most of their time in Quadrant.
The conversation continues online at http://www.dotnetrocks.com/default.aspx?showNum=401
