"Reflection" in M

Wouldn't it be great if you could use reflection to inspect your M program, and add additional data to it. For example, suppose you wanted to add additional metadata to a type declaration, or a computed value.

In the upcoming release of the Oslo CTP, we introduce a new thing we call the M catalog, along with an operator in M called 'about'. 

The M catalog is simply a structured representation of an M program, or what we call the M semantic graph. The schema for the catalog is written in M, and the compiler can generate instances of your M program into that schema. Then it just shows up in the database using mx.exe.

 

Right now, the way to create an image with the M catalog schema is to run m.exe with the 'catalog definition' parameter, or 'catdef' for short. 

 

m.exe -catdef -out:catalog.mx

 

This creates an image that contains extent declarations for things like Modules, Types, Extents, and Computed Values. The catalog also has various helper computed values for querying the catalog. For example, you can ask if a particular computed value is installed (IsComputedValueInstalled). Or, you can ask if a type is an intrinsic, collection or entity type (IsIntrinsicType, IsCollectionType, IsEntityType). Or, one of my favorite, you can ask for all of the references to an extent ('ReferencesToExtent').

 

For now, we don't automatically generate instances for a given M program. You have to explicitly ask for it, and you have to reference the catalog definition generated above. We hope to fix all of that in the next milestone and turn it on by default. Here's a simple example for how to create catalog instances for your M:

 

m.exe myM.m -catalog /r:catalog.mx

 

When you run mx with myM.mx and catalog.mx, the M catalog will show up in the database. After you load, take a look at the database. In particular, one super cool thing is that the catalog also maintains the relationship between the SQL created from the M and their corresponding M declarations. For example, take a look at Tables, Functions, and Views in the Language.Catalog.SQL namespace.

 

That's a brief intro to the M catalog. There will be lots of scenarios coming where we use the catalog to query, analyze and understand M.

 

What if you want to access the catalog from your M program? That's easy enough because the catalog is written in M. So, you can reference and query the extents directly, or call one of the computed values.

 

The coolest reason to do this is to add additional data to your M program. This is what some people call metadata - although I'm not very fond of that term (it's all just data). Let's start with an example.

 

Let's start with one of our classic Oslo scenario models, the WIX model. Let's suppose that I am building a super-duper add-in to WIX that will deploy M to a database. The problem is that the WIX model does not include database deployment information. I could update the WIX model to include this, but that's not a great extensibility story for our developers. Instead, I will build my own little model for my specialization, I will reference M content in the catalog, and I will use about to write instances that reference specific values in the catalog.

 

Here's a simple version of my model. It describes additional information that I want to associate with a module in M:

 

module MDeployer

{

    import Language.Catalog;

   

    DatabaseConfigurations : ({

        Id : Integer32 => AutoNumber;

        DatabaseName : Text;

        ServerName : Text;

        Module : Modules;

    } where identity Id, value.Module in Modules)*;

}

 

Now, I want to write a deployment package the says that a particular M module is deployed to a particular server and database. Here's an example, except what do I write to specify the module name?

 

module DeploymentPackage

{

    import MDeployer;

    import MyModel;

   

    MDeployer::DatabaseConfigurations

    {

        { DatabaseName => "Testdatabase", ServerName => "TestServer", Module => ... }

    }

}

 

You could write a query, for example, 'Module => (Modules where value.Name == "MyModel")'. That's not always easy to remember. What's worse is that it is not strongly typed - what if I misspell "MyModel". I wouldn't find out about that until loading into the database. What I really want is a compile-time check that I have referenced the right M declaration.

 

'about' is the magic to make this easy. 'about' is a function that does what the query does, but it's strongly typed and uses the actual identifiers of the M declaration that you want to reference. Here it is:

 

module DeploymentPackage

{

    import MDeployer;

    import MyModel;

   

    MDeployer::Databases

    {

        {

DatabaseName => "Testdatabase",

ServerName => "TestServer",

Module =>  about(MyModel)

   }

    }

}

 

In summary, about allows you to add additional data to your M that is about your M. It references that content, and when loaded to the database, references your M program that is represented in the M catalog.