What it is The mbeddr.platform is a collection of utilities and MPS extensions useful to many different kinds of systems built on MPS. We have developed it as part of the development of mbeddr, and all the utilities and extensions mentioned here are used extensively in mbeddr.
Some extensions work automatically as soon as you put the platform into your plugins folder. We describe these first. Other extensions require some simple code to be written as part of your language: typically, your language concepts must implement an interface and implement one or two methods, or you just use some new cell in the editor. We describe that second group next. The final group are full blown additional languages, typically in the editor, and require more work to understand and use. We describe these last.
This Page This documentation is intended as an overview to give you a head start. All the examples we refer to are in mbeddr, so we suggest you download mbeddr (not just the platform) as a means for you to see how the extensions work. Here are some of the mechanisms we use in this documentation:
Note that for historic reasons, the various extensions reside in various repositories and languages including git repositories by Sascha Lisson, languages and solutions in the com.mbeddr.mpsutil namespace as well as in one monster langauage com.mbeddr.core.base (and its plugin solution). If you download the platform, all of these are available, the platform packages them together. Nonetheless, we will clean up the structure and modularity of these things in the future -- for historic reasons, some of them are in the wrong place.
If the explanations here plus the sample code is not enough, contact the team and ask for help!
Download Similar to mbeddr as a whole you can get the platform from the repository, or you can download a distribution/binary from the github release page; each release has a download package specifically for the platform.
In the context menu (in the project explorer) of solutions and models you will find an Open Terminal entry. For a solution, it opens a terminal in the solution's root folder. For a model, it opens the terminal in the output folder (where the generated code resides).
On Models and Solutions there is a context menu entry Text Search. When activated, it opens a tool on the right side of MPS where you can enter a search term that is then full-text-searched in the respective solution or model. By default, the searcher looks at names of concepts as well as richtext paragraphs (see below). If you want your language concepts to contribute more text to be searched, implement the ISearchSupport interface.
The Edit menu as well as the context menu of a node contains an entry Copy Node URL. This creates a http URL for the current node that encodes the model name the unique node ID.
http://localhost:8080/select/tutorial/r:0d2d1e96-2ec7-4797-ad03-f95f261d35d7/3978189422528365848/
This URL can be pasted into text editors or moved around by email. Clicking on the node will select the respective node in the MPS editor. Note that this only works if the respective project is already open. The Edit menu also contains an entry Copy Node Info, which creates a human readable "path" to the current node, plus the URL:
project: tutorial (/Users/markusvoelter/Documents/mbeddr/mbeddr.core/code/applications/tutorial) module: mbeddr.tutorial.main model: mbeddr.tutorial.main.analyses node: BasicAnalyses [CBMCAnalysisConfiguration] (root) url: http://localhost:8080/select/tutorial/r:0d2d1e96-2ec7-4797-ad03-f95f261d35d7/3978189422528365848/
Finally, the URL can also be changed by replacing the "select" part with "content". This will return an XML representation of the node that can be pasted into a textual environment (see below).
A node can be copied to an XML representation using the Edit menu. This XML can then be passed around in text-oriented tools. There is also a Paste from XML entry in the Edit menu which recreates an MPS node from the XML representation. This works generically for nodes of any language. Note that the XML format is the same as returned from the URL/content thing mentioned above.
MPS references all nodes by their id, not their name. Still, as humans, we see only the name, and this might be ambiguous. The Unique Names hint can be activated in MPS Preferences, Editor Hints. It shows as much of the full qualified name as required to render a name unique.
Example Concept:
Example BaseLanguage:
In the context menu of the editor you will find an Generated Artifacts Review -> Open Review Tool entry. This will open a MPS tool where you can select one of the generated files. Simply clicking on a line of the text-area containing the generated file will jump to the DSL node from which this line was generated. The functionality is generic and can be used for each high level DSL (e.g. from mbeddr or MPS itself) and generated artifacts(e.g. C code, Makefiles, Java code).
Example 1: Side-by-side mbeddr DSLs and Generated C Code (also check this video):
Example 2: Side-by-side MPS DSLs and Generated Java Code:
MPS supports the use of non-textual notations as a consequence of its projectional editor. This means that programs may contain notations that cannot be typed in the same way as they look on the screen. Examples include sum symbols or fraction bars. Currently these are entered using textual aliases such as sum or frac.
However, this is not easily explorable: users cannot easily find out which such "hidden" aliases are available in a given context.
To solve this issue, we have created the context actions. These look like a palette in a graphical editor. Depending on the user's selection (cursor position), the context actions shows what is available for the user to do. Context actions can include
The context actions language ships with a DSL that supports the convenient deployment of the above mentioned artifacts into the context action palette.
For many applications (and their end users) the default MPS Project Explorer (aka Logical View) is too complicated -- it shows too much "unnecessary" stuff. Also, depending on the use case and the stakeholder, the structure of the primary navigation device must be different, adapted for the use case or stakeholder.
The platform contains a DSL to describe the structure of arbitrary alternative tree views in the project explorer; MPS already provides a drop down menu to select from various views, and the DSL can contribute additional ones.
The DSL supports the definition of arbitrary structures, labels, icons and actions, and also lets you add additional nodes useful for structuring the tree. To see how this DSL works, checkout the custom views defined in com.mbeddr.mpsutil.projectview.views solution as well as our new Favourites view in the com.mbeddr.mpsutil.favourites.plugin plugin solution. Below is a screenshot of that favourites view.
The serializer used to create the XML representations above is accessible programmatically. The classes are called NodeSerializer and NodeDeserializer
Additional editor cells are available to represent Boolean values in the editor with a representation other than "true" and "false". The bool{} cell can be configured with arbitrary strings for the true and false case (such as "yes" and "no" or "on" and "off"). The checkbox{} cell renders an actual checkbox (with a little cross in it).
A classic widget to select a date from a pop-up calendar.
This is only the control, i.e. the developer is responsible for synchronizing the control with the source of the shown date. We provide hooks executed before showing the calendar, for determining whether a date is valid, and to process a selected date.
Sascha has added support for tooltips to MPS. Tooltips are defined using the regular MPS projectional editor and hence have all its notational flexibility. Check out the video below:
To use it, just use the language mentioned above and include a tooltip cell in the editor: in the top slot of the cell put the original editor. In the bottom slot put the cells that make up the tooltip. The tooltips work together particularly well with the querylist in the sense that a querylist can be used to create the contents of the tooltip.
Normally, MPS can only project nodes at the location where they resided physically on the model. So, while the rendering of nodes could be designed arbitrarily, the structure of the projection itself always had to conform to the structure of the underlying AST. In other words, real views were not supported.
Enter querylist. A querylist is a list whose contents are the result of an arbitrary query over the model. The elements in the list can be editable, there are callbacks for what should happen when things are added or deleted, and it is even possible to define wrapper editors for the result elements or override the result element's editor completely. Note that any MPS root editor can project every node only once in an editable way. So if you are sure (by design) that a particular querylist only shows nodes from other roots, you can leave it editable. If it may show nodes from the same root, make sure you select the Duplicate Safe option. Check out existing uses of the querylist for details on how to do this.
We are using this currently in two places. First, we use it to show the signature of inherited operation-like constructs. In this case, the queried elements are rendered in grey, and they are readonly. The following picture shows this.
The second location where we currently make use of this feature is in assessments. So instead of just showing pointers to the result elements in the assessment result, we can now show the actual element inline. If the assessment is used to detect errors, then that error can be fixed directly in the assessment instead of navigating to the respective program location.
Hyperlink cells can be used for references in MPS. In contrast to normal reference cells they automatically become blue and underlined if the mouse hovers over them (no Ctrl is necessary). To make a reference a hyperlink, use the hyperlink-reference style attribute contributed by the hyperlink language.
Breadcrumbs are a way to improve the user's awareness where in a (big) file he is currently editing. It is essentially a "horizontal tree" shown at the top of an editor that reflects the hierarchical structure of an editor's contents. Niko has made an MPS extension that displays a breadcrumb in an MPS editor. By clicking on a label the user can jump to the corresponding level of the containment hierarchy. By clicking on the little triangles one can see the remaining nodes in the hierarchy.
To use breadcrumbs in an editor, only two simple steps are required. Nodes that form the hierarchy (and should show up in the breadcrumb) must implement the interface IBreadcrumb. The root of the editor that should show the breadcrumbs must include a breadcrumb editor cell at the top.
MPS comes with an Outline view that opens when pressing Ctrl-F12. The view contains a customizable list of contents in a root. It can be filtered by typing, and by pressing Enter, the selected element can be selected in the editor. To have your own concepts show up in this view, make them implement IShowUpInOutline.
In MPS, each language concept has its own editor. While this is useful in 99% of all cases, sometimes it is a problem: imagine you want to change the editor of many different language concepts at a time. Then you have to change the editor for all these language concepts. This is tedious and error prone.
As an example, consider projecting a little arrow over all "things with names" in C that are pointers as a means of highlighting pointers throughout the program. This is shown in the example below.
The problem here is that there are separate language concepts for local variable references, global variable references, argument references, etc. All these have to be changed.
Enter conditional editors. A conditional editor is almost like an aspect. It can be applied "around" existing editor, and whether it applies or not is determined by a condition and the applicable concept (similar to a pointcut in AO). The figure below shows the definition of the pointer arrow editor.
It applies to all language concepts that are references and hence implement the IReference interface. It is conditionally applied only if the type of the reference is a pointer. As the editor itself, it shows the arrow (hiding behind the custom cell on top of the already existing editor ([next-editor]).
Using this approach, it is now possible to essentially decorate any editor with any decoration based on arbitrary conditions.
With this extension you can use a non-structured, multi-line text editing cell in MPS editor. It also allows you to insert real, structured program nodes into the otherwise unstructured text, enabling mixed content editing. The details are explained in this paper.
This BaseLanguage extension simplifies usage of variables and Java String.format() inside BaseLanguage Strings. Just add the language com.mbeddr.mpsutil.richstring to your model.
The following example shows referencing a simple variable, embedding an expression inside a string,
and the equivalent of calling String.format()
.
This is a language concept that supports picking paths and files from the file system. The files are picked relative to several different roots (see below). The picker provides code completion, opening the OS file/dir dialog, verifies that a file is valid and has methods to return the absolute path for loading the file.
This provides an integration of PlantUML with MPS. Watch this video to get an overview..
To use it in your own languages, implement the IVisualizable interface in your language concept. Each concept can have several visualizations, called categories. The getCategories() method returns the list (just strings). The getVisualization() method then returns the actual visualition. It gets the user-requested category as an argument, as well as an empty VisGraph object. The getVisualization() method then has to fill the VisGraph with the actual visualiztion. This is done simply by using the add() method to build a native, textual PlantUML visualization. Check out the existing implementations of the interface to learn the deails; in particular, there is a helper method on VisGraph that creates a URL for users to click on in the graph: createUrl(node).
As an additional means of displaying relationships between nodes (next to PlantUML and the tree views) we have integrated the JUNG framework for graph display. As a language developer, you can implement an interface IJGraphProvider and implement a method that creates the graph:
Once you have created a graph, your users have many options to view the graph in an MPS tool window:
The JUNG integration is available as part of mbeddr, but also as part of the mbeddr.platform, so you can use it for your own languages. Check out the Chunk node for an example of how to use it.
A demo video of the viewer can be found here.
To use it in your own language, implement the IJGraphProvider interface. Return a JNGraph from the getGraph() method. Check out the existing implementations, and in particular the DepGraphHelper (called from Chunk.getGraph()) to learn the details. In short, a graph is built in two steps: the first step actually adds the nodes and edges. The second step uses closures to style the graph. Examples can be seen from DepGraphHelper.
Tree views are a proven way of showing and navigating structure in IDEs. MPS of course has several of them for various purposes. We have now added a nice tree view that can be used for various tasks. The screenshot below shows a call stack for functions.
Selecting a node in the tree automatically selects the corresponding node in the editor. In addition, tree nodes can contribute user-defined actions that are shown in the context menu.
As with visualizations, a node can contribute different categories of tree views. All of this can be defined without dealing with any Swing code. The code below shows the implementation of the callgraph based on the interfaces ITreeViewable and ITreeViewRoot.
MPS, like any other language workbench, supports various forms of constraint checks that lead to errors or warnings, annotated directly to the element that fails the constraint check. However, there are some kinds of checks that are different in nature: they may be global and require expensive algorithms to compute. They may be used to create some kind of overview, or report, and using error annotations spread all over the code may not be suitable. You may also want to mark failed constraints as ok and ignore them in the future. To support these use cases, we have added Assessments to mbeddr.
Here is an example assessment. It highlights all requirements in a model that have no effort specified. Not having an effort is a problem, and you may want to keep track of the requirements where you have yet to perform your estimation. The assessment shown below shows an example result (of course, the results are references and you can navigate to the offending requirement):
Note the colors. Green results are those that are marked as ok, so they are judged by the user to not be a real problem. Red results are those that have been added during the last update of the assessment. Black ones have been there from previous updates. Using the colors, you can keep track of the current state of the assessment, as well as of its changes. Assessment results are part of the model, so they are persisted and shared. They are intended to be actively maintained and managed (in contrast to regular error or warning annotations in the code). You can also set the “must be ok” flat to true, in which case the Assessment itself gets an error annotation if the results contain non-ok entries.
The requirements thing above is of course just an example and the assessment facility is extensible. You have to create two concepts:
MPS does not allow default values for properties; this extension adds them.
Add the language com.mbeddr.mpsutil.propertydefault to the constraints
aspect of our concept. Due to limitations in MPS, we first need to add a "standard" constraint
for a property. Then, after selecting the whole constraint and pressing Ctrl-Space, we can select
the default property value, and assign any expression to it.
Sometimes you want to be able to select a location in the model where you want to modify something. For example, in mbeddr's requirements, you may have an intention or refactoring that allows you to move one or more selected requirements to another location. To make this possible, you need a way to choose a node. The targetchooser dialog supports this, as you can see from the following screenshot.
The target chooser supports limiting the scope of what is shown in the tree (one or more solutions or models), lets you filter the contents of the tree (e.g., only requirements modules and requirements) and lets you return an error message in case the user selects an invalid target. The dialog also supports deep search; you can just start typing and the matched node is highlighted, even if the respective subtree has not yet been opened when the search is started.
To understand how to use the target chooser, take a look at the moveToOtherModule intention in the requirements language.
By default, an intention always applies to one node. Only refactorings can apply to several nodes. However, refactorings work differently in the UI and it is hard to explain to users when to use an intention and when to use a refactoring.
To address this problem, the selection intention can be used; it is not defined for a particular concept, but for a particular selection (represented by a Java class in MPS). As of now, there is a selection class for ranges of cells in tables and a class for ranges (lists) of nodes. The latter one ic called NodeRangeSelection. By defining an selection intention for a NodeRangeSelection, you can create an intentions for several nodes at a time.
To understand how to use the selection intention, take a look at the moveToOtherModule intention in the requirements language.
MPS allows you to react to changing the values of properties and references, but it does not allow you to adding or removing a child. This is a conceptual asymmetry, and also a problem in several concrete cases. Model Listeners fix this problem.
They provide a new root for the Constraints model called Model Listeners. They are defined for a particular concept and can react to adding of a child, removing a child and before removing a child (while it is stil attached to its parent).
To understand how to use the it, take a look at the model listeners for Requirement in the Constraints of the requirements language.
MPS already provides a language (jetbrains.mps.baseLanguage.extensionMethods) for defining extension methods, i. e. methods to be called on an Object without being part of the Object's class. We extended this mechanism to provide all static methods of a class as extension methods. They can be invoked on the type of the method's first parameter.
Use the language com.mbeddr.mpsutil.extensionclass and create a new ExtensionClass root node. Point to the Java class containing your static methods. If we like, we can add method annotations to overwrite parameter names, so they are more elaborate than "p0" or "p2".
As a starting point, a dependency on Model com.mbeddr.mpsutil.extensionclass.annotation.apache.commons.lang3 provides extension methods with useful parameter names for Apache Commons StringUtils, ObjectUtils, and StringEscapeUtils.
MPS projectional editor affords users a lot of flexibility in terms of notational flexibility and language composition. This blog is full of examples of this, so we don't have to provide more examples.
However, this comes at a cost: building editor in MPS that don't just look nice but that also are nicely usable can be challenging. In particular, editor developers have to create so-called actions -- wrappers, side transforms, substitutions, delete actions -- that, for example, allow entering expression tree structures linearly. Enabling the insertion of parentheses in cross-tree locations, or splitting a number literal in two is even more challenging. Implementing all of this correctly can be a challenge, even for practiced users of MPS. More importantly, it is easy to forget some of those actions, which then leads to editors that behave inconsistently: good in some cases, and bad in others. Such inconsistencies are a major source of frustration for new MPS users.
To address this issue, we (mostly Sascha) have built the grammar cells infrastructure. These are new editor cells that have enough semantic richness so that the necessary actions can be generated automatically. To implement a nicely usable expression language, developers do not have to write a single line of action code. Check out this video:
Even though language developers have to invest a little bit of effort into learning how to use these new cells, it is certainly simpler to work with than the actions. In particular, it is easier to get things consistent. Check them out, they are in the master branch.
Margin cells are editor cells that are shown beyond the right editor margin, a little bit like comments in Word. In fact, the ReviewNote cell implements exactly Word's comment facility as an example of the margin cell. Here is a screenshot:
To use the review notes, just include the respective language; intentions are available to attach review notes to editor cells.
To put your own cells into the right margin, your cell must implement the IMarginCellContent interface. The editor of your cell may use the margincell-cell-width and margincell-dashed-line-interval style attributes to design the width and line style. You also have to have the actual margincell on the root element of the editor. However, as illustrated by the CommentAnnotationContainer from the review language, you can use an annotation for this. Hence, margin comments can essentially be added to every model, without the model's language being aware of it.
This extension allows the definition of preference pages by using MPS models; so in the preference page you get a normal MPS editor, as exemplified by mbeddr's platform templates preference page:
Such pages are defined using an instance of PreferencePageDescription (check out the root "Platform Templates" as an example). This instance must reside in a plugin solution. The data, i.e., the respective MPS model, is stored as part of the respective project or in MPS globally (this is configurable). It is possible to access the model that contains the preferences via code:
If we needed less sophisticated preferences, mbeddr platform offers the PreferenceForm extension. It allows to define preferences on both application and project scope, and cares for a simple UI and the persistence. For the latter part, it can be seen as an extension to the built-in jetbrains.mps.lang.plugin.structure.PreferencePage.
For using PreferenceForm, create a new PreferenceForm root node inside an MPS plugin.
The following field types are available:
Example:
Resulting preferences:
This extension lets you efficiently write interpreters of your languages. An interpreter is defined as an instance of Interpreter. Check out the concept CInterpreter, which interprets mbeddr C.
Each interpreter consists of the following sections:
Interpreters can be composed; check out CExtInterpreter, which specifies that it must run after the CInterpreter discussed before. The java class CombinedInterpreter can be used to create composite interpreters from several interpreters. Do a Find Usages on the CombinedInterpreter to see how this works.
An introduction to the math notation is in this paper. The image below shows an example of the notation used in C. However, the notational primitives are separate from the language.
To use the notation with your own language, import the above mentioned language into your editor definition. A set of math.* editor cells are then available for use with your concepts in the editor definition. Check out the com.mbeddr.ext.math language for examples of most available editor cells.
An introduction to the table notation is in this paper. Below are a few screenshots of tables used in various languages. Click to enlarge.
Similar to the math notation, the table notation defines a new editor cell "table". The cell is very flexible and can define row-oriented tables (like this reference data cell), as well as tables where the data is not row-oriented (like the decision table, or the table notation for state machines). Check out the concepts DecisionTable and Statemachine (and its table editor).
The tables notation is relatively sophisticated and requires much more documentation than what we can provide right now on this page. More will follow later.
If you have downloaded the recent mbeddr master branch, you will have noticed that, for example, component wiring and state machines can now be edited graphically. The screenshots below show examples of these two notations.
This screenshot shows a few interesting features: you can embed diagrams anywhere in "text", you can use different shapes (at this point drawn by custom Java code), you can use various line styles, the framework supports ports (i.e., connection endpoints on the boxes), inside boxes you can use arbitrary MPS text (or other) editors, and the system also supports edge and endpoint labels. Port labels are also supported, but they are only shown if the mouse is "in the vicinity" of the port to not clutter the diagram. Below is a second screenshot of a bigger diagram:
This one illustrates that the approach scales to reasonable sizes, shows that zooming is supported and also demonstrates the auto layouting capability. The graphical notation also integrates with things such as tooltips. Below is another example diagram that shows a different language:
The definition of a graphical editor is based on the same "cell" abstraction used in other MPS editors: the language for defining editors contains additional cells that are then rendered as a diagram (diagram, diagram.box, diagram.edge). Similar to tables, these abstractions for defining graphical editors rely on queries to make sure that the structure of the graphical editor does not have to directly correspond to the structure of the AST (for example, in terms of ownership). The language also supports hierarchical diagrams, for example, in state machines.
To see example code, check out InstanceConfiguration and Statemachine.
The diagram notation is relatively sophisticated and requires much more documentation than what we can provide right now on this page. More will follow later. A tutorial on non-trivial use of the diagram editor and querylist extensions combined, can be found here: Using the diagram editor and querylist: let’s build a graphical structure editor for MPS.