Configuration

The Generator

Xtext provides lots of generic implementations for your language’s infrastructure but also uses code generation to generate some of the components. Those generated components are for instance the parser, the serializer, the Ecore model and a couple of convenient base classes for content assist etc.

The generator also contributes to shared project resources such as the plugin.xml, Manifest.MF and the Guice modules.

Xtext’s generator leverages the modeling workflow engine (MWE) from EMFT.

A short introduction to MWE

The nice thing about MWE is that it just instantiates Java classes and the configuration is done through setter and adder methods. Given the following Java class :

package foo;
  
public class Person {
     
  private String name;
     
  public void setName(String name) { 
    this.name = name;
  }
     
  private final List<Person> children = new ArrayList<Person>();
     
  public void addChild(Person child) {
    this.children.add(child);
  }
}
  

one can create a family tree this way:

<x class="foo.Person">
  <name value="Grandpa"/>
  <child class="foo.Person" name="Father">
    <child name="Son"/>
  </child>
</x>

These couple of lines will, when interpreted by MWE, result in an object tree consisting of three instances of foo.Person:

The root element can have an arbitrary name, with one exception: If the name is workflow and no class attribute is provided, it is assumed that an instance of org.eclipse.emf.mwe.internal.core.Workflow shall be instantiated. This instance will be the root of a workflow model used in generator workflow configurations. However, as you can see in the example above one can instantiate arbitrary Java object models. This is conceptually very close to the dependency injection and the XML language in the Spring Framework.

As explained above the element name ' x' is ignored in this case. The attribute class tells MWE which class to use in order to create the object node. The created object is populated according to the XML description: For each XML attribute MWE calls a corresponding setter or adder method passing in the value (there are configurable value converters, but usually Boolean and String is all you need). The same procedure is applied for each child element. In the case of XML elements a single attribute value is used and interpreted as if it was an attribute in the parent element. That is:

<foo><name value="bar"/></foo>

means the same as

<foo name="bar"/>

Obviously the latter is far more readable and should be preferred. However, as soon as you want to add multiple values to the same ‘adder’ method you will need to use the first syntax because you cannot define the same attribute twice in XML.

If an element does not have an attribute value, the engine looks for an attribute called class. If it finds one, the class is instantiated by means of the default constructor (there is no support for factories as of now.). If not, the class is inferred by looking at the argument’s type of the current setter method.

Due to this shortcut it is valid to write:

<child name="Son"/>

The addChild method takes a foo.Person as an argument. As this is a concrete class which has a default constructor it can be instantiated.

Tip Whenever you are in an *.mwe file and wonder what kind of configuration the underlying component may accept: Just use JDT’s open type action (CTRL+Shift+T) open the source file of the class in question, use the quick outline view (CTRL+O, use CTRL+O twice to see inherited members as well) and type ‘set’ or ‘add’ and you will see the available modifiers. Note that we plan to replace the XML syntax with an Xtext-based implementation as soon as possible.

This is the basic idea of the MWE language. There are of course a couple of additional concepts and features in the language and we also have not yet talked about the runtime workflow model. Please consult the MWE reference documentation (available through Eclipse help) for additional information.

General Architecture

Now that you know a bit about MWE, you are ready to learn about the concepts and architecture of Xtext’s language generator. An instance of Xtext’s generator is configured with general information about source folders and projects and consists of any number of language configurations. For each language configuration a URI pointing to its grammar file and the file extensions for the DSL must be provided. In addition, a language is configured with a list of IGeneratorFragments. The whole generator is composed of theses fragments. We have fragments for generating parsers, the serializer, the EMF code, the outline view, etc.

Generator Fragments

Each fragment gets the grammar of the language as an EMF model passed in. A fragment is able to generate code in one of the configured locations and contribute to several shared artifacts. The main interface IGeneratorFragment is supported by a convenient abstract base class AbstractGeneratorFragment, which by default delegates to an Xpand template with the same qualified name as the class and delegates some of the calls to Xpand template definitions.

We suggest to have a look at the fragment we have written for label providers ( LabelProviderFragment). It is pretty trivial and at the same time uses the most important call backs. In addition, the structure is not cluttered with too much extra noise so that the whole package (as of Xtext 0.7.0) can serve as a template to write your own fragment.

Configuration

As already explained we use MWE from EMFT in order to instantiate, configure and execute this structure of components. In the following we see an exemplary Xtext generator configuration written in MWE configuration code:


<workflow>
<property file="org/xtext/example/GenerateMyDsl.properties"/>

<property name="runtimeProject" value="../${projectName}"/>

<bean class="org.eclipse.emf.mwe.utils.StandaloneSetup"
platformUri="${runtimeProject}/.."/>

<component class="org.eclipse.emf.mwe.utils.DirectoryCleaner"
directory="${runtimeProject}/src-gen"/>
<component class="org.eclipse.emf.mwe.utils.DirectoryCleaner"
directory="${runtimeProject}.ui/src-gen"/>

<component class="org.eclipse.xtext.generator.Generator">
<pathRtProject value="${runtimeProject}"/>
<pathUiProject value="${runtimeProject}.ui"/>
<projectNameRt value="${projectName}"/>
<projectNameUi value="${projectName}.ui"/>

<language uri="${grammarURI}" fileExtensions="${file.extensions}">
<!-- Java API to access grammar elements 
(required by several other fragments) -->
<fragment class=
"org.eclipse.xtext.generator.grammarAccess.GrammarAccessFragment"/>

<!-- a lot more simple fragments -->
<!-- ...  -->

<!-- a sample fragment with a property -->
<fragment class=
"org.eclipse.xtext.generator.validation.JavaValidatorFragment">
<composedCheck value="org.eclipse.xtext.validation.ImportUriValidator"/>
</fragment>
      
      <!-- more simple fragments -->
<!-- ...  -->      
</language>
</component>
</workflow>

Here the root element is a workflow which accepts bean s and component s. The <property/> element is a first class concept of MWE’s configuration language and essentially acts as a preprocessor, which replaces all occurrences of ${propertyName} with the given value. Property declarations can be defined in-line ( <property name="foo" value="bar"/>) or by means of a property file import ( <property file="foo.properties"/>) which is performed before the actual tree is created. In this example we first import a properties file and after that declare a property runtimeProject which already uses a property imported from the previously imported file.

The method Workflow.setBean() does nothing but provides a means to apply global side-effects, which unfortunately is required by some projects. In this example we do a so called EMF stand alone setup. This class initializes a bunch of things for a non-OSGi environment that are otherwise configured by means of extension points, e.g. EMF’s EPackage.Registry.

Following the <bean/> element there are three <component/> elements. The Workflow.addComponent() method awaits instances of IWorkflowComponent, which is the primary concept of MWE’s workflow model. Xtext’s generator is an instance of IWorkflowComponent and can therefore be used within MWE workflows.

Standard generator fragments

In the following table the most important standard generator fragments are listed. Please refer to the Javadocs for more detailed documentation.

Class Generated Artifacts
EcoreGeneratorFragment EMF code for generated models
XtextAntlrGeneratorFragment ANTLR grammar, parser, lexer and related services.
GrammarAccessFragment Access to the grammar
ResourceFactoryFragment EMF resource factory
ParseTreeConstructorFragment Model-to-text serialization
JavaScopingFragment Java-based scoping
JavaValidatorFragment Java-based model validation
CheckFragment Xpand/Check-based model validation
FormatterFragment Code formatter
LabelProviderFragment Label provider
OutlineNodeAdapterFactoryFragment Outline view configuration
TransformerFragment Outline view configuration
JavaBasedContentAssistFragment Java-based content assist
XtextAntlrUiGeneratorFragment Content assist helper based on ANTLR
SimpleProjectWizardFragment New project wizard

Important Due to IP-Problems with the code generator shipped with ANTLR 3 we’re not allowed to ship this fragment at Eclipse. Therefore you’ll have to download it separately from http://download.itemis.com or use the update site at http://download.itemis.com/updates/.