Getting Started

In this mini-tutorial you will implement your first language with Xtext and create an editor from that. Later, we will create a code generator that is capable of reading the models you create with the DSL editor and process them.

Creating a DSL

Download and install the latest version of Xtext, set up a fresh workspace, take a deep breath and follow the instructions. As soon as you have finished this chapter you will have an editor that understands input files of the following format. We will refer to this snippet as an example of our “target syntax” later on:

type String
type Bool

entity Session {
  property Title: String
  property IsTutorial : Bool
}

entity Conference {
  property Name : String
  property Attendees : Person[]
  property Speakers : Speaker[]
}

entity Person {
  property Name : String
}

entity Speaker extends Person {
  property Sessions : Session[]
}

Create an Xtext project

Use the Xtext wizard to create a new project

File -> New -> Project... -> Xtext -> Xtext project

Choose a meaningful project name, language name and file extension, e.g.

Main project name: org.xtext.example.start
Language name: org.xtext.example.Entities
DSL-File extension: entity

Keep “Create generator project” checked, as we will also create a code generator in a second step.

Click on Finish to create the projects.

Project layout

In the Package Explorer you can see three new projects. In org.xtext.example.start you can define the grammar and configure the runtime aspects of your language. The editor, outline view and code completion goes into org.xtext.example.start.ui. Both projects consist of generated classes derived from your grammar and manual code such as the grammar itself or further classes to differentiate from the default behavior.

It is good to be clear and unambiguous whether the code is generated or is to be manipulated by the developer. Thus, the generated code should be held separately from the manual code. We follow this pattern by having a folder src/ and a folder src-gen/ in each project. Keep in mind not to make changes in the src-gen/ folder. They will be overwritten by the generator.

A third project, org.xtext.example.start.generator will later contain a code generator that leverages the model created with the DSL editor.

Build your own grammar

The wizard will automatically open the example grammar file Entities.xtext from the first project in the editor. A grammar has two purposes. First, it is used to describe the concrete syntax of your language. Second, it contains information about how a parser shall create a model during parsing.

Ignore the generated sample grammar, delete everything after " Model :" to the end. The entry rule for the parser will be called Model. As a Model consists of one or more Entity entries, this rule delegates to another rule named Entity, which will be defined later on. As we can have one ore more entities within a model, the cardinality is “+”. Each rule is terminated with a semicolon. So our first rule reads as

Model :
  Entity+; 

Please note: If you encounter strange errors while copying and pasting these snippets to your Eclipse editor your documentation viewer most likely has inserted characters different from {space} into your clipboard. Reenter these “fillers” or type the text by hand to be sure everything works fine.

An Xtext grammar does not only describe rules for the parser but also the structure of the resulting abstract syntax tree. Usually, each parser rule will create a new node in that tree. The type of that node can be specified after the rule name using the keyword returns. If the type’s name is the same as the rule name, it can be omitted as in our case.

The parser will create a new element of type Model when it enters the rule Model, and a new element of type Entity every time it enters the rule Entity. To connect these AST elements, we have to define the name of a reference. In our case, we call that reference entities. We specify it using the assignment operator " +=", which denotes a multi valued feature.

As a result, we modify the first rule to

Model :
  (elements += Entity)+;

The next rule on our list is the rule Entity. Looking at our target syntax, each entity begins with the keyword entity followed by the entity’s name and an opening curly brace (we will handle the extends clause in a second step). Then, an entity defines a number of properties and ends with a closing curly brace.

Entity returns Entity:
  'entity' name=ID '{'
    (properties+=Property)*
  '}'
;

Instead of creating a new AST node for the name, we rather want the name to be an attribute of the Entity class. Therefore we use the terminal rule ID, which results in a string. The assignment operator " =" denotes a single valued feature, and the asterisk a cardinality of 0..n. In our target syntax, some entities refer to an existing entity as their super type after the keyword extends. Note that this is a cross-reference, as the super type itself must be defined somewhere else. To define a cross-reference we use square brackets. Optional parts have the cardinality “?”. The complete rule now reads:

Entity returns Entity:
  'entity' name=ID ('extends' extends=[Entity])? '{'
    (properties+=Property)*
  '}'
;

We have not specified the rule Property, yet. In our target syntax, properties can refer to simple types such as “String” or “Bool” as well as entities. To make this easy we will first introduce a common supertype Type each Property can refer to.

Change the rule Model and introduce a new rule Type and SimpleType:

Model :
  (elements+=Type)*;

Type:
  SimpleType | Entity;

SimpleType:
  'type' name=ID;
 

Models new consist of types where a Type can either be a SimpleType or the Entity you already know. Our rule Type will just delegate to either of them, using the " |" alternatives operator. The combination of simple data types and entites this way introduces a common super type Type both Entity and SimpleType derive from. This allows you to refer to both types of elements with a single cross-reference.

A Property consist of a keyword, a name, a colon and a cross-reference to an arbitrary Type. The multiplicity is either many or one. The presence of the postfix “[]” (technically a keyword) should trigger a boolean flag in the AST model. This is the purpose of the assignment operator " ?=". Our last parser rule is:

Property:
  'property' name=ID ':' type=[Type] (many?='[]')?;

In the end your grammar editor should look like this:

Generate language artifacts

Save the grammar and make sure that no error markers appear. Then, locate the file GenerateEntities.mwe next to the grammar file in the package explorer view. From its context menu, choose

Run As -> MWE Workflow.

That will trigger the Xtext language generator. You will see its logging messages in the Console view.

Run the generated editor

If code generation succeeded, right-click on the Xtext project and choose

Run As -> Eclipse Application.

This will spawn a new Eclipse workbench with your projects as plug-ins installed. In the new workbench, create a new project ( File -> New -> Project... -> General -> Project) and therein a new file with the file extension you chose in the beginning. This will open the generated entity editor.

To get some hands-on experience with your new DSL editor, type in the following model: