Validation

Static analysis or validation is one of the most interesting aspects when developing a programming language. The users of your languages will be grateful if they get informative feedback as they type. In Xtext there are basically three different kinds of validation.

Automatic Validation

Some implementation aspects (e.g. the grammar, scoping) of a language have an impact on what is required for a document or semantic model to be valid. Xtext automatically takes care of this.

Lexer/Parser: Syntactical Validation

The syntactical correctness of any textual input is validated automatically by the parser. The error messages are generated by the underlying parser technology. One can use the ISyntaxErrorMessageProvider-API to customize this messages. Any syntax errors can be retrieved from the Resource using the common EMF API:

  • org.eclipse.emf.ecore.resource.Resource.getErrors()

  • org.eclipse.emf.ecore.resource.Resource.getWarnings()

Linker: Crosslink Validation

Any broken crosslinks can be checked generically. As crosslink resolution is done lazily (see linking), any broken links are resolved lazily as well. If you want to validate whether all links are valid, you will have to navigate through the model so that all installed EMF proxies get resolved. This is done automatically in the editor.

Similar to syntax errors, any unresolvable crosslinks will be reported and can be obtained through:

  • org.eclipse.emf.ecore.resource.Resource.getErrors()

  • org.eclipse.emf.ecore.resource.Resource.getWarnings()

Serializer: Concrete Syntax Validation

The IConcreteSyntaxValidator validates all constraints that are implied by a grammar. Meeting these constraints for a model is mandatory to be serialized.

Example:

MyRule:
  ({MySubRule} "sub")? (strVal+=ID intVal+=INT)*;

This implies several constraints:

  1. Types: only instances of MyRule and MySubRule are allowed for this rule. Sub-types are prohibited, since the parser never instantiates unknown sub-types.

  2. Features: In case the MyRule and MySubRule have EStructuralFeatures besides strVal and intVal, only strVal and intVal may have non-transient values.

  3. Quantities: The following condition must be true: strVal.size() == intVal.size().

  4. Values: It must be possible to convert all values to valid tokens for terminal rule STRING. The same is true for intVal and INT.

The typical use cases for the concrete syntax validator are validation in non-Xtext-editors that, however, use an XtextResource. This is, for example, the case when combining GMF and Xtext. Another use case is when the semantic model is modified “manually” (not by the parser) and then serialized again. Since it is very difficult for the serializer to provide meaningful error messages, the concrete syntax validator is executed by default before serialization. A textual Xtext editor itself, however, is not a valid use case. Here, the parser ensures that all syntactical constraints are met. Therefore, there is no value in additionally running the concrete syntax validator.

There are some limitations to the concrete syntax validator which result from the fact that it treats the grammar as declarative, which is something the parser doesn’t always do.

  • Grammar rules containing assigned actions (e.g. {MyType.myFeature=current} are ignored. Unassigned actions (e.g. {MyType}), however, are supported.

  • Grammar rules that delegate to one or more rules containing assigned actions via unassigned rule calls are ignored.

  • Orders within list-features can not be validated. e.g. Rule: (foo+=R1 foo+=R2)* implies that foo is expected to contain instances of R1 and R2 in an alternating order.

To use concrete syntax validation you can let Guice inject an instance of IConcreteSyntaxValidator and use it directly. Furthermore, there is an adapter ( ConcreteSyntaxEValidator) allows integrating of the concrete syntax validator as an EValidator. You can, for example, enable it in your runtime module, by adding:

@SingletonBinding(eager = true)
public Class<? extends ConcreteSyntaxEValidator> 
      bindConcreteSyntaxEValidator() {
  return ConcreteSyntaxEValidator.class;
}

To customize error messages please see IConcreteSyntaxDiagnosticProvider and subclass ConcreteSyntaxDiagnosticProvider.

Custom Validation

In addition to the afore mentioned kinds of validation, which are more or less done automatically, you can specify additional constraints specific for your Ecore model. We leverage existing EMF API (mainly EValidator) and have put some convenience stuff on top. Basically all you need to do is to make sure that an EValidator is registered for your EPackage. The registry for EValidators ( EValidator.Registry.INSTANCE) can only be filled programmatically. That means contrary to the EPackage and Resource.Factory registries there is no Equinox extension point to populate the validator registry.

For Xtext we provide a generator fragment for the convenient Java-based EValidator API. Just add the following fragment to your generator configuration and you are good to go:

fragment = org.eclipse.xtext.generator.validation.JavaValidatorFragment {}

The generator will provide you with two Java classes. An abstract class generated to src-gen/ which extends the library class AbstractDeclarativeValidator. This one just registers the EPackages for which this validator introduces constraints. The other class is a subclass of that abstract class and is generated to the src/ folder in order to be edited by you. That’s where you put the constraints in.

The purpose of the AbstractDeclarativeValidator is to allow you to write constraints in a declarative way – as the class name already suggests. That is instead of writing exhaustive if-else constructs or extending the generated EMF switch you just have to add the @Check annotation to any method and it will be invoked automatically when validation takes place. Moreover you can state for what type the respective constraint method is, just by declaring a typed parameter. This also lets you avoid any type casts. In addition to the reflective invocation of validation methods the AbstractDeclarativeValidator provides a couple of convenient assertions.

All in all this is very similar to how JUnit works. Here is an example:

public class DomainmodelJavaValidator 
  extends AbstractDomainmodelJavaValidator {
    
  @Check
  public void checkTypeNameStartsWithCapital(Type type) {
    if (!Character.isUpperCase(type.getName().charAt(0)))
      warning("Name should start with a capital", 
        DomainmodelPackage.TYPE__NAME);
  }
}

You can also implement quick fixes for individual validation errors and warnings. See the chapter on quick fixes for details.

Validation with the Check Language

In addition to the Java-based validation code you can use the language Check (from M2T/Xpand) to implement constraint checks against your model. To do so, you have to configure the generator with the CheckFragment. Please note, that you can combine both types of validation in your project.

fragment = org.eclipse.xtext.generator.validation.CheckFragment {}
  

After regenerating your language artifacts you will find three new files “YourLanguageChecks.chk”, “YourLanguageFastChecks.chk” and “YourLanguageExpensiveChecks.chk” in the src/ folder in the sub-package validation. The checks in these files will be executed when saving a file, while typing (FastChecks) or when triggering the validation explicitly (ExpensiveChecks). When using Check the example of the previous chapter could be written like this.

context Type#name WARNING "Name should start with a capital":
  name.toFirstUpper() == name;  

Each check works in a specific context (here: Type) and can further denote a feature to which a warning or error should be attached to (here: name). Each check could either be a WARNING or an ERROR with a given string to explain the situation. The essential part of each check is an invariant that must hold true for the given context. If it fails the check will produce an issue with the provided explanation.

Please read more about the Check language as well as the underlying expression language in Xpand’s reference documentation which is shipped as Eclipse help.

Validating Manually

As noted above, Xtext uses EMF’s EValidator API to register Java or Check validators. You can run the validators on your model programmatically using EMF’s Diagnostician, e.g.

EObject myModel = myResource.getContents().get(0);
Diagnostic diagnostic = Diagnostician.INSTANCE.validate(myModel);
switch (diagnostic.getSeverity()) {
  case Diagnostic.ERROR:
    System.err.println("Model has errors: ",diagnostic);
    break;
  case Diagnostic.WARNING:
    System.err.println("Model has warnings: ",diagnostic);
}

Test Validators

If you have implemented your validators by extending AbstractDeclarativeValidator, there are helper classes which may assist you when testing your validators.

Testing validators typically works as follows:

  1. The test creates some models which intentionally violate some constraints.

  2. The test runs some choosen @Check-methods from the validator.

  3. The test asserts whether the @Check-methods have raised the expected warnings and errors.

To create models, you can either use EMF’s ResourceSet to load models from your hard disk or you can utilize the Factory (which EMF generates for each EPackage) to construct the needed model elements manually. While the fist option has the advantages that you can edit your models in your textual concrete syntax, the second option has the advantage that you can create partial models.

To run the @Check-methods and ensure they raise the intended errors and warnings, you can utilize ValidatorTester as shown by the following example:

Validator:

public class MyLanguageValidator extends AbstractDeclarativeValidator {
  @Check
  public void checkFooElement(FooElement element) {
    if(element.getBarAttribute().contains("foo"))
      error("Only Foos allowed", element, 
        MyLanguagePackage.FOO_ELEMENT__BAR_ATTRIBUTE, 101);
  }
}

JUnit-Test:

public class MyLanguageValidatorTest extends AbstractXtextTests {

  private ValidatorTester<MyLanguageValidator> tester;

  @Override
  public void setUp() {
    with(MyLanguageStandaloneSetup.class);
    MyLanguageValidator validator = get(MyLanguageValidator.class);
    tester = new ValidatorTester<TestingValidator>(validator);
  }

  public void testError() {
    FooElement model = MyLanguageFactory.eINSTANCE.createFooElement()
    model.setBarAttribute("barbarbarbarfoo");
    
    tester.validator().checkFooElement(model);
    tester.diagnose().assertError(101);
  }
  
  public void testError2() {
    FooElement model = MyLanguageFactory.eINSTANCE.createFooElement()
    model.setBarAttribute("barbarbarbarfoo");
    
    tester.validate(model).assertError(101);
  }
}

This example uses JUnit 3, but since the involved classes from Xtext have no dependency on JUnit whatsoever, JUnit 4 and other testing frameworks will work as well. JUnit runs the setUp()-method before each testcase and thereby helps to create some common state. In this example, the validator ( MyLanguageValidator) is instantiated by means of Google Guice. As we inherit from the AbstractXtextTests there are a plenty of useful methods available and the state of the global EMF singletons will be restored in the tearDown(). Afterwards, the ValidatorTester is created and parameterized with the actual validator. It acts as a wrapper for the validator, ensures that the validator has a valid state and provides convenient access to the validator itself ( tester.validator()) as well as to the utility classes which assert diagnostics created by the validator ( tester.diagnose()). Please be aware that you have to call validator() before you can call diagnose(). However, you can call validator() multiple times in a row.

While validator() allows to call the validator’s @Check-methods directly, validate(model) leaves it to the framework to call the applicable @Check-methods. However, to avoid side-effects between tests, it is recommended to call the @Check-methods directly.

diagnose() and validate(model) return an object of type AssertableDiagnostics which provides several assert-methods to verify whether the expected diagnostics are present: