Eclipse.org Eclipse.org - Device Kit

Validation Testing Introduction

Getting Started with Validation Testing

The following steps are needed to create and run a validation test suite.

  1. Implement one or more TestAgent bundles, without an activator, with one TestCase or a TestSuite with a collection of TestCases per TestAgent.
  2. For deployment in an OSGi environment, implement a TestAgentBundle, TestAgentManaged or a TestAgentFactory activator bundle for each TestAgent.
  3. Write an XML test script.
  4. Create a deployment configuration to deploy the application under test, the TestAgents, TestController and TestManager.
  5. Run the test script and verify the test report.

These steps will be described in more detail in the following sections, and examples will be given.

Step 1: Implement a new TestAgent

The first step is to create a new bundle project that implements a concrete subclass of TestAgent and a specialization of the TestAgentService interface (in order to be able to identify this specific test agent in an OSGi environment). For the subclass of TestAgent the only requirement is to implement the framework methods that return a description and a test class. The test class must follow the JUnit 3 convention, that is, either provide a static suite() method or provide one or more instance methods starting with “test”.

import java.util.Dictionary;
import java.util.Hashtable;
import org.eclipse.soda.dk.testagent.TestAgent;
import org.eclipse.soda.dk.testagent.service.TestSynchronizationService;
 
public class ExampleTestAgent extends TestAgent {
 
      public static final String DESCRIPTION = " Example TestAgent";
 
      public String getDescription() {
            return DESCRIPTION;
      }
 
      protected Class loadTestClass() {
            return ExampleTestSuite.class;
      }
 
}

Please note that the Device Kit provides a wizard to create the projects for a new TestAgent with the skeleton code for a TestSuite with a single TestCase.

Figure 1 - New TestAgent wizard

Here is an example of a TestSuite. It is an example of a ConfiguredTestSuite, taken from org.eclipse.soda.dk.testagent.test, which is the self-test for TestAgent. In this example you see that the test suite runs certain test cases only if certain capabilities are available in the execution environment.

public class TestAgentTestSuite extends ConfiguredTestSuite {
 
      public static Test suite() {
            TestAgentTestSuite suite = new TestAgentTestSuite();
            suite.addTest(ConfigurationTestCase.class);
            if (suite.getContext().getSynchronizationService() != null) {
                  suite.addTest(SynchronizationTestCase.class);
            }
            if (suite.getContext().getNotificationService() != null) {
                  suite.addTest(NotificationProbeTestCase.class);
            }
            return suite;
      }
 
}

You can open the source code for ConfigurationTestCase, SynchronizationTestCase, and NotificationProbeTestCase to see basic examples of using the configuration, synchronization and notification features provided by the TestAgent framework.

Following the common coding patterns in the Device Kit, the ExampleTestAgentService interface should be defined as follows:

import org.eclipse.soda.dk.testagent.service.TestAgentService;
public interface ExampleTestService extends TestAgentService {
      public static final String SERVICE_NAME = ExampleeTestService.class.getName();
 
}

Step 2: Implement activators for the new TestAgent

To implement the ExampleTestAgentBundle, ExampleTestAgentManaged and ExampleTestAgentFactory activator bundles, create a new bundle project for each and extend the framework activator class provided by TestAgentBundle, TestAgentManaged and TestAgentFactory. The minimum requirement is to provide an implementation for:

  1. getExportedServiceNames, to list the interfaces that will be used to register the TestAgent.
  2. createProperties, to define the specific registration properties for this service. Note that the SERVICE_PID property will only be used for unmanaged services.
  3. createService, to create a new instance of your TestAgent.

The Device Kit wizard for creating a new TestAgent also creates the desired activator projects.

Figure 2 - Creating Activator projects with the New TestAgent wizard

Here is the example for TestAgentTestFactory.

import java.util.Hashtable;
import org.eclipse.soda.dk.testagent.TestAgent;
import org.eclipse.soda.dk.testagent.factory.TestAgentFactory;
import org.eclipse.soda.dk.testagent.service.TestAgentService;
import org.eclipse.soda.dk.testagent.test.TestAgentTest;
import org.eclipse.soda.dk.testagent.test.service.TestAgentTestService;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.Constants;
 
public class TestAgentTestFactory extends TestAgentFactory implements
            BundleActivator {
 
      public static final String CLASS_NAME = "org.eclipse.soda.dk.testagent.test.factory.TestAgentTestFactory"; //$NON-NLS-1$
 
      public static final String[] EXPORTED_SERVICE_NAMES = new String[] {
            TestAgentService.SERVICE_NAME,
            TestAgentTestService.SERVICE_NAME
      };
 
      public String[] getExportedServiceNames() {
            return EXPORTED_SERVICE_NAMES;
      }
 
      public Hashtable createProperties() {
            final Hashtable properties = super.createProperties();
            properties.put(Constants.SERVICE_PID, TestAgentTestService.SERVICE_NAME);
            properties.put(Constants.SERVICE_DESCRIPTION, TestAgentTest.DESCRIPTION);
            properties.put(Constants.SERVICE_VENDOR, "Eclipse"); //$NON-NLS-1$
            properties.put(Constants.SERVICE_RANKING, "6"); //$NON-NLS-1$
            return properties;
      }
 
      public TestAgent createService() {
            return new TestAgentTest();
      }
}

If it is necessary to acquire additional optional or non-optional services and make them available to the test via the TestContext, one can do so by overriding the applicable methods of the abstract framework activator class that is being used.

Step 3: Write an XML test script

The test script language is introduced with an example. This example is a test script to exercise the TestAgent, TestController and TestManager themselves by running sequential and parallel tests implemented in the TestAgentTest.

 Figure 3 - Example Test Script

Please note that the Device Kit wizard for creating a new TestAgent also creates a sample test script in each of the activator projects of the new agent

Although not shown in this example, it is not only possible to use wildcards in the configuration OSGi LDAP filter specification, but also in the test filter specification. In case multiple tests match the filter all matching tests are run concurrently if the attribute match=”multiple” is specified.

          <test filter="(id=*)" match="multiple">
          </test>

The attributes attributes for the configuration element are the following:

        <configuration factoryPid="..." pid="..." filter="..." bundleLocation="..." match="single|multiple">
          ...
        </configuration>

  1. ConfigurationAction type="create":

  2. ConfigurationAction type="createOrUpdate":

  3. ConfigurationAction type="update" (yes, there also is an update):

  4. ConfigurationAction type="delete":

The type attribute of the <testAction> element is optional and has possible values "sequential" and "parallel". Sequential is the default.

The <test> element has optional attributes: id, filter, class and match.

The <property> element has optional attributes: type and cardinality.

The <title>, <version>, <vendor> and <description> elements are optional and can appear in any order.

Note: the XML schema definition (dktest.xsd) with detailed documentation annotations can be found in the reference section of the Device Kit documentation.

Step 4: Creating a launch configuration for running the new TestAgent

To create a launch configuration for running the test script of the previous section in the Equinox OSGi runtime the following bundles must be loaded.

Please note that the Device Kit wizard for creating a new TestAgent also creates a sample launch configuration in each of the activator projects of the new agent.

Step 5: Running the test script in the Equinox OSGi runtime

To run the example test script using the servlet-based user interface of the TestManager, perform the following steps:

  1. Run the Equinox launch configuration created in the previous section.
  2. Open the main servlet of the Device Kit, http://localhost/device

    3.  Figure 4 - Main servlet of the Device Kit


  3. Navigate to the http://localhost/device/test/manager
    4. servlet.
  4. Select the test script for uploading to the TestManager.

    5.  Figure 5 - Uploading the test script


  5. Observe the execution of the test in the test log panel and interact with the tests through the control panel.

    6.  Figure 6 - Running and interacting with the test


  6. Verify the test summary printed at the end of the test.

Running a test script as a standard Java application

To run the second example test script in a non-OSGi environment, perform the following steps:

  1. Create a launch configuration or script with the TestManager, TestController and TestAgent jars on the classpath.
  2. Run main class org.eclipse.soda.dk.testmanager.TestManager with the URL or path name of the test script optionally as program argument (test.xml is the default).
    Example:

    java org.eclipse.soda.dk.testmanager.TestManager test.xml

  3. Observe the execution of the test and interact with the tests through the console.
  4. Verify the test summary printed at the end of the test.

 

A version of the example test script that would exercise the TestAgentTest in a non-OSGi environment would looks as follows. Note that in such an environment configuration actions are not supported and TestAgents must be identified by their fully qualified class name.

<?xml version="1.0" encoding="UTF-8"?>
<validationTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="dktest.xsd">
  <title>Test of the Device Kit Test Framework</title>
  <version>1.0.0</version>
  <vendor>Eclipse</vendor>
  <description>Validation test for TestManager, TestController and TestAgent</description>
  <actions>
    <testAction type="sequential">
      <tests>
        <test class="org.eclipse.soda.dk.testagent.test.TestAgentTest">
          <properties>
            <property key="id" value="TestAgentTest-1"/>
              <property key="testKey" value="testValue"/>
          </properties>
        </test>
        <test class="org.eclipse.soda.dk.testagent.test.TestAgentTest">
          <properties>
            <property key="id" value="TestAgentTest-2"/>
              <property key="testKey" value="otherValue"/>
              <property key="expectedValue" value="otherValue"/>
          </properties>
        </test>
      </tests>
    </testAction>
    <testAction type="parallel">
      <tests>
        <test id="parA" class="org.eclipse.soda.dk.testagent.test.TestAgentTest">
          <properties>
            <property key="id" value="TestAgentTest-1"/>
              <property key="testKey" value="testValue"/>
          </properties>
        </test>
        <test id="ParB" class="org.eclipse.soda.dk.testagent.test.TestAgentTest">
          <properties>
            <property key="id" value="TestAgentTest-2"/>
              <property key="testKey" value="otherValue"/>
              <property key="expectedValue" value="otherValue"/>
          </properties>
        </test>
      </tests>
    </testAction>
  </actions>

</validationTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="dktest.xsd">

Example scripts for Standalone, Bundle-, Managed- and Factory-based testing

Depending on how the test is deployed, the test script will be slightly different. The following examples illustrate the differences for running the TestAgentTest in a non-OSGi environment and using either the Bundle, the Managed or the Factory-based activator in an OSGi environment.

The differences are highlighted in boldface.

Please note that the Device Kit wizard for creating a new TestAgent creates these sample test scripts in each of the activator projects of the new agent.

Sample script for testing in a standard Java runtime environment

<?xml version="1.0" encoding="UTF-8"?>
<validationTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="dktest.xsd">
  <title>TestAgentTest</title>
  <version>1.0.0</version>
  <vendor>Eclipse</vendor>
  <description>Static test for TestAgent</description>
  <actions>
    <testAction type="sequential">
      <tests>
        <test id="TestAgentTest" class="org.eclipse.soda.dk.testagent.test.TestAgentTest">
          <properties>
              <property key="testKey" value="testValue"/>
          </properties>
        </test>
      </tests>
    </testAction>
  </actions>
</validationTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="dktest.xsd">

Sample script for testing with a Bundle activator

<?xml version="1.0" encoding="UTF-8"?>
<validationTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="dktest.xsd">
  <title>TestAgentTest</title>
  <version>1.0.0</version>
  <vendor>Eclipse</vendor>
  <description>Bundle test for TestAgent</description>
  <actions>
    <!-- Invoke the TestAgentTest. -->
    <testAction type="sequential">
      <tests>
        <test id="TestAgentTest" filter="(service.pid=org.eclipse.soda.dk.testagent.test.service.TestAgentTestService)">
          <properties>
              <property key="testKey" value="testValue"/>
          </properties>
        </test>
      </tests>
    </testAction>
  </actions>
</validationTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="dktest.xsd">

Sample script for testing with a Managed activator

<?xml version="1.0" encoding="UTF-8"?>
<validationTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="dktest.xsd">
  <title>TestAgentTest</title>
  <version>1.0.0</version>
  <vendor>eclipse</vendor>
  <description>Managed test for TestAgent</description>
  <actions>
    <!-- Create or update test configurations. -->
    <configurationAction type="createOrUpdate">
      <configurations>
        <configuration pid="org.eclipse.soda.dk.testagent.test.managed.TestAgentTestManaged">
          <properties>
              <property key="id" value="TestAgentTest"/>
          </properties>
        </configuration>
      </configurations>
    </configurationAction>
    <!-- Invoke the TestAgentTest. -->
    <testAction type="sequential">
      <tests>
        <test id="TestAgentTest" filter="(id=TestAgentTest)">
          <properties>
              <property key="testKey" value="testValue"/>
          </properties>
        </test>
      </tests>
    </testAction>
  </actions>

</validationTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="dktest.xsd">

Sample script for testing with a Factory activator

<?xml version="1.0" encoding="UTF-8"?>
<validationTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="dktest.xsd">
  <title>TestAgentTest</title>
  <version>1.0.0</version>
  <vendor>Eclipse</vendor>
  <description>Factory-based test for TestAgent</description>
  <actions>
    <!-- Create new test configurations. -->
    <configurationAction type="create">
      <configurations>
        <configuration factoryPid="org.eclipse.soda.dk.testagent.test.factory.TestAgentTestFactory">
          <properties>
            <property key="testId" value="TestAgentTest-1"/>
          </properties>
        </configuration>
      </configurations>
    </configurationAction>
    <!-- Invoke the TestAgentTest. -->
    <testAction type="sequential">
      <tests>
        <test id="TestAgentTest" filter="(testId=TestAgentTest-1)">
          <properties>
              <property key="testKey" value="testValue"/>
          </properties>
        </test>
      </tests>
    </testAction>
    <!-- Delete all created test configurations. -->
    <configurationAction type="delete">
      <configurations>
        <configuration filter="(service.pid=org.eclipse.soda.dk.testagent.test.factory.*)"/>
      </configurations>
    </configurationAction>
  </actions>
</validationTest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="dktest.xsd">

Copyright © 2007 IBM. All Rights Reserved.