Debugging Test Problems

## 6 Debugging Test Problems

There are a number of reasons that tests can fail to execute properly. This chapter provides some approaches for dealing with these failures. Please note that most of these suggestions are only relevant when running the test harness in GUI mode.

This chapter includes the following topics:

- Overview

- Test Tree

- Folder Information

- Test Information

- Report Files

- Configuration Failures

### 6.1 Overview

The goal of a test run is for all tests in the test suite that are not filtered out to have passing results. If the root test suite folder contains tests with errors or failing results, you must troubleshoot and correct the cause to satisfactorily complete the test run.

- Errors: Tests with errors could not be executed by the JavaTest harness. These errors usually occur because the test environment is not properly configured.

- Failures: Tests that fail were executed but had failing results.

The Test Manager GUI provides you with a number of tools for effectively troubleshooting a test run. See the JavaTest User’s Guide and JavaTest online help for detailed descriptions of the tools described in this chapter. Ant test execution tasks provide command-line users with immediate test execution feedback to the display. Available JTR report files and log files can also help command-line users troubleshoot test run problems.

For every test run, the JavaTest harness creates a set of report files in the reports directory, which you specified by setting the `report.dir` property in the `<TS_HOME>/bin/ts.jte` file. The report files contain information about the test description, environment, messages, properties used by the test, status of the test, and test result. After a test run is completed, the JavaTest harness writes HTML reports for the test run. You can view these files in the JavaTest ReportBrowser when running in GUI mode, or in the Web browser of your choice outside the JavaTest interface. To see all of the HTML report files, enter the URL of the `report.html` file. This file is the root file that links to all of the other HTML reports.

The JavaTest harness also creates a `summary.txt` file in the report directory that you can open in any text editor. The `summary.txt` file contains a list of all tests that were run, their test results, and their status messages.

The work directory, which you specified by setting the `work.dir` property in the `<TS_HOME>/bin/ts.jte` file, contains several files that were deposited there during test execution: `harness.trace`, `log.txt`, `lastRun.txt`, and `testsuite`. Most of these files provide information about the harness and environment in which the tests were executed.

Note | You can set `harness.log.traceflag=true` in `<TS_HOME>/bin/ts.jte` to get more debugging information.

If a large number of tests failed, you should read Configuration Failures to see if a configuration issue is the cause of the failures.

### 6.2 Test Tree

Use the test tree in the JavaTest GUI to identify specific folders and tests that had errors or failing results. Color codes are used to indicate status as follows:

- Green: Passed

- Blue: Test Error

- Red: Failed to pass test

- White: Test not run

- Gray: Test filtered out (not run)

### 6.3 Folder Information

Click a folder in the test tree in the JavaTest GUI to display its tabs.

Choose the Error and the Failed tabs to view the lists of all tests in and under a folder that were not successfully run. You can double-click a test in the lists to view its test information.

### 6.4 Test Information

To display information about a test in the JavaTest GUI, click its icon in the test tree or double-click its name in a folder status tab. The tab contains detailed information about the test run and, at the bottom of the window, a brief status message identifying the type of failure or error. This message may be sufficient for you to identify the cause of the error or failure.

If you need more information to identify the cause of the error or failure, use the following tabs listed in order of importance:

- Test Run Messages contains a Message list and a Message section that display the messages produced during the test run.

- Test Run Details contains a two-column table of name/value pairs recorded when the test was run.

- Configuration contains a two-column table of the test environment name/value pairs derived from the configuration data actually used to run the test.

Note | You can set `harness.log.traceflag=true` in `<TS_HOME>/bin/ts.jte` to get more debugging information.

### 6.5 Report Files

Report files are another good source of troubleshooting information. You may view the individual test results of a batch run in the JavaTest Summary window, but there are also a wide range of HTML report files that you can view in the JavaTest ReportBrowser or in the external browser or your choice following a test run. See Section 5.5, "Test Reports," for more information.

### 6.6 Configuration Failures

Configuration failures are easily recognized because many tests fail the same way. When all your tests begin to fail, you may want to stop the run immediately and start viewing individual test output. However, in the case of full-scale launching problems where no tests are actually processed, report files are usually not created (though sometimes a small `harness.trace` file in the report directory is written).

<!--

    Copyright (c) 2020 Oracle and/or its affiliates. All rights reserved.

    This program and the accompanying materials are made available under the
    terms of the Eclipse Public License v. 2.0, which is available at
    http://www.eclipse.org/legal/epl-2.0.

    This Source Code may also be made available under the following Secondary
    Licenses when the conditions for such availability set forth in the
    Eclipse Public License v. 2.0 are satisfied: GNU General Public License,
    version 2 with the GNU Classpath Exception, which is available at
    https://www.gnu.org/software/classpath/license.html.

    SPDX-License-Identifier: EPL-2.0 OR GPL-2.0 WITH Classpath-exception-2.0

-→

### 6.7 Troubleshooting Tips

This section provides some tips for troubleshooting errors that may be encountered.

- Verify that the Ant `config.vi` and `enable.jacc` configuration targets executed correctly.

- If there are several failures during a test run, check the various output for hints about what caused the failures. A common problem is the absence of the Server Faces log file. This log file should be created in the directory defined by the `log.file.location` property in the `ts.jte` file. There should be a log file called `JACCLog.txt` in this directory. The `JACCLog.txt` consists of record entries containing permission infomation that will be used to verify the TCK tests for proper compliance. This log file typically gets populated with Server Faces-based security information when test archives are deployed. Then, during test execution, the `JACCLog.txt` file is read and used for validating that Server Faces behavior is correct.

- Simultaneously deploying all Server Faces TCK test archives may cause false failures. If unexpected failures occur during a TCK run when all Server Faces archives were deployed, these failures could be caused by interference from tests and archives that are defined multiple times. If such situational failures do occur, undeploy all archives, remove the `JACCLog.txt` file, recycle your server (if necessary), and rerun only the tests in the directory that showed failures.

- Check that the following JVM variables, which should have been set by invoking the `enable.jacc` Ant target, have been set in the application server :

  - `-Dlog.file.location` (this comes from the `<TS_HOME>/bin/ts.jte` property)

  - `-Djakarta.security.jacc.policy.provider=com.sun.ts.tests.jacc.provider.TSPolicy`

  - `-Djakarta.security.jacc.PolicyConfigurationFactory.provider=com.sun.ts.tests.jacc.provider.TSPolicyConfigurationFactoryImpl`

  - `-Dvendor.jakarta.security.jacc.policy.provider=com.sun.enterprise.security.provider.PolicyWrapper`

  - `-Dvendor.jakarta.security.jacc.PolicyConfigurationFactory.provider=com.sun.enterprise.security.provider.PolicyConfigurationFactoryImpl`

Note | The values for the `-Dvendor.jakarta.security.jacc.policy.provider` and `-Dvendor.jakarta.security.jacc.PolicyConfigurationFactory.provider` JVM variables will need to be set specific to the application server in use.

Copyright © 2018, 2020 Oracle and/or its affiliates. All rights reserved.