NAME

fix_test_suite.pl -- remove failing tests from test suite until all tests pass.

SYNOPSIS

  fix_test_suite.pl -p project_id -d suite_dir [-f include_file_pattern] [-v version_id] [-s test_suite_src] [-t tmp_dir] [-A] [-D]

-p project_id: The id of the project for which the generated test suites are analyzed. See Project module for available project IDs.
-d suite_dir: The directory that contains the test suite archives. See Test suites.
-f include_file_pattern: The pattern of the file names of the test classes that should be included (optional). Per default all files (*.java) are included.
-v version_id: Only analyze test suites for this version id (optional). Per default all test suites for the given project id are analyzed.
-s test_suite_src: Only analyze test suites originating from this source (optional). A test suite source is a specific tool or configuration (e.g., evosuite-branch). Per default all test suite sources for the given project id are considered.
-t tmp_dir: The temporary root directory to be used to check out program versions (optional). The default is /tmp.
-A: Assertions: Try to remove failing assertions first, before removing the entire test method (optional). By default failing test methods are entirely removed.
-D: Debug: Enable verbose logging and do not delete the temporary check-out directory (optional).

Runs the following worflow for each provided test suite (i.e., each test suite archive in suite_dir):

If a test suite was fixed, its original archive is backed up and replaced with the fixed version.

To be considered for the analysis, a test suite has to be provided as an archive in suite_dir. Format of the archive file name:

project_id-version_id-test_suite_src(\.test_id)?\.tar\.bz2

Note that test_id is optional, the default is 1.

Examples:

This script logs all information to fix_tests.log in the test suite directory SUITE_DIR.