Pysys

--------------
- Fixed a defect in the unix process helper module to correct a file
handle leak in the write end of the stdin pipe.

--------------
- Added the ability to run tests concurrently through the -n | --threads
option to the pysys launcher run target. Tests to be run are placed on a
request queue and processed by the designated number of worker threads.
The results of each test are then placed on a result queue, collated and
displayed in the order in which they would run serially. Depending on
the nature of the application under test, the recommended number of
threads to designate when using this option is no more than two times
the number of CPUs. Note also that care needs to be made when running
tests in parallel, so as to ensure no shared resources are accessed
in a non-atomic way, e.g using direct references to os.environ() in one
test when another test modifies the environment directly etc.
- The constructor to the pysys.baserunner.BaseRunner class was changed to
include the threads parameter, i.e. ::

def __init__(self, record, purge, cycle, mode, threads, outsubdir, descriptors, xargs)

This parameter is required for the runner to create the required
threadpool before running a set of tests in parallel. Any custom runner
classes extending the base runner will need to be updated to incorporate
this change.
- Removed module specific loggers from pysys in order to support running
tests in parallel. There is now a single logger used within the
framework, and which can be referenced directly from the pysys.constants
module. Attached to this logger are two handler types; one for logging
to stdout, and one for logging to the run
log file saved in the output subdirectory of each test. The stdout
handler is set to only log to stdout from the main thread, whilst the
run log file handlers are set to log to the output subdirectory of a
test only on the creating thread.
- Added exception handling to the pysys.process.user module when trying to
stop all processes on destruction. When a background process takes
longer to stop than the default timeout period, the thrown
pysys.exceptions.ProcessTimeout exception was uncaught causing abnormal
exit from the test run.

--------------
- The clean target has been updated to accept the -a | --all command line
option to allow deleting all derived files produced when running a set
of testcases, i.e. both the testcase output subdirectory and any
compiled test class modules.
- The waitForSignal method of the ProcessUser class, subclassed by both
the BaseTestand BaseRunner classes, has been updated to return a list of
match objects on invocation. By using tagged regular expressions in the
expr parameter of the method call, this allows retrieval of portions of
the matched data e.g. to extract expressions in the file to use later in
the validation routines.
- All references to pysys.constants.TRUE and pysys.constants.FALSE have
been replaced by the native Python True and False literals. The values
of the constants have been set to True and False respectively so as to
maintain backwards compatibility.

--------------
- The PySys test and PySys project files have been renamed by default from
.pysystest to pysystest.xm, and .pysysproject to pysysproject.xml
respectively. Backwards compatibility is maintained for the previous
file naming convention, though this will be deprecated in a later
release; it is strongly advised that the new naming convention is
adopted. New tests made using the PySys launcher will by default use the
new naming convention. This change was made due to issues on Windows
systems displaying and recognising hidden files, and files without
specified extensions e.g. within the Eclipse framework, for display in
internet browsers etc.
- The clean mode of operation has been added to the pysys.py launcher. This
allows removal of testcase output subdirectories, e.g. before importing
into a source code control system. The -o option allows specifying the
output subdirectory name to be deleted, which defaults to the platform
identifier if not specified.
- The test output summary writer interface has been changed so that the
test output is written and updated during the test execution; previously
a call to the writer was only made on completion of the test run. This
allows monitoring the test output summary during the test execution to
monitor the run time status of the tests.
- Added the XMLFileResultsWriter class to the pysys.writer module. This
performs logging of the test output summary in an XML format, suitable
for display via XLST in a web browser. A simple XSL stylesheet is
included with the PySys distribution to provide better display in
internet browsers.
- Added the ability to specify custom test output summary writers in the
PySys project file via the <writer> tag. For an example see the
.pysysproject file in the pysys-examples distribution. Should no
<writer> be specified in the project file, the default
XMLFileResultsWriter will be used. Multiple writers may be specified in
the PySys project file.
- Added exception logging on parsing errors in the PySys project file, e.g.
when the file in badly formed due to invalid XML tokens etc.
- Added variable argument passing to the process.monitor.ProcessMonitor
class constructor so that operating specific arguments can be passed
into the class on instantiation. The wrapper method
pysys.basetest.BaseTest.startProcessMonitor has also been updated to
allow pass through of the variable arguments.
- The win32 process.monitor module has been changed so that on windows
systems the percentage CPU usage is not normalised by default by the
number of available processors, e.g. on a 4 core processor if 2 cores
were fully utilized the CPU usage was previously output as 50% - the
change means that the reported usage will now be 200% (a value of 100%
indicates that one core is fully utilitised). This makes the output
consistent with that reported on unix systems. Should the
previous behavior be required the numProcessors argument can be passed
to the pysys.basetest.BaseTest.startProcessMonitor method in order to
normalise the CPU usage statistics by the number of processors. On
windows systems the number of processors can be obtained from the
NUM_PROCESSORS environment variable.
- Added comments to the PySys Project file distributed with the example
testcases, to detail the possible configuration options.

--------------
- The lastgrep method has been added to pysys.utils.filegrep, and the
assertLastGrep method has been added to the BaseTest class. This allows
test validation to be performed based on regular expression matching on
the last line of an input file to the assertLastGrep method.
- The win32 process monitor has been modified to calculate the percentage
CPU usage statistics as a sum over all available processors. A CPU usage
of 100% represents the process fully utilising all available processors.
- The win32 process monitor now also logs the handle count of a process.

--------------
- Fixed a bug in pysys.process.user.ProcessUser destructor to explicitly
set the process list to null to allow process handles to be cleaned up
on destruction. This bug only seemed to be exhibited when the process
handle of a process returned in the startProcess() method was set as a
data attribute to an instance of the class. This handle was then both a
data attribute of the class, and was contained in a list data attribute
of the class. Under these conditions the handles were not being released
correctly.
- The print mode of the pysys.py launcher now supports printing out the
test user defined modes, and the printing out of tests that can be run
in a given mode.