Testtools

~~~~~

In this release we bring some very interesting improvements:

* new matchers for exceptions, sets, lists, dicts and more.

* experimental (works but the contract isn't supported) twisted reactor
support.

* The built in runner can now list tests and filter tests (the -l and
--load-list options).

Changes
-------

* addUnexpectedSuccess is translated to addFailure for test results that don't
know about addUnexpectedSuccess. Further, it fails the entire result for
all testtools TestResults (i.e. wasSuccessful() returns False after
addUnexpectedSuccess has been called). Note that when using a delegating
result such as ThreadsafeForwardingResult, MultiTestResult or
ExtendedToOriginalDecorator then the behaviour of addUnexpectedSuccess is
determined by the delegated to result(s).
(Jonathan Lange, Robert Collins, 654474, 683332)

* startTestRun will reset any errors on the result. That is, wasSuccessful()
will always return True immediately after startTestRun() is called. This
only applies to delegated test results (ThreadsafeForwardingResult,
MultiTestResult and ExtendedToOriginalDecorator) if the delegated to result
is a testtools test result - we cannot reliably reset the state of unknown
test result class instances. (Jonathan Lange, Robert Collins, 683332)

* Responsibility for running test cleanups has been moved to ``RunTest``.
This change does not affect public APIs and can be safely ignored by test
authors. (Jonathan Lange, 662647)

Improvements
------------

* New matchers:

* ``EndsWith`` which complements the existing ``StartsWith`` matcher.
(Jonathan Lange, 669165)

* ``MatchesException`` matches an exception class and parameters. (Robert
Collins)

* ``KeysEqual`` matches a dictionary with particular keys. (Jonathan Lange)

* ``assertIsInstance`` supports a custom error message to be supplied, which
is necessary when using ``assertDictEqual`` on Python 2.7 with a
``testtools.TestCase`` base class. (Jelmer Vernooij)

* Experimental support for running tests that return Deferreds.
(Jonathan Lange, Martin [gz])

* Provide a per-test decorator, run_test_with, to specify which RunTest
object to use for a given test. (Jonathan Lange, 657780)

* Fix the runTest parameter of TestCase to actually work, rather than raising
a TypeError. (Jonathan Lange, 657760)

* Non-release snapshots of testtools will now work with buildout.
(Jonathan Lange, 613734)

* Malformed SyntaxErrors no longer blow up the test suite. (Martin [gz])

* ``MismatchesAll.describe`` no longer appends a trailing newline.
(Michael Hudson-Doyle, 686790)

* New helpers for conditionally importing modules, ``try_import`` and
``try_imports``. (Jonathan Lange)

* ``Raises`` added to the ``testtools.matchers`` module - matches if the
supplied callable raises, and delegates to an optional matcher for validation
of the exception. (Robert Collins)

* ``raises`` added to the ``testtools.matchers`` module - matches if the
supplied callable raises and delegates to ``MatchesException`` to validate
the exception. (Jonathan Lange)

* Tests will now pass on Python 2.6.4 : an ``Exception`` change made only in
2.6.4 and reverted in Python 2.6.5 was causing test failures on that version.
(Martin [gz], 689858).

* ``testtools.TestCase.useFixture`` has been added to glue with fixtures nicely.
(Robert Collins)

* ``testtools.run`` now supports ``-l`` to list tests rather than executing
them. This is useful for integration with external test analysis/processing
tools like subunit and testrepository. (Robert Collins)

* ``testtools.run`` now supports ``--load-list``, which takes a file containing
test ids, one per line, and intersects those ids with the tests found. This
allows fine grained control of what tests are run even when the tests cannot
be named as objects to import (e.g. due to test parameterisation via
testscenarios). (Robert Collins)

* Update documentation to say how to use testtools.run() on Python 2.4.
(Jonathan Lange, 501174)

* ``text_content`` conveniently converts a Python string to a Content object.
(Jonathan Lange, James Westby)

~~~~~

Lots of little cleanups in this release; many small improvements to make your
testing life more pleasant.

Improvements
------------

* Cleanups can raise ``testtools.MultipleExceptions`` if they have multiple
exceptions to report. For instance, a cleanup which is itself responsible for
running several different internal cleanup routines might use this.

* Code duplication between assertEqual and the matcher Equals has been removed.

* In normal circumstances, a TestCase will no longer share details with clones
of itself. (Andrew Bennetts, bug 637725)

* Less exception object cycles are generated (reduces peak memory use between
garbage collection). (Martin [gz])

* New matchers 'DoesNotStartWith' and 'StartsWith' contributed by Canonical
from the Launchpad project. Written by James Westby.

* Timestamps as produced by subunit protocol clients are now forwarded in the
ThreadsafeForwardingResult so correct test durations can be reported.
(Martin [gz], Robert Collins, 625594)

* With unittest from Python 2.7 skipped tests will now show only the reason
rather than a serialisation of all details. (Martin [gz], 625583)

* The testtools release process is now a little better documented and a little
smoother. (Jonathan Lange, 623483, 623487)

~~~~~

Nothing major in this release, just enough small bits and pieces to make it
useful enough to upgrade to.

In particular, a serious bug in assertThat() has been fixed, it's easier to
write Matchers, there's a TestCase.patch() method for those inevitable monkey
patches and TestCase.assertEqual gives slightly nicer errors.

Improvements
------------

* 'TestCase.assertEqual' now formats errors a little more nicely, in the
style of bzrlib.

* Added `PlaceHolder` and `ErrorHolder`, TestCase-like objects that can be
used to add results to a `TestResult`.

* 'Mismatch' now takes optional description and details parameters, so
custom Matchers aren't compelled to make their own subclass.

* jml added a built-in UTF8_TEXT ContentType to make it slightly easier to
add details to test results. See bug 520044.

* Fix a bug in our built-in matchers where assertThat would blow up if any
of them failed. All built-in mismatch objects now provide get_details().

* New 'Is' matcher, which lets you assert that a thing is identical to
another thing.

* New 'LessThan' matcher which lets you assert that a thing is less than
another thing.

* TestCase now has a 'patch()' method to make it easier to monkey-patching
objects in tests. See the manual for more information. Fixes bug 310770.

* MultiTestResult methods now pass back return values from the results it
forwards to.

~~~~~

This release fixes some obscure traceback formatting issues that probably
weren't affecting you but were certainly breaking our own test suite.

Changes
-------

* Jamu Kakar has updated classes in testtools.matchers and testtools.runtest
to be new-style classes, fixing bug 611273.

Improvements
------------

* Martin[gz] fixed traceback handling to handle cases where extract_tb returns
a source line of None. Fixes bug 611307.

* Martin[gz] fixed an unicode issue that was causing the tests to fail,
closing bug 604187.

* testtools now handles string exceptions (although why would you want to use
them?) and formats their tracebacks correctly. Thanks to Martin[gz] for
fixing bug 592262.

~~~~~

This release overhauls the traceback formatting layer to deal with Python 2
line numbers and traceback objects often being local user encoded strings
rather than unicode objects. Test discovery has also been added and Python 3.1
is also supported. Finally, the Mismatch protocol has been extended to let
Matchers collaborate with tests in supplying detailed data about failures.

Changes
-------

* testtools.utils has been renamed to testtools.compat. Importing
testtools.utils will now generate a deprecation warning.

Improvements
------------

* Add machinery for Python 2 to create unicode tracebacks like those used by
Python 3. This means testtools no longer throws on encountering non-ascii
filenames, source lines, or exception strings when displaying test results.
Largely contributed by Martin[gz] with some tweaks from Robert Collins.

* James Westby has supplied test discovery support using the Python 2.7
TestRunner in testtools.run. This requires the 'discover' module. This
closes bug 250764.

* Python 3.1 is now supported, thanks to Martin[gz] for a partial patch.
This fixes bug 592375.

* TestCase.addCleanup has had its docstring corrected about when cleanups run.

* TestCase.skip is now deprecated in favour of TestCase.skipTest, which is the
Python2.7 spelling for skip. This closes bug 560436.

* Tests work on IronPython patch from Martin[gz] applied.

* Thanks to a patch from James Westby testtools.matchers.Mismatch can now
supply a get_details method, which assertThat will query to provide
additional attachments. This can be used to provide additional detail
about the mismatch that doesn't suite being included in describe(). For
instance, if the match process was complex, a log of the process could be
included, permitting debugging.

* testtools.testresults.real._StringException will now answer __str__ if its
value is unicode by encoding with UTF8, and vice versa to answer __unicode__.
This permits subunit decoded exceptions to contain unicode and still format
correctly.

~~~~~

More matchers, Python 2.4 support, faster test cloning by switching to copy
rather than deepcopy and better output when exceptions occur in cleanups are
the defining characteristics of this release.

Improvements
------------

* New matcher "Annotate" that adds a simple string message to another matcher,
much like the option 'message' parameter to standard library assertFoo
methods.

* New matchers "Not" and "MatchesAll". "Not" will invert another matcher, and
"MatchesAll" that needs a successful match for all of its arguments.

* On Python 2.4, where types.FunctionType cannot be deepcopied, testtools will
now monkeypatch copy._deepcopy_dispatch using the same trivial patch that
added such support to Python 2.5. The monkey patch is triggered by the
absence of FunctionType from the dispatch dict rather than a version check.
Bug 498030.

* On windows the test 'test_now_datetime_now' should now work reliably.

* TestCase.getUniqueInteger and TestCase.getUniqueString now have docstrings.

* TestCase.getUniqueString now takes an optional prefix parameter, so you can
now use it in circumstances that forbid strings with '.'s, and such like.

* testtools.testcase.clone_test_with_new_id now uses copy.copy, rather than
copy.deepcopy. Tests that need a deeper copy should use the copy protocol to
control how they are copied. Bug 498869.

* The backtrace test result output tests should now pass on windows and other
systems where os.sep is not '/'.

* When a cleanUp or tearDown exception occurs, it is now accumulated as a new
traceback in the test details, rather than as a separate call to addError /
addException. This makes testtools work better with most TestResult objects
and fixes bug 335816.