============================
Breaking Changes
----------------
- :issue:`1316`: ``TestReport.longrepr`` is now always an instance of ``ReprExceptionInfo``. Previously it was a ``str`` when a test failed with ``pytest.fail(..., pytrace=False)``.
- :issue:`5965`: symlinks are no longer resolved during collection and matching `conftest.py` files with test file paths.
Resolving symlinks for the current directory and during collection was introduced as a bugfix in 3.9.0, but it actually is a new feature which had unfortunate consequences in Windows and surprising results in other platforms.
The team decided to step back on resolving symlinks at all, planning to review this in the future with a more solid solution (see discussion in
:pull:`6523` for details).
This might break test suites which made use of this feature; the fix is to create a symlink
for the entire test tree, and not only to partial files/tress as it was possible previously.
- :issue:`6505`: ``Testdir.run().parseoutcomes()`` now always returns the parsed nouns in plural form.
Originally ``parseoutcomes()`` would always returns the nouns in plural form, but a change
meant to improve the terminal summary by using singular form single items (``1 warning`` or ``1 error``)
caused an unintended regression by changing the keys returned by ``parseoutcomes()``.
Now the API guarantees to always return the plural form, so calls like this:
.. code-block:: python
result = testdir.runpytest()
result.assert_outcomes(error=1)
Need to be changed to:
.. code-block:: python
result = testdir.runpytest()
result.assert_outcomes(errors=1)
- :issue:`6903`: The ``os.dup()`` function is now assumed to exist. We are not aware of any
supported Python 3 implementations which do not provide it.
- :issue:`7040`: ``-k`` no longer matches against the names of the directories outside the test session root.
Also, ``pytest.Package.name`` is now just the name of the directory containing the package's
``__init__.py`` file, instead of the full path. This is consistent with how the other nodes
are named, and also one of the reasons why ``-k`` would match against any directory containing
the test suite.
- :issue:`7122`: Expressions given to the ``-m`` and ``-k`` options are no longer evaluated using Python's :func:`eval`.
The format supports ``or``, ``and``, ``not``, parenthesis and general identifiers to match against.
Python constants, keywords or other operators are no longer evaluated differently.
- :issue:`7135`: Pytest now uses its own ``TerminalWriter`` class instead of using the one from the ``py`` library.
Plugins generally access this class through ``TerminalReporter.writer``, ``TerminalReporter.write()``
(and similar methods), or ``_pytest.config.create_terminal_writer()``.
The following breaking changes were made:
- Output (``write()`` method and others) no longer flush implicitly; the flushing behavior
of the underlying file is respected. To flush explicitly (for example, if you
want output to be shown before an end-of-line is printed), use ``write(flush=True)`` or
``terminal_writer.flush()``.
- Explicit Windows console support was removed, delegated to the colorama library.
- Support for writing ``bytes`` was removed.
- The ``reline`` method and ``chars_on_current_line`` property were removed.
- The ``stringio`` and ``encoding`` arguments was removed.
- Support for passing a callable instead of a file was removed.
- :issue:`7224`: The `item.catch_log_handler` and `item.catch_log_handlers` attributes, set by the
logging plugin and never meant to be public, are no longer available.
The deprecated ``--no-print-logs`` option and ``log_print`` ini option are removed. Use ``--show-capture`` instead.
- :issue:`7226`: Removed the unused ``args`` parameter from ``pytest.Function.__init__``.
- :issue:`7418`: Removed the `pytest_doctest_prepare_content` hook specification. This hook
hasn't been triggered by pytest for at least 10 years.
- :issue:`7438`: Some changes were made to the internal ``_pytest._code.source``, listed here
for the benefit of plugin authors who may be using it:
- The ``deindent`` argument to ``Source()`` has been removed, now it is always true.
- Support for zero or multiple arguments to ``Source()`` has been removed.
- Support for comparing ``Source`` with an ``str`` has been removed.
- The methods ``Source.isparseable()`` and ``Source.putaround()`` have been removed.
- The method ``Source.compile()`` and function ``_pytest._code.compile()`` have
been removed; use plain ``compile()`` instead.
- The function ``_pytest._code.source.getsource()`` has been removed; use
``Source()`` directly instead.
Deprecations
------------
- :issue:`7210`: The special ``-k '-expr'`` syntax to ``-k`` is deprecated. Use ``-k 'not expr'``
instead.
The special ``-k 'expr:'`` syntax to ``-k`` is deprecated. Please open an issue
if you use this and want a replacement.
- :issue:`4049`: ``pytest_warning_captured`` is deprecated in favor of the ``pytest_warning_recorded`` hook.
Features
--------
- :issue:`1556`: pytest now supports ``pyproject.toml`` files for configuration.
The configuration options is similar to the one available in other formats, but must be defined
in a ``[tool.pytest.ini_options]`` table to be picked up by pytest:
.. code-block:: toml
pyproject.toml
[tool.pytest.ini_options]
minversion = "6.0"
addopts = "-ra -q"
testpaths = [
"tests",
"integration",
]
More information can be found :ref:`in the docs <config file formats>`.
- :issue:`3342`: pytest now includes inline type annotations and exposes them to user programs.
Most of the user-facing API is covered, as well as internal code.
If you are running a type checker such as mypy on your tests, you may start
noticing type errors indicating incorrect usage. If you run into an error that
you believe to be incorrect, please let us know in an issue.
The types were developed against mypy version 0.780. Versions before 0.750
are known not to work. We recommend using the latest version. Other type
checkers may work as well, but they are not officially verified to work by
pytest yet.
- :issue:`4049`: Introduced a new hook named `pytest_warning_recorded` to convey information about warnings captured by the internal `pytest` warnings plugin.
This hook is meant to replace `pytest_warning_captured`, which is deprecated and will be removed in a future release.
- :issue:`6471`: New command-line flags:
* `--no-header`: disables the initial header, including platform, version, and plugins.
* `--no-summary`: disables the final test summary, including warnings.
- :issue:`6856`: A warning is now shown when an unknown key is read from a config INI file.
The `--strict-config` flag has been added to treat these warnings as errors.
- :issue:`6906`: Added `--code-highlight` command line option to enable/disable code highlighting in terminal output.
- :issue:`7245`: New ``--import-mode=importlib`` option that uses :mod:`importlib` to import test modules.
Traditionally pytest used ``__import__`` while changing ``sys.path`` to import test modules (which
also changes ``sys.modules`` as a side-effect), which works but has a number of drawbacks, like requiring test modules
that don't live in packages to have unique names (as they need to reside under a unique name in ``sys.modules``).
``--import-mode=importlib`` uses more fine grained import mechanisms from ``importlib`` which don't
require pytest to change ``sys.path`` or ``sys.modules`` at all, eliminating much of the drawbacks
of the previous mode.
We intend to make ``--import-mode=importlib`` the default in future versions, so users are encouraged
to try the new mode and provide feedback (both positive or negative) in issue :issue:`7245`.
You can read more about this option in :std:ref:`the documentation <import-modes>`.
- :issue:`7305`: New ``required_plugins`` configuration option allows the user to specify a list of plugins, including version information, that are required for pytest to run. An error is raised if any required plugins are not found when running pytest.
Improvements
------------
- :issue:`4375`: The ``pytest`` command now suppresses the ``BrokenPipeError`` error message that
is printed to stderr when the output of ``pytest`` is piped and and the pipe is
closed by the piped-to program (common examples are ``less`` and ``head``).
- :issue:`4391`: Improved precision of test durations measurement. ``CallInfo`` items now have a new ``<CallInfo>.duration`` attribute, created using ``time.perf_counter()``. This attribute is used to fill the ``<TestReport>.duration`` attribute, which is more accurate than the previous ``<CallInfo>.stop - <CallInfo>.start`` (as these are based on ``time.time()``).
- :issue:`4675`: Rich comparison for dataclasses and `attrs`-classes is now recursive.
- :issue:`6285`: Exposed the `pytest.FixtureLookupError` exception which is raised by `request.getfixturevalue()`
(where `request` is a `FixtureRequest` fixture) when a fixture with the given name cannot be returned.
- :issue:`6433`: If an error is encountered while formatting the message in a logging call, for
example ``logging.warning("oh no!: %s: %s", "first")`` (a second argument is
missing), pytest now propagates the error, likely causing the test to fail.
Previously, such a mistake would cause an error to be printed to stderr, which
is not displayed by default for passing tests. This change makes the mistake
visible during testing.
You may suppress this behavior temporarily or permanently by setting
``logging.raiseExceptions = False``.
- :issue:`6817`: Explicit new-lines in help texts of command-line options are preserved, allowing plugins better control
of the help displayed to users.
- :issue:`6940`: When using the ``--duration`` option, the terminal message output is now more precise about the number and duration of hidden items.
- :issue:`6991`: Collected files are displayed after any reports from hooks, e.g. the status from ``--lf``.
- :issue:`7091`: When ``fd`` capturing is used, through ``--capture=fd`` or the ``capfd`` and
``capfdbinary`` fixtures, and the file descriptor (0, 1, 2) cannot be
duplicated, FD capturing is still performed. Previously, direct writes to the
file descriptors would fail or be lost in this case.
- :issue:`7119`: Exit with an error if the ``--basetemp`` argument is empty, is the current working directory or is one of the parent directories.
This is done to protect against accidental data loss, as any directory passed to this argument is cleared.
- :issue:`7128`: `pytest --version` now displays just the pytest version, while `pytest --version --version` displays more verbose information including plugins. This is more consistent with how other tools show `--version`.
- :issue:`7133`: :meth:`caplog.set_level() <pytest.LogCaptureFixture.set_level>` will now override any :confval:`log_level` set via the CLI or configuration file.
- :issue:`7159`: :meth:`caplog.set_level() <pytest.LogCaptureFixture.set_level>` and :meth:`caplog.at_level() <pytest.LogCaptureFixture.at_level>` no longer affect
the level of logs that are shown in the *Captured log report* report section.
- :issue:`7348`: Improve recursive diff report for comparison asserts on dataclasses / attrs.
- :issue:`7385`: ``--junitxml`` now includes the exception cause in the ``message`` XML attribute for failures during setup and teardown.
Previously:
.. code-block:: xml
<error message="test setup failure">
Now:
.. code-block:: xml
<error message="failed on setup with "ValueError: Some error during setup"">
Bug Fixes
---------
- :issue:`1120`: Fix issue where directories from :fixture:`tmpdir` are not removed properly when multiple instances of pytest are running in parallel.
- :issue:`4583`: Prevent crashing and provide a user-friendly error when a marker expression (`-m`) invoking of :func:`eval` raises any exception.
- :issue:`4677`: The path shown in the summary report for SKIPPED tests is now always relative. Previously it was sometimes absolute.
- :issue:`5456`: Fix a possible race condition when trying to remove lock files used to control access to folders
created by :fixture:`tmp_path` and :fixture:`tmpdir`.
- :issue:`6240`: Fixes an issue where logging during collection step caused duplication of log
messages to stderr.
- :issue:`6428`: Paths appearing in error messages are now correct in case the current working directory has
changed since the start of the session.
- :issue:`6755`: Support deleting paths longer than 260 characters on windows created inside :fixture:`tmpdir`.
- :issue:`6871`: Fix crash with captured output when using :fixture:`capsysbinary`.
- :issue:`6909`: Revert the change introduced by :pull:`6330`, which required all arguments to ``pytest.mark.parametrize`` to be explicitly defined in the function signature.
The intention of the original change was to remove what was expected to be an unintended/surprising behavior, but it turns out many people relied on it, so the restriction has been reverted.
- :issue:`6910`: Fix crash when plugins return an unknown stats while using the ``--reportlog`` option.
- :issue:`6924`: Ensure a ``unittest.IsolatedAsyncioTestCase`` is actually awaited.
- :issue:`6925`: Fix `TerminalRepr` instances to be hashable again.
- :issue:`6947`: Fix regression where functions registered with :meth:`unittest.TestCase.addCleanup` were not being called on test failures.
- :issue:`6951`: Allow users to still set the deprecated ``TerminalReporter.writer`` attribute.
- :issue:`6956`: Prevent pytest from printing `ConftestImportFailure` traceback to stdout.
- :issue:`6991`: Fix regressions with `--lf` filtering too much since pytest 5.4.
- :issue:`6992`: Revert "tmpdir: clean up indirection via config for factories" :issue:`6767` as it breaks pytest-xdist.
- :issue:`7061`: When a yielding fixture fails to yield a value, report a test setup error instead of crashing.
- :issue:`7076`: The path of file skipped by ``pytest.mark.skip`` in the SKIPPED report is now relative to invocation directory. Previously it was relative to root directory.
- :issue:`7110`: Fixed regression: ``asyncbase.TestCase`` tests are executed correctly again.
- :issue:`7126`: ``--setup-show`` now doesn't raise an error when a bytes value is used as a ``parametrize``
parameter when Python is called with the ``-bb`` flag.
- :issue:`7143`: Fix :meth:`pytest.File.from_parent <_pytest.nodes.Node.from_parent>` so it forwards extra keyword arguments to the constructor.
- :issue:`7145`: Classes with broken ``__getattribute__`` methods are displayed correctly during failures.
- :issue:`7150`: Prevent hiding the underlying exception when ``ConfTestImportFailure`` is raised.
- :issue:`7180`: Fix ``_is_setup_py`` for files encoded differently than locale.
- :issue:`7215`: Fix regression where running with ``--pdb`` would call :meth:`unittest.TestCase.tearDown` for skipped tests.
- :issue:`7253`: When using ``pytest.fixture`` on a function directly, as in ``pytest.fixture(func)``,
if the ``autouse`` or ``params`` arguments are also passed, the function is no longer
ignored, but is marked as a fixture.
- :issue:`7360`: Fix possibly incorrect evaluation of string expressions passed to ``pytest.mark.skipif`` and ``pytest.mark.xfail``,
in rare circumstances where the exact same string is used but refers to different global values.
- :issue:`7383`: Fixed exception causes all over the codebase, i.e. use `raise new_exception from old_exception` when wrapping an exception.
Improved Documentation
----------------------
- :issue:`7202`: The development guide now links to the contributing section of the docs and `RELEASING.rst` on GitHub.
- :issue:`7233`: Add a note about ``--strict`` and ``--strict-markers`` and the preference for the latter one.
- :issue:`7345`: Explain indirect parametrization and markers for fixtures.
Trivial/Internal Changes
------------------------
- :issue:`7035`: The ``originalname`` attribute of ``_pytest.python.Function`` now defaults to ``name`` if not
provided explicitly, and is always set.
- :issue:`7264`: The dependency on the ``wcwidth`` package has been removed.
- :issue:`7291`: Replaced ``py.iniconfig`` with :pypi:`iniconfig`.
- :issue:`7295`: ``src/_pytest/config/__init__.py`` now uses the ``warnings`` module to report warnings instead of ``sys.stderr.write``.
- :issue:`7356`: Remove last internal uses of deprecated *slave* term from old ``pytest-xdist``.
- :issue:`7357`: ``py``>=1.8.2 is now required.