6.0.2

pytest 6.0.2 (2020-09-04)

Bug Fixes

• #7148: Fixed --log-cli potentially causing unrelated print output to be swallowed.
• #7672: Fixed log-capturing level restored incorrectly if caplog.set_level is called more than once.
• #7686: Fixed NotSetType.token being used as the parameter ID when the parametrization list is empty.
  Regressed in pytest 6.0.0.
• #7707: Fix internal error when handling some exceptions that contain multiple lines or the style uses multiple lines (--tb=line for example).

6.0.1

pytest 6.0.1 (2020-07-30)

Bug Fixes

• #7394: Passing an empty help value to Parser.add_option is now accepted instead of crashing when running pytest --help.
  Passing None raises a more informative TypeError.
• #7558: Fix pylint not-callable lint on pytest.mark.parametrize() and the other builtin marks:
  skip, skipif, xfail, usefixtures, filterwarnings.
• #7559: Fix regression in plugins using TestReport.longreprtext (such as pytest-html) when TestReport.longrepr is not a string.
• #7569: Fix logging capture handler's level not reset on teardown after a call to caplog.set_level().

6.0.0

pytest 6.0.0 (2020-07-28)

(Please see the full set of changes for this release also in the 6.0.0rc1 notes below)

Breaking Changes

• #5584: PytestDeprecationWarning are now errors by default.

  Following our plan to remove deprecated features with as little disruption as
  possible, all warnings of type PytestDeprecationWarning now generate errors
  instead of warning messages.

  The affected features will be effectively removed in pytest 6.1, so please consult the
  Deprecations and Removals
  section in the docs for directions on how to update existing code.

  In the pytest 6.0.X series, it is possible to change the errors back into warnings as a
  stopgap measure by adding this to your pytest.ini file:

  [pytest]
  filterwarnings =
      ignore::pytest.PytestDeprecationWarning
  

  But this will stop working when pytest 6.1 is released.

  If you have concerns about the removal of a specific feature, please add a
  comment to #5584.

• #7472: The exec_() and is_true() methods of _pytest._code.Frame have been removed.

Features

• #7464: Added support for NO_COLOR and FORCE_COLOR environment variables to control colored output.

Improvements

• #7467: --log-file CLI option and log_file ini marker now create subdirectories if needed.
• #7489: The pytest.raises function has a clearer error message when match equals the obtained string but is not a regex match. In this case it is suggested to escape the regex.

Bug Fixes

• #7392: Fix the reported location of tests skipped with @pytest.mark.skip when --runxfail is used.
• #7491: tmpdir and tmp_path no longer raise an error if the lock to check for
  stale temporary directories is not accessible.
• #7517: Preserve line endings when captured via capfd.
• #7534: Restored the previous formatting of TracebackEntry.__str__ which was changed by accident.

Improved Documentation

• #7422: Clarified when the usefixtures mark can apply fixtures to test.
• #7441: Add a note about -q option used in getting started guide.

Trivial/Internal Changes

• #7389: Fixture scope package is no longer considered experimental.

pytest 6.0.0rc1 (2020-07-08)

Breaking Changes

• #1316: TestReport.longrepr is now always an instance of ReprExceptionInfo. Previously it was a str when a test failed with pytest.fail(..., pytrace=False).

• #5965: symlinks are no longer resolved during collection and matching conftest.py files with test file paths.

  Resolving symlinks for the current directory and during collection was introduced as a bugfix in 3.9.0, but it actually is a new feature which had unfortunate consequences in Windows and surprising results in other platforms.

  The team decided to step back on resolving symlinks at all, planning to review this in the future with a more solid solution (see discussion in
  #6523 for details).

  This might break test suites which made use of this feature; the fix is to create a symlink
  for the entire test tree, and not only to partial files/tress as it was possible previously.

• #6505: Testdir.run().parseoutcomes() now always returns the parsed nouns in plural form.

  Originally parseoutcomes() would always returns the nouns in plural form, but a change
  meant to improve the terminal summary by using singular form single items (1 warning or 1 error)
  caused an unintended regression by changing the keys returned by parseoutcomes().

  Now the API guarantees to always return the plural form, so calls like this:

  result = testdir.runpytest()
  result.assert_outcomes(error=1)
  

  Need to be changed to:

  result = testdir.runpytest()
  result.assert_outcomes(errors=1)
  

• #6903: The os.dup() function is now assumed to exist. We are not aware of any
  supported Python 3 implementations which do not provide it.

• #7040: -k no longer matches against the names of the directories outside the test session root.

  Also, pytest.Package.name is now just the name of the directory containing the package's
  __init__.py file, instead of the full path. This is consistent with how the other nodes
  are named, and also one of the reasons why -k would match against any directory containing
  the test suite.

• #7122: Expressions given to the -m and -k options are no longer evaluated using Python's eval.
  The format supports or, and, not, parenthesis and general identifiers to match against.
  Python constants, keywords or other operators are no longer evaluated differently.

• #7135: Pytest now uses its own TerminalWriter class instead of using the one from the py library.
  Plugins generally access this class through TerminalReporter.writer, TerminalReporter.write()
  (and similar methods), or _pytest.config.create_terminal_writer().

  The following breaking changes were made:

  • Output (write() method and others) no longer flush implicitly; the flushing behavior
    of the underlying file is respected. To flush explicitly (for example, if you
    want output to be shown before an end-of-line is printed), use write(flush=True) or
    terminal_writer.flush().
  • Explicit Windows console support was removed, delegated to the colorama library.
  • Support for writing bytes was removed.
  • The reline method and chars_on_current_line property were removed.
  • The stringio and encoding arguments was removed.
  • Support for passing a callable instead of a file was removed.

• #7224: The item.catch_log_handler and item.catch_log_handlers attributes, set by the
  logging plugin and never meant to be public , are no longer available.

  The deprecated --no-print-logs option is removed. Use --show-capture instead.

• #7226: Removed the unused args parameter from pytest.Function.__init__.

• #7418: Removed the pytest_doctest_prepare_content hook specification. This hook
  hasn't been triggered by pytest for at least 10 years.

• #7438: Some changes were made to the internal _pytest._code.source, listed here
  for the benefit of plugin authors who may be using it:

  • The deindent argument to Source() has been removed, now it is always true.
  • Support for zero or multiple arguments to Source() has been removed.
  • Support for comparing Source with an str has been removed.
  • The methods Source.isparseable() and Source.putaround() have been removed.
  • The method Source.compile() and function _pytest._code.compile() have
    been removed; use plain compile() instead.
  • The function _pytest._code.source.getsource() has been removed; use
    Source() directly instead.

Deprecations

• #7210: The special -k '-expr' syntax to -k is deprecated. Use -k 'not expr'
  instead.

  The special -k 'expr:' syntax to -k is deprecated. Please open an issue
  if you use this and want a replacement.

• #4049: pytest_warning_captured is deprecated in favor of the pytest_warning_recorded hook.

Features

• #1556: pytest now supports pyproject.toml files for configuration.

  The configuration options is similar to the one available in other formats, but must be defined
  in a [tool.pytest.ini_options] table to be picked up by pytest:

  # pyproject.toml
  [tool.pytest.ini_options]
  minversion = "6.0"
  addopts = "-ra -q"
  testpaths = [
      "tests",
      "integration",
  ]
  

  More information can be found in the docs.

• #3342: pytest now includes inline type annotations and exposes them to user programs.
  Most of the user-facing API is covered, as well as internal code.

  If you are running a type checker such as mypy on your tests, you may start
  noticing type errors indicating incorrect usage. If you run into an error that
  you believe to be incorrect, please let us know in an issu...

6.0.0rc1

pytest 6.0.0rc1 (2020-07-08)

Breaking Changes

• #1316: TestReport.longrepr is now always an instance of ReprExceptionInfo. Previously it was a str when a test failed with pytest.fail(..., pytrace=False).

• #5965: symlinks are no longer resolved during collection and matching conftest.py files with test file paths.

  Resolving symlinks for the current directory and during collection was introduced as a bugfix in 3.9.0, but it actually is a new feature which had unfortunate consequences in Windows and surprising results in other platforms.

  The team decided to step back on resolving symlinks at all, planning to review this in the future with a more solid solution (see discussion in
  #6523 for details).

  This might break test suites which made use of this feature; the fix is to create a symlink
  for the entire test tree, and not only to partial files/tress as it was possible previously.

• #6505: Testdir.run().parseoutcomes() now always returns the parsed nouns in plural form.

  Originally parseoutcomes() would always returns the nouns in plural form, but a change
  meant to improve the terminal summary by using singular form single items (1 warning or 1 error)
  caused an unintended regression by changing the keys returned by parseoutcomes().

  Now the API guarantees to always return the plural form, so calls like this:

  result = testdir.runpytest()
  result.assert_outcomes(error=1)
  

  Need to be changed to:

  result = testdir.runpytest()
  result.assert_outcomes(errors=1)
  

• #6903: The os.dup() function is now assumed to exist. We are not aware of any
  supported Python 3 implementations which do not provide it.

• #7040: -k no longer matches against the names of the directories outside the test session root.

  Also, pytest.Package.name is now just the name of the directory containing the package's
  __init__.py file, instead of the full path. This is consistent with how the other nodes
  are named, and also one of the reasons why -k would match against any directory containing
  the test suite.

• #7122: Expressions given to the -m and -k options are no longer evaluated using Python's eval.
  The format supports or, and, not, parenthesis and general identifiers to match against.
  Python constants, keywords or other operators are no longer evaluated differently.

• #7135: Pytest now uses its own TerminalWriter class instead of using the one from the py library.
  Plugins generally access this class through TerminalReporter.writer, TerminalReporter.write()
  (and similar methods), or _pytest.config.create_terminal_writer().

  The following breaking changes were made:

  • Output (write() method and others) no longer flush implicitly; the flushing behavior
    of the underlying file is respected. To flush explicitly (for example, if you
    want output to be shown before an end-of-line is printed), use write(flush=True) or
    terminal_writer.flush().
  • Explicit Windows console support was removed, delegated to the colorama library.
  • Support for writing bytes was removed.
  • The reline method and chars_on_current_line property were removed.
  • The stringio and encoding arguments was removed.
  • Support for passing a callable instead of a file was removed.

• #7224: The item.catch_log_handler and item.catch_log_handlers attributes, set by the
  logging plugin and never meant to be public , are no longer available.

  The deprecated --no-print-logs option is removed. Use --show-capture instead.

• #7226: Removed the unused args parameter from pytest.Function.__init__.

• #7418: Removed the pytest_doctest_prepare_content hook specification. This hook
  hasn't been triggered by pytest for at least 10 years.

• #7438: Some changes were made to the internal _pytest._code.source, listed here
  for the benefit of plugin authors who may be using it:

  • The deindent argument to Source() has been removed, now it is always true.
  • Support for zero or multiple arguments to Source() has been removed.
  • Support for comparing Source with an str has been removed.
  • The methods Source.isparseable() and Source.putaround() have been removed.
  • The method Source.compile() and function _pytest._code.compile() have
    been removed; use plain compile() instead.
  • The function _pytest._code.source.getsource() has been removed; use
    Source() directly instead.

Deprecations

• #7210: The special -k '-expr' syntax to -k is deprecated. Use -k 'not expr'
  instead.

  The special -k 'expr:' syntax to -k is deprecated. Please open an issue
  if you use this and want a replacement.

• #4049: pytest_warning_captured is deprecated in favor of the pytest_warning_recorded hook.

Features

• #1556: pytest now supports pyproject.toml files for configuration.

  The configuration options is similar to the one available in other formats, but must be defined
  in a [tool.pytest.ini_options] table to be picked up by pytest:

  # pyproject.toml
  [tool.pytest.ini_options]
  minversion = "6.0"
  addopts = "-ra -q"
  testpaths = [
      "tests",
      "integration",
  ]
  

  More information can be found in the docs.

• #3342: pytest now includes inline type annotations and exposes them to user programs.
  Most of the user-facing API is covered, as well as internal code.

  If you are running a type checker such as mypy on your tests, you may start
  noticing type errors indicating incorrect usage. If you run into an error that
  you believe to be incorrect, please let us know in an issue.

  The types were developed against mypy version 0.780. Versions before 0.750
  are known not to work. We recommend using the latest version. Other type
  checkers may work as well, but they are not officially verified to work by
  pytest yet.

• #4049: Introduced a new hook named pytest_warning_recorded to convey information about warnings captured by the internal pytest warnings plugin.

  This hook is meant to replace pytest_warning_captured, which is deprecated and will be removed in a future release.

• #6471: New command-line flags:

  • `--no-header`: disables the initial header, including platform, version, and plugins.
  • `--no-summary`: disables the final test summary, including warnings.

• #6856: A warning is now shown when an unknown key is read from a config INI file.

  The --strict-config flag has been added to treat these warnings as errors.

• #6906: Added --code-highlight command line option to enable/disable code highlighting in terminal output.

• #7245: New --import-mode=importlib option that uses importlib to import test modules.

  Traditionally pytest used __import__ while changing sys.path to import test modules (which
  also changes sys.modules as a side-effect), which works but has a number of drawbacks, like requiring test modules
  that don't live in packages to have unique names (as they need to reside under a unique name in sys.modules).

  --import-mode=importlib uses more fine grained import mechanisms from importlib which don't
  require pytest to change sys.path or sys.modules at all, eliminating much of the drawbacks
  of the previous mode.

  We intend to make --import-mode=importlib the default in future versions, so users are encouraged
  to try the new mode and provide feedback (both positive or negative) in issue #7245.

  You can read more about this option in the documentation.

• #7305: New required_plugins configuration option allows the user to specify a list of plugins, including version information, that are required for pytest to run. An error is raised if any required plugins are not found when running pytest.

Improvements

• #4375: The pytest command now suppresses the BrokenPipeError error message that
  is printed to stderr when the output of pytest is piped and and the pipe is
  closed by the piped...

pytest 4.6.11 (2020-06-04)

Bug Fixes

• #6334: Fix summary entries appearing twice when f/F and s/S report chars were used at the same time in the -r command-line option (for example -rFf).

  The upper case variants were never documented and the preferred form should be the lower case.

• #7310: Fix UnboundLocalError: local variable 'letter' referenced before assignment in _pytest.terminal.pytest_report_teststatus()
  when plugins return report objects in an unconventional state.

  This was making pytest_report_teststatus() skip
  entering if-block branches that declare the letter variable.

  The fix was to set the initial value of the letter before
  the if-block cascade so that it always has a value.

5.4.3

pytest 5.4.3 (2020-06-02)

Bug Fixes

• #6428: Paths appearing in error messages are now correct in case the current working directory has
  changed since the start of the session.
• #6755: Support deleting paths longer than 260 characters on windows created inside tmpdir.
• #6956: Prevent pytest from printing ConftestImportFailure traceback to stdout.
• #7150: Prevent hiding the underlying exception when ConfTestImportFailure is raised.
• #7215: Fix regression where running with --pdb would call the tearDown methods of unittest.TestCase
  subclasses for skipped tests.

5.4.2

pytest 5.4.2 (2020-05-08)

Bug Fixes

• #6871: Fix crash with captured output when using the capsysbinary fixture <capsysbinary>.
• #6924: Ensure a unittest.IsolatedAsyncioTestCase is actually awaited.
• #6925: Fix TerminalRepr instances to be hashable again.
• #6947: Fix regression where functions registered with TestCase.addCleanup were not being called on test failures.
• #6951: Allow users to still set the deprecated TerminalReporter.writer attribute.
• #6992: Revert "tmpdir: clean up indirection via config for factories" #6767 as it breaks pytest-xdist.
• #7110: Fixed regression: asyncbase.TestCase tests are executed correctly again.
• #7143: Fix File.from_constructor so it forwards extra keyword arguments to the constructor.
• #7145: Classes with broken __getattribute__ methods are displayed correctly during failures.
• #7180: Fix _is_setup_py for files encoded differently than locale.

pytest 4.6.10 (2020-05-08)

• #6870: New Config.invocation_args attribute containing the unchanged arguments passed to pytest.main().

  Remark: while this is technically a new feature and according to our policy it should not have been backported, we have opened an exception in this particular case because it fixes a serious interaction with pytest-xdist, so it can also be considered a bugfix.

5.4.1

pytest 5.4.1 (2020-03-13)

Bug Fixes

• #6909: Revert the change introduced by #6330, which required all arguments to @pytest.mark.parametrize to be explicitly defined in the function signature.

  The intention of the original change was to remove what was expected to be an unintended/surprising behavior, but it turns out many people relied on it, so the restriction has been reverted.

• #6910: Fix crash when plugins return an unknown stats while using the --reportlog option.

pytest 5.4.0 (2020-03-12)

Breaking Changes

• #6316: Matching of -k EXPRESSION to test names is now case-insensitive.

• #6443: Plugins specified with -p are now loaded after internal plugins, which results in their hooks being called before the internal ones.

  This makes the -p behavior consistent with PYTEST_PLUGINS.

• #6637: Removed the long-deprecated pytest_itemstart hook.

  This hook has been marked as deprecated and not been even called by pytest for over 10 years now.

• #6673: Reversed / fix meaning of "+/-" in error diffs. "-" means that sth. expected is missing in the result and "+" means that there are unexpected extras in the result.

• #6737: The cached_result attribute of FixtureDef is now set to None when
  the result is unavailable, instead of being deleted.

  If your plugin performs checks like hasattr(fixturedef, 'cached_result'),
  for example in a pytest_fixture_post_finalizer hook implementation, replace
  it with fixturedef.cached_result is not None. If you del the attribute,
  set it to None instead.

Deprecations

• #3238: Option --no-print-logs is deprecated and meant to be removed in a future release. If you use --no-print-logs, please try out --show-capture and
  provide feedback.

  --show-capture command-line option was added in pytest 3.5.0 and allows to specify how to
  display captured output when tests fail: no, stdout, stderr, log or all (the default).

• #571: Deprecate the unused/broken [pytest_collect_directory]{.title-ref} hook.
  It was misaligned since the removal of the Directory collector in 2010
  and incorrect/unusable as soon as collection was split from test execution.

• #5975: Deprecate using direct constructors for Nodes.

  Instead they are new constructed via Node.from_parent.

  This transitional mechanism enables us to detangle the very intensely
  entangled Node relationships by enforcing more controlled creation/configruation patterns.

  As part of that session/config are already disallowed parameters and as we work on the details we might need disallow a few more as well.

  Subclasses are expected to use [super().from_parent]{.title-ref} if they intend to expand the creation of [Nodes]{.title-ref}.

• #6779: The TerminalReporter.writer attribute has been deprecated and should no longer be used. This
  was inadvertently exposed as part of the public API of that plugin and ties it too much
  with py.io.TerminalWriter.

Features

• #4597: New --capture=tee-sys <capture-method>{.interpreted-text role="ref"} option to allow both live printing and capturing of test output.
• #5712: Now all arguments to @pytest.mark.parametrize need to be explicitly declared in the function signature or via indirect.
  Previously it was possible to omit an argument if a fixture with the same name existed, which was just an accident of implementation and was not meant to be a part of the API.
• #6454: Changed default for [-r]{.title-ref} to [fE]{.title-ref}, which displays failures and errors in the short test summary <pytest.detailed_failed_tests_usage>{.interpreted-text role="ref"}. [-rN]{.title-ref} can be used to disable it (the old behavior).
• #6469: New options have been added to the junit_logging{.interpreted-text role="confval"} option: log, out-err, and all.
• #6834: Excess warning summaries are now collapsed per file to ensure readable display of warning summaries.

Improvements

• #1857: pytest.mark.parametrize accepts integers for ids again, converting it to strings.

• #449: Use "yellow" main color with any XPASSED tests.

• #4639: Revert "A warning is now issued when assertions are made for None".

  The warning proved to be less useful than initially expected and had quite a
  few false positive cases.

• #5686: tmpdir_factory.mktemp now fails when given absolute and non-normalized paths.

• #5984: The pytest_warning_captured hook now receives a location parameter with the code location that generated the warning.

• #6213: pytester: the testdir fixture respects environment settings from the monkeypatch fixture for inner runs.

• #6247: --fulltrace is honored with collection errors.

• #6384: Make [--showlocals]{.title-ref} work also with [--tb=short]{.title-ref}.

• #6653: Add support for matching lines consecutively with LineMatcher <_pytest.pytester.LineMatcher>{.interpreted-text role="attr"}'s ~_pytest.pytester.LineMatcher.fnmatch_lines{.interpreted-text role="func"} and ~_pytest.pytester.LineMatcher.re_match_lines{.interpreted-text role="func"}.

• #6658: Code is now highlighted in tracebacks when pygments is installed.

  Users are encouraged to install pygments into their environment and provide feedback, because
  the plan is to make pygments a regular dependency in the future.

• #6795: Import usage error message with invalid [-o]{.title-ref} option.

• #759: pytest.mark.parametrize supports iterators and generators for ids.

Bug Fixes

• #310: Add support for calling [pytest.xfail()]{.title-ref} and [pytest.importorskip()]{.title-ref} with doctests.

• #3823: --trace now works with unittests.

• #4445: Fixed some warning reports produced by pytest to point to the correct location of the warning in the user's code.

• #5301: Fix --last-failed to collect new tests from files with known failures.

• #5928: Report PytestUnknownMarkWarning at the level of the user's code, not pytest's.

• #5991: Fix interaction with --pdb and unittests: do not use unittest's TestCase.debug().

• #6334: Fix summary entries appearing twice when f/F and s/S report chars were used at the same time in the -r command-line option (for example -rFf).

  The upper case variants were never documented and the preferred form should be the lower case.

• #6409: Fallback to green (instead of yellow) for non-last items without previous passes with colored terminal progress indicator.

• #6454: [--disable-warnings]{.title-ref} is honored with [-ra]{.title-ref} and [-rA]{.title-ref}.

• #6497: Fix bug in the comparison of request key with cached key in fixture.

  A construct if key == cached_key: can fail either because == is explicitly disallowed, or for, e.g., NumPy arrays, where the result of a == b cannot generally be converted to [bool]{.title-ref}.
  The implemented fix replaces [==]{.title-ref} with is.

• #6557: Make capture output streams .write() method return the same return value from original streams.

• #6566: Fix EncodedFile.writelines to call the underlying buffer's writelines method.

• #6575: Fix internal crash when faulthandler starts initialized
  (for example with PYTHONFAULTHANDLER=1 environment variable set) and faulthandler_timeout defined
  in the configuration file.

• #6597: Fix node ids which contain a parametrized empty-string variable.

• #6646: Assertion rewriting hooks are (re)stored for the current item, which fixes them being still used after e.g. pytester's testdir.runpytest <_pytest.pytester.Testdir.runpytest>{.interpreted-text role="func"} etc.

• #6660: pytest.exit() <_pytest.outcomes.exit>{.interpreted-text role="func"} is handled when emitted from the pytest_sessionfinish <_pytest.hookspec.pytest_sessionfinish>{.interpreted-text role="func"} hook. This includes quitting from a debugger.

• #6752: When :pypytest.raises{.int...