Changelogs » Avocado-framework

Avocado-framework

71.0

Users/Test Writers
==================

* Avocado can now run on systems with nothing but Python 3 (and
"quasi-standard-library" module ``setuptools``).  This means that it
won't require extra packages, and should be easier to deploy on
containers, embedded systems, etc.  Optional plugins may have
additional requirements.

* A new and still experimental test runner implementation, known as
"N(ext) Runner" has been introduced.  It brings a number of
different concepts, increasing the decoupling between a test (and
its runner) and the job.  For more information, please refer to
:ref:`the early documentation <nrunner>`.

* The new :func:`avocado.cancel_on` decorator has been added to the
Test APIs, allowing you to define the conditions for a test to be
considered canceled.  See one example :ref:`here
<turning_errors_into_cancels>`.

* The ``glib`` plugin got a configuration option its safe/unsafe
operation, that is, whether it will execute binaries in an attempt to
find the whole list of tests.  Look for the ``glib.conf`` shipped
with the plugin to enable the unsafe mode.

* Avocado can now use tags inside Python Unittests, and not only on
its own Instrumented tests.  It's expected that other forms or
providing tags for other types of tests will also be introduced in
the near future.

* The HTML report will now show, as a handy pop-up, the contents of the
test whiteboard.  If you set, say, performance metrics there, you'll
able to see straight from the report.

* The HTML report now has filtering support by test status, and can
show all records in the table.

* The :mod:`avocado.utils.runtime` module, a badly designed mechanism
for sharing Avocado runtime settings with the utility libraries, has
been removed.

* The test runner feature that would allow binaries to be run
transparently inside GDB was removed.  The reason for dropping such
a feature have to do with how it limits the test runner to run one
test at a time, and the use of the :mod:`avocado.utils.runtime`
mechanism, also removed.

* Initial examples for writing custom jobs, using the so called Job
API, have been added to ``examples/jobs``.  These APIs are still
non-public (under core), but they're supposed to become public
and supported soon.

* By means of a new plugin (``merge_files``, of type ``job.prepost``),
when using the :ref:`output check record <output_check_record>`
features, duplicate files created by different tests/variants will
be consolidated into unique files.

Bug Fixes
=========

* The HTML plugin now correctly shows the date for tests that were
never executed because of interrupted jobs.

* A temporarily workaround for a stack overflow problem in Python 3.7
has been addressed.

* The pict plugin (a varianter implementaion) now properly yields the
variants paths as a list.

* A Python 3 related fix to mod:`avocado.utils.software_manager`, that
was using Python 2 ``next`` on ``get_source``.

* A Python 3 related fix to the docker plugin, that wasn't caught
earlier.

Utility APIs
============

* :mod:`avocado.utils.partition` now allows ``mkfs`` and ``mount``
flags to be set.

* :func:`avocado.utils.cpu.get_cpu_vendor_name` now returns the CPU
vendor name for POWER9.

* :mod:`avocado.utils.asset` now allows a given location, as well as a
list, to be given, simplifying the most common use case.

* :meth:`avocado.utils.process.SubProcess.stop` now supports setting
a timeout.  Please refer to the documentation for the important
details on its behavior.

* :mod:`avocado.utils.memory` now properly handles hugepages for POWER
platform.

Internal Changes
================

* Removal of the ``stevedore`` library dependency (previously used for
the dispatcher/plugins infrastructure).

* ``make check`` now runs selftests using the experimental N(ext)
Runner.

* Formal support for Python 3.7, which is now on our CI checks,
documentation and module information.

* The Yaml to Mux plugin now uses a safe version of the Yaml loader,
so that the execution of arbitrary Python code from Yaml input is
now no longer possible.

* Codecov coverage reports for have been enabled for Avocado, and
can be seen on every pull request.

* New tests have been added to many of the optional plugins.

* Various pylint compliance improvements, including w0231, w0235,
w0706, w0715 and w0221.

* Avocado's selftests now use tempfile.TemporaryDirectory instead of
mkdtemp and shutil.rmtree.

* :class:`avocado.core.job.Job` instantiation now takes a ``config``
dictionary parameter, instead of a ``argparse.Namespace`` instance,
and keeps it in a ``config`` attribute.

* :class:`avocado.core.job.Job` instances don't have a ``references``
attribute anymore.  That information is available in the ``config``
attribute, that is, ``myjob.config['references']``.

* Basic checks for Fedora and RHEL 8 using Cirrus CI have been added,
and will be shown on every pull request.

For more information, please check out the complete
`Avocado changelog
<https://github.com/avocado-framework/avocado/compare/70.0...71.0>`_.

39.0

The major changes introduced on this version are listed below.
- Support for running tests in Docker container.  Now, in addition to
running tests on a (libvirt based) Virtual Machine or on a remote host,
you can now run tests in transient Docker containers.  The usage is as
simple as::

$ avocado run mytests.py --docker ldoktor/fedora-avocado

The container will be started, using `ldoktor/fedora-avocado` as
the image.  This image contains a Fedora based system with Avocado
already installed, and it's provided at the official Docker hub.
- Introduction of the "Fail Fast" feature.

By running a job with the `--failfast` flag, the job will be
interrupted after the very first test failure.  If your job only
makes sense if it's a complete PASS, this feature can save you a lot
of time.
- Avocado supports replaying previous jobs, selected by using their
Job IDs.  Now, it's also possible to use the special keyword
`latest`, which will cause Avocado to rerun the very last job.
- Python's standard signal handling is restored for SIGPIPE, and thus
for all tests running on Avocado.

In previous releases, Avocado introduced a change that set the
default handler to SIGPIPE, which caused the application to be
terminated.  This seemed to be the right approach when testing how
the Avocado app would behave on broken pipes on the command line,
but it introduced side effects to a lot of Python code.  Instead of
exceptions, the affected Python code would receive the signal themselves.

This is now reverted to the Python standard, and the signal behavior
of Python based tests running on Avocado should not surprise anyone.
- The project release notes are now part of the official
documentation.  That means that users can quickly find when a given
change was introduced.

Together with those changes listed, a total of 38 changes made into
this release.  For more information, please check out the complete
`Avocado changelog
<https://github.com/avocado-framework/avocado/compare/38.0...39.0>`_.

Sprint Theme

After so much love that we had on the previous version, let's twist
things a bit with an antagonist title.  Info on this pretty good movie
by Tarantino can be found at:

http://www.imdb.com/title/tt3460252/?ref_=nm_flmg_wr_2

https://www.youtube.com/watch?v=6_UI1GzaWv0

The story line:

In the dead of a Wyoming winter, a bounty hunter and his prisoner
find shelter in a cabin currently inhabited by a collection of
nefarious characters.

Release Meeting

The Avocado release meetings are now open to the community via
Hangouts on Air.  The meetings are recorded and made available on the
`Avocado Test Framework YouTube channel
<https://www.youtube.com/channel/UC-RVZ_HFTbEztDM7wNY4NfA>`_.

For this release, you can watch the meeting on `this link
<https://www.youtube.com/watch?v=GotEH7SmHSw>`_.

38.0

Another development cycle has just finished, and our community will
receive this new release containing a nice assortment of bug fixes and
new features.
- The download of assets in tests now allow for an expiration time.
This means that tests that need to download any kind of external
asset, say a tarball, can now automatically benefit from the
download cache, but can also keep receiving new versions
automatically.

Suppose your asset uses an asset named `myproject-daily.tar.bz2`,
and that your test runs 50 times a day.  By setting the expire time
to `1d` (1 day), your test will benefit from cache on most runs, but
will still fetch the new version when the the 24 hours from the
first download have passed.

For more information, please check out the documentation on the
`expire` parameter to the `fetch_asset()` method[1].
- Environment variables can be propagated into tests running on remote
systems.  It's a known fact that one way to influence application behavior,
including test, is to set environment variables.  A command line such as::


$ MYAPP_DEBUG=1 avocado run myapp_test.py


Will work as expected on a local system.  But Avocado also allows
running tests on remote machines, and up until now, it has been
lacking a way to propagate environment variables to the remote
system.

Now, you can use::


$ MYAPP_DEBUG=1 avocado run --env-keep MYAPP_DEBUG \
--remote-host test-machine myapp_test.py

- The plugin interfaces have been moved into the
`avocado.core.plugin_interfaces` module.  This means that plugin
writers now have to import the interface definitions this namespace,
example::


...
from avocado.core.plugin_interfaces import CLICmd

class MyCommand(CLICmd):
...


This is a way to keep ourselves honest, and say that there's no
difference from plugin interfaces to Avocado's core implementation,
that is, they may change at will.  For greater stability, one should
be tracking the LTS releases.

Also, it effectively makes all plugins the same, whether they're
implemented and shipped as part of Avocado, or as part of external
projects.
- A contrib script for running kvm-unit-tests.  As some people are
aware, Avocado has indeed a close relation to virtualization
testing.  Avocado-VT is one obvious example, but there are other
virtualization related test suites can Avocado can run.

This release adds a contrib script that will fetch, download,
compile and run kvm-unit-tests using Avocado's external runner
feature.  This gives results in a better granularity than the
support that exists in Avocado-VT, which gives only a single
PASS/FAIL for the entire test suite execution.

For more information, please check out the Avocado changelog[2].

Also, while we focused on Avocado, let's also not forget that
Avocado-VT maintains it's own fast pace of incoming niceties.
- s390 support: Avocado-VT is breaking into new grounds, and now has
support for the s390 architecture.  Fedora 23 for s390 has been added
as a valid guest OS, and s390-virtio has been added as a new machine
type.
- Avocado-VT is now more resilient against failures to persist its
environment file, and will only give warnings instead of errors when
it fails to save it.
- An improved implementation of the "job lock" plugin, which prevents
multiple Avocado jobs with VT tests to run simultaneously.  Since
there's no finer grained resource locking in Avocado-VT, this is a
global lock that will prevent issues such as image corruption when
two jobs are run at the same time.

This new implementation will now check if existing lock files are
stale, that is, they are leftovers from previous run.  If the
processes associated with these files are not present, the stale
lock files are deleted, removing the need to clean them up manually.
It also outputs better debugging information when failures to
acquire lock.

The complete list of changes to Avocado-VT are available on its
changelog[3].

While not officially part of this release, this development cycle saw
the introduction of new tests on our avocado-misc-tests. Go check it
out!

Finally, since Avocado and Avocado-VT are not newly born anymore, we
decided to update information mentioning KVM-Autotest, virt-test on so
on around the web.  This will hopefully redirect new users to the
Avocado community and avoid confusion.

Install avocado

Instructions are available in our documentation on how to install
either with packages or from source[4].

Updated RPM packages are be available in the project repos for EPEL 6,
EPEL 7, Fedora 22, Fedora 23 and the newly released Fedora 24.

Please note that on the next release, we'll drop support for Fedora 22
packages.

Happy hacking and testing!

---

[1] http://avocado-framework.readthedocs.io/en/38.0/WritingTests.html
[2] https://github.com/avocado-framework/avocado/compare/37.0...38.0
[3] https://github.com/avocado-framework/avocado-vt/compare/37.0...38.0
[4] http://avocado-framework.readthedocs.io/en/38.0/GetStartedGuide.htmlinstalling-avocado

37.0

details on the specific RFC[9].

The main changes in Avocado for this release are:
- A big round of fixes and on machine readable output formats, such
as xunit (aka JUnit) and JSON.  The xunit output, for instance,
now includes tests with schema checking.  This should make sure
interoperability is even better on this release.
- Much more robust handling of test references, aka test URLs.
Avocado now properly handles very long test references, and also
test references with non-ascii characters.
- The avocado command line application now provides richer exit
status[1].  If your application or custom script depends on the
avocado exit status code, you should be fine as avocado still
returns zero for success and non-zero for errors.  On error
conditions, though, the exit status code are richer and made of
combinable (ORable) codes.  This way it's possible to detect that,
say, both a test failure and a job timeout occurred in a single
execution.
- [SECURITY RELATED] The remote execution of tests (including in
Virtual Machines) now allows for proper checks of host keys[2].
Without these checks, avocado is susceptible to a man-in-the-middle
attack, by connecting and sending credentials to the wrong machine.
This check is _disabled_ by default, because users depend on this
behavior when using machines without any prior knowledge such as
cloud based virtual machines.  Also, a bug in the underlying SSH
library may prevent existing keys to be used if these are in ECDSA
format[3].  There's an automated check in place to check for the
resolution of the third party library bug.  Expect this feature to
be _enabled_ by default in the upcoming releases.
- Pre/Post Job hooks.  Avocado now defines a proper interface for
extension/plugin writers to execute actions while a Job is runnning.
Both Pre and Post hooks have access to the Job state (actually, the
complete Job instance).  Pre job hooks are called before tests are
run, and post job hooks are called at the very end of the job (after
tests would have usually finished executing).
- Pre/Post job scripts[4].  As a feature built on top of the Pre/Post job
hooks described earlier, it's now possible to put executable scripts
in a configurable location, such as `/etc/avocado/scripts/job/pre.d`
and have them called by Avocado before the execution of tests.  The
executed scripts will receive some information about the job via
environment variables[5].
- The implementation of proper Test-IDs[6] in the test result
directory.

Also, while not everything is (yet) translated into code, this release
saw various and major RFCs, which are definitely shaping the future of
Avocado.  Among those:
- Introduce proper test IDs[6]
- Pre/Post _test_ hooks[7]
- Multi-stream tests[8]
- Avocado maintainability and integration with avocado-vt[9]
- Improvements to job status (completely implemented)[10]

For a complete list of changes please check the Avocado changelog[11].

For Avocado-VT, please check the full Avocado-VT changelog[12].

Install avocado

Instructions are available in our documentation on how to install
either with packages or from source[13].

Updated RPM packages are be available in the project repos for
Fedora 22, Fedora 23, EPEL 6 and EPEL 7.

Packages

As a heads up, we still package the latest version of the various
Avocado sub projects, such as the very popular Avocado-VT and the
pretty much experimental Avocado-Virt and Avocado-Server projects.

For the upcoming releases, there will be changes in our package
offers, with a greater focus on long term stability packages for
Avocado.  Other packages may still be offered as a convenience, or
may see a change of ownership.  All in the best interest of our users.
If you have any concerns or questions, please let us know.

Happy hacking and testing!

---

[1] http://avocado-framework.readthedocs.org/en/35.0/ResultFormats.htmlexit-codes
[2] https://github.com/avocado-framework/avocado/blob/35.0/etc/avocado/avocado.confL41
[3] https://github.com/avocado-framework/avocado/blob/35.0/selftests/functional/test_thirdparty_bugs.pyL17
[4] http://avocado-framework.readthedocs.org/en/35.0/ReferenceGuide.htmljob-pre-and-post-scripts
[5] http://avocado-framework.readthedocs.org/en/35.0/ReferenceGuide.htmlscript-execution-environment
[6] https://www.redhat.com/archives/avocado-devel/2016-March/msg00024.html
[7] https://www.redhat.com/archives/avocado-devel/2016-April/msg00000.html
[8] https://www.redhat.com/archives/avocado-devel/2016-April/msg00042.html
[9] https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html
[10] https://www.redhat.com/archives/avocado-devel/2016-April/msg00010.html
[11] https://github.com/avocado-framework/avocado/compare/0.34.0...35.0
[13] https://github.com/avocado-framework/avocado-vt/compare/0.34.0...35.0
[12] http://avocado-framework.readthedocs.org/en/35.0/GetStartedGuide.htmlinstalling-avocado

35.0

This is another proud announcement: Avocado release 35.0, aka "Mr
Robot", is now out!

This release, while a "regular" release, will also serve as a beta for
our first "long term stability" (aka "lts") release.  That means that
the next release, will be version "36.0lts" and will receive only bug
fixes and minor improvements.  So, expect release 35.0 to be pretty
much like "36.0lts" feature-wise.  New features will make into the

0.34.0

Hello to all test enthusiasts out there, specially to those that
cherish, care or are just keeping an eye on the greenest test
framework there is: Avocado release 0.34.0, aka The Hour of the Star,
is now out!

The main changes in Avocado for this release are:
- A complete overhaul of the logging and output implementation. This
means that all Avocado output uses the standard Python logging library
making it very consistent and easy to understand [1].
- Based on the logging and output overhaul, the command line test
runner is now very flexible with its output. A user can choose
exactly what should be output. Examples include application output
only, test output only, both application and test output or any
other combination of the builtin streams. The user visible command
line option that controls this behavior is `--show`, which is an
application level option, that is, it's available to all avocado
commands. [2]
- Besides the builtin streams, test writers can use the standard
Python logging API to create new streams. These streams can be shown
on the command line as mentioned before, or persisted automatically
in the job results by means of the `--store-logging-stream` command
line option. [3][4]
- The new `avocado.core.safeloader` module, intends to make it easier
to to write new test loaders for various types of Python
code. [5][6]
- Based on the new `avocado.core.safeloader` module, a contrib script
called `avocado-find-unittests`, returns the name of
unittest.TestCase based tests found on a given number of Python
source code files. [7]
- Avocado is now able to run its own selftest suite. By leveraging the
`avocado-find-unittests` contrib script and the External Runner [8]
feature. A Makefile target is available, allowing developers to run
`make selfcheck` to have the selftest suite run by Avocado. [9]
- Partial Python 3 support. A number of changes were introduced that
allow concurrent Python 2 and 3 support on the same code base.  Even
though the support for Python 3 is still _incomplete_, the `avocado`
command line application can already run some limited commands at
this point.
- Asset fetcher utility library. This new utility library, and
INSTRUMENTED test feature, allows users to transparently request
external assets to be used in tests, having them cached for later
use. [10]
- Further cleanups in the public namespace of the avocado Test class.
- [BUG FIX] Input from the local system was being passed to remote
systems when running tests with either in remote systems or VMs.
- [BUG FIX] HTML report stability improvements, including better
Unicode handling and support for other versions of the Pystache
library.
- [BUG FIX] Atomic updates of the "latest" job symlink, allows for
more reliable user experiences when running multiple parallel jobs.
- [BUG FIX] The avocado.core.data_dir module now dynamically checks
the configuration system when deciding where the data directory
should be located. This allows for later updates, such as when
giving one extra `--config` parameter in the command line, to be
applied consistently throughout the framework and test code.
- [MAINTENANCE] The CI jobs now run full checks on each commit on
any proposed PR, not only on its topmost commit. This gives higher
confidence that a commit in a series is not causing breakage that
a later commit then inadvertently fixes.

For a complete list of changes please check the Avocado changelog[11].

For Avocado-VT, please check the full Avocado-VT changelog[12].

Avocado Videos

As yet another way to let users know about what's available in
Avocado, we're introducing short videos with very targeted content on
our very own YouTube channel:

https://www.youtube.com/channel/UCP4xob52XwRad0bU_8V28rQ

The first video available demonstrates a couple of new features
related to the advanced logging mechanisms, introduced on this
release:

https://www.youtube.com/watch?v=8Ur_p5p6YiQ

Install avocado

Instructions are available in our documentation on how to install
either with packages or from source[13].

Updated RPM packages are be available in the project repos for
Fedora 22, Fedora 23, EPEL 6 and EPEL 7.

Happy hacking and testing!

---

[1] http://avocado-framework.readthedocs.org/en/0.34.0/LoggingSystem.html
[2] http://avocado-framework.readthedocs.org/en/0.34.0/LoggingSystem.htmltweaking-the-ui
[3] http://avocado-framework.readthedocs.org/en/0.34.0/LoggingSystem.htmlstoring-custom-logs
[4] http://avocado-framework.readthedocs.org/en/0.34.0/WritingTests.htmladvanced-logging-capabilities
[5] https://github.com/avocado-framework/avocado/blob/0.34.0/avocado/core/safeloader.py
[6] http://avocado-framework.readthedocs.org/en/0.34.0/api/core/avocado.core.htmlmodule-avocado.core.safeloader
[7] https://github.com/avocado-framework/avocado/blob/0.34.0/contrib/avocado-find-unittests
[8] http://avocado-framework.readthedocs.org/en/0.34.0/GetStartedGuide.htmlrunning-tests-with-an-external-runner
[9] https://github.com/avocado-framework/avocado/blob/0.34.0/MakefileL33
[10] http://avocado-framework.readthedocs.org/en/0.34.0/WritingTests.htmlfetching-asset-files
[11] https://github.com/avocado-framework/avocado/compare/0.33.0...0.34.0
[12] https://github.com/avocado-framework/avocado-vt/compare/0.33.0...0.34.0
[13] http://avocado-framework.readthedocs.org/en/latest/GetStartedGuide.htmlinstalling-avocado

0.31.0

Hi everyone! Right on time for the holidays, Avocado reaches the end
of Sprint 31, and together with it, we're very happy to announce a brand
new release! This version brings stability fixes and improvements to
both Avocado and Avocado-VT, some new features and a major redesign of
our plugin architecture.

For Avocado the main changes are:
- It's now possible to register callback functions to be executed when
all tests finish, that is, at the end of a particular job[1].
- The software manager utility library received a lot of love on the
Debian side of things. If you're writing tests that install software
packages on Debian systems, you may be in for some nice treats and
much more reliable results.
- Passing malformed commands (such as ones that can not be properly
split by the standard shlex library) to the process utility library
is now better dealt with.
- The test runner code received some refactors and it's a lot easier
to follow. If you want to understand how the Avocado test runner
communicates with the processes that run the test themselves, you
may have a much better code reading experience now.
- Updated inspektor to the latest and greatest, so that our code is
kept is shiny and good looking (and performing) as possible.
- Fixes to the utility GIT library when using a specific local branch
name.
- Changes that allow our selftest suite to run properly on virtualenvs.
- Proper installation requirements definition for Python 2.6 systems.
- A completely new plugin architecture[2]. Now we offload all plugin
discovery and loading to the Stevedore library. Avocado now defines
precise (and simpler) interfaces for plugin writers. Please be aware
that the public and documented interfaces for plugins, at the
moment, allows adding new commands to the avocado command line app,
or adding new options to existing commands. Other functionality can
be achived by "abusing" the core avocado API from within
plugins. Our goal is to expand the interfaces so that other areas of
the framework can be extended just as easily.

For a complete list of changes please check the Avocado changelog[3].

Avocado-VT received just too many fixes and improvements to
list. Please refer to the changelog[4] for more information.

Install avocado

Instructions are available in our documentation on how to install
either with packages or from source[5].

Within a couple of hours, updated RPM packages will be available in
the project repos for Fedora 22, Fedora 23, EPEL 6 and EPEL 7.

Happy hacking and testing!

---

[1] http://avocado-framework.readthedocs.org/en/0.31.0/ReferenceGuide.htmljob-cleanup
[2] http://avocado-framework.readthedocs.org/en/0.31.0/Plugins.html
[3] https://github.com/avocado-framework/avocado/compare/0.30.0...0.31.0
[4] https://github.com/avocado-framework/avocado-vt/compare/0.30.0...0.31.0
[5] http://avocado-framework.readthedocs.org/en/0.31.0/GetStartedGuide.html

0.30.0

Hello! Avocado reaches the end of Sprint 30, and with it, we have a new release available! This version brings stability fixes and improvements to both Avocado and Avocado-vt.

As software doesn't spring out of life itself, we'd like to acknowledge the major contributions by Lucas (AKA lmr) since the dawn of time for Avocado (and earlier projects like Autotest and virt-test). Although the Avocado team at Red Hat was hit by some changes, we're already extremely happy to see that this major contributor (and good friend) has not gone too far.

Now back to the more informational part of the release notes. For Avocado the main changes are:
- New RPM repository location, check the docs[1] for instructions on how to install the latest releases
- Makefile rules for building RPMs are now based on mock, to ensure sound dependencies
- Packaged versions are now available for Fedora 22, newly released Fedora 23, EL6 and EL7
- The software manager utility library now supports DNF
- The avocado test runner now supports a dry run mode, which allows users to check how a job would be executed, including tests that would be found and parameters that would be passed to it. This is currently complementary to the avocado list command.
- The avocado test runner now supports running simple tests with parameters. This may come in handy for simple use cases when Avocado will wrap a test suite, but the test suite needs some command line arguments.

Avocado-vt also received many bugfixes[3]. Please refer to the changelog for more information.

Install avocado

Instructions are available in our documentation on how to install either with packages or from source[1].

Happy hacking and testing!

---

[1] http://avocado-framework.readthedocs.org/en/0.30.0/GetStartedGuide.html
[2] https://github.com/avocado-framework/avocado/compare/0.29.0...0.30.0
[3] https://github.com/avocado-framework/avocado-vt/compare/0.29.0...0.30.0

0.29.0

[2] http://avocado-framework.readthedocs.org/en/0.29.0/ReferenceGuide.htmltest-resolution
[3] http://avocado-framework.readthedocs.org/en/0.29.0/Loaders.html
[4] https://github.com/avocado-framework/avocado/compare/0.28.0...0.29.0

0.27.1

Hi guys, we're up to a new avocado release! It's basically a bugfix release, with a few usability tweaks.
- The avocado human output received some extra tweaks. Here's how it looks now:

$ avocado run passtest
JOB ID     : f186c729dd234c8fdf4a46f297ff0863684e2955
JOB LOG    : /home/lmr/avocado/job-results/job-2015-08-15T08.09-f186c72/job.log
TESTS      : 1
(1/1) passtest.py:PassTest.test: PASS (0.00 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
JOB HTML   : /home/lmr/avocado/job-results/job-2015-08-15T08.09-f186c72/html/results.html
TIME       : 0.00 s
- Bugfixes. You may refer to [1] for the full list of 58 commits.

Changes in avocado-vt:
- Bugfixes. In particular, a lot of issues related to --vt-type libvirt were fixed and now that backend is fully functional.

0.27.0

Avocado 0.27.0 released!

Hi guys, here I am, announcing yet another avocado release! The most exciting news for this release is that our avocado-vt plugin was merged with the virt-test project. The avocado-vt plugin will be very important for QEMU/KVM/Libvirt developers, so the main avocado received updates to better support the goal of having a good quality avocado-vt.

Changes in avocado:
- The avocado human output received some tweaks and it's more compact, while still being informative. Here's an example:


JOB ID     : f2f5060440bd57cba646c1f223ec8c40d03f539b
JOB LOG    : /home/user/avocado/job-results/job-2015-07-27T17.13-f2f5060/job.log
JOB HTML   : /home/user/avocado/job-results/job-2015-07-27T17.13-f2f5060/html/results.html
TESTS      : 1
(1/1) passtest.py:PassTest.test: PASS (0.00 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
TIME       : 0.00 s

- The avocado test loader was refactored and behaves more consistently in different test loading scenarios.
- The `utils` API received new modules and functions:
- NEW avocado.utils.cpu: APIs related to CPU information on linux boxes [1]
- NEW avocado.utils.git: APIs to clone/update git repos [2]
- NEW avocado.utils.iso9660: Get information about ISO files [3]
- NEW avocado.utils.service: APIs to control services on linux boxes (systemv and systemd) [4]
- NEW avocado.utils.output: APIs that help avocado based CLI programs to display results to users [5]
- UPDATE avocado.utils.download: Add url_download_interactive
- UPDATE avocado.utils.download: Add new params to get_file
- Bugfixes. You may refer to [6] for the full list of 64 commits.

Changes in avocado-vt:
- Merged virt-test into avocado-vt. Basically, the virt-test core library (virttest) replaced most uses of autotest by equivalent avocado API calls, and its code was brought up to the virt-test repository [7]. This means, among other things, that you can simply install avocado-vt through RPM and enjoy all the virt tests without having to clone another repository manually to bootstrap your tests. More details about the process will be sent on an e-mail to the avocado and virt-test mailing lists. Please go to [8] for instructions on how to get started with all our new tools.

See you in a couple of weeks for our next release! Happy testing!

The avocado development team

Links

[1] http://avocado-framework.readthedocs.org/en/latest/api/utils/avocado.utils.htmlmodule-avocado.utils.cpu
[2] http://avocado-framework.readthedocs.org/en/latest/api/utils/avocado.utils.htmlmodule-avocado.utils.git
[3] http://avocado-framework.readthedocs.org/en/latest/api/utils/avocado.utils.htmlmodule-avocado.utils.iso9660
[4] http://avocado-framework.readthedocs.org/en/latest/api/utils/avocado.utils.htmlmodule-avocado.utils.service
[5] http://avocado-framework.readthedocs.org/en/latest/api/utils/avocado.utils.htmlmodule-avocado.utils.output
[6] https://github.com/avocado-framework/avocado/compare/0.26.0...0.27.0
[7] https://github.com/avocado-framework/avocado-vt/commit/20dd39ef00db712f78419f07b10b8f8edbd19942
[8] http://avocado-vt.readthedocs.org/en/latest/GetStartedGuide.html

0.26.0

Avocado 0.26.0 released!

Hi guys, I'm here to announce avocado 0.26.0. This release was dedicated to
polish aspects of the avocado user experience, such as documentation and
behavior.

Changes
- Now avocado tests that raise exceptions that don't inherit from
`avocado.core.exceptions.TestBaseException` now will be marked as ERRORs.
This change was made to make avocado to have clearly defined test statuses.
A new decorator, `avocado.fail_on_error` was added to let arbitary exceptions
to raise errors, if users need a more relaxed behavior.
- The avocado.Test() utility method skip() now can only be called from inside
the setUp() method. This was made because by definition, if we get to the
test execution step, by definition it can't be skipped anymore. It's important
to keep the concepts clear and well separated if we want to give users a good
experience.
- More documentation polish and updates. Make sure you check out our documentation
website http://avocado-framework.readthedocs.org/en/latest/.
- A number of avocado command line options and help text was reviewed and
updated.
- A new, leaner and mobile friendly version of the avocado website is live.
Please check http://avocado-framework.github.io/ for more information.
- We have the first version of the avocado dashboard! avocado dashboard is the
initial version of an avocado web interface, and will serve as the frontend
to our testing database. You can check out a screenshot here
https://cloud.githubusercontent.com/assets/296807/8536678/dc5da720-242a-11e5-921c-6abd46e0f51e.png

![screenshot from 2015-07-06 22-02-36](https://cloud.githubusercontent.com/assets/296807/8536678/dc5da720-242a-11e5-921c-6abd46e0f51e.png)
- And the usual bugfixes. You can take a look at the full list of 68 commits here:
https://github.com/avocado-framework/avocado/compare/0.25.0...0.26.0

0.25.0

Avocado 0.25.0 released!

Hi guys, I'm here to announce the newest avocado release, 0.25.0. This is an
important milestone in avocado development, and we would like to invite
you to be a part of the development process, by contributing PRs, testing
and giving feedback on the test runner's usability and new plugins we came
up with.

What to expect

This is the first release aimed for general use. We did our best to deliver
a coherent and enjoyable experience, but keep in mind that it's a young
project, so please set your expectations accordingly. What is expected to work
well:
- Running avocado 'instrumented' tests
- Running arbitrary executables as tests
- Automatic test discovery and run of tests on directories
- xUnit/JSON report

Known Issues
- HTML report of test jobs with multiplexed tests has a minor naming display
issue that is scheduled to be fixed by next release.
- avocado-vt might fail to load if virt-test was not properly bootstrapped.
Make sure you always run bootstrap in the virt-test directory on any
virt-test git updates to prevent the issue. Next release will have more
mechanisms to give the user better error messages on tough to judge
situations (virt-test repo with stale or invalid config files that need
update).

Changes
- The Avocado API has been greatly streamlined. After a long discussion and
several rounds of reviews and planning, now we have a clear separation of
what is intended as functions useful for test developers and plugin/core
developers:
- avocado.core is intended for plugin/core developers. Things are more fluid
on this space, so that we can move fast with development
- avocado.utils is a generic library, with functions we found out to be
useful for a variety of tests and core code alike.
- avocado has some symbols exposed at its top level, with the test API:
- our Test() class, derived from the unittest.TestCase() class
- a main() entry point, similar to unittest.main()
- VERSION, that gives the user the avocado version (eg 0.25.0).
Those symbols and classes/APIs will be changed more carefully, and release
notes will certainly contain API update notices. In other words, we'll be
a lot more mindful of changes in this area, to reduce the maintenance cost
of writing avocado tests.

We believe this more strict separation between the available APIs will help
test developers to quickly identify what they need for test development, and
reduce following a fast moving target, what usually happens when we have a
new project that does not have clear policies behind its API design.
- There's a new plugin added to the avocado project: avocado-vt. This plugin
acts as a wrapper for the virt-test test suite
(https://github.com/autotest/virt-test), allowing people to use avocado to
list and run the tests available for that test suite. This allows people to
leverage a number of the new cool avocado features for the virt tests
themselves:
- HTML reports, a commonly asked feature for the virt-test suite. You can
see a screenshot of what the report looks like here:
https://cloud.githubusercontent.com/assets/296807/7406339/7699689e-eed7-11e4-9214-38a678c105ec.png
- You can run virt-tests on arbitrary order, and multiple instances of a
given test, something that is also currently not possible with the virt
test runner (also a commonly asked feature for the suite.
- System info collection. It's a flexible feature, you get to configure
easily what gets logged/recorded between tests.
- The avocado multiplexer (test matrix representation/generation system)
also received a lot of work and fixes during this
release. One of the most visible (and cool) features of 0.25.0 is the new,
improved --tree representation of the multiplexer file:


$ avocado multiplex examples/mux-environment.yaml -tc
┗━━ run
┣━━ hw
┃    ┣━━ cpu
┃    ┃    ╠══ intel
┃    ┃    ║     → cpu_CFLAGS: -march=core2
┃    ┃    ╠══ amd
┃    ┃    ║     → cpu_CFLAGS: -march=athlon64
┃    ┃    ╚══ arm
┃    ┃          → cpu_CFLAGS: -mabi=apcs-gnu -march=armv8-a -mtune=arm8
┃    ┗━━ disk
┃         ╠══ scsi
┃         ║     → disk_type: scsi
┃         ╚══ virtio
┃               → disk_type: virtio
┣━━ distro
┃    ╠══ fedora
┃    ║     → init: systemd
┃    ╚══ mint
┃          → init: systemv
┗━━ env
╠══ debug
║     → opt_CFLAGS: -O0 -g
╚══ prod
→ opt_CFLAGS: -O2


We hope you find the multiplexer useful and enjoyable.
- If an avocado plugin fails to load, due to factors such as missing
dependencies, environment problems and misconfiguration, in order to notify
users and make them mindful of what it takes to fix the root causes for the
loading errors, those errors are displayed in the avocado stderr stream.

However, often we can't fix the problem right now and don't need
the constant stderr nagging. If that's the case, you can set in your local
config file:


[plugins]
Suppress notification about broken plugins in the app standard error.
Add the name of each broken plugin you want to suppress the notification
in the list. The names can be easily seen from the stderr messages. Example:
avocado.core.plugins.htmlresult  ImportError No module named pystache
add 'avocado.core.plugins.htmlresult' as an element of the list below.
skip_broken_plugin_notification = []

- Our documentation has received a big review, that led to a number of
improvements. Those can be seen online
(http://avocado-framework.readthedocs.org/en/latest/), but if you feel so
inclined, you can build the documentation for local viewing, provided that
you have the sphinx python package installed by executing:

$ make -C docs html

Of course, if you find places where our documentation needs
fixes/improvements, please send us a PR and we'll gladly review it.
- As one would expect, many bugs were fixed. You can take a look at the full
list of 156 commits here:
https://github.com/avocado-framework/avocado/compare/0.24.0...0.25.0

0.24.0

We are proud to announce the new Avocado 0.24.0 release! This time, we had made a number of API changes to reflect some good practices in API development, and changes to the multiplexer, that is making large steps towards a production ready state. Let's break down the changes:

Core/General API isolation

In order to avoid user confusion, now the `avocado` API has a clear separation with regards to what it's supposed to be used by test developers, and what is supposed to be used by plugin/core application developers:
- `avocado.core` is restricted to plugin/core app developers
- `avocado` can be used by both test developers and plugin/core app developers

Breakdown of changes:
- `avocado.core.data_dir` -> `avocado.data_dir`
- `avocado.plugins` -> `avocado.core.plugins`
- `avocado.remote` -> `avocado.core.remote`
- `avocado.job` -> `avocado.core.job`
- `avocado.job.main` -> `avocado.main`
- `avocado.loader` -> `avocado.core.loader`
- `avocado.output` -> `avocado.core.output`
- `avocado.loader` -> `avocado.core.loader`
- `avocado.parser` -> `avocado.core.parser`
- `avocado.runner` -> `avocado.core.runner`
- `avocado.sysinfo` -> `avocado.core.sysinfo`
- `avocado.external.gdbmi_parser` -> `avocado.core.gdbmi_parser`
- `avocado.external.spark` -> `avocado.core.spark`
- `avocado.cli.app` -> `avocado.core.app`
- `avocado.exceptions` -> `avocado.core.exceptions`

Multiplexer

The multiplexer has gone through changes on the parameters retrieval API. The new way of retrieving params is through the `get` API, that has the form::


get(key, path, default)


The `avocado.test.Test.params` object is no longer a dictionary, and it's not possible to refer to the values through the shortcut `avocado.test.Test.params.key` anymore.

Example of old usage::


c_file = self.get_data_path(self.params.get('source', 'doublefree.c'))


This statement has to be updated to::


c_file = self.get_data_path(self.params.get('source', default='doublefree.c'))


Otherwise the default `doublefree.c` value will be taken as the retrieval path, then c_file would be `None`.

The multiplexer file format also has a new `!mux` tag, that means to recursively multiplex all nodes below the specified tag. By default now, we don't multiplex until the `!mux` tag is specified. Let's consider the following file::


hw:
cpu:
intel:
cpu_CFLAGS: '-march=core2'
amd:
cpu_CFLAGS: '-march=athlon64'
arm:
cpu_CFLAGS: '-mabi=apcs-gnu -march=armv8-a -mtune=arm8'
disk:
scsi:
disk_type: 'scsi'
virtio:
disk_type: 'virtio'
distro:
fedora:
init: 'systemd'
mint:
init: 'systemv'
env:
debug:
opt_CFLAGS: '-O0 -g'
prod:
opt_CFLAGS: '-O2'


Here, the nodes `cpu`, `disk`, `distro` and `env` are the ones we actually want to combine, while `hw` is just for organization purposes (we don't want to combine `hw` with `distro`, really).

New version, with `!mux`:


hw:
cpu: !mux
intel:
cpu_CFLAGS: '-march=core2'
amd:
cpu_CFLAGS: '-march=athlon64'
arm:
cpu_CFLAGS: '-mabi=apcs-gnu -march=armv8-a -mtune=arm8'
disk: !mux
scsi:
disk_type: 'scsi'
virtio:
disk_type: 'virtio'
distro: !mux
fedora:
init: 'systemd'
mint:
init: 'systemv'
env: !mux
debug:
opt_CFLAGS: '-O0 -g'
prod:
opt_CFLAGS: '-O2'


This way we can keep using `hw` as a organization label, not interfering with our final purpose of combining only `cpu`, `disk`, `distro` and `env`. The variants generated with the new tag arrangement are:


Variant 1:    /hw/cpu/intel, /hw/disk/scsi, /distro/fedora, /env/debug
Variant 2:    /hw/cpu/intel, /hw/disk/scsi, /distro/fedora, /env/prod
Variant 3:    /hw/cpu/intel, /hw/disk/scsi, /distro/mint, /env/debug
Variant 4:    /hw/cpu/intel, /hw/disk/scsi, /distro/mint, /env/prod
Variant 5:    /hw/cpu/intel, /hw/disk/virtio, /distro/fedora, /env/debug
Variant 6:    /hw/cpu/intel, /hw/disk/virtio, /distro/fedora, /env/prod
Variant 7:    /hw/cpu/intel, /hw/disk/virtio, /distro/mint, /env/debug
Variant 8:    /hw/cpu/intel, /hw/disk/virtio, /distro/mint, /env/prod
Variant 9:    /hw/cpu/amd, /hw/disk/scsi, /distro/fedora, /env/debug
Variant 10:    /hw/cpu/amd, /hw/disk/scsi, /distro/fedora, /env/prod
Variant 11:    /hw/cpu/amd, /hw/disk/scsi, /distro/mint, /env/debug
Variant 12:    /hw/cpu/amd, /hw/disk/scsi, /distro/mint, /env/prod
Variant 13:    /hw/cpu/amd, /hw/disk/virtio, /distro/fedora, /env/debug
Variant 14:    /hw/cpu/amd, /hw/disk/virtio, /distro/fedora, /env/prod
Variant 15:    /hw/cpu/amd, /hw/disk/virtio, /distro/mint, /env/debug
Variant 16:    /hw/cpu/amd, /hw/disk/virtio, /distro/mint, /env/prod
Variant 17:    /hw/cpu/arm, /hw/disk/scsi, /distro/fedora, /env/debug
Variant 18:    /hw/cpu/arm, /hw/disk/scsi, /distro/fedora, /env/prod
Variant 19:    /hw/cpu/arm, /hw/disk/scsi, /distro/mint, /env/debug
Variant 20:    /hw/cpu/arm, /hw/disk/scsi, /distro/mint, /env/prod
Variant 21:    /hw/cpu/arm, /hw/disk/virtio, /distro/fedora, /env/debug
Variant 22:    /hw/cpu/arm, /hw/disk/virtio, /distro/fedora, /env/prod
Variant 23:    /hw/cpu/arm, /hw/disk/virtio, /distro/mint, /env/debug
Variant 24:    /hw/cpu/arm, /hw/disk/virtio, /distro/mint, /env/prod


The tree representation also changed to print the `!mux` points of branching, to further help the user to understand the structure of the tree::


$ scripts/avocado multiplex --tree examples/mux-environment.yaml
Config file tree structure:

/-intel
|
/cpu-<>--amd
|      |
/hw       \-arm
|  |
|  |        /-scsi
|   \disk-<>
|           \-virtio
-|
|          /-fedora
|-distro-<>
|          \-mint
|
|       /-debug
\env-<>
\-prod


The more detailed breakdown of the 145 commits can be seen here:

https://github.com/avocado-framework/avocado/compare/0.23.0...0.24.0

Happy testing!

0.23.0

Hello everyone! I'm here to announce the new Avocado 0.23.0 release! This time around, we have a few changes worth of note:
- Now avocado looks more like nose and other unittest based test runners: You can define a number of test methods prefixed with `test`, and the runner will simply run them. Also, for compatibility with the base unittest class, now `setup` and `cleanup` are `setUp` and `tearDown`.

Let's see how an old avocado test looked like:


!/usr/bin/python
from avocado import test
from avocado import job


class OldTest(test.Test):

def setup(self):
self.hello = "Hi there!"

def action(self):
self.assertEqual(self.hello, "Hi there!")

def cleanup(self):
self.log.debug('Cleanup')

if __name__ == '__main__':
job.main()


The new avocado tests look like this:


!/usr/bin/python
from avocado import test
from avocado import job


class MultipleTests(test.Test):

def setUp(self):
self.hello = "Hi there!"

def test_hello(self):
self.assertEqual(self.hello, "Hi there!")

def testIdentity(self):
self.assertTrue(1, 1)

def tearDown(self):
self.log.debug('Cleanup')

if __name__ == '__main__':
job.main()

- `avocado run`: Simplified command line options, cutting off a number of rarely used options and moving them to config file entries
- `avocado run`: Added the `--job-timeout` option, that allows users to specify the maximum time an avocado test job might take.
- `avocado run`: Consolidated the options `--remote-timeout` and `--vm-timeout` into `--job-timeout`.
- Sysinfo feature: The sysinfo feature went through refactoring and simplification - Now it is possible to specify what information from the system is to be collected through config files

See how to configure your collectibles:


[sysinfo.collectibles]
File with list of commands that will be executed and have their output collected
commands = /etc/avocado/sysinfo/commands
File with list of files that will be collected verbatim
files = /etc/avocado/sysinfo/files
File with list of commands that will run alongside the job/test
profilers = /etc/avocado/sysinfo/profilers


The default `commands` file looks like:


df -mP
dmesg -c
uname -a
lspci -vvnn
gcc --version
ld --version
mount
hostname
uptime
dmidecode
ifconfig -a
brctl show
ip link
numactl --hardware show
lscpu
fdisk -l

- Multiplexer: A new params retrieval system is in place. Now, instead of `self.params.key` you must use the `self.params.get(key, path, default)` API. A compatibility layer with the old dict structure was made, but we encourage you to migrate your param retrieval calls from `self.params.key` or `self.params['key'] to`self.params.get('key').

The full list of commits can be seen here:

https://github.com/avocado-framework/avocado/compare/0.21.0...0.23.0

Happy testing!

0.21.0

- Avocado is now compatible with python 2.6 and distros that ship 2.6
as their default runtime, such as RHEL 6 and CentOS 6. Our CI
environment was updated to test under 2.6 and 2.7 targets
- Avocado rest client application (lets you talk to the avocado server
application)
- A new option was added to disable colored output in the config file,
runner.output.colored
- A new option was added to add commands that will profile new test job
runs, sysinfo.collect.profiler_commands
- The documentation was updated with GDB instrumentation examples
(besides the normal docs update on new features/patchsets)
- API Change: avocado.utils.io -> avocado.utils.genio
- API Change: avocado.utils.process.find_command ->
avocado.utils.path.find_command
- API Change: avocado.utils.process.CmdNotFoundError ->
avocado.utils.path.CmdNotFoundError
- BUGFIX: remote machine could get stuck waiting on human input on
paginator
- BUGFIX: Avocado crashes on certain test defined exceptions
- BUGFIX: Internal HTML report links when using remote/vm plugins
- BUGFIX: Job directory generated even when 'avocado run' had option
validation errors
- BUGFIX: When installing avocado as an RPM, some example tests stop
working
- BUGFIX: Avocado crashes if a remote test is missing

The complete list with the 130 commits can be seen here:

https://github.com/avocado-framework/avocado/compare/0.20.1...0.21.0

0.17.0

New features:

avocado:
- HTML job report plugin, that gives users the ability to generate a nice
and good looking report of the testing activities.
- --wrapper feature, that allows users to write scripts to wrap the execution
of test commands. You can use it to profile your QEMU instance during the
test with perf/ltrace, for example.
- Support for the GDB remote protocol inside avocado.
- A new config plugin, that allows users to see the current configuration
keys set for their particular instances of avocado, and help you to modify
values if you wish so.
- Tweaks and documentation improvements in the requirements.txt file, and the
split of the file into other requirements files, with more specific purpose.
- Fix a deadlock that occurred when the main test code has an early abort.
- Fix a temporary dir issue, that had potential security implications.

avocado-virt plugin:
- Introduction of QEMU command line templates, to make it easier for people to
provide specific command lines to be run inside avocado virt specific tests.
- Fix a bug in migration process end detection, the code is now more robust and
reliable.

avocado-server:
- Introduction of software components and test environment data models,
preparing the server for a big feature: Automated bisection of failed tests.

infrastructure:
- Avocado has now a web page, courtesy of github pages:

http://avocado-framework.github.io/