PyUp Safety actively tracks 274,040 Python packages for vulnerabilities and notifies you when to upgrade.
Bug Fixes ========= * Avocado can now find tests on classes that are imported using relative ``import`` statements with multiple classes. Previously only the first class imported in such a statement was properly processed. * ``avocado run`` will now create test suites without an automatic (and usually very verbose) name, but instead without a name, given that there will be only one suite on such jobs. This restores the ``avocado run`` behavior users expected and are used to. * Hint files are now being respected again, this time within the context of test suite creation. * Filtering by tags is now working properly when using the resolver, that is, when using ``avocado list --resolver -t $tag -- $reference``. * Test suites now properly respect the configuration given to them, as opposed to using a configuration composed by the default registered option values. * Fixed the "elapsed time" produced by the ``avocado-instrumented`` nrunner runner (that is, ``avocado-runner-avocado-instrumented``). * ``avocado --verbose list --resolver -- $reference`` has reinstated the presentation of failed resolution information, which is useful for understanding why a test reference was not resolved into a test. * The "legacy replay plugin", that is, ``avocado run --replay``, can now replay a subset of tests based on their status. * The ``avocado diff`` command won't crash anymore if given ``sysinfo`` files with binary content. It will log the issue, and not attempt to present binary differences. * The HTML report generated by ``avocado diff`` now runs properly and won't crash. * The asset fetcher plugin won't crash anymore due to differences in the AST based node attributes. * :class:`avocado.utils.process.FDDrainer` now properly respects the presence and absence of newlines produced when running new processes via :func:`avocado.utils.process.run` and friends. This also fixes tests that relied on the "output check" feature because of missing newlines. * The ``nrunner`` plugin will now always display test status in the most natural order, that is, ``STARTED`` before ``PASS`` or ``FAIL``. * The ``nrunner`` plugin will now properly set the job status in case of test failures, resulting in the job (and ``avocado run``) exit status to properly signal failures. * A vast documentation review was performed, with many fixes and improvements. For more information, please check out the complete Avocado changelog: https://github.com/avocado-framework/avocado/compare/81.0...82.0
Users/Test Writers ================== * ``INSTRUMENTED`` tests using the :func:`avocado.core.test.Test.fetch_asset` can take advantage of plugins that will attempt to download (and cache) assets before the test execution. This should make the overall test execution more reliable, and give better test execution times as the download time will be excluded. Users can also manually execute the ``avocado assets`` command to manually fetch assets from tests. * The still experimental "N(ext) Runner" support for Avocado Instrumented tests is more complete and supports tag filtering and passing tags to the tests. * A new architecture for "finding" tests has been introduced as an alternative to the :mod:`avocado.core.loader` code. It's based around the :mod:`avocado.core.resolver`, and it's currently used in the still experimental "N(ext) Runner". It currently supports tests of the following types: ``avocado-instrumented``, ``exec-test``, ``glib``, ``golang``, ``python-unittest`` and ``robot``. You can experiment it by running ``avocado nlist``, similarly to how ``avocado list`` is used. * Avocado ``sysinfo`` feature file will now work out of the box on ``pip`` based installations. Previously, it would require configuration files tweaks to adjust installation paths. * A massive documentation overhaul, now designed around guides to different target audiences. The "User's Guide", "Test Writer's Guide" and "Contributor's Guide" can be easily found as first lever sections contain curated content for those audiences. Bug Fixes ========= * Content supposed to be UI only could leak into TAP files, making them invalid. * Avocado's ``sysinfo`` feature will now run commands without a shell, resulting in more proper captured output, without shell related content. * :func:`avocado.utils.process.SubProcess.send_signal` will now send a signal to itself correctly even when using ``sudo`` mode. Utility APIs ============ * The :mod:`avocado.utils.vmimage` library now allows a user to define the ``qemu-img`` binary that will be used for creating snapshot images via the :data:`avocado.utils.vmimage.QEMU_IMG` variable. * The :mod:`avocado.utils.configure_network` module introduced a number of utilities, including MTU configuration support, a method for validating network among peers, IPv6 support, etc. * The :func:`avocado.utils.configure_network.set_ip` function now supports different interface types through a ``interface_type`` parameter, while still defaulting to ``Ethernet``. Internal Changes ================ * Package support for Enterprise Linux 8. * Increased CI coverage, having tests now run on four different hardware architectures: amd64 (x86_64), arm64 (aarch64), ppc64le and s390x. * Packit support adding extended CI coverage, with RPM packages being built for Pull Requests and results shown on GitHub. * Pylint checks for w0703 were enabled. * Runners, such as the remote runner, vm runner, docker runner, and the default local runner now conform to a "runner" interface and can be seen as proper plugins with ``avocado plugins``. * Avocado's configuration parser will now treat values with relative paths as a special value, and evaluate their content in relation to the Python's distribution directory where Avocado is installed. For more information, please check out the complete `Avocado changelog <https://github.com/avocado-framework/avocado/compare/72.0...73.0>`_.
Users/Test Writers ================== * Avocado can now run on systems with nothing but Python 3 (and "quasi-standard-library" module ``setuptools``). This means that it won't require extra packages, and should be easier to deploy on containers, embedded systems, etc. Optional plugins may have additional requirements. * A new and still experimental test runner implementation, known as "N(ext) Runner" has been introduced. It brings a number of different concepts, increasing the decoupling between a test (and its runner) and the job. For more information, please refer to :ref:`the early documentation <nrunner>`. * The new :func:`avocado.cancel_on` decorator has been added to the Test APIs, allowing you to define the conditions for a test to be considered canceled. See one example :ref:`here <turning_errors_into_cancels>`. * The ``glib`` plugin got a configuration option its safe/unsafe operation, that is, whether it will execute binaries in an attempt to find the whole list of tests. Look for the ``glib.conf`` shipped with the plugin to enable the unsafe mode. * Avocado can now use tags inside Python Unittests, and not only on its own Instrumented tests. It's expected that other forms or providing tags for other types of tests will also be introduced in the near future. * The HTML report will now show, as a handy pop-up, the contents of the test whiteboard. If you set, say, performance metrics there, you'll able to see straight from the report. * The HTML report now has filtering support by test status, and can show all records in the table. * The :mod:`avocado.utils.runtime` module, a badly designed mechanism for sharing Avocado runtime settings with the utility libraries, has been removed. * The test runner feature that would allow binaries to be run transparently inside GDB was removed. The reason for dropping such a feature have to do with how it limits the test runner to run one test at a time, and the use of the :mod:`avocado.utils.runtime` mechanism, also removed. * Initial examples for writing custom jobs, using the so called Job API, have been added to ``examples/jobs``. These APIs are still non-public (under core), but they're supposed to become public and supported soon. * By means of a new plugin (``merge_files``, of type ``job.prepost``), when using the :ref:`output check record <output_check_record>` features, duplicate files created by different tests/variants will be consolidated into unique files. Bug Fixes ========= * The HTML plugin now correctly shows the date for tests that were never executed because of interrupted jobs. * A temporarily workaround for a stack overflow problem in Python 3.7 has been addressed. * The pict plugin (a varianter implementaion) now properly yields the variants paths as a list. * A Python 3 related fix to mod:`avocado.utils.software_manager`, that was using Python 2 ``next`` on ``get_source``. * A Python 3 related fix to the docker plugin, that wasn't caught earlier. Utility APIs ============ * :mod:`avocado.utils.partition` now allows ``mkfs`` and ``mount`` flags to be set. * :func:`avocado.utils.cpu.get_cpu_vendor_name` now returns the CPU vendor name for POWER9. * :mod:`avocado.utils.asset` now allows a given location, as well as a list, to be given, simplifying the most common use case. * :meth:`avocado.utils.process.SubProcess.stop` now supports setting a timeout. Please refer to the documentation for the important details on its behavior. * :mod:`avocado.utils.memory` now properly handles hugepages for POWER platform. Internal Changes ================ * Removal of the ``stevedore`` library dependency (previously used for the dispatcher/plugins infrastructure). * ``make check`` now runs selftests using the experimental N(ext) Runner. * Formal support for Python 3.7, which is now on our CI checks, documentation and module information. * The Yaml to Mux plugin now uses a safe version of the Yaml loader, so that the execution of arbitrary Python code from Yaml input is now no longer possible. * Codecov coverage reports for have been enabled for Avocado, and can be seen on every pull request. * New tests have been added to many of the optional plugins. * Various pylint compliance improvements, including w0231, w0235, w0706, w0715 and w0221. * Avocado's selftests now use tempfile.TemporaryDirectory instead of mkdtemp and shutil.rmtree. * :class:`avocado.core.job.Job` instantiation now takes a ``config`` dictionary parameter, instead of a ``argparse.Namespace`` instance, and keeps it in a ``config`` attribute. * :class:`avocado.core.job.Job` instances don't have a ``references`` attribute anymore. That information is available in the ``config`` attribute, that is, ``myjob.config['references']``. * Basic checks for Fedora and RHEL 8 using Cirrus CI have been added, and will be shown on every pull request. For more information, please check out the complete `Avocado changelog <https://github.com/avocado-framework/avocado/compare/70.0...71.0>`_.
The major changes introduced on this version are listed below. - Support for running tests in Docker container. Now, in addition to running tests on a (libvirt based) Virtual Machine or on a remote host, you can now run tests in transient Docker containers. The usage is as simple as:: $ avocado run mytests.py --docker ldoktor/fedora-avocado The container will be started, using `ldoktor/fedora-avocado` as the image. This image contains a Fedora based system with Avocado already installed, and it's provided at the official Docker hub. - Introduction of the "Fail Fast" feature. By running a job with the `--failfast` flag, the job will be interrupted after the very first test failure. If your job only makes sense if it's a complete PASS, this feature can save you a lot of time. - Avocado supports replaying previous jobs, selected by using their Job IDs. Now, it's also possible to use the special keyword `latest`, which will cause Avocado to rerun the very last job. - Python's standard signal handling is restored for SIGPIPE, and thus for all tests running on Avocado. In previous releases, Avocado introduced a change that set the default handler to SIGPIPE, which caused the application to be terminated. This seemed to be the right approach when testing how the Avocado app would behave on broken pipes on the command line, but it introduced side effects to a lot of Python code. Instead of exceptions, the affected Python code would receive the signal themselves. This is now reverted to the Python standard, and the signal behavior of Python based tests running on Avocado should not surprise anyone. - The project release notes are now part of the official documentation. That means that users can quickly find when a given change was introduced. Together with those changes listed, a total of 38 changes made into this release. For more information, please check out the complete `Avocado changelog <https://github.com/avocado-framework/avocado/compare/38.0...39.0>`_. Sprint Theme After so much love that we had on the previous version, let's twist things a bit with an antagonist title. Info on this pretty good movie by Tarantino can be found at: http://www.imdb.com/title/tt3460252/?ref_=nm_flmg_wr_2 https://www.youtube.com/watch?v=6_UI1GzaWv0 The story line: In the dead of a Wyoming winter, a bounty hunter and his prisoner find shelter in a cabin currently inhabited by a collection of nefarious characters. Release Meeting The Avocado release meetings are now open to the community via Hangouts on Air. The meetings are recorded and made available on the `Avocado Test Framework YouTube channel <https://www.youtube.com/channel/UC-RVZ_HFTbEztDM7wNY4NfA>`_. For this release, you can watch the meeting on `this link <https://www.youtube.com/watch?v=GotEH7SmHSw>`_.
Another development cycle has just finished, and our community will receive this new release containing a nice assortment of bug fixes and new features. - The download of assets in tests now allow for an expiration time. This means that tests that need to download any kind of external asset, say a tarball, can now automatically benefit from the download cache, but can also keep receiving new versions automatically. Suppose your asset uses an asset named `myproject-daily.tar.bz2`, and that your test runs 50 times a day. By setting the expire time to `1d` (1 day), your test will benefit from cache on most runs, but will still fetch the new version when the the 24 hours from the first download have passed. For more information, please check out the documentation on the `expire` parameter to the `fetch_asset()` method. - Environment variables can be propagated into tests running on remote systems. It's a known fact that one way to influence application behavior, including test, is to set environment variables. A command line such as:: $ MYAPP_DEBUG=1 avocado run myapp_test.py Will work as expected on a local system. But Avocado also allows running tests on remote machines, and up until now, it has been lacking a way to propagate environment variables to the remote system. Now, you can use:: $ MYAPP_DEBUG=1 avocado run --env-keep MYAPP_DEBUG \ --remote-host test-machine myapp_test.py - The plugin interfaces have been moved into the `avocado.core.plugin_interfaces` module. This means that plugin writers now have to import the interface definitions this namespace, example:: ... from avocado.core.plugin_interfaces import CLICmd class MyCommand(CLICmd): ... This is a way to keep ourselves honest, and say that there's no difference from plugin interfaces to Avocado's core implementation, that is, they may change at will. For greater stability, one should be tracking the LTS releases. Also, it effectively makes all plugins the same, whether they're implemented and shipped as part of Avocado, or as part of external projects. - A contrib script for running kvm-unit-tests. As some people are aware, Avocado has indeed a close relation to virtualization testing. Avocado-VT is one obvious example, but there are other virtualization related test suites can Avocado can run. This release adds a contrib script that will fetch, download, compile and run kvm-unit-tests using Avocado's external runner feature. This gives results in a better granularity than the support that exists in Avocado-VT, which gives only a single PASS/FAIL for the entire test suite execution. For more information, please check out the Avocado changelog. Also, while we focused on Avocado, let's also not forget that Avocado-VT maintains it's own fast pace of incoming niceties. - s390 support: Avocado-VT is breaking into new grounds, and now has support for the s390 architecture. Fedora 23 for s390 has been added as a valid guest OS, and s390-virtio has been added as a new machine type. - Avocado-VT is now more resilient against failures to persist its environment file, and will only give warnings instead of errors when it fails to save it. - An improved implementation of the "job lock" plugin, which prevents multiple Avocado jobs with VT tests to run simultaneously. Since there's no finer grained resource locking in Avocado-VT, this is a global lock that will prevent issues such as image corruption when two jobs are run at the same time. This new implementation will now check if existing lock files are stale, that is, they are leftovers from previous run. If the processes associated with these files are not present, the stale lock files are deleted, removing the need to clean them up manually. It also outputs better debugging information when failures to acquire lock. The complete list of changes to Avocado-VT are available on its changelog. While not officially part of this release, this development cycle saw the introduction of new tests on our avocado-misc-tests. Go check it out! Finally, since Avocado and Avocado-VT are not newly born anymore, we decided to update information mentioning KVM-Autotest, virt-test on so on around the web. This will hopefully redirect new users to the Avocado community and avoid confusion. Install avocado Instructions are available in our documentation on how to install either with packages or from source. Updated RPM packages are be available in the project repos for EPEL 6, EPEL 7, Fedora 22, Fedora 23 and the newly released Fedora 24. Please note that on the next release, we'll drop support for Fedora 22 packages. Happy hacking and testing! ---  http://avocado-framework.readthedocs.io/en/38.0/WritingTests.html  https://github.com/avocado-framework/avocado/compare/37.0...38.0  https://github.com/avocado-framework/avocado-vt/compare/37.0...38.0  http://avocado-framework.readthedocs.io/en/38.0/GetStartedGuide.htmlinstalling-avocado
details on the specific RFC. The main changes in Avocado for this release are: - A big round of fixes and on machine readable output formats, such as xunit (aka JUnit) and JSON. The xunit output, for instance, now includes tests with schema checking. This should make sure interoperability is even better on this release. - Much more robust handling of test references, aka test URLs. Avocado now properly handles very long test references, and also test references with non-ascii characters. - The avocado command line application now provides richer exit status. If your application or custom script depends on the avocado exit status code, you should be fine as avocado still returns zero for success and non-zero for errors. On error conditions, though, the exit status code are richer and made of combinable (ORable) codes. This way it's possible to detect that, say, both a test failure and a job timeout occurred in a single execution. - [SECURITY RELATED] The remote execution of tests (including in Virtual Machines) now allows for proper checks of host keys. Without these checks, avocado is susceptible to a man-in-the-middle attack, by connecting and sending credentials to the wrong machine. This check is _disabled_ by default, because users depend on this behavior when using machines without any prior knowledge such as cloud based virtual machines. Also, a bug in the underlying SSH library may prevent existing keys to be used if these are in ECDSA format. There's an automated check in place to check for the resolution of the third party library bug. Expect this feature to be _enabled_ by default in the upcoming releases. - Pre/Post Job hooks. Avocado now defines a proper interface for extension/plugin writers to execute actions while a Job is runnning. Both Pre and Post hooks have access to the Job state (actually, the complete Job instance). Pre job hooks are called before tests are run, and post job hooks are called at the very end of the job (after tests would have usually finished executing). - Pre/Post job scripts. As a feature built on top of the Pre/Post job hooks described earlier, it's now possible to put executable scripts in a configurable location, such as `/etc/avocado/scripts/job/pre.d` and have them called by Avocado before the execution of tests. The executed scripts will receive some information about the job via environment variables. - The implementation of proper Test-IDs in the test result directory. Also, while not everything is (yet) translated into code, this release saw various and major RFCs, which are definitely shaping the future of Avocado. Among those: - Introduce proper test IDs - Pre/Post _test_ hooks - Multi-stream tests - Avocado maintainability and integration with avocado-vt - Improvements to job status (completely implemented) For a complete list of changes please check the Avocado changelog. For Avocado-VT, please check the full Avocado-VT changelog. Install avocado Instructions are available in our documentation on how to install either with packages or from source. Updated RPM packages are be available in the project repos for Fedora 22, Fedora 23, EPEL 6 and EPEL 7. Packages As a heads up, we still package the latest version of the various Avocado sub projects, such as the very popular Avocado-VT and the pretty much experimental Avocado-Virt and Avocado-Server projects. For the upcoming releases, there will be changes in our package offers, with a greater focus on long term stability packages for Avocado. Other packages may still be offered as a convenience, or may see a change of ownership. All in the best interest of our users. If you have any concerns or questions, please let us know. Happy hacking and testing! ---  http://avocado-framework.readthedocs.org/en/35.0/ResultFormats.htmlexit-codes  https://github.com/avocado-framework/avocado/blob/35.0/etc/avocado/avocado.confL41  https://github.com/avocado-framework/avocado/blob/35.0/selftests/functional/test_thirdparty_bugs.pyL17  http://avocado-framework.readthedocs.org/en/35.0/ReferenceGuide.htmljob-pre-and-post-scripts  http://avocado-framework.readthedocs.org/en/35.0/ReferenceGuide.htmlscript-execution-environment  https://www.redhat.com/archives/avocado-devel/2016-March/msg00024.html  https://www.redhat.com/archives/avocado-devel/2016-April/msg00000.html  https://www.redhat.com/archives/avocado-devel/2016-April/msg00042.html  https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html  https://www.redhat.com/archives/avocado-devel/2016-April/msg00010.html  https://github.com/avocado-framework/avocado/compare/0.34.0...35.0  https://github.com/avocado-framework/avocado-vt/compare/0.34.0...35.0  http://avocado-framework.readthedocs.org/en/35.0/GetStartedGuide.htmlinstalling-avocado
This is another proud announcement: Avocado release 35.0, aka "Mr Robot", is now out! This release, while a "regular" release, will also serve as a beta for our first "long term stability" (aka "lts") release. That means that the next release, will be version "36.0lts" and will receive only bug fixes and minor improvements. So, expect release 35.0 to be pretty much like "36.0lts" feature-wise. New features will make into the
Hello to all test enthusiasts out there, specially to those that cherish, care or are just keeping an eye on the greenest test framework there is: Avocado release 0.34.0, aka The Hour of the Star, is now out! The main changes in Avocado for this release are: - A complete overhaul of the logging and output implementation. This means that all Avocado output uses the standard Python logging library making it very consistent and easy to understand . - Based on the logging and output overhaul, the command line test runner is now very flexible with its output. A user can choose exactly what should be output. Examples include application output only, test output only, both application and test output or any other combination of the builtin streams. The user visible command line option that controls this behavior is `--show`, which is an application level option, that is, it's available to all avocado commands.  - Besides the builtin streams, test writers can use the standard Python logging API to create new streams. These streams can be shown on the command line as mentioned before, or persisted automatically in the job results by means of the `--store-logging-stream` command line option.  - The new `avocado.core.safeloader` module, intends to make it easier to to write new test loaders for various types of Python code.  - Based on the new `avocado.core.safeloader` module, a contrib script called `avocado-find-unittests`, returns the name of unittest.TestCase based tests found on a given number of Python source code files.  - Avocado is now able to run its own selftest suite. By leveraging the `avocado-find-unittests` contrib script and the External Runner  feature. A Makefile target is available, allowing developers to run `make selfcheck` to have the selftest suite run by Avocado.  - Partial Python 3 support. A number of changes were introduced that allow concurrent Python 2 and 3 support on the same code base. Even though the support for Python 3 is still _incomplete_, the `avocado` command line application can already run some limited commands at this point. - Asset fetcher utility library. This new utility library, and INSTRUMENTED test feature, allows users to transparently request external assets to be used in tests, having them cached for later use.  - Further cleanups in the public namespace of the avocado Test class. - [BUG FIX] Input from the local system was being passed to remote systems when running tests with either in remote systems or VMs. - [BUG FIX] HTML report stability improvements, including better Unicode handling and support for other versions of the Pystache library. - [BUG FIX] Atomic updates of the "latest" job symlink, allows for more reliable user experiences when running multiple parallel jobs. - [BUG FIX] The avocado.core.data_dir module now dynamically checks the configuration system when deciding where the data directory should be located. This allows for later updates, such as when giving one extra `--config` parameter in the command line, to be applied consistently throughout the framework and test code. - [MAINTENANCE] The CI jobs now run full checks on each commit on any proposed PR, not only on its topmost commit. This gives higher confidence that a commit in a series is not causing breakage that a later commit then inadvertently fixes. For a complete list of changes please check the Avocado changelog. For Avocado-VT, please check the full Avocado-VT changelog. Avocado Videos As yet another way to let users know about what's available in Avocado, we're introducing short videos with very targeted content on our very own YouTube channel: https://www.youtube.com/channel/UCP4xob52XwRad0bU_8V28rQ The first video available demonstrates a couple of new features related to the advanced logging mechanisms, introduced on this release: https://www.youtube.com/watch?v=8Ur_p5p6YiQ Install avocado Instructions are available in our documentation on how to install either with packages or from source. Updated RPM packages are be available in the project repos for Fedora 22, Fedora 23, EPEL 6 and EPEL 7. Happy hacking and testing! ---  http://avocado-framework.readthedocs.org/en/0.34.0/LoggingSystem.html  http://avocado-framework.readthedocs.org/en/0.34.0/LoggingSystem.htmltweaking-the-ui  http://avocado-framework.readthedocs.org/en/0.34.0/LoggingSystem.htmlstoring-custom-logs  http://avocado-framework.readthedocs.org/en/0.34.0/WritingTests.htmladvanced-logging-capabilities  https://github.com/avocado-framework/avocado/blob/0.34.0/avocado/core/safeloader.py  http://avocado-framework.readthedocs.org/en/0.34.0/api/core/avocado.core.htmlmodule-avocado.core.safeloader  https://github.com/avocado-framework/avocado/blob/0.34.0/contrib/avocado-find-unittests  http://avocado-framework.readthedocs.org/en/0.34.0/GetStartedGuide.htmlrunning-tests-with-an-external-runner  https://github.com/avocado-framework/avocado/blob/0.34.0/MakefileL33  http://avocado-framework.readthedocs.org/en/0.34.0/WritingTests.htmlfetching-asset-files  https://github.com/avocado-framework/avocado/compare/0.33.0...0.34.0  https://github.com/avocado-framework/avocado-vt/compare/0.33.0...0.34.0  http://avocado-framework.readthedocs.org/en/latest/GetStartedGuide.htmlinstalling-avocado
Hi everyone! Right on time for the holidays, Avocado reaches the end of Sprint 31, and together with it, we're very happy to announce a brand new release! This version brings stability fixes and improvements to both Avocado and Avocado-VT, some new features and a major redesign of our plugin architecture. For Avocado the main changes are: - It's now possible to register callback functions to be executed when all tests finish, that is, at the end of a particular job. - The software manager utility library received a lot of love on the Debian side of things. If you're writing tests that install software packages on Debian systems, you may be in for some nice treats and much more reliable results. - Passing malformed commands (such as ones that can not be properly split by the standard shlex library) to the process utility library is now better dealt with. - The test runner code received some refactors and it's a lot easier to follow. If you want to understand how the Avocado test runner communicates with the processes that run the test themselves, you may have a much better code reading experience now. - Updated inspektor to the latest and greatest, so that our code is kept is shiny and good looking (and performing) as possible. - Fixes to the utility GIT library when using a specific local branch name. - Changes that allow our selftest suite to run properly on virtualenvs. - Proper installation requirements definition for Python 2.6 systems. - A completely new plugin architecture. Now we offload all plugin discovery and loading to the Stevedore library. Avocado now defines precise (and simpler) interfaces for plugin writers. Please be aware that the public and documented interfaces for plugins, at the moment, allows adding new commands to the avocado command line app, or adding new options to existing commands. Other functionality can be achived by "abusing" the core avocado API from within plugins. Our goal is to expand the interfaces so that other areas of the framework can be extended just as easily. For a complete list of changes please check the Avocado changelog. Avocado-VT received just too many fixes and improvements to list. Please refer to the changelog for more information. Install avocado Instructions are available in our documentation on how to install either with packages or from source. Within a couple of hours, updated RPM packages will be available in the project repos for Fedora 22, Fedora 23, EPEL 6 and EPEL 7. Happy hacking and testing! ---  http://avocado-framework.readthedocs.org/en/0.31.0/ReferenceGuide.htmljob-cleanup  http://avocado-framework.readthedocs.org/en/0.31.0/Plugins.html  https://github.com/avocado-framework/avocado/compare/0.30.0...0.31.0  https://github.com/avocado-framework/avocado-vt/compare/0.30.0...0.31.0  http://avocado-framework.readthedocs.org/en/0.31.0/GetStartedGuide.html
Hello! Avocado reaches the end of Sprint 30, and with it, we have a new release available! This version brings stability fixes and improvements to both Avocado and Avocado-vt. As software doesn't spring out of life itself, we'd like to acknowledge the major contributions by Lucas (AKA lmr) since the dawn of time for Avocado (and earlier projects like Autotest and virt-test). Although the Avocado team at Red Hat was hit by some changes, we're already extremely happy to see that this major contributor (and good friend) has not gone too far. Now back to the more informational part of the release notes. For Avocado the main changes are: - New RPM repository location, check the docs for instructions on how to install the latest releases - Makefile rules for building RPMs are now based on mock, to ensure sound dependencies - Packaged versions are now available for Fedora 22, newly released Fedora 23, EL6 and EL7 - The software manager utility library now supports DNF - The avocado test runner now supports a dry run mode, which allows users to check how a job would be executed, including tests that would be found and parameters that would be passed to it. This is currently complementary to the avocado list command. - The avocado test runner now supports running simple tests with parameters. This may come in handy for simple use cases when Avocado will wrap a test suite, but the test suite needs some command line arguments. Avocado-vt also received many bugfixes. Please refer to the changelog for more information. Install avocado Instructions are available in our documentation on how to install either with packages or from source. Happy hacking and testing! ---  http://avocado-framework.readthedocs.org/en/0.30.0/GetStartedGuide.html  https://github.com/avocado-framework/avocado/compare/0.29.0...0.30.0  https://github.com/avocado-framework/avocado-vt/compare/0.29.0...0.30.0
 http://avocado-framework.readthedocs.org/en/0.29.0/ReferenceGuide.htmltest-resolution  http://avocado-framework.readthedocs.org/en/0.29.0/Loaders.html  https://github.com/avocado-framework/avocado/compare/0.28.0...0.29.0
Hi guys, we're up to a new avocado release! It's basically a bugfix release, with a few usability tweaks. - The avocado human output received some extra tweaks. Here's how it looks now: $ avocado run passtest JOB ID : f186c729dd234c8fdf4a46f297ff0863684e2955 JOB LOG : /home/lmr/avocado/job-results/job-2015-08-15T08.09-f186c72/job.log TESTS : 1 (1/1) passtest.py:PassTest.test: PASS (0.00 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB HTML : /home/lmr/avocado/job-results/job-2015-08-15T08.09-f186c72/html/results.html TIME : 0.00 s - Bugfixes. You may refer to  for the full list of 58 commits. Changes in avocado-vt: - Bugfixes. In particular, a lot of issues related to --vt-type libvirt were fixed and now that backend is fully functional.
Avocado 0.27.0 released! Hi guys, here I am, announcing yet another avocado release! The most exciting news for this release is that our avocado-vt plugin was merged with the virt-test project. The avocado-vt plugin will be very important for QEMU/KVM/Libvirt developers, so the main avocado received updates to better support the goal of having a good quality avocado-vt. Changes in avocado: - The avocado human output received some tweaks and it's more compact, while still being informative. Here's an example: JOB ID : f2f5060440bd57cba646c1f223ec8c40d03f539b JOB LOG : /home/user/avocado/job-results/job-2015-07-27T17.13-f2f5060/job.log JOB HTML : /home/user/avocado/job-results/job-2015-07-27T17.13-f2f5060/html/results.html TESTS : 1 (1/1) passtest.py:PassTest.test: PASS (0.00 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 TIME : 0.00 s - The avocado test loader was refactored and behaves more consistently in different test loading scenarios. - The `utils` API received new modules and functions: - NEW avocado.utils.cpu: APIs related to CPU information on linux boxes  - NEW avocado.utils.git: APIs to clone/update git repos  - NEW avocado.utils.iso9660: Get information about ISO files  - NEW avocado.utils.service: APIs to control services on linux boxes (systemv and systemd)  - NEW avocado.utils.output: APIs that help avocado based CLI programs to display results to users  - UPDATE avocado.utils.download: Add url_download_interactive - UPDATE avocado.utils.download: Add new params to get_file - Bugfixes. You may refer to  for the full list of 64 commits. Changes in avocado-vt: - Merged virt-test into avocado-vt. Basically, the virt-test core library (virttest) replaced most uses of autotest by equivalent avocado API calls, and its code was brought up to the virt-test repository . This means, among other things, that you can simply install avocado-vt through RPM and enjoy all the virt tests without having to clone another repository manually to bootstrap your tests. More details about the process will be sent on an e-mail to the avocado and virt-test mailing lists. Please go to  for instructions on how to get started with all our new tools. See you in a couple of weeks for our next release! Happy testing! The avocado development team Links  http://avocado-framework.readthedocs.org/en/latest/api/utils/avocado.utils.htmlmodule-avocado.utils.cpu  http://avocado-framework.readthedocs.org/en/latest/api/utils/avocado.utils.htmlmodule-avocado.utils.git  http://avocado-framework.readthedocs.org/en/latest/api/utils/avocado.utils.htmlmodule-avocado.utils.iso9660  http://avocado-framework.readthedocs.org/en/latest/api/utils/avocado.utils.htmlmodule-avocado.utils.service  http://avocado-framework.readthedocs.org/en/latest/api/utils/avocado.utils.htmlmodule-avocado.utils.output  https://github.com/avocado-framework/avocado/compare/0.26.0...0.27.0  https://github.com/avocado-framework/avocado-vt/commit/20dd39ef00db712f78419f07b10b8f8edbd19942  http://avocado-vt.readthedocs.org/en/latest/GetStartedGuide.html
Avocado 0.26.0 released! Hi guys, I'm here to announce avocado 0.26.0. This release was dedicated to polish aspects of the avocado user experience, such as documentation and behavior. Changes - Now avocado tests that raise exceptions that don't inherit from `avocado.core.exceptions.TestBaseException` now will be marked as ERRORs. This change was made to make avocado to have clearly defined test statuses. A new decorator, `avocado.fail_on_error` was added to let arbitary exceptions to raise errors, if users need a more relaxed behavior. - The avocado.Test() utility method skip() now can only be called from inside the setUp() method. This was made because by definition, if we get to the test execution step, by definition it can't be skipped anymore. It's important to keep the concepts clear and well separated if we want to give users a good experience. - More documentation polish and updates. Make sure you check out our documentation website http://avocado-framework.readthedocs.org/en/latest/. - A number of avocado command line options and help text was reviewed and updated. - A new, leaner and mobile friendly version of the avocado website is live. Please check http://avocado-framework.github.io/ for more information. - We have the first version of the avocado dashboard! avocado dashboard is the initial version of an avocado web interface, and will serve as the frontend to our testing database. You can check out a screenshot here https://cloud.githubusercontent.com/assets/296807/8536678/dc5da720-242a-11e5-921c-6abd46e0f51e.png ![screenshot from 2015-07-06 22-02-36](https://cloud.githubusercontent.com/assets/296807/8536678/dc5da720-242a-11e5-921c-6abd46e0f51e.png) - And the usual bugfixes. You can take a look at the full list of 68 commits here: https://github.com/avocado-framework/avocado/compare/0.25.0...0.26.0
Avocado 0.25.0 released! Hi guys, I'm here to announce the newest avocado release, 0.25.0. This is an important milestone in avocado development, and we would like to invite you to be a part of the development process, by contributing PRs, testing and giving feedback on the test runner's usability and new plugins we came up with. What to expect This is the first release aimed for general use. We did our best to deliver a coherent and enjoyable experience, but keep in mind that it's a young project, so please set your expectations accordingly. What is expected to work well: - Running avocado 'instrumented' tests - Running arbitrary executables as tests - Automatic test discovery and run of tests on directories - xUnit/JSON report Known Issues - HTML report of test jobs with multiplexed tests has a minor naming display issue that is scheduled to be fixed by next release. - avocado-vt might fail to load if virt-test was not properly bootstrapped. Make sure you always run bootstrap in the virt-test directory on any virt-test git updates to prevent the issue. Next release will have more mechanisms to give the user better error messages on tough to judge situations (virt-test repo with stale or invalid config files that need update). Changes - The Avocado API has been greatly streamlined. After a long discussion and several rounds of reviews and planning, now we have a clear separation of what is intended as functions useful for test developers and plugin/core developers: - avocado.core is intended for plugin/core developers. Things are more fluid on this space, so that we can move fast with development - avocado.utils is a generic library, with functions we found out to be useful for a variety of tests and core code alike. - avocado has some symbols exposed at its top level, with the test API: - our Test() class, derived from the unittest.TestCase() class - a main() entry point, similar to unittest.main() - VERSION, that gives the user the avocado version (eg 0.25.0). Those symbols and classes/APIs will be changed more carefully, and release notes will certainly contain API update notices. In other words, we'll be a lot more mindful of changes in this area, to reduce the maintenance cost of writing avocado tests. We believe this more strict separation between the available APIs will help test developers to quickly identify what they need for test development, and reduce following a fast moving target, what usually happens when we have a new project that does not have clear policies behind its API design. - There's a new plugin added to the avocado project: avocado-vt. This plugin acts as a wrapper for the virt-test test suite (https://github.com/autotest/virt-test), allowing people to use avocado to list and run the tests available for that test suite. This allows people to leverage a number of the new cool avocado features for the virt tests themselves: - HTML reports, a commonly asked feature for the virt-test suite. You can see a screenshot of what the report looks like here: https://cloud.githubusercontent.com/assets/296807/7406339/7699689e-eed7-11e4-9214-38a678c105ec.png - You can run virt-tests on arbitrary order, and multiple instances of a given test, something that is also currently not possible with the virt test runner (also a commonly asked feature for the suite. - System info collection. It's a flexible feature, you get to configure easily what gets logged/recorded between tests. - The avocado multiplexer (test matrix representation/generation system) also received a lot of work and fixes during this release. One of the most visible (and cool) features of 0.25.0 is the new, improved --tree representation of the multiplexer file: $ avocado multiplex examples/mux-environment.yaml -tc ┗━━ run ┣━━ hw ┃ ┣━━ cpu ┃ ┃ ╠══ intel ┃ ┃ ║ → cpu_CFLAGS: -march=core2 ┃ ┃ ╠══ amd ┃ ┃ ║ → cpu_CFLAGS: -march=athlon64 ┃ ┃ ╚══ arm ┃ ┃ → cpu_CFLAGS: -mabi=apcs-gnu -march=armv8-a -mtune=arm8 ┃ ┗━━ disk ┃ ╠══ scsi ┃ ║ → disk_type: scsi ┃ ╚══ virtio ┃ → disk_type: virtio ┣━━ distro ┃ ╠══ fedora ┃ ║ → init: systemd ┃ ╚══ mint ┃ → init: systemv ┗━━ env ╠══ debug ║ → opt_CFLAGS: -O0 -g ╚══ prod → opt_CFLAGS: -O2 We hope you find the multiplexer useful and enjoyable. - If an avocado plugin fails to load, due to factors such as missing dependencies, environment problems and misconfiguration, in order to notify users and make them mindful of what it takes to fix the root causes for the loading errors, those errors are displayed in the avocado stderr stream. However, often we can't fix the problem right now and don't need the constant stderr nagging. If that's the case, you can set in your local config file: [plugins] Suppress notification about broken plugins in the app standard error. Add the name of each broken plugin you want to suppress the notification in the list. The names can be easily seen from the stderr messages. Example: avocado.core.plugins.htmlresult ImportError No module named pystache add 'avocado.core.plugins.htmlresult' as an element of the list below. skip_broken_plugin_notification =  - Our documentation has received a big review, that led to a number of improvements. Those can be seen online (http://avocado-framework.readthedocs.org/en/latest/), but if you feel so inclined, you can build the documentation for local viewing, provided that you have the sphinx python package installed by executing: $ make -C docs html Of course, if you find places where our documentation needs fixes/improvements, please send us a PR and we'll gladly review it. - As one would expect, many bugs were fixed. You can take a look at the full list of 156 commits here: https://github.com/avocado-framework/avocado/compare/0.24.0...0.25.0
We are proud to announce the new Avocado 0.24.0 release! This time, we had made a number of API changes to reflect some good practices in API development, and changes to the multiplexer, that is making large steps towards a production ready state. Let's break down the changes: Core/General API isolation In order to avoid user confusion, now the `avocado` API has a clear separation with regards to what it's supposed to be used by test developers, and what is supposed to be used by plugin/core application developers: - `avocado.core` is restricted to plugin/core app developers - `avocado` can be used by both test developers and plugin/core app developers Breakdown of changes: - `avocado.core.data_dir` -> `avocado.data_dir` - `avocado.plugins` -> `avocado.core.plugins` - `avocado.remote` -> `avocado.core.remote` - `avocado.job` -> `avocado.core.job` - `avocado.job.main` -> `avocado.main` - `avocado.loader` -> `avocado.core.loader` - `avocado.output` -> `avocado.core.output` - `avocado.loader` -> `avocado.core.loader` - `avocado.parser` -> `avocado.core.parser` - `avocado.runner` -> `avocado.core.runner` - `avocado.sysinfo` -> `avocado.core.sysinfo` - `avocado.external.gdbmi_parser` -> `avocado.core.gdbmi_parser` - `avocado.external.spark` -> `avocado.core.spark` - `avocado.cli.app` -> `avocado.core.app` - `avocado.exceptions` -> `avocado.core.exceptions` Multiplexer The multiplexer has gone through changes on the parameters retrieval API. The new way of retrieving params is through the `get` API, that has the form:: get(key, path, default) The `avocado.test.Test.params` object is no longer a dictionary, and it's not possible to refer to the values through the shortcut `avocado.test.Test.params.key` anymore. Example of old usage:: c_file = self.get_data_path(self.params.get('source', 'doublefree.c')) This statement has to be updated to:: c_file = self.get_data_path(self.params.get('source', default='doublefree.c')) Otherwise the default `doublefree.c` value will be taken as the retrieval path, then c_file would be `None`. The multiplexer file format also has a new `!mux` tag, that means to recursively multiplex all nodes below the specified tag. By default now, we don't multiplex until the `!mux` tag is specified. Let's consider the following file:: hw: cpu: intel: cpu_CFLAGS: '-march=core2' amd: cpu_CFLAGS: '-march=athlon64' arm: cpu_CFLAGS: '-mabi=apcs-gnu -march=armv8-a -mtune=arm8' disk: scsi: disk_type: 'scsi' virtio: disk_type: 'virtio' distro: fedora: init: 'systemd' mint: init: 'systemv' env: debug: opt_CFLAGS: '-O0 -g' prod: opt_CFLAGS: '-O2' Here, the nodes `cpu`, `disk`, `distro` and `env` are the ones we actually want to combine, while `hw` is just for organization purposes (we don't want to combine `hw` with `distro`, really). New version, with `!mux`: hw: cpu: !mux intel: cpu_CFLAGS: '-march=core2' amd: cpu_CFLAGS: '-march=athlon64' arm: cpu_CFLAGS: '-mabi=apcs-gnu -march=armv8-a -mtune=arm8' disk: !mux scsi: disk_type: 'scsi' virtio: disk_type: 'virtio' distro: !mux fedora: init: 'systemd' mint: init: 'systemv' env: !mux debug: opt_CFLAGS: '-O0 -g' prod: opt_CFLAGS: '-O2' This way we can keep using `hw` as a organization label, not interfering with our final purpose of combining only `cpu`, `disk`, `distro` and `env`. The variants generated with the new tag arrangement are: Variant 1: /hw/cpu/intel, /hw/disk/scsi, /distro/fedora, /env/debug Variant 2: /hw/cpu/intel, /hw/disk/scsi, /distro/fedora, /env/prod Variant 3: /hw/cpu/intel, /hw/disk/scsi, /distro/mint, /env/debug Variant 4: /hw/cpu/intel, /hw/disk/scsi, /distro/mint, /env/prod Variant 5: /hw/cpu/intel, /hw/disk/virtio, /distro/fedora, /env/debug Variant 6: /hw/cpu/intel, /hw/disk/virtio, /distro/fedora, /env/prod Variant 7: /hw/cpu/intel, /hw/disk/virtio, /distro/mint, /env/debug Variant 8: /hw/cpu/intel, /hw/disk/virtio, /distro/mint, /env/prod Variant 9: /hw/cpu/amd, /hw/disk/scsi, /distro/fedora, /env/debug Variant 10: /hw/cpu/amd, /hw/disk/scsi, /distro/fedora, /env/prod Variant 11: /hw/cpu/amd, /hw/disk/scsi, /distro/mint, /env/debug Variant 12: /hw/cpu/amd, /hw/disk/scsi, /distro/mint, /env/prod Variant 13: /hw/cpu/amd, /hw/disk/virtio, /distro/fedora, /env/debug Variant 14: /hw/cpu/amd, /hw/disk/virtio, /distro/fedora, /env/prod Variant 15: /hw/cpu/amd, /hw/disk/virtio, /distro/mint, /env/debug Variant 16: /hw/cpu/amd, /hw/disk/virtio, /distro/mint, /env/prod Variant 17: /hw/cpu/arm, /hw/disk/scsi, /distro/fedora, /env/debug Variant 18: /hw/cpu/arm, /hw/disk/scsi, /distro/fedora, /env/prod Variant 19: /hw/cpu/arm, /hw/disk/scsi, /distro/mint, /env/debug Variant 20: /hw/cpu/arm, /hw/disk/scsi, /distro/mint, /env/prod Variant 21: /hw/cpu/arm, /hw/disk/virtio, /distro/fedora, /env/debug Variant 22: /hw/cpu/arm, /hw/disk/virtio, /distro/fedora, /env/prod Variant 23: /hw/cpu/arm, /hw/disk/virtio, /distro/mint, /env/debug Variant 24: /hw/cpu/arm, /hw/disk/virtio, /distro/mint, /env/prod The tree representation also changed to print the `!mux` points of branching, to further help the user to understand the structure of the tree:: $ scripts/avocado multiplex --tree examples/mux-environment.yaml Config file tree structure: /-intel | /cpu-<>--amd | | /hw \-arm | | | | /-scsi | \disk-<> | \-virtio -| | /-fedora |-distro-<> | \-mint | | /-debug \env-<> \-prod The more detailed breakdown of the 145 commits can be seen here: https://github.com/avocado-framework/avocado/compare/0.23.0...0.24.0 Happy testing!
Hello everyone! I'm here to announce the new Avocado 0.23.0 release! This time around, we have a few changes worth of note: - Now avocado looks more like nose and other unittest based test runners: You can define a number of test methods prefixed with `test`, and the runner will simply run them. Also, for compatibility with the base unittest class, now `setup` and `cleanup` are `setUp` and `tearDown`. Let's see how an old avocado test looked like: !/usr/bin/python from avocado import test from avocado import job class OldTest(test.Test): def setup(self): self.hello = "Hi there!" def action(self): self.assertEqual(self.hello, "Hi there!") def cleanup(self): self.log.debug('Cleanup') if __name__ == '__main__': job.main() The new avocado tests look like this: !/usr/bin/python from avocado import test from avocado import job class MultipleTests(test.Test): def setUp(self): self.hello = "Hi there!" def test_hello(self): self.assertEqual(self.hello, "Hi there!") def testIdentity(self): self.assertTrue(1, 1) def tearDown(self): self.log.debug('Cleanup') if __name__ == '__main__': job.main() - `avocado run`: Simplified command line options, cutting off a number of rarely used options and moving them to config file entries - `avocado run`: Added the `--job-timeout` option, that allows users to specify the maximum time an avocado test job might take. - `avocado run`: Consolidated the options `--remote-timeout` and `--vm-timeout` into `--job-timeout`. - Sysinfo feature: The sysinfo feature went through refactoring and simplification - Now it is possible to specify what information from the system is to be collected through config files See how to configure your collectibles: [sysinfo.collectibles] File with list of commands that will be executed and have their output collected commands = /etc/avocado/sysinfo/commands File with list of files that will be collected verbatim files = /etc/avocado/sysinfo/files File with list of commands that will run alongside the job/test profilers = /etc/avocado/sysinfo/profilers The default `commands` file looks like: df -mP dmesg -c uname -a lspci -vvnn gcc --version ld --version mount hostname uptime dmidecode ifconfig -a brctl show ip link numactl --hardware show lscpu fdisk -l - Multiplexer: A new params retrieval system is in place. Now, instead of `self.params.key` you must use the `self.params.get(key, path, default)` API. A compatibility layer with the old dict structure was made, but we encourage you to migrate your param retrieval calls from `self.params.key` or `self.params['key'] to`self.params.get('key'). The full list of commits can be seen here: https://github.com/avocado-framework/avocado/compare/0.21.0...0.23.0 Happy testing!
- Avocado is now compatible with python 2.6 and distros that ship 2.6 as their default runtime, such as RHEL 6 and CentOS 6. Our CI environment was updated to test under 2.6 and 2.7 targets - Avocado rest client application (lets you talk to the avocado server application) - A new option was added to disable colored output in the config file, runner.output.colored - A new option was added to add commands that will profile new test job runs, sysinfo.collect.profiler_commands - The documentation was updated with GDB instrumentation examples (besides the normal docs update on new features/patchsets) - API Change: avocado.utils.io -> avocado.utils.genio - API Change: avocado.utils.process.find_command -> avocado.utils.path.find_command - API Change: avocado.utils.process.CmdNotFoundError -> avocado.utils.path.CmdNotFoundError - BUGFIX: remote machine could get stuck waiting on human input on paginator - BUGFIX: Avocado crashes on certain test defined exceptions - BUGFIX: Internal HTML report links when using remote/vm plugins - BUGFIX: Job directory generated even when 'avocado run' had option validation errors - BUGFIX: When installing avocado as an RPM, some example tests stop working - BUGFIX: Avocado crashes if a remote test is missing The complete list with the 130 commits can be seen here: https://github.com/avocado-framework/avocado/compare/0.20.1...0.21.0
New features: avocado: - HTML job report plugin, that gives users the ability to generate a nice and good looking report of the testing activities. - --wrapper feature, that allows users to write scripts to wrap the execution of test commands. You can use it to profile your QEMU instance during the test with perf/ltrace, for example. - Support for the GDB remote protocol inside avocado. - A new config plugin, that allows users to see the current configuration keys set for their particular instances of avocado, and help you to modify values if you wish so. - Tweaks and documentation improvements in the requirements.txt file, and the split of the file into other requirements files, with more specific purpose. - Fix a deadlock that occurred when the main test code has an early abort. - Fix a temporary dir issue, that had potential security implications. avocado-virt plugin: - Introduction of QEMU command line templates, to make it easier for people to provide specific command lines to be run inside avocado virt specific tests. - Fix a bug in migration process end detection, the code is now more robust and reliable. avocado-server: - Introduction of software components and test environment data models, preparing the server for a big feature: Automated bisection of failed tests. infrastructure: - Avocado has now a web page, courtesy of github pages: http://avocado-framework.github.io/