This release does not add new features compared to BenchExec 1.22, but removes several deprecated features and brings several other backwards-incompatible changes to make BenchExec more consistent and user-friendly:
- Support for Python 3.2 and 3.3 is removed, the minimal Python version is now 3.4.
Additionally, `runexec`/`RunExecutor` continue to support Python 2.7 [until end of 2019](https://github.com/sosy-lab/benchexec/issues/438).
- Support for running benchmarks as a different user with `sudo` is removed (parameters `--user`/`--users`).
Use container mode as better method for isolating runs.
- [Container mode](doc/container.md) is enabled by default.
It can be disabled with `--no-container`, but this decreases reliability of benchmarking.
- If the `cpuacct` cgroup is not available, CPU-time measurements and limits are not supported.
- Either container mode or the `freezer` cgroup are required to ensure protection against fork bombs.
- [Niceness](http://man7.org/linux/man-pages/man2/nice.2.html) of benchmarked process is not changed, previously it was increased by 5.
- Changes to input of `benchexec`:
- The memory limit given to `benchexec` requires an explicitly specified unit.
- Support for `<test>` tags, `<sourcefiles>` tags, and variables named `${sourcefile_*}` removed from benchmark definitions.
Use `<rundefinition>`, `<tasks>`, and `${inputfile_*}` instead.
- Variables named `${taskdef_*}` are defined only if task-definition files are used, and variables named `${inputfile_*}` only otherwise.
- Changes to `table-generator`:
- A column named `memUsage` is automatically renamed to `memory`.
- A column named `memory` is automatically converted to Megabytes.
Both conversions are only applied if no `<column>` tags are used.
- Changes to [run-result data](doc/run-results.md):
- In case of aborted or failed runs, no dummy results (e.g., `cputime` of 0s) are present.
- The memory results of `benchexec` are named `memory`, not `memUsage`.
- Memory results have the unit `B` explicitly specified.
Furthermore, units are present in all attributes of the result XML files where they were still missing.
- Result item `exitcode` is removed, only `RunExecutor.execute_run()` still returns it, but as an object instance instead of an `int`.
Use `returnvalue` and `exitsignal` instead.
- Module `benchexec.test_tool_wrapper` is removed, use `benchexec.test_tool_info` instead.
- BenchExec (both `benchexec` and `runexec`) terminates itself cleanly after aborting all runs if it receives one of the signals `SIGTERM`, `SIGINT` (Ctrl+C), or `SIGQUIT`.
Additionally, this release adds a fix for the container that is used since BenchExec 1.20 for the tool-info module. In this container, the environment variable `HOME` did not point to `/home/benchexec` as expected but to the user's real home directory. This broke tools like Ultimate if the `/home` was configured to be hidden or read-only.
Furthermore, we declare the following features deprecated and plan on removing them for [BenchExec 3.0](https://github.com/sosy-lab/benchexec/milestone/8), which is expected to be released in January 2020:
- Support for Python 2.7 and 3.4 (cf. [438](https://github.com/sosy-lab/benchexec/issues/438))
- Support for checking correctness of run results and computing scores if task-definition files are *not* used (cf. [439](https://github.com/sosy-lab/benchexec/issues/439))
Please respond in the respective issue if one of these deprecations is a problem for you.