Benchexec

Latest version: v3.21

Safety actively analyzes 629004 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 6 of 10

1.22

Not secure
- More robust handling of Ctrl+C in `benchexec`.
For example, output files are now always fully written, whereas previously pressing Ctrl+C at the wrong time could result in truncated files. A side effect of this is that if you call `benchexec.benchexec.BenchExec().start()` in own Python code, you must now add a signal handler for `SIGINT`. The same was already true for users of `RunExecutor`, this is now documented.
- Fix Ctrl+C for `benchexec` in container mode.
In BenchExec 1.21, one would need to press Ctrl+C twice to stop `benchexec`.
- Fix unreliable container mode on Python 3.7.
- Some robustness improvements and fixes of rare deadlocks.
- Decreased overhead of `benchexec` while runs are executing.

1.21

Not secure
This release contains only a few bug fixes:

- Forwarding signals to the benchmarked process (and thus, stopping runs via Ctrl+C), was broken on Python 2.
- If the freezer cgroup was available but mounted in a separate hierarchy, it was not used reliably as protection against fork bombs when killing processes.
- Since BenchExec 1.19, an exception would occur if a non-existing command was started in container mode.
- Since BenchExec 1.19, copying output files from a container would occur while subprocesses are still running and would be counted towards the walltime limit. This is fixed, although subprocesses will still be running if the freezer cgroup is not available (cf. 433).

1.20

Not secure
- If `benchexec --container` is used, all code that is part of the tool-info module (as well as all processes started by it) are now run in a separate container with the same layout and restrictions as the run container.
Note, however, that it is not the same container, so any modifications made by the tool-info module to files on disk are *not* visible in the runs!
The `test_tool_info` utility also has gained a parameter `--container` for testing how a tool-info module behaves in a container.
- Nested containers are now supported.
Due to a change to the internal implementation of the container mode, commands like the following succeed now:
`containerexec -- containerexec --hidden-dir /sys -- /bin/bash`.
(Some parts of `/sys` need to be excluded because of kernel limitations.)
Note that nesting `runexec` or `benchexec` is still not supported, because nested cgroups are not implemented, so any cgroup-related features (resource limitations and measurements) are missing. But nesting `containerexec` and `runexec --container` (or vice-versa) now works.
- `/etc/hostname` in container now also shows the container's host name that exists since BenchExec 1.19.
- Change how CPUs with several NUMA nodes per CPU are handled:
BenchExec will now treat each NUMA node like a separate CPU package and avoid creating runs that span several NUMA nodes. Thanks alohamora!

1.19

Not secure
- In container mode, all temp directories are now on a `tmpfs` "RAM disk".
This affects everything written to directories in the hidden or overlay modes. Files written there are now included in the memory measurements and the memory limit! The advantage is that performance should be more deterministic, especially if several runs use much I/O in parallel. This feature can be disabled with `--no-tmpfs`.
- `/dev/shm` and `/run/shm` are now available inside the container and provide a `tmpfs` instance (even with `--no-tmpfs`) as required by some tools for shared memory.
- Container mode now recommends [LXCFS](https://github.com/lxc/lxcfs) and automatically uses it if available for a better container isolation (e.g., uptime measures container uptime, not host uptime). On Debian/Ubuntu, just use `sudo apt install lxcfs`.
- Several small bug fixes and other improvements of isolation for container mode (e.g., host name in container is no longer the real host name).
- Add `benchexec --no-hyperthreading`, which restricts core assignments to a single virtual core per physical CPU core (all other sibling cores will stay unused). Thanks alohamora!

1.18

Not secure
- Add result `done` that tools can output if the standard results `true`/`false`/`unknown`
are not applicable (for example because no property was checked),
and the run completed successfully.
- In container mode, `--keep-system-config` is no longer necessary if overlayfs
is not used for `/etc`, and thus it is is no longer automatically implied in such cases.
- Benchmark definitions support a new attribute `displayName` with a human-readable name
that will be shown in tables.
- A new variable `${taskdef_name}` can now be used in places where variable substitution is supported.
- Table-generator supports `%` as unit for numerical values.
- Some improvements for score handling outside of SV-COMP (i.e., if scores are not calculated by BenchExec).
- New tool-info modules for Test-Comp'19
- Several small bug fixes and improvements

1.17

Not secure
- Tasks can now be defined in a YAML-based format, cf. [the documentation](https://github.com/sosy-lab/benchexec/blob/master/doc/benchexec.md#task-definition-files).
This supports tasks with several input files, and allows providing metadata such as expected verdicts
in a structured format instead of encoded in the file name.
The format will be extended to handle more information in the future.
- The wall-time limit can now be specified separately from the CPU-time limit for `benchexec` as command-line parameter or in the benchmark definition.
- Support for SV-COMP'19 property `memcleanup`.
- In containers, properly handle `/run/systemd/resolve`, which is necessary for DNS resolution on systems with `systemd-resolved`.
- Avoid warnings for mountpoints below inaccessible directories in containers.
- Improvements for handling `NaN` and `Inf` values in `table-generator`.
- Log output of BenchExec will now have colors if `coloredlogs` is installed.
- New tool-info modules and updates for SV-COMP'19.

Page 6 of 10

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.