Easybuild

Detailed release notes available at https://easybuild.readthedocs.io/en/latest/Release_notes.html#easybuild-v4-3-0-september-13th-2020)

Detailed release notes available at https://easybuild.readthedocs.io/en/latest/Release_notes.html#easybuild-v4-2-2-july-8th-2020

Detailed release notes available at https://easybuild.readthedocs.io/en/latest/Release_notes.html#easybuild-v4-2-1-may-20th-2020

EasyBuild v4.2.0 is primarily a feature/update release, but also includes various bug fixes.

Highlights of the changes in this release include:

* (experimental) support for `--try-update-deps`, to upgrade dependencies based on available easyconfigs;

* a locking mechanism to prevent two simultaneous installations of the same easyconfig file to the same installation path (see https://easybuild.readthedocs.io/en/latest/Locks.html for more information);

* significant speedup for `-D`/`--dry-run` by avoiding useless '`module show`' calls;

* support for creating an index & using it to speed up searching for easyconfigs (see https://easybuild.readthedocs.io/en/latest/Easyconfigs_index.html for more information);

* additional GitHub integration features:
- support for targeting the easyblocks and framework repositories with `eb --new-pr` (via `--pr-target-repo`)
- support for including easyblocks from an (open) pull request via `--include-easyblocks-from-pr`
- for more information, https://easybuild.readthedocs.io/en/latest/Integration_with_GitHub.html

* probing of external modules for missing metadata that is not provided via an external module metadata file (mostly relevant for integration with the Cray Programming Environment);

* leveraging the [`archspec`](https://github.com/archspec/archspec) Python library (if it is installed) to show the "code name" of the processor in your system in the output of "`eb --show-system-info`" and in test reports uploaded to PRs via `--upload-test-report`;

* support for using templates like `%(cudaver)s` for the CUDA version (if it's included as direct dependency);

* fix for `UnicodeDecodeError` that was raised when reading a patch file that contained non-ASCII characters (if EasyBuild was being run with Python 3);

* fixed filtering of already installed extensions under `--skip` (mostly relevant for Python packages installed as bundles of extensions);

* also support `%(installdir)s` and `%(builddir)s` templates for extensions;

* take into account toolchain hierarchy when dumping easyconfig files, to avoid hardcoded subtoolchains in '`*dependencies`' easyconfig parameters (relevant when using `--try-*` options);

* take into account dependencies marked as external modules when composing template values like `%(pyver)s`;

* deprecate `rmtree2` function provided by `easybuild.tools.filetools` (the `remove_dir` function should be used instead);

* 7 new software-specific easyblocks: BerkeleyGW, CMake, ELSI, LAMMPS, libdrm, Mesa, SEPP;

* updates, enhancements & bug fixes for the `CMakeMake`, `PythonPackage` and `PythonBundle` generic easyblocks;

* updates, enhancements & bug fixes in the software-specific easyblocks for Clang, Ferret, Hypre, impi, libxml2, MATLAB, numexpr, OpenBabel, OpenBLAS, OpenCV, OpenFOAM, ParMETIS, PETSc, Python, QuantumESPRESSO, Qt, ROOT, Siesta, tbb, TensorFlow, TensorRT, Trinity, VMD, XCrySDen;

* added support for no less than 114 (!) new software packages, incl. ADIOS, ArviZ, Bonito, ELSI, FuSeq, GraphMap2, LAMMPS, MedPy, motionSegmentation, OpenAI-Gym, PyMC3, PyRe, PyTorch-Geometric, RAxML-NG, Shapely, STEAK, UQTk, Zip, ...;

* updates for already supported software, incl. Arrow 0.16.0, BUSCO 4.0.5, Clang 10.0.0, ELPA 2019.11.001, Ferret 7.5.0, GCC(core) 9.3.0, GROMACS 2020, GTK+ 3.24.13, Hypre 2.18.2, IQ-TREE 1.6.12, Java 13(.0.2), LLVM 9.0.1 + 10.0.0, likwid 5.0.1, medaka 0.12.0, numba 0.47.0, Octave 5.1.0, OpenBLAS 0.3.8, OpenBabel 3.0.0, OpenCV 4.2.0, OpenFOAM-Extend 4.1-20191120, PETSc 3.12.4, PyTorch 1.4.0, Qhull 2019.1, QuantumESPRESSO 6.5, R-bundle-Bioconductor 3.10, Racon 1.4.10, ReFrame 2.21, SLEPc 3.12.2, SimpleElastix 1.1.0, SimpleITK 1.2.4, TensorFlow 1.15.2 + 2.0.1, Trinity 2.10.0, XGBoost 0.90, xarray 0.15.1, ...;

* update Java/1.8 wrapper to Java/1.8.0_241;

* enhanced Java/11 wrapper to support both x86_64 and POWER;

* fixed source URL for ISL component in `GCCcore` easyconfigs;

* checksum fixes in easyconfigs for R 3.6.0 and 3.6.2;

* revert removal of AVX512 vmovd with 64-bit operands in binutils 2.32 easyconfigs;

* fixed packaging issue that resulted in CVS easyconfigs to be missing from EasyBuild releases;

* updated checksum/homepage in tbb easyconfigs after move to '`oneapi-src`' repository;

* various additional updates, enhancements & bug fixes;

This brings the total number of supported software packages to 1,912!
An up-to-date list of supported software is available at https://easybuild.readthedocs.io/en/latest/version-specific/Supported_software.html .

A detailed overview of all changes, which links to the respective pull requests, is available in the release notes at https://easybuild.readthedocs.io/en/latest/Release_notes.html#easybuild-v4-2-0-april-14th-2020 .

These changes result from various contributions, made by 26 different contributors:

* 56 merged pull requests for EasyBuild framework
* 63 merged pull requests for easyblocks
* 381 merged pull requests for easyconfigs

Thanks to everyone who contributed to this release in one way or another!

This release fixes a problem where the GitHub token that EasyBuild uses for some of the GitHub integration features (like `--from-pr`, `--new-pr`, etc.) got included in plain text in the 'top-level' EasyBuild log file when the `--debug` configuration option is enabled, potentially leaving it exposed to be used by others.

**We strongly encourage that you revoke the GitHub tokens you are using currently, via https://github.com/settings/tokens, and to replace them using a new token (using "eb --install-github-token --force").**

More information is available in the [security advisory that was published](https://github.com/easybuilders/easybuild-framework/security/advisories/GHSA-2wx6-wc87-rmjm).

Detailed release notes at https://easybuild.readthedocs.io/en/latest/Release_notes.html#easybuild-v4-1-2-march-16th-2020 .

This release is primarily a bug fix/update release, but it also includes a couple of minor enhancements.

Highlights of the changes in EasyBuild v4.1.1 include:

* support for "`eb --show-ec`" to show the contents of the specified easyconfig file, and "`eb --copy-ec`" to copy an easyconfig file to a specified location;

* a new `--cuda-compute-capabilities` configuration option, which can be used to specify with which CUDA compute capabilities GPU software should be built (for now this is only relevant for TensorFlow);

* various fixes for permission issues that occurred when (re-)installing software with EasyBuild using multiple different users;

* avoiding needless warnings that commonly arise in EasyBuild log files;

* one new software-specific easyblock for the cryptography Python package;

* various fixes for installing software on POWER systems;

* making the TensorFlow easyblock print a clear warning when no custom CUDA compute capabilities are specified (either via the new `--cuda-compute-capabilities` configuration option, or via the TensorFlow-specific `cuda_compute_capabilities` easyconfig parameter);

- by default recent TensorFlow versions only build with the 3.5 and 7.0 CUDA compute capabilities, which implies potentially significant performance loss on systems using NVIDIA Tesla GPUs like P100;

* updates, enhancements & bug fixes for the `Bundle`, `CMakeMake`, `conda`, `PythonPackage` and `VersionIndependentPythonPackage` generic easyblocks;

* updates, enhancements & bug fixes for the software-specific easyblocks for Bazel, CUDA, ESMF, FSL, GCC, GROMACS, HDF5, iccifort, imkl, impi, MATLAB, MRtrix, NCL, netCDF, OpenFOAM, Python, SAMtools, SWIG, tbb, TensorFlow, WPS, and WRF;

* fixed problem that occurred when installing EasyBuild with EasyBuild running on top of Python 3;

* added support for 27 new software packages, incl. breseq, CrossMap, horton, LEMON, OpenSlide, pythran, Qualimap, ...;

* updates for already supported software, incl. Beast 1.10.4, Clang 9.0.1, ESMF 8.0.0, FSL 6.0.3, GDAL 3.0.2, GEOS 3.8.0, GROMACS 2019.4, GSL 2.6, Julia 1.3.1, mayavi 4.7.1, molmod 1.4.5, netCDF-C++4 4.3.1, netCDF-Fortran 4.5.2, OpenFOAM 7, OpenFOAM v1912, OpenMM 7.4.1, OpenMPI 4.0.2, PLUMED 2.5.3, PROJ 6.2.1, pocl 1.4, QuickFF 2.2.4, R 3.6.2 w/ foss/2019b and fosscuda/2019b, ReFrame 2.20, SAMtools 1.10, SUNDIALS 5.1.0, Salmon 1.0.0, SuiteSparse 5.6.0, TensorFlow 2.1.0 w/ fosscuda/2019b, WPS 4.1, WRF 4.1.3, ...;

* added easyconfigs for intel/2020.00 toolchain;

* added missing wcwidth extension to Python 2.7.15 + 2.7.16 easyconfigs;

* updated Java/1.8 wrapper to Java/1.8.0_231 (for x86_64);

* enable building of CMake in parallel;

* actually use Java dependency when building/running Bazel (rather the JDK included with Bazel);

* fix source URLs in various easyconfigs, incl. ant, libsodium, libxc, Mesa, QCA, xorg-macros, ...;

* add missing extensions in easyconfigs for cyvcf2 and pyfits;

* add alternate valid SHA256 checksums for kallisto 0.43.1 + MASS 7.3-51.4 & rda 1.0.2-2.1 extensions in R 3.6.0 easyconfigs;

* make sure we use provided Clang dependency for building pocl;

* require that sanity_pip_check is enabled in easyconfigs for Python packages that are touched by pull requests;

* use True (boolean value) rather than 'True' (string value) for boolean easyconfig parameters;

* various additional updates, enhancements & bug fixes;

This brings the total number of supported software packages to 1,798!
An up-to-date list of supported software is available at https://easybuild.readthedocs.io/en/latest/version-specific/Supported_software.html .

A detailed overview of all changes, which links to the respective pull requests, is available in the release notes at https://easybuild.readthedocs.io/en/latest/Release_notes.html .


These changes result from various contributions, made by 22 different contributors:

- 27 merged pull requests for EasyBuild framework
- 47 merged pull requests for easyblocks
- 197 merged pull requests for easyconfigs

Thanks to everyone who contributed to this release in one way or another!