Cupy

This is the release note of v7.0.0rc1. See [here](https://github.com/cupy/cupy/milestone/60?closed=1) for the complete list of solved issues and merged PRs.

Announcements

This time, we will keep the current branches for active development (`master` for v7.x, `v6` for v6.x) after the RC. We will maintain v6.x series until Python2 EOL, so we do not cut the new development version for now to avoid increasing the number of branches to maintain. New features will be included directly into v7 for a while, and maintenance changes will be backported to v6.

Highlights

- Experimental support of AMD GPUs are added (1094). See [the installation guide](https://docs-cupy.chainer.org/en/latest/install_rocm.html) for how to install CuPy with AMD support. Note that this feature is still experimental, and we do not guarantee the API stability.

Changes without compatibility

- Avoid casting inputs to `cupy.ndarray` in `cupy.pad` (2504)
- From this release, `cupy.pad` does no longer convert the input to `cupy.ndarray` automatically. This is done under the design principle of not implicitly synchronizing the host and the device, which most of the other APIs are also following.

New Features

- Experimental support of AMD GPU via HIP (ROCm2.7.0+) (1094)
- Adds `nvcc` as a `RawKernel` backend (1941, thanks sjperkins and leofang!)
- Support cuTENSOR 0.2 (2341)
- Implement `isin` and `in1d` (2388, thanks UmashankarTriforce!)
- Support `scipy.ndimage` compatible `convolve` and `correlate` (2483)
- Added `cupy.cuda.memory.get_allocator` interface (2489)
- Handle PCI bus ID (2531, thanks jameshclrk!)
- Expand coverage of cuSolverSP APIs (2539)
- Add cuSPARSE routines for preconditioners (2542)

Enhancements

- Fix division by zero in mean/std/var functions for 0-length dimensions (2201, thanks pentschev!)
- Improve error message in `cupy.linalg.inv` (2342)
- Replace `cupy.pad` with a heavily refactored version from NumPy 1.17 (2399, thanks grlee77!)
- Fix `cupy.repeat` error message about `repeats` argument type (2400)
- Ignore warning caused by `fastrlock` (2488)
- Update `__cuda_array_interface__` to protocol version 2 (2491, thanks leofang!)
- Allow `axis=None` in concatenate (2496, thanks liwt31!)
- Fix `testing.numpy_cupy_` decorators for skips (2498)
- Avoid implicit cast inputs to `cupy.ndarray` in `cupy.pad` (2504)
- Cholesky decomposition to support complex values (2509)
- Enhance shuffle-test of `testing.for_dtypes_combination` (2511)
- Allow to use `real` and `imag` on CUDA kernels (2520)
- Support complex numbers in `cupy.linalg.qr()` (2526, thanks leofang!)
- Fix bug in CUB + Native support of complex numbers in CUB (2538, thanks leofang!)
- Support `cupyx.fallback_mode` as an experimental feature (2541)
- Support stream in CUB (2555, thanks leofang!)

Performance Improvements

- Performance improvement for `cupy.var` complex inputs (2484)
- Enable fast CUB-based reductions in more cases (`cupy.linalg.norm`, etc.) (2517, thanks grlee77!)

Code Fixes

- Remove unnecessary check of cuSOLVER (2529, thanks grlee77!)

Documentation

- Fix dead links in NumPy docs in random functions (2384)
- Update `install_rocm.rst` (2512)
- Fix some typo (2523, thanks garanews!)

Tests

- Fix memory pool disabled during tests (2452)
- Skip bool-bool inputs in `cupy.cross` test (2503)
- Fix error in `test_build.py` (2514, thanks leofang!)
- Move CI requirements to CuPy repository (2533)
- Fix for NumPy 1.14.x compatibility (2544)
- Workaround bug in NumPy 1.12.x or earlier (2545)

This is the release note of v7.0.0b4. See [here](https://github.com/cupy/cupy/milestone/57?closed=1) for the complete list of solved issues and merged PRs.

New Features

- Add `cupy.cross` (2366, thanks UmashankarTriforce!)
- Add `cupy.may_share_memory` (2417)
- Add occupancy driver APIs (2424, thanks leofang!)
- Add `scatter_max` and `scatter_min` (2427)
- Support texture memory in `RawKernel` (2432, thanks leofang!)

Enhancements

- Extend `CuDNNError` to have more debugging information (2404)
- Add complex dtype support to `cupy.std` and `cupy.var` (2411, thanks grlee77!)
- Support complex dtypes in `cupy.linalg.inv()` (2468, thanks leofang!)

Performance Improvements

- Use CUB to speed up `sum`/`min`/`max` (2090)
- Avoid creating empty numpy ndarray in `common_type` (2307)
- `cupy.linalg.norm`: update docstring and improve performance for `ord=2` cases (2479, thanks grlee77!)

Bug Fixes

- Fix bug in `coosort` (2410, thanks econtal!)
- Fix multithreading issue in `cupy.cuda.cufft.get_current_plan()` (2435, thanks leofang!)
- Normalize hidden layer strides in cuDNN RNN (2442)
- Copy inputs in ufunc if they share the same memory with outputs (2460)

Code Fixes

- Fix cuSOLVER `devInfo` dtype in `inv` and unify how those are specified (2454)
- Fix code convention of statistics routines (2459)
- Remove unnecessary `ndarray` private methods (2465)

Documentation

- Document `CUTENSOR_PATH` environment variable (2386)
- Add appropriate pointer types in Cython to the contribute guide (2455, thanks leofang!)
- Fix invalid escape sequence (2470)

Installation

- Fix NumPy version in Dockerfile (2430)

Examples

- Broadcast `ValueError` for `n-clusters > 2` in k-means example (2453, thanks casheera!)

Tests

- Simplify `cupy.fuse` tests (2339)
- Revert "Skip some ndarray-elementwise-op tests as temporary fix" (2383)
- Drop Python 2 Travis CI configuration (2428)
- Drop Python 2 PFN CI configuration (2429)
- Change URL to place test assets (2434)
- Add NumPy 1.17.1 to skip list of 0-length `ifft` test (2441)
- Remove unnecessary skip in 0-length `ifft` test (2443)
- Fix `TestDLTensorMemory.test_delete` (2451)
- Fix data race in advanced indexing test (2472)
- Fix `pytest` 5.x version errors (2473)
- Remove global state from tests (2475, thanks leofang!)

This is the release note of v7.0.0b3. See [here](https://github.com/cupy/cupy/milestone/55?closed=1) for the complete list of solved issues and merged PRs.

Highlights

`cupy.RawModule` has been introduced to allow users access low-level features (CUDA modules).

Dropping Support of Python 2

Due to the end-of-life (EOL) of Python 2 in January 2020, Python 2 support has been dropped in this release. CuPy v6.x continues to support Python 2. See the [blog post](https://chainer.org/announcement/2019/08/21/python2.html) for details.

New Features

- Add `cupy.nanargmin` and `cupy.nanargmax` (2222, thanks harshalchaudhari35!)
- Support cuDNN Fused Ops API (2246)
- Add `fallback_mode.ndarray` (2272, thanks Piyush-555!)
- Add NCCL broadcast (2303)
- Add `take_along_axis` (2314)
- Add `cupy.nanmean` (2319, thanks Piyush-555!)
- Add more cuSolver APIs for dense linear solver (2320)
- Provide full coverage for NCCL APIs in CuPy (2325, thanks leofang!)
- Add `cupy.nanvar` and `cupy.nanstd` (2344, thanks Piyush-555!)
- Allow getting and setting CUDA kernel attributes (2369, thanks leofang, grlee77 and andravin!)
- Add `cupy.RawModule()` (2389, thanks leofang!)
- Add `diagonal` method for `cupyx.scipy.sparse.dia_matrix` (2398, thanks grlee77!)

Enhancements

- Support for shape argument in `*_like` functions (2171, thanks pentschev!)
- Support batched matrix inverse (2300)
- Add wrapper for `numpy.vectorize` in `fallback_mode` (2350, thanks Piyush-555!)
- Allow cuFFT plans to be used as a context manager; Set stream before executing cuFFT plans (2362, thanks leofang!)
- Raise `DeprecationWarning` on 0-dim arrays in `numpy.nonzero` to match NumPy 1.17 behavior (2394)
- Change check condition of `accept_error` (2396)

Bug Fixes

- Make exceptions picklable (2318)
- Make `cupy.tensordot` use Tensor Core also in case of compute-capability > 70 (2328)

Code Fixes

- Fix unused imports and import orders (2312)
- Fix variable names in `tri` kernel (2326)
- Some simplifications using `isnan` (2364)
- Small fix of `fusion.pyx` and docs (2393, thanks xuzijian629!)

Documentation

- Add new functions to API reference (2308)
- Fix for `linalg.svd` documentation (2321, thanks IvanYashchuk!)
- Fix markup in `linalg.svd` docs (2323)
- Fix doctest failure related to `__array_function__` support (2352)
- Add 1.17 to supported NumPy versions (2368)

Tests

- Make wheel of master for CI (2144)
- Add ChainerCV's tests to pfnCI (2186)
- Fix tests of `linalg.svd` (2338)
- Check return type in `test_type_routines.py` (2358)
- Skip some FFT tests on NumPy 1.17.0 due to a NumPy bug (2363, thanks grlee77!)
- Remove `random.power` test with forbidden value (2375)
- Skip some tests in `TestArrayElementwiseOp` as temporary fix (2376)
- Avoid using `numpy.nonzero` in `_make_decorator` (2385)
- Increase numbers of retries for K-S tests (2397)

Others

- Drop support for Python 2.7 and 3.4 (2343)

This is the release note of v7.0.0b2. See [here](https://github.com/cupy/cupy/milestone/54?closed=1) for the complete list of solved issues and merged PRs.

Highlights

- `cupy.cutensor` has been introduced that wraps [cuTENSOR](https://developer.nvidia.com/gtc/2019/video/S9593), allowing high-performance tensor operations. Examples are available [here](https://github.com/cupy/cupy/tree/master/examples/cutensor).

Changes without compatibility

- `cupy.load` now specifies `allow_pickle=False` by default to follow the security fix made in NumPy 1.16.3 (see [numpy/numpy 13359](https://github.com/numpy/numpy/pull/13359) and [cupy/cupy #2290](2290) for details). Most users should not be affected by this change; users loading `ndarray` serialized using pickle may need to explicitly specify `allow_pickle=True`.

New Features

- Support cuTENSOR (2210)
- Add `nansum` and `nanprod` support (2252, thanks pentschev!)

Enhancements

- Raise error when no available algorithm found by `cudnnFindConvolution*` (2234)
- Add NHWC layout support to batch normalization (2235)
- Remove unused code (2255)
- Support non-square matrices in `lu_factor` (2286, thanks econtal!)
- Remove `cupy.ndarray.{nansum/nanprod}` (2292)
- Remove warning about nvcc absence (2299)

Performance Improvements

- Add support for merge path algorithm (`csrmvEx`) when `csr_matrix` multiply with a dense vector (2287, thanks wonghang!)

Bug Fixes

- Add wrappers for `can_cast`, `common_type` and `result_type` functions (2249, thanks pentschev!)
- Make `__cuda_array_interface__()['strides']` be `tuple` (2260, thanks leofang!)
- Avoid `CUDNN_STATUS_BAD_PARAMS` (2261, thanks himkt!)
- Support `allow_pickle` in `cupy.load` and `cupy.save` (2290)

Documentation

- Update installation guide (2184)
- Document interoperability with `mpi4py` (2270, thanks leofang!)

Installation

- Make nvcc code generation target configure by env var (2293)

Tests

- Fix test failure when cuDNN is unavailable (2284)
- Enhance tests for type routines (2306)

This is the release note of v7.0.0b1. See [here](https://github.com/cupy/cupy/milestone/52?closed=1) for the complete list of solved issues and merged PRs.

Highlights

Host to device copy from NumPy ndarrays is now allowed as an experimental feature with the syntax `cupy_array[:] = numpy_array`. Set the environment variable `CUPY_EXPERIMENTAL_SLICE_COPY` to try it out.

Notes

- Tensor Core in cuDNN convolution is tentatively disabled for Turing GPUs due to some test failures. The issue is under investigation and hopefully fixed in a future version.

New Features

- Allow copying in the format `cupy_array[:] = numpy_array` (2079, thanks pentschev!)
- Add `linalg.lstsq` (2165, thanks cjekel!)
- Add `fallback_mode` (2229, thanks Piyush-555!)
- Add `CUDNN_POOLING_MAX_DETERMINISTIC` (2239)
- Add API to retrieve installation info (2245)

Enhancements

- Improve `atomicAdd` in histogram and sample (1345)
- Set current device in `cupy.ndarray.get()`/`set()` (2169, thanks hyabe!)
- Improve out-of-memory error message (2242)
- Simplify `_prepare_multiple_array_indexing` (2254)
- Fix `_preprocess_args` to avoid calling `hasattr` (2256)
- Add `constant` modification to `vector_equal` (2257)

Performance Improvements

- Optimize the initialization of `List[ndarray]` in `cupy.array` (2081)

Bug Fixes

- Revert "Fix usage check of Tensor Core" (2197)
- Fix `power` for large integrals (2204)
- Avoid division by zero in `tensordot`, allowing 0-length arrays (2209, thanks pentschev!)
- Make `RandomState.permutation` compatible with `random.permutation` (2250)

Code Fixes

- Coding style fix (2211)

Documentation

- Add `cupy-cuda101` to README (2196)
- Fix: duplicate object description of cupy (2233)

Installation

- Add setup option to copy include files in wheel (2208)
- Bump version to v7.0.0b1 (2266)

Tests

- Ignore invalid axis type test in NumPy 1.12.x or earlier (2192)

Others

- Fix `array_split` with non-equally dividing sections (2207)

This is the release note of v7.0.0a1. See [here](https://github.com/cupy/cupy/milestone/50?closed=1) for the complete list of solved issues and merged PRs.

Highlights

* CuPy memory pool now supports setting hard-limit quota for the amount of GPU memory allocated. Refer to the [reference](https://docs-cupy.chainer.org/en/latest/reference/memory.html#limiting-gpu-memory-usage) for the details.

New Features

- Support cuDNN CTC functions (1769, thanks aonotas!)
- Support NHWC format in convolution (1885)
- Add `hostRegister` and `hostUnregister` (2102)
- Implement limit to memory pool (2113)
- Add `strides_check` option in array testing functions (2150)

Enhancements

- Fix `ascontiguousarray` with 0-dim array input (2078)
- Emit kernel names with type names (2151)
- Support complex dtypes in `cupy.where` (2175, thanks AntoineDujardin!)

Bug Fixes

- Fix `__cuda_array_interface__` data pointer for sliced arrays (2129, thanks pentschev!)
- Fix usage check of Tensor Core (2168)
- Avoid using Tensor Core with cuDNN deterministic mode in convolution backward (2174)

Code Fixes

- Avoid unnecessary weak pointer for null stream (1539)
- Avoid PyThread in `stream.pyx` (1945)
- Add a comment to the testing condition of `einsum` (2131)
- Fix style (2167)

Documentation

- Add upgrade guide for v6 (2182)

Installation

- Fix compile error on CUDA 10.1 and GCC 7 or 8 (2147, thanks grafi-tt!)

Examples

- Make k-means example's custom kernels simpler (2145)
- Make k-means sample code cleaner for educational purpose (2146)

Tests

- Fix testing condition of `diff` and `unwrap` (2124)
- Add a test for `assert_array_equal(strides_check=True)` (2156)
- Fix test failure when cudnn is unavailable (2161)