Dask-cuda

🚨 Breaking Changes

- Introduce incompatible-types and enables spilling of CuPy arrays ([856](https://github.com/rapidsai/dask-cuda/pull/856)) [madsbk](https://github.com/madsbk)

🐛 Bug Fixes

- Resolve build issues / consistency with conda-forge packages ([883](https://github.com/rapidsai/dask-cuda/pull/883)) [charlesbluca](https://github.com/charlesbluca)
- Increase test_worker_force_spill_to_disk timeout ([857](https://github.com/rapidsai/dask-cuda/pull/857)) [pentschev](https://github.com/pentschev)

📖 Documentation

- Remove description from non-existing `--nprocs` CLI argument ([852](https://github.com/rapidsai/dask-cuda/pull/852)) [pentschev](https://github.com/pentschev)

🚀 New Features

- Add --pre-import/pre_import argument ([854](https://github.com/rapidsai/dask-cuda/pull/854)) [pentschev](https://github.com/pentschev)
- Remove support for UCX < 1.11.1 ([830](https://github.com/rapidsai/dask-cuda/pull/830)) [pentschev](https://github.com/pentschev)

🛠️ Improvements

- Raise `ImportError` when platform is not Linux ([885](https://github.com/rapidsai/dask-cuda/pull/885)) [pentschev](https://github.com/pentschev)
- Temporarily disable new `ops-bot` functionality ([880](https://github.com/rapidsai/dask-cuda/pull/880)) [ajschmidt8](https://github.com/ajschmidt8)
- Pin `dask` & `distributed` ([878](https://github.com/rapidsai/dask-cuda/pull/878)) [galipremsagar](https://github.com/galipremsagar)
- Upgrade min `dask` & `distributed` versions ([872](https://github.com/rapidsai/dask-cuda/pull/872)) [galipremsagar](https://github.com/galipremsagar)
- Add `.github/ops-bot.yaml` config file ([871](https://github.com/rapidsai/dask-cuda/pull/871)) [ajschmidt8](https://github.com/ajschmidt8)
- Make Dask CUDA work with the new WorkerMemoryManager abstraction ([870](https://github.com/rapidsai/dask-cuda/pull/870)) [shwina](https://github.com/shwina)
- Implement ProxifyHostFile.evict() ([862](https://github.com/rapidsai/dask-cuda/pull/862)) [madsbk](https://github.com/madsbk)
- Introduce incompatible-types and enables spilling of CuPy arrays ([856](https://github.com/rapidsai/dask-cuda/pull/856)) [madsbk](https://github.com/madsbk)
- Spill to disk clean up ([853](https://github.com/rapidsai/dask-cuda/pull/853)) [madsbk](https://github.com/madsbk)
- ProxyObject to support matrix multiplication ([849](https://github.com/rapidsai/dask-cuda/pull/849)) [madsbk](https://github.com/madsbk)
- Unpin max dask and distributed ([847](https://github.com/rapidsai/dask-cuda/pull/847)) [galipremsagar](https://github.com/galipremsagar)
- test_gds: skip if GDS is not available ([845](https://github.com/rapidsai/dask-cuda/pull/845)) [madsbk](https://github.com/madsbk)
- ProxyObject implement __array_function__ ([843](https://github.com/rapidsai/dask-cuda/pull/843)) [madsbk](https://github.com/madsbk)
- Add option to track RMM allocations ([842](https://github.com/rapidsai/dask-cuda/pull/842)) [shwina](https://github.com/shwina)

🐛 Bug Fixes

- Ignore `DeprecationWarning` from `distutils.Version` classes ([823](https://github.com/rapidsai/dask-cuda/pull/823)) [pentschev](https://github.com/pentschev)
- Handle explicitly disabled UCX transports ([820](https://github.com/rapidsai/dask-cuda/pull/820)) [pentschev](https://github.com/pentschev)
- Fix regex pattern to match to in test_on_demand_debug_info ([819](https://github.com/rapidsai/dask-cuda/pull/819)) [pentschev](https://github.com/pentschev)
- Fix skipping GDS test if cucim is not installed ([813](https://github.com/rapidsai/dask-cuda/pull/813)) [pentschev](https://github.com/pentschev)
- Unpin Dask and Distributed versions ([810](https://github.com/rapidsai/dask-cuda/pull/810)) [pentschev](https://github.com/pentschev)
- Update to UCX-Py 0.24 ([805](https://github.com/rapidsai/dask-cuda/pull/805)) [pentschev](https://github.com/pentschev)

📖 Documentation

- Fix Dask-CUDA version to 22.02 ([835](https://github.com/rapidsai/dask-cuda/pull/835)) [jakirkham](https://github.com/jakirkham)
- Merge branch-21.12 into branch-22.02 ([829](https://github.com/rapidsai/dask-cuda/pull/829)) [pentschev](https://github.com/pentschev)
- Clarify `LocalCUDACluster`&39;s `n_workers` docstrings ([812](https://github.com/rapidsai/dask-cuda/pull/812)) [pentschev](https://github.com/pentschev)

🚀 New Features

- Pin `dask` & `distributed` versions ([832](https://github.com/rapidsai/dask-cuda/pull/832)) [galipremsagar](https://github.com/galipremsagar)
- Expose rmm-maximum_pool_size argument ([827](https://github.com/rapidsai/dask-cuda/pull/827)) [VibhuJawa](https://github.com/VibhuJawa)
- Simplify UCX configs, permitting UCX_TLS=all ([792](https://github.com/rapidsai/dask-cuda/pull/792)) [pentschev](https://github.com/pentschev)

🛠️ Improvements

- Add avg and std calculation for time and throughput ([828](https://github.com/rapidsai/dask-cuda/pull/828)) [quasiben](https://github.com/quasiben)
- sizeof test: increase tolerance ([825](https://github.com/rapidsai/dask-cuda/pull/825)) [madsbk](https://github.com/madsbk)
- Query UCX-Py from gpuCI versioning service ([818](https://github.com/rapidsai/dask-cuda/pull/818)) [pentschev](https://github.com/pentschev)
- Standardize Distributed config separator in get_ucx_config ([806](https://github.com/rapidsai/dask-cuda/pull/806)) [pentschev](https://github.com/pentschev)
- Fixed `ProxyObject.__del__` to use the new Disk IO API from 791 ([802](https://github.com/rapidsai/dask-cuda/pull/802)) [madsbk](https://github.com/madsbk)
- GPUDirect Storage (GDS) support for spilling ([793](https://github.com/rapidsai/dask-cuda/pull/793)) [madsbk](https://github.com/madsbk)
- Disk IO interface ([791](https://github.com/rapidsai/dask-cuda/pull/791)) [madsbk](https://github.com/madsbk)

🐛 Bug Fixes

- Remove automatic `doc` labeler ([807](https://github.com/rapidsai/dask-cuda/pull/807)) [pentschev](https://github.com/pentschev)
- Add create_cuda_context UCX config from Distributed ([801](https://github.com/rapidsai/dask-cuda/pull/801)) [pentschev](https://github.com/pentschev)
- Ignore deprecation warnings from pkg_resources ([784](https://github.com/rapidsai/dask-cuda/pull/784)) [pentschev](https://github.com/pentschev)
- Fix parsing of device by UUID ([780](https://github.com/rapidsai/dask-cuda/pull/780)) [pentschev](https://github.com/pentschev)
- Avoid creating CUDA context in LocalCUDACluster parent process ([765](https://github.com/rapidsai/dask-cuda/pull/765)) [pentschev](https://github.com/pentschev)
- Remove gen_cluster spill tests ([758](https://github.com/rapidsai/dask-cuda/pull/758)) [pentschev](https://github.com/pentschev)
- Update memory_pause_fraction in test_spill ([757](https://github.com/rapidsai/dask-cuda/pull/757)) [pentschev](https://github.com/pentschev)

📖 Documentation

- Add troubleshooting page with PCI Bus ID issue description ([777](https://github.com/rapidsai/dask-cuda/pull/777)) [pentschev](https://github.com/pentschev)

🚀 New Features

- Handle UCX-Py FutureWarning on UCX < 1.11.1 deprecation ([799](https://github.com/rapidsai/dask-cuda/pull/799)) [pentschev](https://github.com/pentschev)
- Pin max `dask` & `distributed` versions ([794](https://github.com/rapidsai/dask-cuda/pull/794)) [galipremsagar](https://github.com/galipremsagar)
- Update to UCX-Py 0.23 ([752](https://github.com/rapidsai/dask-cuda/pull/752)) [pentschev](https://github.com/pentschev)

🛠️ Improvements

- Fix spill-to-disk triggered by Dask explicitly ([800](https://github.com/rapidsai/dask-cuda/pull/800)) [madsbk](https://github.com/madsbk)
- Fix Changelog Merge Conflicts for `branch-21.12` ([797](https://github.com/rapidsai/dask-cuda/pull/797)) [ajschmidt8](https://github.com/ajschmidt8)
- Use unittest.mock.patch for all os.environ tests ([787](https://github.com/rapidsai/dask-cuda/pull/787)) [pentschev](https://github.com/pentschev)
- Logging when RMM allocation fails ([782](https://github.com/rapidsai/dask-cuda/pull/782)) [madsbk](https://github.com/madsbk)
- Tally IDs instead of device buffers directly ([779](https://github.com/rapidsai/dask-cuda/pull/779)) [madsbk](https://github.com/madsbk)
- Avoid proxy object aliasing ([775](https://github.com/rapidsai/dask-cuda/pull/775)) [madsbk](https://github.com/madsbk)
- Test of sizeof proxy object ([774](https://github.com/rapidsai/dask-cuda/pull/774)) [madsbk](https://github.com/madsbk)
- gc.collect when spilling on demand ([771](https://github.com/rapidsai/dask-cuda/pull/771)) [madsbk](https://github.com/madsbk)
- Reenable explicit comms tests ([770](https://github.com/rapidsai/dask-cuda/pull/770)) [madsbk](https://github.com/madsbk)
- Simplify JIT-unspill and writing docs ([768](https://github.com/rapidsai/dask-cuda/pull/768)) [madsbk](https://github.com/madsbk)
- Increase CUDAWorker close timeout ([764](https://github.com/rapidsai/dask-cuda/pull/764)) [pentschev](https://github.com/pentschev)
- Ignore known but expected test warnings ([759](https://github.com/rapidsai/dask-cuda/pull/759)) [pentschev](https://github.com/pentschev)
- Spilling on demand ([756](https://github.com/rapidsai/dask-cuda/pull/756)) [madsbk](https://github.com/madsbk)
- Revert "Temporarily skipping some tests because of a bug in Dask ([753)" (754](https://github.com/rapidsai/dask-cuda/pull/753)" (#754)) [madsbk](https://github.com/madsbk)
- Temporarily skipping some tests because of a bug in Dask ([753](https://github.com/rapidsai/dask-cuda/pull/753)) [madsbk](https://github.com/madsbk)
- Removing the `FrameProxyObject` workaround ([751](https://github.com/rapidsai/dask-cuda/pull/751)) [madsbk](https://github.com/madsbk)
- Use cuDF Frame instead of Table ([748](https://github.com/rapidsai/dask-cuda/pull/748)) [madsbk](https://github.com/madsbk)
- Remove proxy object locks ([747](https://github.com/rapidsai/dask-cuda/pull/747)) [madsbk](https://github.com/madsbk)
- Unpin `dask` & `distributed` in CI ([742](https://github.com/rapidsai/dask-cuda/pull/742)) [galipremsagar](https://github.com/galipremsagar)
- Update SSHCluster usage in benchmarks with new CUDAWorker ([326](https://github.com/rapidsai/dask-cuda/pull/326)) [pentschev](https://github.com/pentschev)

🐛 Bug Fixes

- Drop test setting UCX global options via Dask config ([738](https://github.com/rapidsai/dask-cuda/pull/738)) [pentschev](https://github.com/pentschev)
- Prevent CUDA context errors when testing on single-GPU ([737](https://github.com/rapidsai/dask-cuda/pull/737)) [pentschev](https://github.com/pentschev)
- Handle `ucp` import error during `initialize()` ([729](https://github.com/rapidsai/dask-cuda/pull/729)) [pentschev](https://github.com/pentschev)
- Check if CUDA context was created in distributed.comm.ucx ([722](https://github.com/rapidsai/dask-cuda/pull/722)) [pentschev](https://github.com/pentschev)
- Fix registering correct dispatches for `cudf.Index` ([718](https://github.com/rapidsai/dask-cuda/pull/718)) [galipremsagar](https://github.com/galipremsagar)
- Register `percentile_lookup` for `FrameProxyObject` ([716](https://github.com/rapidsai/dask-cuda/pull/716)) [galipremsagar](https://github.com/galipremsagar)
- Leave interface unset when ucx_net_devices unset in LocalCUDACluster ([711](https://github.com/rapidsai/dask-cuda/pull/711)) [pentschev](https://github.com/pentschev)
- Update to UCX-Py 0.22 ([710](https://github.com/rapidsai/dask-cuda/pull/710)) [pentschev](https://github.com/pentschev)
- Missing fixes to Distributed config namespace refactoring ([703](https://github.com/rapidsai/dask-cuda/pull/703)) [pentschev](https://github.com/pentschev)
- Reset UCX-Py after rdmacm tests run ([702](https://github.com/rapidsai/dask-cuda/pull/702)) [pentschev](https://github.com/pentschev)
- Skip DGX InfiniBand tests when "rc" transport is unavailable ([701](https://github.com/rapidsai/dask-cuda/pull/701)) [pentschev](https://github.com/pentschev)
- Update UCX config namespace ([695](https://github.com/rapidsai/dask-cuda/pull/695)) [pentschev](https://github.com/pentschev)
- Bump isort hook version ([682](https://github.com/rapidsai/dask-cuda/pull/682)) [charlesbluca](https://github.com/charlesbluca)

📖 Documentation

- Update more docs for UCX 1.11+ ([720](https://github.com/rapidsai/dask-cuda/pull/720)) [pentschev](https://github.com/pentschev)
- Forward-merge branch-21.08 to branch-21.10 ([707](https://github.com/rapidsai/dask-cuda/pull/707)) [jakirkham](https://github.com/jakirkham)

🚀 New Features

- Warn if CUDA context is created on incorrect device with `LocalCUDACluster` ([719](https://github.com/rapidsai/dask-cuda/pull/719)) [pentschev](https://github.com/pentschev)
- Add `--benchmark-json` option to all benchmarks ([700](https://github.com/rapidsai/dask-cuda/pull/700)) [charlesbluca](https://github.com/charlesbluca)
- Remove Distributed tests from CI ([699](https://github.com/rapidsai/dask-cuda/pull/699)) [pentschev](https://github.com/pentschev)
- Add device memory limit argument to benchmarks ([683](https://github.com/rapidsai/dask-cuda/pull/683)) [charlesbluca](https://github.com/charlesbluca)
- Support for LocalCUDACluster with MIG ([674](https://github.com/rapidsai/dask-cuda/pull/674)) [akaanirban](https://github.com/akaanirban)

🛠️ Improvements

- Pin max `dask` and `distributed` versions to `2021.09.1` ([735](https://github.com/rapidsai/dask-cuda/pull/735)) [galipremsagar](https://github.com/galipremsagar)
- Implements a ProxyManagerDummy for convenience ([733](https://github.com/rapidsai/dask-cuda/pull/733)) [madsbk](https://github.com/madsbk)
- Add `__array_ufunc__` support for `ProxyObject` ([731](https://github.com/rapidsai/dask-cuda/pull/731)) [galipremsagar](https://github.com/galipremsagar)
- Use `has_cuda_context` from Distributed ([723](https://github.com/rapidsai/dask-cuda/pull/723)) [pentschev](https://github.com/pentschev)
- Fix deadlock and simplify proxy tracking ([712](https://github.com/rapidsai/dask-cuda/pull/712)) [madsbk](https://github.com/madsbk)
- JIT-unspill: support spilling to/from disk ([708](https://github.com/rapidsai/dask-cuda/pull/708)) [madsbk](https://github.com/madsbk)
- Tests: replacing the obsolete cudf.testing._utils.assert_eq calls ([706](https://github.com/rapidsai/dask-cuda/pull/706)) [madsbk](https://github.com/madsbk)
- JIT-unspill: warn when spill to disk triggers ([705](https://github.com/rapidsai/dask-cuda/pull/705)) [madsbk](https://github.com/madsbk)
- Remove max version pin for `dask` & `distributed` on development branch ([693](https://github.com/rapidsai/dask-cuda/pull/693)) [galipremsagar](https://github.com/galipremsagar)
- ENH Replace gpuci_conda_retry with gpuci_mamba_retry ([675](https://github.com/rapidsai/dask-cuda/pull/675)) [dillon-cullinan](https://github.com/dillon-cullinan)

🐛 Bug Fixes

- Use aliases to check for installed UCX version ([692](https://github.com/rapidsai/dask-cuda/pull/692)) [pentschev](https://github.com/pentschev)
- Don&39;t install Dask main branch in CI for 21.08 release ([687](https://github.com/rapidsai/dask-cuda/pull/687)) [pentschev](https://github.com/pentschev)
- Skip test_get_ucx_net_devices_raises on UCX >= 1.11.0 ([685](https://github.com/rapidsai/dask-cuda/pull/685)) [pentschev](https://github.com/pentschev)
- Fix NVML index usage in CUDAWorker/LocalCUDACluster ([671](https://github.com/rapidsai/dask-cuda/pull/671)) [pentschev](https://github.com/pentschev)
- Add --protocol flag to dask-cuda-worker ([670](https://github.com/rapidsai/dask-cuda/pull/670)) [jacobtomlinson](https://github.com/jacobtomlinson)
- Fix `assert_eq` related imports ([663](https://github.com/rapidsai/dask-cuda/pull/663)) [galipremsagar](https://github.com/galipremsagar)
- Small tweaks to make compatible with dask-mpi ([656](https://github.com/rapidsai/dask-cuda/pull/656)) [jacobtomlinson](https://github.com/jacobtomlinson)
- Remove Dask version pin ([647](https://github.com/rapidsai/dask-cuda/pull/647)) [pentschev](https://github.com/pentschev)
- Fix CUDA_VISIBLE_DEVICES tests ([638](https://github.com/rapidsai/dask-cuda/pull/638)) [pentschev](https://github.com/pentschev)
- Add `make_meta_dispatch` handling ([637](https://github.com/rapidsai/dask-cuda/pull/637)) [galipremsagar](https://github.com/galipremsagar)
- Update UCX-Py version in CI to 0.21.* ([636](https://github.com/rapidsai/dask-cuda/pull/636)) [pentschev](https://github.com/pentschev)

📖 Documentation

- Deprecation warning for ucx_net_devices=&39;auto&39; on UCX 1.11+ ([681](https://github.com/rapidsai/dask-cuda/pull/681)) [pentschev](https://github.com/pentschev)
- Update documentation on InfiniBand with UCX >= 1.11 ([669](https://github.com/rapidsai/dask-cuda/pull/669)) [pentschev](https://github.com/pentschev)
- Merge branch-21.06 ([622](https://github.com/rapidsai/dask-cuda/pull/622)) [pentschev](https://github.com/pentschev)

🚀 New Features

- Treat Deprecation/Future warnings as errors ([672](https://github.com/rapidsai/dask-cuda/pull/672)) [pentschev](https://github.com/pentschev)
- Update parse_bytes imports to resolve deprecation warnings ([662](https://github.com/rapidsai/dask-cuda/pull/662)) [pentschev](https://github.com/pentschev)

🛠️ Improvements

- Pin max `dask` & `distributed` versions ([686](https://github.com/rapidsai/dask-cuda/pull/686)) [galipremsagar](https://github.com/galipremsagar)
- Fix DGX tests warnings on RMM pool size and file not closed ([673](https://github.com/rapidsai/dask-cuda/pull/673)) [pentschev](https://github.com/pentschev)
- Remove dot calling style for pytest ([661](https://github.com/rapidsai/dask-cuda/pull/661)) [quasiben](https://github.com/quasiben)
- get_device_memory_objects(): dispatch on cudf.core.frame.Frame ([658](https://github.com/rapidsai/dask-cuda/pull/658)) [madsbk](https://github.com/madsbk)
- Fix `21.08` forward-merge conflicts ([655](https://github.com/rapidsai/dask-cuda/pull/655)) [ajschmidt8](https://github.com/ajschmidt8)
- Fix conflicts in `643` ([644](https://github.com/rapidsai/dask-cuda/pull/644)) [ajschmidt8](https://github.com/ajschmidt8)

🐛 Bug Fixes

- Handle `import`ing relocated dispatch functions ([623](https://github.com/rapidsai/dask-cuda/pull/623)) [jakirkham](https://github.com/jakirkham)
- Fix DGX tests for UCX 1.9 ([619](https://github.com/rapidsai/dask-cuda/pull/619)) [pentschev](https://github.com/pentschev)
- Add PROJECTS var ([614](https://github.com/rapidsai/dask-cuda/pull/614)) [ajschmidt8](https://github.com/ajschmidt8)

📖 Documentation

- Bump docs copyright year ([616](https://github.com/rapidsai/dask-cuda/pull/616)) [charlesbluca](https://github.com/charlesbluca)
- Update RTD site to redirect to RAPIDS docs ([615](https://github.com/rapidsai/dask-cuda/pull/615)) [charlesbluca](https://github.com/charlesbluca)
- Document DASK_JIT_UNSPILL ([604](https://github.com/rapidsai/dask-cuda/pull/604)) [madsbk](https://github.com/madsbk)

🚀 New Features

- Disable reuse endpoints with UCX >= 1.11 ([620](https://github.com/rapidsai/dask-cuda/pull/620)) [pentschev](https://github.com/pentschev)

🛠️ Improvements

- Adding profiling to dask shuffle ([625](https://github.com/rapidsai/dask-cuda/pull/625)) [arunraman](https://github.com/arunraman)
- Update `CHANGELOG.md` links for calver ([618](https://github.com/rapidsai/dask-cuda/pull/618)) [ajschmidt8](https://github.com/ajschmidt8)
- Fixing Dataframe merge benchmark ([617](https://github.com/rapidsai/dask-cuda/pull/617)) [madsbk](https://github.com/madsbk)
- Fix DGX tests for UCX 1.10+ ([613](https://github.com/rapidsai/dask-cuda/pull/613)) [pentschev](https://github.com/pentschev)
- Update docs build script ([612](https://github.com/rapidsai/dask-cuda/pull/612)) [ajschmidt8](https://github.com/ajschmidt8)