Dask-cuda

🚨 Breaking Changes

- Pin `dask` and `distributed` for release ([1153](https://github.com/rapidsai/dask-cuda/pull/1153)) [galipremsagar](https://github.com/galipremsagar)
- Update minimum `pandas` and `numpy` pinnings ([1139](https://github.com/rapidsai/dask-cuda/pull/1139)) [galipremsagar](https://github.com/galipremsagar)

🐛 Bug Fixes

- Rectify `dask-core` pinning in pip requirements ([1155](https://github.com/rapidsai/dask-cuda/pull/1155)) [galipremsagar](https://github.com/galipremsagar)
- Monkey patching all locations of `get_default_shuffle_algorithm` ([1142](https://github.com/rapidsai/dask-cuda/pull/1142)) [madsbk](https://github.com/madsbk)
- Update usage of `get_worker()` in tests ([1141](https://github.com/rapidsai/dask-cuda/pull/1141)) [pentschev](https://github.com/pentschev)
- Update `rmm_cupy_allocator` usage ([1138](https://github.com/rapidsai/dask-cuda/pull/1138)) [jakirkham](https://github.com/jakirkham)
- Serialize of `ProxyObject` to pickle fixed attributes ([1137](https://github.com/rapidsai/dask-cuda/pull/1137)) [madsbk](https://github.com/madsbk)
- Explicit-comms: update monkey patching of Dask ([1135](https://github.com/rapidsai/dask-cuda/pull/1135)) [madsbk](https://github.com/madsbk)
- Fix for bytes/str discrepancy after PyNVML update ([1118](https://github.com/rapidsai/dask-cuda/pull/1118)) [pentschev](https://github.com/pentschev)

🚀 New Features

- Allow specifying dashboard address in benchmarks ([1147](https://github.com/rapidsai/dask-cuda/pull/1147)) [pentschev](https://github.com/pentschev)
- Add argument to enable RMM alloaction tracking in benchmarks ([1145](https://github.com/rapidsai/dask-cuda/pull/1145)) [pentschev](https://github.com/pentschev)
- Reinstate `--death-timeout` CLI option ([1140](https://github.com/rapidsai/dask-cuda/pull/1140)) [charlesbluca](https://github.com/charlesbluca)
- Extend RMM async allocation support ([1116](https://github.com/rapidsai/dask-cuda/pull/1116)) [pentschev](https://github.com/pentschev)
- Allow using stream-ordered and managed RMM allocators in benchmarks ([1012](https://github.com/rapidsai/dask-cuda/pull/1012)) [pentschev](https://github.com/pentschev)

🛠️ Improvements

- Pin `dask` and `distributed` for release ([1153](https://github.com/rapidsai/dask-cuda/pull/1153)) [galipremsagar](https://github.com/galipremsagar)
- Update minimum `pandas` and `numpy` pinnings ([1139](https://github.com/rapidsai/dask-cuda/pull/1139)) [galipremsagar](https://github.com/galipremsagar)
- Drop Python 3.7 handling for pickle protocol 4 ([1132](https://github.com/rapidsai/dask-cuda/pull/1132)) [jakirkham](https://github.com/jakirkham)
- Adapt to rapidsai/rmm1221 which moves allocator callbacks ([1129](https://github.com/rapidsai/dask-cuda/pull/1129)) [wence-](https://github.com/wence-)
- Merge `branch-23.02` into `branch-23.04` ([1128](https://github.com/rapidsai/dask-cuda/pull/1128)) [ajschmidt8](https://github.com/ajschmidt8)
- Template Conda recipe&39;s `about` metadata ([1121](https://github.com/rapidsai/dask-cuda/pull/1121)) [jakirkham](https://github.com/jakirkham)
- Fix GHA build workflow ([1120](https://github.com/rapidsai/dask-cuda/pull/1120)) [AjayThorve](https://github.com/AjayThorve)
- Reduce error handling verbosity in CI tests scripts ([1113](https://github.com/rapidsai/dask-cuda/pull/1113)) [AjayThorve](https://github.com/AjayThorve)
- Update shared workflow branches ([1112](https://github.com/rapidsai/dask-cuda/pull/1112)) [ajschmidt8](https://github.com/ajschmidt8)
- Remove gpuCI scripts. ([1111](https://github.com/rapidsai/dask-cuda/pull/1111)) [bdice](https://github.com/bdice)
- Unpin `dask` and `distributed` for development ([1110](https://github.com/rapidsai/dask-cuda/pull/1110)) [galipremsagar](https://github.com/galipremsagar)
- Move date to build string in `conda` recipe ([1103](https://github.com/rapidsai/dask-cuda/pull/1103)) [ajschmidt8](https://github.com/ajschmidt8)

🚨 Breaking Changes

- Pin `dask` and `distributed` for release ([1106](https://github.com/rapidsai/dask-cuda/pull/1106)) [galipremsagar](https://github.com/galipremsagar)

🐛 Bug Fixes

- pre-commit: Update isort version to 5.12.0 ([1098](https://github.com/rapidsai/dask-cuda/pull/1098)) [wence-](https://github.com/wence-)
- explicit-comms: don&39;t mix `-` and `_` in config ([1096](https://github.com/rapidsai/dask-cuda/pull/1096)) [madsbk](https://github.com/madsbk)
- Update `cudf.Buffer` pointer access method ([1094](https://github.com/rapidsai/dask-cuda/pull/1094)) [pentschev](https://github.com/pentschev)
- Update tests for Python 3.10 ([1086](https://github.com/rapidsai/dask-cuda/pull/1086)) [pentschev](https://github.com/pentschev)
- Use `pkgutil.iter_modules` to get un-imported module for `test_pre_import` ([1085](https://github.com/rapidsai/dask-cuda/pull/1085)) [charlesbluca](https://github.com/charlesbluca)
- Make proxy tests with `LocalCUDACluster` asynchronous ([1084](https://github.com/rapidsai/dask-cuda/pull/1084)) [pentschev](https://github.com/pentschev)
- Ensure consistent results from `safe_sizeof()` in test ([1071](https://github.com/rapidsai/dask-cuda/pull/1071)) [madsbk](https://github.com/madsbk)
- Pass missing argument to groupby benchmark compute ([1069](https://github.com/rapidsai/dask-cuda/pull/1069)) [mattf](https://github.com/mattf)
- Reorder channel priority. ([1067](https://github.com/rapidsai/dask-cuda/pull/1067)) [bdice](https://github.com/bdice)
- Fix owner check when the owner is a cupy array ([1061](https://github.com/rapidsai/dask-cuda/pull/1061)) [wence-](https://github.com/wence-)

🛠️ Improvements

- Pin `dask` and `distributed` for release ([1106](https://github.com/rapidsai/dask-cuda/pull/1106)) [galipremsagar](https://github.com/galipremsagar)
- Update shared workflow branches ([1105](https://github.com/rapidsai/dask-cuda/pull/1105)) [ajschmidt8](https://github.com/ajschmidt8)
- Proxify: make duplicate check optional ([1101](https://github.com/rapidsai/dask-cuda/pull/1101)) [madsbk](https://github.com/madsbk)
- Fix whitespace & add URLs in `pyproject.toml` ([1092](https://github.com/rapidsai/dask-cuda/pull/1092)) [jakirkham](https://github.com/jakirkham)
- pre-commit: spell, whitespace, and mypy check ([1091](https://github.com/rapidsai/dask-cuda/pull/1091)) [madsbk](https://github.com/madsbk)
- shuffle: use cuDF&39;s `partition_by_hash()` when available ([1090](https://github.com/rapidsai/dask-cuda/pull/1090)) [madsbk](https://github.com/madsbk)
- add initial docs build ([1089](https://github.com/rapidsai/dask-cuda/pull/1089)) [AjayThorve](https://github.com/AjayThorve)
- Remove `--get-cluster-configuration` option, check for scheduler in `dask cuda config` ([1088](https://github.com/rapidsai/dask-cuda/pull/1088)) [charlesbluca](https://github.com/charlesbluca)
- Add timeout to `pytest` command ([1082](https://github.com/rapidsai/dask-cuda/pull/1082)) [ajschmidt8](https://github.com/ajschmidt8)
- shuffle-benchmark: add `--partition-distribution` ([1081](https://github.com/rapidsai/dask-cuda/pull/1081)) [madsbk](https://github.com/madsbk)
- Ensure tests run for Python `3.10` ([1080](https://github.com/rapidsai/dask-cuda/pull/1080)) [ajschmidt8](https://github.com/ajschmidt8)
- Use TrackingResourceAdaptor to get better debug info ([1079](https://github.com/rapidsai/dask-cuda/pull/1079)) [madsbk](https://github.com/madsbk)
- Improve shuffle-benchmark ([1074](https://github.com/rapidsai/dask-cuda/pull/1074)) [madsbk](https://github.com/madsbk)
- Update builds for CUDA `11.8` and Python `310` ([1072](https://github.com/rapidsai/dask-cuda/pull/1072)) [ajschmidt8](https://github.com/ajschmidt8)
- Shuffle by partition to reduce memory usage significantly ([1068](https://github.com/rapidsai/dask-cuda/pull/1068)) [madsbk](https://github.com/madsbk)
- Enable copy_prs. ([1063](https://github.com/rapidsai/dask-cuda/pull/1063)) [bdice](https://github.com/bdice)
- Add GitHub Actions Workflows ([1062](https://github.com/rapidsai/dask-cuda/pull/1062)) [bdice](https://github.com/bdice)
- Unpin `dask` and `distributed` for development ([1060](https://github.com/rapidsai/dask-cuda/pull/1060)) [galipremsagar](https://github.com/galipremsagar)
- Switch to the new dask CLI ([981](https://github.com/rapidsai/dask-cuda/pull/981)) [jacobtomlinson](https://github.com/jacobtomlinson)

🚨 Breaking Changes

- Make local_directory a required argument for spilling impls ([1023](https://github.com/rapidsai/dask-cuda/pull/1023)) [wence-](https://github.com/wence-)

🐛 Bug Fixes

- Fix `parse_memory_limit` function call ([1055](https://github.com/rapidsai/dask-cuda/pull/1055)) [galipremsagar](https://github.com/galipremsagar)
- Work around Jupyter errors in CI ([1041](https://github.com/rapidsai/dask-cuda/pull/1041)) [pentschev](https://github.com/pentschev)
- Fix version constraint ([1036](https://github.com/rapidsai/dask-cuda/pull/1036)) [wence-](https://github.com/wence-)
- Support the new `Buffer` in cudf ([1033](https://github.com/rapidsai/dask-cuda/pull/1033)) [madsbk](https://github.com/madsbk)
- Install Dask nightly last in CI ([1029](https://github.com/rapidsai/dask-cuda/pull/1029)) [pentschev](https://github.com/pentschev)
- Fix recorded time in merge benchmark ([1028](https://github.com/rapidsai/dask-cuda/pull/1028)) [wence-](https://github.com/wence-)
- Switch pre-import not found test to sync definition ([1026](https://github.com/rapidsai/dask-cuda/pull/1026)) [pentschev](https://github.com/pentschev)
- Make local_directory a required argument for spilling impls ([1023](https://github.com/rapidsai/dask-cuda/pull/1023)) [wence-](https://github.com/wence-)
- Fixes for handling MIG devices ([950](https://github.com/rapidsai/dask-cuda/pull/950)) [pentschev](https://github.com/pentschev)

📖 Documentation

- Merge 22.10 into 22.12 ([1016](https://github.com/rapidsai/dask-cuda/pull/1016)) [pentschev](https://github.com/pentschev)
- Merge 22.08 into 22.10 ([1010](https://github.com/rapidsai/dask-cuda/pull/1010)) [pentschev](https://github.com/pentschev)

🚀 New Features

- Allow specifying fractions as RMM pool initial/maximum size ([1021](https://github.com/rapidsai/dask-cuda/pull/1021)) [pentschev](https://github.com/pentschev)
- Add feature to get cluster configuration ([1006](https://github.com/rapidsai/dask-cuda/pull/1006)) [quasiben](https://github.com/quasiben)
- Add benchmark option to use dask-noop ([994](https://github.com/rapidsai/dask-cuda/pull/994)) [wence-](https://github.com/wence-)

🛠️ Improvements

- Ensure linting checks for whole repo in CI ([1053](https://github.com/rapidsai/dask-cuda/pull/1053)) [pentschev](https://github.com/pentschev)
- Pin `dask` and `distributed` for release ([1046](https://github.com/rapidsai/dask-cuda/pull/1046)) [galipremsagar](https://github.com/galipremsagar)
- Remove `pytest-asyncio` dependency ([1045](https://github.com/rapidsai/dask-cuda/pull/1045)) [pentschev](https://github.com/pentschev)
- Migrate as much as possible to `pyproject.toml` ([1035](https://github.com/rapidsai/dask-cuda/pull/1035)) [jakirkham](https://github.com/jakirkham)
- Re-implement shuffle using staging ([1030](https://github.com/rapidsai/dask-cuda/pull/1030)) [madsbk](https://github.com/madsbk)
- Explicit-comms-shuffle: fine control of task scheduling ([1025](https://github.com/rapidsai/dask-cuda/pull/1025)) [madsbk](https://github.com/madsbk)
- Remove stale labeler ([1024](https://github.com/rapidsai/dask-cuda/pull/1024)) [raydouglass](https://github.com/raydouglass)
- Unpin `dask` and `distributed` for development ([1005](https://github.com/rapidsai/dask-cuda/pull/1005)) [galipremsagar](https://github.com/galipremsagar)
- Support cuDF&39;s built-in spilling ([984](https://github.com/rapidsai/dask-cuda/pull/984)) [madsbk](https://github.com/madsbk)

🐛 Bug Fixes

- Revert "Update rearrange_by_column patch for explicit comms" ([1001](https://github.com/rapidsai/dask-cuda/pull/1001)) [rjzamora](https://github.com/rjzamora)
- Address CI failures caused by upstream distributed and cupy changes ([993](https://github.com/rapidsai/dask-cuda/pull/993)) [rjzamora](https://github.com/rjzamora)
- DeviceSerialized.__reduce_ex__: convert frame to numpy arrays ([977](https://github.com/rapidsai/dask-cuda/pull/977)) [madsbk](https://github.com/madsbk)

📖 Documentation

- Remove line-break that&39;s breaking link ([982](https://github.com/rapidsai/dask-cuda/pull/982)) [ntabris](https://github.com/ntabris)
- Dask-cuda best practices ([976](https://github.com/rapidsai/dask-cuda/pull/976)) [quasiben](https://github.com/quasiben)

🚀 New Features

- Add Groupby benchmark ([979](https://github.com/rapidsai/dask-cuda/pull/979)) [rjzamora](https://github.com/rjzamora)

🛠️ Improvements

- Pin `dask` and `distributed` for release ([1003](https://github.com/rapidsai/dask-cuda/pull/1003)) [galipremsagar](https://github.com/galipremsagar)
- Update rearrange_by_column patch for explicit comms ([992](https://github.com/rapidsai/dask-cuda/pull/992)) [rjzamora](https://github.com/rjzamora)
- benchmarks: Add option to suppress output of point to point data ([985](https://github.com/rapidsai/dask-cuda/pull/985)) [wence-](https://github.com/wence-)
- Unpin `dask` and `distributed` for development ([971](https://github.com/rapidsai/dask-cuda/pull/971)) [galipremsagar](https://github.com/galipremsagar)

🚨 Breaking Changes

- Fix useless property ([944](https://github.com/rapidsai/dask-cuda/pull/944)) [wence-](https://github.com/wence-)

🐛 Bug Fixes

- Fix `distributed` error related to `loop_in_thread` ([963](https://github.com/rapidsai/dask-cuda/pull/963)) [galipremsagar](https://github.com/galipremsagar)
- Add `__rmatmul__` to `ProxyObject` ([960](https://github.com/rapidsai/dask-cuda/pull/960)) [jakirkham](https://github.com/jakirkham)
- Always use versioneer command classes in setup.py ([948](https://github.com/rapidsai/dask-cuda/pull/948)) [wence-](https://github.com/wence-)
- Do not dispatch removed `cudf.Frame._index` object ([947](https://github.com/rapidsai/dask-cuda/pull/947)) [pentschev](https://github.com/pentschev)
- Fix useless property ([944](https://github.com/rapidsai/dask-cuda/pull/944)) [wence-](https://github.com/wence-)
- LocalCUDACluster&39;s memory limit: `None` means no limit ([943](https://github.com/rapidsai/dask-cuda/pull/943)) [madsbk](https://github.com/madsbk)
- ProxyManager: support `memory_limit=None` ([941](https://github.com/rapidsai/dask-cuda/pull/941)) [madsbk](https://github.com/madsbk)
- Remove deprecated `loop` kwarg to `Nanny` in `CUDAWorker` ([934](https://github.com/rapidsai/dask-cuda/pull/934)) [pentschev](https://github.com/pentschev)
- Import `cleanup` fixture in `test_dask_cuda_worker.py` ([924](https://github.com/rapidsai/dask-cuda/pull/924)) [pentschev](https://github.com/pentschev)

📖 Documentation

- Switch docs to use common `js` & `css` code ([967](https://github.com/rapidsai/dask-cuda/pull/967)) [galipremsagar](https://github.com/galipremsagar)
- Switch `language` from `None` to `"en"` in docs build ([939](https://github.com/rapidsai/dask-cuda/pull/939)) [galipremsagar](https://github.com/galipremsagar)

🚀 New Features

- Add communications bandwidth to benchmarks ([938](https://github.com/rapidsai/dask-cuda/pull/938)) [pentschev](https://github.com/pentschev)

🛠️ Improvements

- Pin `dask` & `distributed` for release ([965](https://github.com/rapidsai/dask-cuda/pull/965)) [galipremsagar](https://github.com/galipremsagar)
- Test memory_limit=None for CUDAWorker ([946](https://github.com/rapidsai/dask-cuda/pull/946)) [wence-](https://github.com/wence-)
- benchmarks: Record total number of workers in dataframe ([945](https://github.com/rapidsai/dask-cuda/pull/945)) [wence-](https://github.com/wence-)
- Benchmark refactoring: tidy data and multi-node capability via `--scheduler-file` ([940](https://github.com/rapidsai/dask-cuda/pull/940)) [wence-](https://github.com/wence-)
- Add util functions to simplify printing benchmarks results ([937](https://github.com/rapidsai/dask-cuda/pull/937)) [pentschev](https://github.com/pentschev)
- Add --multiprocessing-method option to benchmarks ([933](https://github.com/rapidsai/dask-cuda/pull/933)) [wence-](https://github.com/wence-)
- Remove click pinning ([932](https://github.com/rapidsai/dask-cuda/pull/932)) [charlesbluca](https://github.com/charlesbluca)
- Remove compiler variables ([929](https://github.com/rapidsai/dask-cuda/pull/929)) [ajschmidt8](https://github.com/ajschmidt8)
- Unpin `dask` & `distributed` for development ([927](https://github.com/rapidsai/dask-cuda/pull/927)) [galipremsagar](https://github.com/galipremsagar)

🚨 Breaking Changes

- Upgrade `numba` pinning to be in-line with rest of rapids ([912](https://github.com/rapidsai/dask-cuda/pull/912)) [galipremsagar](https://github.com/galipremsagar)

🐛 Bug Fixes

- Reduce `test_cudf_cluster_device_spill` test and speed it up ([918](https://github.com/rapidsai/dask-cuda/pull/918)) [pentschev](https://github.com/pentschev)
- Update ImportError tests with --pre-import ([914](https://github.com/rapidsai/dask-cuda/pull/914)) [pentschev](https://github.com/pentschev)
- Add xfail mark to `test_pre_import_not_found` ([908](https://github.com/rapidsai/dask-cuda/pull/908)) [pentschev](https://github.com/pentschev)
- Increase spill tests timeout to 30 seconds ([901](https://github.com/rapidsai/dask-cuda/pull/901)) [pentschev](https://github.com/pentschev)
- Fix errors related with `distributed.worker.memory.terminate` ([900](https://github.com/rapidsai/dask-cuda/pull/900)) [pentschev](https://github.com/pentschev)
- Skip tests on import error for some optional packages ([899](https://github.com/rapidsai/dask-cuda/pull/899)) [pentschev](https://github.com/pentschev)
- Update auto host_memory computation when threads per worker > 1 ([896](https://github.com/rapidsai/dask-cuda/pull/896)) [ayushdg](https://github.com/ayushdg)
- Update black to 22.3.0 ([889](https://github.com/rapidsai/dask-cuda/pull/889)) [charlesbluca](https://github.com/charlesbluca)
- Remove legacy `check_python_3` ([886](https://github.com/rapidsai/dask-cuda/pull/886)) [pentschev](https://github.com/pentschev)

📖 Documentation

- Add documentation for `RAPIDS_NO_INITIALIZE` ([898](https://github.com/rapidsai/dask-cuda/pull/898)) [charlesbluca](https://github.com/charlesbluca)
- Use upstream warning functions for CUDA initialization ([894](https://github.com/rapidsai/dask-cuda/pull/894)) [charlesbluca](https://github.com/charlesbluca)

🛠️ Improvements

- Pin `dask` and `distributed` for release ([922](https://github.com/rapidsai/dask-cuda/pull/922)) [galipremsagar](https://github.com/galipremsagar)
- Pin `dask` & `distributed` for release ([916](https://github.com/rapidsai/dask-cuda/pull/916)) [galipremsagar](https://github.com/galipremsagar)
- Upgrade `numba` pinning to be in-line with rest of rapids ([912](https://github.com/rapidsai/dask-cuda/pull/912)) [galipremsagar](https://github.com/galipremsagar)
- Removing test of `cudf.merge_sorted()` ([905](https://github.com/rapidsai/dask-cuda/pull/905)) [madsbk](https://github.com/madsbk)
- Disable `include-ignored` coverage warnings ([903](https://github.com/rapidsai/dask-cuda/pull/903)) [pentschev](https://github.com/pentschev)
- Fix ci/local script ([902](https://github.com/rapidsai/dask-cuda/pull/902)) [Ethyling](https://github.com/Ethyling)
- Use conda to build python packages during GPU tests ([897](https://github.com/rapidsai/dask-cuda/pull/897)) [Ethyling](https://github.com/Ethyling)
- Pull `requirements.txt` into Conda recipe ([893](https://github.com/rapidsai/dask-cuda/pull/893)) [jakirkham](https://github.com/jakirkham)
- Unpin `dask` & `distributed` for development ([892](https://github.com/rapidsai/dask-cuda/pull/892)) [galipremsagar](https://github.com/galipremsagar)
- Build packages using mambabuild ([846](https://github.com/rapidsai/dask-cuda/pull/846)) [Ethyling](https://github.com/Ethyling)