Ark-analysis

This latest release contains the revised Pixie pixel and cell clustering algorithm.


Pixie Paper Implementation
- Next Release - v0.6.0 alex-l-kong ([945](https://github.com/angelolab/ark-analysis/pull/945))
- Temporarily remove pixel preprocessing normalization for Candace's paper alex-l-kong ([913](https://github.com/angelolab/ark-analysis/pull/913))
- Pixie notebook text changes cliu72 ([942](https://github.com/angelolab/ark-analysis/pull/942))

🚀 Features

- Refactor Pixie modules alex-l-kong ([940](https://github.com/angelolab/ark-analysis/pull/940))
- Split spatial analysis module camisowers ([938](https://github.com/angelolab/ark-analysis/pull/938))
- Ark Refactor srivarra ([935](https://github.com/angelolab/ark-analysis/pull/935))
- Github Actions - Merge Queue srivarra ([932](https://github.com/angelolab/ark-analysis/pull/932))
- Sunset Pairwise Distance Notebook srivarra ([929](https://github.com/angelolab/ark-analysis/pull/929))
- Mixing score camisowers ([836](https://github.com/angelolab/ark-analysis/pull/836))
- Integrate existing Pixie cell clustering process with generic cell clustering process alex-l-kong ([885](https://github.com/angelolab/ark-analysis/pull/885))

🐛 Bug Fixes

- Add a hidden parameter to overwrite certain portions of the pipeline alex-l-kong ([914](https://github.com/angelolab/ark-analysis/pull/914))

🧰 Maintenance

- :arrow\_up: Bump release-drafter/release-drafter from 5.20.1 to 5.22.0 dependabot ([916](https://github.com/angelolab/ark-analysis/pull/916))

📚️ Documentation

- Make sure `data_types.md` displays newlines alex-l-kong ([930](https://github.com/angelolab/ark-analysis/pull/930))
alex-l-kong, camisowers, cliu72, dependabot, dependabot[bot] and srivarra

The latest version of ark-analysis include many improvements to the Pixie pipeline such as allowing the ability to rerun SOM training, and allowing users to skip many of the `regionprops` fields which are computationally expensive to generate.

🚀 Features

- Remove deterministic argument from `som` function in `pyFlowSOM` alex-l-kong ([893](https://github.com/angelolab/ark-analysis/pull/893))
- GitHub Actions srivarra ([895](https://github.com/angelolab/ark-analysis/pull/895))
- Ensure `cluster_helpers.PixelSOMCluster` receives the `fovs` argument alex-l-kong ([905](https://github.com/angelolab/ark-analysis/pull/905))
- Allow user to set an explicit skip flag for custom regionprops and marker extraction functionality in notebook 1 alex-l-kong ([897](https://github.com/angelolab/ark-analysis/pull/897))

🐛 Bug Fixes

- Turn off minor ticks for pixel and cell cluster overlay colorbar alex-l-kong ([912](https://github.com/angelolab/ark-analysis/pull/912))
- Remove zero column pixel cluster columns from cell clustering pipeline alex-l-kong ([884](https://github.com/angelolab/ark-analysis/pull/884))
- Ensure z-score slider adjusts colorbar range when changed alex-l-kong ([904](https://github.com/angelolab/ark-analysis/pull/904))
- Ensure pixel SOM training reruns if new markers specified alex-l-kong ([903](https://github.com/angelolab/ark-analysis/pull/903))
- Correct 99.9% normalization to 100% normalization for "many-zero" columns alex-l-kong ([894](https://github.com/angelolab/ark-analysis/pull/894))

🧰 Maintenance

- Bump ipython from 8.4.0 to 8.10.0 in /docs dependabot ([908](https://github.com/angelolab/ark-analysis/pull/908))
alex-l-kong, dependabot, and srivarra

This latest version of Ark Analysis includes moving shared utility functions across [angelolab/ark-analysis](https://github.com/angelolab/ark-analysis), [angelolab/toffy](https://github.com/angelolab/toffy), and [angelolab/mibi-bin-tools](https://github.com/angelolab/mibi-bin-tools) to their new home in [angelolab/tmi](https://github.com/angelolab/tmi). In addition some other updates include the ability to convert to and from OME-TIFFs and adding `inverse` as a custom method to allow for more recent versions of `matplotlib`. In addition a majority of the requirements were adjusted to be less strict.

🚀 Features

- Added `nuc_seg_suffix` for `filter_with_nuclear_mask` srivarra ([879](https://github.com/angelolab/ark-analysis/pull/879))
- Add `inverse` method to `ZScoreNorm` to integrate with current `matplotlib` alex-l-kong ([871](https://github.com/angelolab/ark-analysis/pull/871))
- OME-TIFF Read / Write srivarra ([819](https://github.com/angelolab/ark-analysis/pull/819))
- Load Utils - Expanding Acceptable File Types srivarra ([878](https://github.com/angelolab/ark-analysis/pull/878))

🧰 Maintenance

- Next Release v0.5.1 srivarra ([889](https://github.com/angelolab/ark-analysis/pull/889))
- Adding TMI srivarra ([825](https://github.com/angelolab/ark-analysis/pull/825))
alex-l-kong and srivarra

The latest release of Ark Analysis mainly focused on removing `R` dependencies. In order to do so, we have created a small version of FlowSOM over at [angelolab/pyFlowSOM](https://github.com/angelolab/pyFlowSOM).

Other major features include more example datasets, and performance improvements.
Notable bug fixes include removing duplicate FOV loading and making the mantis file extension independent from the segmentation file extension.
Documentation improvements include improved git cloning / installation instructions, and ejecting an imposter Pixie author (ඞ).


🚀 Features

- Load Utils - Expanding Acceptable File Types srivarra ([878](https://github.com/angelolab/ark-analysis/pull/878))
- Example Dataset - Cropped FOV 0 srivarra ([859](https://github.com/angelolab/ark-analysis/pull/859))
- Integrate `pyFlowSOM` with Pixie alex-l-kong ([863](https://github.com/angelolab/ark-analysis/pull/863))
- Clean up Pixie pipeline alex-l-kong ([856](https://github.com/angelolab/ark-analysis/pull/856))
- Make `segmentation_dir` definition more clear for pixel and cell clustering processes alex-l-kong ([855](https://github.com/angelolab/ark-analysis/pull/855))
- Additional Input Validation srivarra ([841](https://github.com/angelolab/ark-analysis/pull/841))
- Add native Python consensus clustering process alex-l-kong ([839](https://github.com/angelolab/ark-analysis/pull/839))
- Subset pixel channel averaging to remove bottleneck alex-l-kong ([823](https://github.com/angelolab/ark-analysis/pull/823))
- Example Dataset - Spatial Datasets srivarra ([832](https://github.com/angelolab/ark-analysis/pull/832))
- Use `random_state` parameter in KMeans clustering for reproducibility alex-l-kong ([834](https://github.com/angelolab/ark-analysis/pull/834))
- Added Parameter for Seg Mask Suffix for Mantis srivarra ([828](https://github.com/angelolab/ark-analysis/pull/828))
- Rename `_feature_0` to `_whole_cell` and `_feature_1` to `_nuclear` alex-l-kong ([786](https://github.com/angelolab/ark-analysis/pull/786))
- Refactor distance matrix saving alex-l-kong ([803](https://github.com/angelolab/ark-analysis/pull/803))

🐛 Bug Fixes

- Make Mantis mask file extension independent from segmentation file extension alex-l-kong ([870](https://github.com/angelolab/ark-analysis/pull/870))
- `seg_suffix` Requires Explicit File Extension srivarra ([848](https://github.com/angelolab/ark-analysis/pull/848))
- Fix neighborhood matrix indexing due to subsetting alex-l-kong ([824](https://github.com/angelolab/ark-analysis/pull/824))
- Removed duplicate mantis fov loading srivarra ([838](https://github.com/angelolab/ark-analysis/pull/838))
- Don't load the pixel, cell, and neighborhood FOVs in batches for mask generation alex-l-kong ([816](https://github.com/angelolab/ark-analysis/pull/816))
- Ensure neighborhood inertia and silhouette score generation works with varying `min_k` alex-l-kong ([820](https://github.com/angelolab/ark-analysis/pull/820))
- Travis patchelf pypi\_deploy Updates srivarra ([805](https://github.com/angelolab/ark-analysis/pull/805))

🧰 Maintenance

- Next Release v0.5.0 srivarra ([877](https://github.com/angelolab/ark-analysis/pull/877))

📚️ Documentation

- Tagged Installation Instructions srivarra ([840](https://github.com/angelolab/ark-analysis/pull/840))
- Example Dataset - Adding more info to Overwrite Existing srivarra ([826](https://github.com/angelolab/ark-analysis/pull/826))
- Fix Pixie authorship ngreenwald ([813](https://github.com/angelolab/ark-analysis/pull/813))
- Cell Table Assumptions srivarra ([808](https://github.com/angelolab/ark-analysis/pull/808))
alex-l-kong, ngreenwald and srivarra

Users will now be able to tile their images with a script, and use the notebooks with example data. In addition this release adds more validation checks for user provided paths, optimization and a handful of bug fixes.

🚀 Features

- Standardize usage of `.tiff` extension alex-l-kong ([801](https://github.com/angelolab/ark-analysis/pull/801))
- Adjust load tiled imgs for toffy use camisowers ([802](https://github.com/angelolab/ark-analysis/pull/802))
- Compressed `csv` Option in Notebook 1 srivarra ([800](https://github.com/angelolab/ark-analysis/pull/800))
- Increase validate paths usage camisowers ([796](https://github.com/angelolab/ark-analysis/pull/796))
- Testbook Adjustments srivarra ([791](https://github.com/angelolab/ark-analysis/pull/791))
- Local Caching srivarra ([794](https://github.com/angelolab/ark-analysis/pull/794))
- Notebook Update Tests with string args srivarra ([782](https://github.com/angelolab/ark-analysis/pull/782))
- Example Dataset - Notebook 4 srivarra ([763](https://github.com/angelolab/ark-analysis/pull/763))
- Make non-parallel loops canon for Pixie alex-l-kong ([779](https://github.com/angelolab/ark-analysis/pull/779))
- Example Dataset - Notebook 3 srivarra ([760](https://github.com/angelolab/ark-analysis/pull/760))
- Image tiling script camisowers ([702](https://github.com/angelolab/ark-analysis/pull/702))
- Example Dataset - Notebook 2 srivarra ([736](https://github.com/angelolab/ark-analysis/pull/736))
- Remove mibitiff option in notebooks camisowers ([772](https://github.com/angelolab/ark-analysis/pull/772))
- Optimize dataframe merging camisowers ([762](https://github.com/angelolab/ark-analysis/pull/762))
- Add inertia and silhouette score progress bars for neighborhood analysis notebook alex-l-kong ([758](https://github.com/angelolab/ark-analysis/pull/758))
- Fiber segmentation ackagel ([569](https://github.com/angelolab/ark-analysis/pull/569))
- Add Read the Docs status badge to README alex-l-kong ([755](https://github.com/angelolab/ark-analysis/pull/755))

🐛 Bug Fixes

- In k-means neighborhood notebook, fix excluded\_channels=None cliu72 ([788](https://github.com/angelolab/ark-analysis/pull/788))
- Run `pytest` as a separate job to avoid `scripts` from getting overwritten alex-l-kong ([776](https://github.com/angelolab/ark-analysis/pull/776))
- Patch up PyPI requirements alex-l-kong ([753](https://github.com/angelolab/ark-analysis/pull/753))
- Settings name update camisowers ([741](https://github.com/angelolab/ark-analysis/pull/741))

🧰 Maintenance

- Next Release - v0.4.4 srivarra ([804](https://github.com/angelolab/ark-analysis/pull/804))
ackagel, alex-l-kong, camisowers, cliu72 and srivarra