Flowkit

Dependencies:

* FlowIO requirement updated to v1.0.1 (fixes issue exporting some FCS files)

API changes:

* new GatingStrategy method find_matching_gate_paths to find gate paths given a gate name
* new Session method find_matching_gate_paths & get_child_gate_ids

Bug fixes:

* Corrected error types (several instances of ValueError changed to TypeError)
* BUGFIX: Fixed issue where adding new gates to a Session where a WSP was imported would cause an error (issue 110)

Documentation:

* Several docstrings updated and fixed for ReadTheDocs
* Example notebooks moved to docs folder & added to ReadTheDocs

New features:
* Support for Python 3.10 (drops support for Python 3.6)
* All C extensions moved to FlowUtils (and FlowUtils now provides pre-built wheels - should avoid the need for a compiler for most users)
* New function `extract_wsp_sample_data` to parse FlowJo 10 wsp & extract Sample metadata (keywords) to a Python dictionary
* New GatingStrategy method `get_max_depth` to get max depth of gate tree
* New Session method `summary` for a quick summary of what the session contains
* Sample class now supports exporting new FCS files with metadata (89)

API changes:
* Sample attribute `channel_lin_log` removed, lin / log metadata now available as "pne" column in Sample.channels attribute
* Sample class: `export` method has new kwarg `include_metadata` for including all key/value pairs in self.metadata in the output FCS file

Bug fixes:
* Fixed typo in FCS_STANDARD_KEYWORDS for 'date' keyword
* Fixed parsing of FlowJo workspaces where sample groups have "gate-less" samples (91)
* Replace special quote characters that caused a UnicodeDecodeError on some Windows installs (97)

Dependencies:
* FlowIO requirement updated to 1.0.0
* FlowUtils requirement updated to 1.0.0 (with pre-built binaries for most platforms)
* NumPy minimum requirement of 1.20 when using pre-built FlowUtils binaries (required for NumPy C API compatibility)
* Fix deprecation warnings from Pandas 1.4 (squeeze kwarg removed from `read_csv`)

Acknowledgements:
* Special thanks to sbenz for help testing this release on arm64 Macs (Apple Silicon)

* Sample Class
* Error is raised when retrieving compensated or transformed events prior to applying a compensation matrix or a transform (used to issue just a warning)
* plot_scatter_matrix argument order changed to be more consistent with other plot methods
* Session Class
* Analyzing multiple samples using multiprocessing now uses a more conservative estimate of memory to avoid running out of system memory
* BUGFIX: Fixed failure to properly parse FlowJo workspaces containing an ellipse gate.
* Documentation was updated to include a table of contents with links to methods for the complex classes: Sample, GatingStrategy, GatingResults, & Session

* GatingResults get methods now accept a `gate_path` argument to disambiguate re-used gates
* BUGFIX: fixed issue parsing FlowJo 10 wsp files with reused gates that contain a child gate (issue 84)
* Suppressed output for analyzing Session samples when verbose=False
* Suppress FlowKit generated warnings when running tests

New features:
* Support for Python 3.9
* FlowKit version now available via top-level attribute `__version__`
* GatingStrategy:
* optimized processing order of gates to avoid re-processing
* optional caching of pre-processed events to speed up processing of gates
* Matrix class now has an `inverse` method
* WSPBiexTransform now has an `inverse` method
* Significant speed increase for parsing FlowJo 10 WSP files with lots of bi-ex transforms
* Sample method `plot_contour` now has optional `fill` argument to created filled contours
* Sample now re-applies a transform when Sample is given a new compensation
* Analyzing samples in a Session using multiprocessing will now try to dynamically allocate CPUs based on available memory to avoid out of memory errors

Removed features:
* Anomalous event detection removed from FlowKit, now lives in separate FlowQC project

API changes:
* gates module:
* All gate types now take a pandas DataFrame as input (possible to use gates independently now)
* Sample class:
* `subsample` argument added back to Sample constructor (better to have subsample ready for plot methods)
* Sample plot methods (except histogram and plot channel) now default to `subsample=True`
* The various methods for getting NumPy arrays of events are now encapsulated into the single `get_events` methods that takes a `source` argument (consistent w/ other Sample methods)
* Method `get_channel_data` renamed to `get_channel_events`
* Added option to cache original events, they are no longer stored by default (saves memory)
* Attribute `anomalous_indices` renamed to `flagged_indices`, and now set by user for any purpose using `set_flagged_events` method
* Attribute `channels` changed from a dictionary to a more readable pandas DataFrame
* New plotting method `plot_channel` for plotting channel data with "events" as x-axis (similar to plotting vs Time)
* GatingStrategy class:
* `gate_path` is now always a tuple (not a list)
* Uses of `gate_id` replaced with `gate_name`: a gate ID is a tuple of (gate_name, gate_path)
* Method `get_gate_indices` renamed to `get_gate_membership` (since it returns a boolean array)
* Session class:
* Plotting methods no longer analyze samples, `analyze_samples` method must be called prior
* Method `get_gate_indices` renamed to `get_gate_membership` (since it returns a boolean array)
* Dimension class
* Attribute `label` renamed to `id`
* Method `get_gate` now returns the containing QuadrantGate when a Quadrant ID is given
* Function `calculate_extent` removed from public API (was never intended for external use)

Bug fixes:
* Improved support for Windows
* multiprocessing now working
* fixed issue where parent gate results were not being properly retrieved, causing lots of re-processing
* fixed memory leak in points_in_polygon C extension (used for processing PolygonGate instances)
* fixed issue in C extension code that would sometimes cause mis-allocation of NumPy array sizes

Documentation:
* New & comprehensive tutorial notebook series for new FlowKit users
* New notebook demonstrating how to create a custom Sample gate in a Session
* More docstrings for classes and methods

Dependencies:
* FlowIO requirement updated to 0.9.12 (significantly faster reading of FCS files)
* FlowUtils requirement updated to 0.9.5
* new requirement networkx >= 2.3
* new requirement psutil ~= 5.8.0

New features:
* It's now easier to assign multiple samples to a Session group
* Session.assign_samples can take either a Sample ID or list of Sample IDs
* Session.add_samples now takes an optional group argument for assigning to a group when loading Samples
* New Session method `get_wsp_gated_events` for conveniently getting a list of DFs of compensated & transformed gated events
* More informative error message when reloading the same samples (include file name in error message)
* Transforms now support handling of 1-D data arrays

API changes:
* Session: renamed method `assign_sample` to `assign_samples` now that it supports a list of sample IDs
* Session methods `get_sample_ids()` & `get_group_sample_ids()` now have a `loaded_only` option to only return sample IDs of loaded samples
* Session: method `get_gate` now has a `sample_id` option allowing retrieval of custom sample gates in a sample group's gating strategy

Bug fixes:
* Eliminate Pandas PerformanceWarning when creating GatingResults instances (occurred when getting gate indices)
* Fixed issue where Session.export_wsp fails to export rectangle gates that include transforms
* Fixed issue where Session.get_group_sample_ids() incorrectly reports samples as members when that are not part of that group
* Test data and code are no longer included in packaged

Documentation:
* Showcase exporting to wsp in replicate flowjo example notebook
* More Session methods documented with docstrings
* Cleaned up examples folder so it is easier for users to browse
* New example notebooks for:
* Demonstrating a comparison of clustering methods (Leiden vs Louvain)
* Demonstrating dimension reduction on processed event data

Dependencies:
* FlowUtils requirement updated to 0.9.4
* NumPy requirement updated to >=1.19
* Seaborn requirement updated to >=0.11
* Pandas requirement updated to >=1.1