Pyuvdata

Added
- Support for new lunar ellipsoids in `lunarsky` version 0.2.2.
- Added new keyword handling for v.6 of the MIR data format within `MirParser`.

Changed
- Updated minimum dependency versions: astropy>=6.0, h5py>=3.4, numpy>=1.23,
pyerfa>=2.0.1.1, scipy>=1.7.3, python>=3.10
- Updated minimum optional dependency versions: hdf5plugin>=3.2.0, pytest>=6.2.5,
python-casacore>=3.5.2, pyyaml>=5.4.1, lunarsky>=0.2.2
- Speed up for reading MWA correlator FITS files into `UVData`.
- Now compatible with pytest>=8.0 but require pytest-cases>=3.8.3.

Fixed
- Fixed a bug where SMA-based coordinates were recorded as FK5 for sidereal sources
(should be ICRS).
- Bug in which passing 'ee' or 'nn' pols to UVData.new() would error, even if
`x_orientation` was passed.

Added
- Added a new redundancy finding algorithm based on the HERA gridding algorithm.
This new algorithm is used by default but the older clustering algorithm is still
available.
- Added support for initializing `MirParser` and `MirMetaData` objects from `Path`
objects.
- Added a brief tutorial on how to convert SMA MIR data into MS format (with some notes
on SMA specific keywords than can be used).
- Added the `Mir.generate_sma_antpos_dict` method for reading in SMA-formatted antenna
position files into a dict (which can be passed to `UVData.update_antenna_positions`).
- Added handling for older formatted (so called "ASIC-era") MIR files from SMA, as well
as more recent file formats that preceded pyuvdata support (circa 2020, v1/v2 of the
MIR file format).
- Added the ability to save/load selection masks in `MirParser` objects.
- Fixed a bug when trying to write a MS with a legacy array shape (but note legacy shape
will be deprecated in upcoming releases).
- Improved support for MIRIAD UV files, and added `use_miriad_convention` flag to the
UVFITS writer code, so files with the MIRIAD `BASELINE` convention can be created.
- A new `freq_interp_kind` parameter to `UVBeam.interp`, `UVBeam._interp_az_za_rect_spline`
and `UVBeam._interp_healpix_bilinear` to allow the frequency interpolation
specification to be passed into the methods. Note this defaults to "cubic" rather
than "linear" (the old default for the attribute of the same name on UVBeam objects)
because several groups have found that a linear interpolation leads to nasty
artifacts in visibility simulations for EoR applications.
- A new `UVBeam.new()` method (based on new function `new_uvbeam`) that creates a new,
self-consistent `UVBeam` object from scratch from a set of flexible input parameters.
- Added a the `UVData.update_antenna_positions` method to enable making antenna
position updates with corresponding updates the uvw-coordinates and visibility phases.
- Added a switch to `UVData.write_ms` called `flip_conj`, which allows a user to write
out data with the baseline conjugation scheme flipped from the standard `UVData`
convention.
- Added the `utils.determine_pol_order` method for determining polarization
order based on a specified scheme ("AIPS" or "CASA").
- Added a `check_surface_based_positions` positions method for verifying antenna
positions are near surface of whatever celestial body their positions are referenced to
(either the Earth or Moon, currently).

Changed
- Made the determination of whether or not to create a flex-pol dataset when reading
in a MIR more robust (particularly with pre-V3 data formats).
- Reading in of MIR data sets into `UVData` objects will now use pyuvdata-calculated
values for `lst_array` via (`UVData.set_lsts_from_time_array`) instead of those read in
from the file due to known precision issues in the later (~25 ms), so long as the two
agree within this known precision limit.
- `UVFlag.to_baseline` and `UVFlag.to_antenna` are now more robust to differences
in antenna metadata sorting.
- Made `MirParser` more robust against metadata indexing errors.
- Made `MirParser` handling of autocorrelation data more robust.
- Improved hooks for handling and loading of COMPASS solutions into `MirParser` objects.
- Fixed handling of weights to be consistent with tsys scaling method in `MirParser`.
- Changed `MWACorrFits.corrcorrect_simps` method to use the `scipy.integrate.simpson`
method rather than the `scipy.integrate.simps` method to fix deprecation warnings.
- added support for python 3.12, dropped support for python 3.8.
- Updated minimum dependency versions: pyyaml>=5.3
- Changed `UVData.write_ms` to sort polarizations based on CASA-preferred ordering.
- Added some functionality to the `utils._convert_to_slices` method to enable quick
assessment of whether an indexing array can be replaced by a single slice.
- Increased the tolerance to 75 mas (equivalent to 5 ms time error) for a warning about
values in `lst_array` not conforming to expectations for `UVData`, `UVCal`, and `UVFlag`
(was 1 mas) inside of `check`. Additionally, added a keyword to `check` enable the
tolerance value to be user-specified.
- Changed the behavior of checking of telescope location to look at the combination of
`antenna_positions` and `telescope_location` together for `UVData`, `UVCal`, and `UVFlag`.
Additionally, failing this check results in a warning (was an error).

Deprecated
- The `freq_interp_kind` attribute on UVBeams.
- The `spw_array` and `Nspws` attributes on UVBeam objects. Also the
`unset_spw_params` and `set_spw_params` parameters to the `use_future_array_shapes`
and `use_current_array_shapes` methods on UVBeam objects.
- Upper case feed names (e.g. "N" or "E") in UVBeam.feed_array. This was never
fully tested and didn't work properly.
- Having `freq_range` defined on non-wide-band gain style UVCal objects.
- Having `freq_array` and `channel_width` defined on wide-band UVCal objects.

Fixed
- A bug where `time_array` was not being correctly calculated for older (pre-V3) MIR
data formats.
- A small bug in UVFlag that could occur when reading in an older UVFlag HDF5
file with missing antenna metadata.
- A small bug (mostly affecting continuous integration) that threw an error when the
IERS service was down and when tests were using `test.check_warnings` to look for no
warnings thrown (i.e., where `match=None`).
- A couple of small bugs related to handling of the `freq_range` parameter in the
`reorder_freqs` and `__add__` methods on `UVCal`.

Added
- Added a `uvw_track_generator` method within `utils` for calculating the expected
uvws (and a few other values) without needing to instantiate a whole `UVData` object.
- Added a convenience function called `compare_value` in `UVParameter` that enables
one to do value checking with tolerances accounted for.
- New `mwa_metafits_file` and `telescope_name` optional parameters to `UVFlag.read` and
`UVFlag.__init__` to help with setting telescope metadata for old UVFlag files that are
missing it.
- MWA antenna metadata to our known telescopes to allow them to be filled in for old
UVFlag files. This is a stopgap solution, a better approach is to pass an MWA metafits
file to the new `mwa_metafits_file` parameter.
- Support for recarrays in `UVParameter` objects and in `UVBase`, needed for pyradiosky.
- Support for setting the astrometry library for various object methods including `set_lsts_from_time_array`, file read methods and others.
- Properly round-trip the telescope frame through UVH5, UVFITS and MS files.

Fixed
- A bug in apparent coordinate calculation that resulted in small errors/loss of
precision due to the way times were passed when using `erfa` astrometry library.
- A small correction due to polar drift for LST calculation when using the `erfa`
astrometry library.
- Fixed a bug in `utils.calc_app_coords` that occurred when supplying an astropy `Time`
object for the `time_array` argument.
- Fixed a bug in `UVData.write_ms` that caused the writer to crash if prior measurement
set history was not formatted in the currently expected fashion.

Added
- Support for multiple spectral windows in `UVData.frequency_average`, including a new
parameter `respect_spws` for controlling whether averaging crosses spectral window
boundaries.
- Better handling in `UVData.frequency_average` when averaging by a number of channels
that does not divide evenly into the number of channels in each spectral window.
- Compatibility with Python 3.11

Changed
- The following `UVFlag` parameters are now required: `Nspws`, `channel_width`,
`spw_array`, `telescope_name`, `telescope_location`, `antenna_names`, `antenna_numbers`,
and `antenna_positions`.
- The `quality_array` on UVCal objects is no longer required.

Deprecated
- The `sky_field` attribute on `UVCal`.

Fixed
- A bug in LST calculation that led to small discontinuities in LSTs calculated using
the `erfa` or `novas` astrometry libraries.
- Error when setting `UVBeam.freq_interp_kind` to an integer.
- Error when reading `mwa_corr_fits` files from the new MWAX correlator

Removed
- The `phase_uvw` and `unphase_uvw` utility methods associated with old style phasing.
- The `with_conjugates` option to the `get_baseline_redundancies` utility method in
favor of the `include_conjugates` option.
- Support for the `interpolation_function` attribute on UVBeams.
- Support for the "unphased" catalog type in UVData.phase_center_catalog in favor of
the "unprojected" catalog type.
- The `UVData.unphase_to_drift` method in favor of `UVData.unproject_phase` method.
- Support for using old style phasing on UVData objects (the `UVData.fix_phase` method
to fix datasets that were phased with the old style of phasing remains as well as the
`fix_old_proj` option on read for uvfits, miriad and uvh5 file types).
- Phasing related keywords in the following `UVData` methods: `__add__`, `__iadd__`,
`fast_concat`, `read` and `from_file`. Multiple phase centers are now fully supported
so datasets can always be combined without phasing. If desired, the full object can be
rephased to a single phase center afterwards.
- The `use_cotter_flags` and `flag_small_sig_ants` options to the `UVData.read` and
`UVData.from_file` methods for MWA correlator fits files in favor of the
`use_aoflagger_flags` and `flag_small_auto_ants` options respectively.
- The `spoof_nonessential` option to `UVData.write_uvfits` method as the previously
spoofed parameters are now properly calculated.

Added
- New dependency `docstring-parser` to enable dynamic docstring generation for
methods that mostly wrap other methods.
- A new `UVCal.new()` method (based on new function `new_uvcal`) that creates a new,
self-consistent `UVCal` object from scratch from a set of flexible input parameters.
- A new `UVData.new()` method (based on new function `new_uvdata`) that creates a new,
self-consistent `UVData` object from scratch from a set of flexible input parameters.
- A new `fast_concat` method on `UVCal`.
- A new generic `read` method on `UVCal` that, like the `read` methods on our other
objects, supports all file types and a new `from_file` class method to allow one-line
reading of UVCal files.

Deprecated
- The `input_flag_array` attribute on `UVCal`.
- Support for the 'unknown' cal_type in UVCal.
- Reading in multiple files to `UVCal` using file-type specific read methods
(e.g. `read_calfits`) in favor of the generic `read` method.

Fixed
- Removed error when `time_axis_faster_than_bls=True` and `Ntimes=1`. In the case of
`Ntimes=1`, it is inconsequential whether time-axis moves first or not, so it should
not be an error.

Added
- The `catalog_name` keyword has been added to the `UVData.read` and `UVData.select`
methods to allow users to select on source name.

Changed
- The keywords for `UVData.read` have been changed for MIR-specific values to more
generally match those founds with other data types.
- The `UVData.read_mir` method has been overhauled to mak reading in of MIR data more
efficiency in memory/processor usage.

Fixed
- Frequency frame information for MS datasets is now correctly recorded as "TOPO".