Pyuvdata

Added
- A new `FastUVH5Meta` object that enables quick partial reading of uvh5 file metadata.
This is used by default in the `UVData.read_uvh5` method to initialize a `UVData`
object, without any apparent change for the user. To get the benefits of the new class,
use it directly to interface with the metadata portion of a `.uvh5` file.

Changed
- Updated numpy requirements to `>=1.20`.
- Updated scipy requirements to `>=1.5`.

Fixed
- Handling of antenna_names and antenna_numbers in `read_fhd` and `read_fhd_cal`.

Added
- A new `interpolation_function` parameter to `UVBeam.interp` and `UVBeam.to_healpix`
to allow the function name to be passed into the methods, with sensible defaulting.
- Support for selecting on phase center IDs, including on read.
- Several new attributes to UVFlag: `telescope_name`, `telescope_location`,
`antenna_names`, `antenna_numbers`, `antenna_positions`, `channel_width`, `spw_array`,
`flex_spw_id_array`, enabling support for flexible spectral windows. All of these except
`flex_spw_id_array` will eventually be required for all UVFlag object types, as will
`Nspws` (which used to only be required for baseline and antenna types). These
parameters all already exist on UVData and UVCal objects and UVFlag objects initialized
from them will inherit them. UVFlag objects will also inherit these parameters when
they are converted between types using UVData and UVCalobjects.
- The `UVFlag.set_telescope_params` method, similar to the ones on UVData and UVCal,
to set several of these new parameters.

Changed
- Increases the h5py minimum version to 3.1.
- Increase the minimum compatible version for lunarsky to 0.2.1 to fix astropy
deprecation warnings.

Fixed
- Fixed some bugs related to handling of the `UVCal.time_range` parameter in
`UVCal.__add__` and other methods. Ensured that it is a list when
`UVCal.future_array_shapes` is False.
- Fixed a bug in `UVCal.__add__` when data are sorted differently along any axis or
interleaved along an axis.
- Fixed a bug in reading MS files with non-UTC time scales.
- Fixed some bugs in UVData, UVFlag and UVCal `__add__` and `select` methods for objects
with only one spectral window and `flex_spw_id_array` defined. Also fixed some handling
in those methods on UVFlag for objects with multiple spectral windows.
- Fix a bug when reading FHD files into UVData objects when the layout file was not
passed but the antenna_positions are in the known telescopes.
- Fixed a bug where the beamfits partial read parameters were not available in the
`UVBeam.from_file` class method and were not passed properly when reading multiple files.
- Fixed a bug where objects created from scratch with the old phase attributes weren't
properly getting converted to the new `phase_center_catalog` representation. That
conversion now happens in the check so will happen before any `write`. There could still
be errors if some methods are called before a check is done (or if the check is turned
off).
- Fix a bug in how `frame_pa` was calculated in `phase` and `_set_app_coords_helper` for
multi_phase_center objects (it was using the old `phase_center_frame` attribute).
- Fix a bug where trying to select lsts or lst_ranges on read didn't work for some file
types.
- Severe performance hit when calling `polnum2str` and its variants for many baselines.

Deprecated
- Reading files into objects without setting `use_future_array_shapes` now results in
deprecation warnings.
- The utility functions `phase_uvw` and `unphase_uvw` associated with the deprecated
old phasing method.
- The `flex_spw_id_array` will be required on all UVData and UVFlag and all
non-wide-band UVCal objects in version 3.0.
- Deprecated the `interpolation_function` attribute on UVBeams.
- Deprecated the older phase attributes (`phase_type`, `phase_center_ra`,
`phase_center_dec`, `phase_center_frame`, `phase_center_epoch`, `object_name`) in favor
of the `phase_center_catalog`. The older phase attributes will be removed in version 3.0.
- The `lst_from_uv` function in the uvflag module.

Removed
- Removed deprecated handling for UVCal objects without `telescope_location`,
`antenna_positions` and `lst_array` parameters defined.

Changed
- Updated how wheels for PyPI are built.

Added
- A frame attribute to the telescope_location parameters on UVData, UVCal and UVFlag
objects to support observatories on the moon (or anywhere not on earth).

Fixed
- A bug in `UVData.read_uvh5` where `multi_phase_center` was being used to imply the
existence of the `phase_center_catalog` header item, rather than checking for the
presence of that item. This conflicted with the uvh5 memo.

Added
- A new subclass of UVParameter for SkyCoord objects, needed in pyradiosky.
- Added `UVData.convert_to_flex_pol` method to enable writing uvh5 files with the
polarization axis changing slowest as desired by HERA. Also updated
`UVData.remove_flex_pol` to properly undo the `UVData.convert_to_flex_pol` operation.
- Added an option to `UVData.read` and `UVData.from_file` methods when reading uvh5
type files to call `remove_flex_pol` as part of the read. This is defaulted to `True`
so that when HERA data is written with flex_pol the change will not disrupt users, but
other users of uvh5 may want to set it to `False`.
- Added support to make bitshuffle compression easier when writing UVData objects to
uvh5 files
- Support for partial reads on beamfits files. This can be done on frequency, azimuth
or zenith angle (if beam is in az/za coordinates).
- Logging statements (INFO level) in `UVData.check`.

Changed
- Significantly increased uvh5 reading and writing speed.
- Added `use_future_array_shapes` method to UVBeam to allow users to convert to using
the future array shapes now, with support throughout UVBeam methods and related utility
functions. Also added `use_current_array_shapes` to revert to the standard shapes.
- Major performance improvement in the `UVBeam.to_healpix` method.
- Performance improvement when doing a `UVData.select` using the `bls` parameter (~70%
improvement in the `_select_preprocess` function)

Fixed
- A bug in `UVData.check` when `check_autos` is True and the object is a flex pol object.

Added
- Added `use_future_array_shapes` method to UVFlag to allow users to convert to using the
future array shapes now, with support throughout UVFlag methods and related utility
functions. Also added a `use_future_array_shapes` parameter to UVFlag's `__init__`,
`read`, `from_uvdata` and `from_uvcal` methods to allow the conversion to be done at the
time the object is made. Added `use_current_array_shapes` to revert to the standard shapes.
- Baseline number calculations for antenna numbers > 2047.
- New uvh5 version 1.1 definition in memo and code to use the future phasing info
(i.e. the phase_center_catalog) rather than the older parameters.
- Added the `normalize_by_autos` method to `UVData`, which allows one to normalize
cross-correlations (i.e., covert values from arbitrary scale to correlation coefficients)
if auto-correlations are present in the data.
- Added the `write` method to `MirParser`, which allows for `MirParser` to write out
Mir-formatted data to disk.
- Added the `select` method to `MirParser`, which provides a much simpler interface for
selecting subsets of data to load.
- Added the `rechunk` method to `MirParser`, which allows data to be spectrally averaged
(either after already loading or to be averaged "on-the-fly" when data are read from
disk).
- Added the `redoppler_data` method to `MirParser`, which enables frequency shifting of
the data.
- Added new `__add__` and `__iadd__` methods to `MirParser` for combining data sets.
- Added new functionality to `UVBeam.check` to verify that power beams for the auto
polarizations (and pstokes) are real-only, along with an option to force them to be
real-only if non-zero imaginary components are detected.
- Coarse band correction for mwax to `mwa_corr_fits`
- Reordering methods for UVCal: `reorder_antennas`, `reorder_freqs`, `reorder_times`
and `reorder_jones`.

Changed
- Always write out uvh5 version 1.0 or greater files (which use the future array shapes) with `UVData.write_uvh5`.
- Use "unprojected" rather than "unphased" for the cat_type in the phase_center_catalog
to describe zenith driftscan data that have no w-projections applied.
- The `MirParser` class has been significantly overhauled, which significantly reduces
memory usage and improves processing speed.
- Updated minimum dependency versions: numpy>=1.19, scipy>=1.3, optional dependencies:
python-casacore>=3.3, pyyaml>=5.1, astropy-healpix>=0.6
- `UVBase` object now require that individual attribute names match that given in
`UVParameter._name`.
- `UVData.fix_phase` now raises a warning when called.
- Changed `UVData.write_uvfits` to allow for one to write out datasets in UVFITS format
without "spoofing" (via setting `spoof_nonessential=True`) UVFITS-specific values.
- Methods that ensure that attributes are set up properly on `UVBeam` are now called in
the `UVBeam.check` method, making it easier to fill in UVBeam objects from scratch.
- Changed uvfits reading and writing to no longer subtract/add 1 to the antenna
numbers. The subtraction/addition of 1 was initially done because FITS uses one-based
indexing but antenna numbers are physical numbers not indices, so the numbers should not
be adjusted.
- Changed the way baseline numbers are calculated from antenna numbers to conform to the
standard in uvfits and miriad. The inconsistency with those files was noticed when we
corrected the handling of antenna numbers in uvfits.
- The `spw_order` parameter for the `UVData.reorder_freqs` method now accepts an index
array rather than an array of spw numbers, making it match the other reorder methods.
- Updated the astropy requirement to >= 5.0.4
- Dropped support for python 3.7

Fixed
- Store the phase_center_catalog as a set of nested datasets in uvh5 files rather than
as a json blob.
- A bug in `UVData.__add__` where flexible spectral window datasets were not combined
correctly if they contained overlapping frequencies in different spectral windows.
- A bug in `UVData.print_phase_center_info` that occasionally resulted in incorrect
values being reported for RA/Az/Longitudinal coordinates.
- A bug in `UVData.select` that could cause `UVData.check` to fail if
`UVData._scan_number_array` was set.
- A bug in `UVBeam.select` where after selecting down to only auto polarization power
beams the `UVBeam.data_array` remained complex instead of real.
- A bug in `UVBeam.__add__` where adding an object with cross pol power beams to an
object with only auto pol power beams could result in loss of the imaginary part of the
cross pol power beams.
- Testing of piping keywords through `read` by `test_mwa_corr_fits`.
- Incorrect piping of `flag_dc_offset` keyword.
- `mwa_corr_fits` handing of the `BSCALE` keyword in gpubox files.

Deprecated
- The "unphased" cat_type in phase_center_catalog.