Pydda

PyDDA is moving to an Xarray-based data structure (a PyDDA Grid) for all of its retrievals. This will make it easier for PyDDA to take in any xarray-based dataset as inputs. In addition, we are now also using xarray DataTrees to support grid nesting using a tree structure. Documentation and examples on this nesting have been added to the user guide and example gallery.

The migration of your code to PyDDA 2.0 will involve small changes to the pipeline. Instead of using pyart.io.read_grid to read your grid, simply use pydda.io.read_grid! In addition, you can convert a PyART Grid to a PyDDA Grid) by using pydda.io.read_from_pyart_grid. In addition, PyDDA will automatically convert PyART grids that are specified as inputs to PyDDA grids in the get_dd_wind_field function, though it is recommended to use PyDDA grids as inputs since the initalization and constraint functions use PyDDA Grids.

* Moved to ruff for linting and added pre-commit hooks to better ensure code quality in PRs.
* Expanded unit testing of Jax/TensorFlow cost functions to resolve stability/syntax issues with these engines (i.e. issue 94)

Today we have released PyDDA 1.3.0. This version has three major bugfixes:
* The weights from the final radar in the specified list were being set to zero by default. Now the coverage for the last radar is calculated correctly (Addressing https://github.com/openradar/PyDDA/issues/89 ).
* The yz-section plotting functions were not plotting the contours onto the specified axis, but rather whatever was the matplotlib current axis. This is now fixed.
* Smoothness constraints between the three engines are now consistent with each other (numpy engine was not dividing by grid size). I recommend setting Cx, Cy, and Cz to 1e-4 for best results.

New features:
* You can now specify the input initial state as u, v, and w fields in the first input Grid to `get_dd_wind_field`. The initialization procedures now dump the initial state into this variables in your Py-ART Grids automatically for you so that you don't have to specify these three parameters in get_dd_wind_field anymore. This reduces the number of minimum parameters for` get_dd_wind_field` to 1. In addition, this is useful if you want to save the state of the retrieval after many iterations, then apply low pass filters before resuming the retrieval to remove high frequency noise.
* get_dd_wind_field now returns 2 arguments, the list of grids and the parameters used for the retrieval. This allows the user to see the weights that PyDDA pre-calculates for each grid. Therefore, when you are using `get_dd_wind_field`, you'll want to call it using `grids, params = get_dd_wind_field([grids], ...)`.

In this release, we have added an augmented Lagrangian solver to PyDDA that allows the user to be able to perform retrievals without having to select weights beforehand. Right now this solver has been tested using the mass continuity and radar observational cost functions. In order to enable this functionality, set the engine to "auglag" and then set the Co and Cm weights to 1.0.

We are glad to finally release PyDDA 1.0. In this version we have fixed the problems noted by issues 58 ,63 , and 64. In addition, we have been working hard to enhance the optimizer inside PyDDA. Therefore, we now provide the user with an option to use either Jax or TensorFlow (in addition to the current SciPy engine) to solve the optimization problem.

There are several advantages to the Jax and TensorFlow-based engines that we *highly* encourage our users to use. With Jax and TensorFlow, we are able to use automatic differentiation to calculate the gradients to the cost function. This makes the gradient calculation less susceptible to roundoff and boundary errors. In addition, Jax and TensorFlow both support CUDA-enabled GPUs. Therefore, if you are using a GPU, PyDDA is now capable of harnessing it in order to dramatically speed up the wind retrieval calculation. Even on a CPU-based system, the TensorFlow-based algorithm typically converges faster than the original SciPy-based algorithm. Therefore, we strongly encourage users to use these two other engines in their retrievals.

TensorFlow and Jax are optional dependencies. You need both TensorFlow 2.6.0 and tensorflow-probability in order to use the TensorFlow functionality.

In this new release we now support:
* Integration of point observations as a constraint
* Automatic download of ASOS data from the Iowa Mesonet Archive for a given grid as a source of point observations
* Fixes to the documentation
* Reorganization of the cost function/gradient code to make development easier