Jwql

===================

New Features
------------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Added API documentation for Bias, Bad Pixel, and Readnoise Monitors

Web Application
~~~~~~~~~~~~~~~

- Added webpage that describes how to use the JWQL web app API
- Added webpage that enables a user to query the ``jwqldb`` database contents
- Enabled more search options and aesthetic improvements for anomaly query webpage
- Added webpage for displaying Readnoise Monitor results with ``bokeh`` plots
- Added webpage for displaying Bad Pixel Monitor results with ``bokeh`` plots

``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Added support for Python versions 3.7 and 3.8
- Added unit tests for Readnoise Monitor
- Added unit tests for Bias Monitor
- Added unit tests for Bad Pixel Monitor
- Added unit tests for ``bokeh`` templating library


Bug Fixes
---------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Updated broken links to JWST instrument JDox pages

Web Application
~~~~~~~~~~~~~~~

- Fixed various issues that was cuasing authentication through ``auth.mast`` to fail

``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Fixed bug that was causing Dark Monitor to crash on recently added apertures from ``pysiaf``
- Fixed several bugs in ``bokeh`` templating library
- Fixed bug that was causing unit tests for ``permissions.py`` to fail
- Fixed bug that was causing ``most_recent_search`` function in Dark Monitor to fail

===================

New Features
------------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- The ``jwql_web_app`` PowerPoint presentation has been updated
- The bokeh templating software now has full API documentation
- Updated ``README`` and About webpage to reflect changes to development team members

Web Application
~~~~~~~~~~~~~~~

- Added webpage to view FITS headers of each extension for a given dataset
- Added webpage for displaying Dark Monitor results with ``bokeh`` plots
- Added webpage for viewing contents of a given JWQL database table
- Added webpage for querying and displaying anomaly results
- Added slider bar for easily navigating through integrations in preview image display
- The list of anomalies one can flag in a preview image is now instrument specific


``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- The ``jwql-3.6`` conda environment now uses the ``astroconda`` channel instead of ``astroconda-dev``
- Added Bias Monitor module, which currently monitors the bias levels for NIRCam
- Added Readnoise Monitor module, which monitors readnoise for all instruments except FGS
- Added Bad Pixel Monitor module, which tracks bad pixels for all instruments
- Cron job logs now include a print out of the complete ``conda`` environment being used


Bug Fixes
---------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Fixed broken link to ``numpydoc`` docstring convention in Style Guide

===================

New Features
------------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Added slides from July 2019 TIPS presentation to ``presentations/`` directory


``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Updated dark monitor to support all five JWST instruments, instead of only NIRCam
- Changed the ``jwql-3.5`` and ``jwql-3.6`` conda environments to be more simple and to work on Linux distributions
- Added library code for creating instrument monitoring ``bokeh`` plots with new ``bokeh`` templating software


Bug Fixes
---------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Fixed various bugs that were causing the ``sphinx`` API documentation to crash on ReadTheDocs

===================

New Features
------------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Updated ``README`` to include instructions on package installation via ``pip``.

Web Application
~~~~~~~~~~~~~~~

- Updated all webpages to conform to Web Application Accessibility Guidelines.
- Upgraded to ``django`` version 2.2.
- ``bokeh`` is now imported in ``base`` template so that the version being used is consistent across all HTML templates.

``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- The ``jwql`` package is now available on PyPI (https://pypi.org/project/jwql/) and installable via ``pip``.
- Updated Jenkins configuration file to include in-line comments and descriptions.
- Added ``utils`` function to validate the ``config.json`` file during import of ``jwql`` package.
- Added support for monitoring contents of the ``jwql`` central storage area in the filesystem monitor.


Bug Fixes
---------

Web Application
~~~~~~~~~~~~~~~

- Fixed position error of JWQL version display in footer.

``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Fixed spelling error in dark monitor database column names.
- Fixed dark monitor to avoid processing files that are not in the filesystem.

===================

New Features
------------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Updated the notebook exemplifying how to perform an engineering database (EDB) telemetry query.
- Updated the README for the ``style_guide`` directory.

Web Application
~~~~~~~~~~~~~~~

- Added form on preview image pages to allow users to submit image anomalies.
- Added buttons for users to download the results of EDB telemetry queries as CSV files.
- Enabled users to search for or navigate to program numbers without requiring leading zeros (i.e. "756" is now treated equivalently to "00756").
- Enabled authentication for EDB queries via the web login (rather than requiring authentication information to be present in the configuration file).
- Added custom 404 pages.
- Added adaptive redirect feature so that users are not sent back to the homepage after login.
- Added more descriptive errors if a user tries to run the web application without filling out the proper fields in the configuration file.

``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Replaced all EDB interface code within ``jwql`` with the new ``jwedb`` `package<https://github.com/spacetelescope/jwst-dms-edb>`_.
- Fully incorporated Python 3.5 testing into the Jenkins test suite.

Bug Fixes
---------

Web Application
~~~~~~~~~~~~~~~

- Fixed bug in which dashboard page would throw an error.
- Fixed incorrect dashboard axis labels.

===================

New Features
------------

Project & API Documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Added guidelines to the style guide for logging the execution of instrument monitors
- Added example usage of logging in the ``example.py`` module

Web Application
~~~~~~~~~~~~~~~

- Modified various web app views to enable faster loading times
- Modified archive and preview image views to only display data for an authenticated user
- Added views for MIRI and NIRSpec Data Trending Monitors, which monitors the behavior of select MIRI and NIRSpec Engineering Database mnemonics over time

``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Added Dark Monitor module, which monitors the dark current and hot pixel populations for each JWST instrument
- Added software for producing MIRI and NIRSpec Data Trending Monitors (described above)
- Modified ``generate_preview_images`` module to support the creation of preview images for stage 3 data products
- Refactored ``monitor_filesystem`` to utilize PostgreSQL database tables to store archive filesystem statistics
- Configured ``codecov`` for the project. The project homepage can be found at https://codecov.io/gh/spacetelescope/jwql
- Modified ``logging_functions`` module to enable dev, test, and production logging environments
- Added convenience decorator to ``logging_functions`` module to time the execution of a function or method
- Modified ``monitor_cron_jobs`` module to make use of updated ``logging_functions``

Bug Fixes
---------

Web Application
~~~~~~~~~~~~~~~

- Fixed API views to only return the basenames of file paths, instead of full directory names

``jwql`` Repository
~~~~~~~~~~~~~~~~~~~

- Fixed ``logging_functions`` module to properly parse new format of ``INSTALL_REQUIRES`` dependency in ``setup.py`` for logging system dependencies and their versions
- Fixed ``Jenkinsfile`` to not allow for one failed unit test in Jenkins builds