Payu

- Manifest support has been added. This feature will generate a manifest
which enables more rigorous checksums for model inputs and greater
reproducibility of runs.
- `PAYU_REPRODUCE` environment variable denotes a reproducible run using
manifests.
- Job statistics and environment variables are now saved as output files.
Several new environmental variables have also been added to facilitate
logging of jobs. This information is also logged as
- Minimal implementation of a Scheduler class. Currently only bare support
for PBS.
- Core dumps can be explicitly enabled using the `coredump` config flag.

Bugfixes:
- Many calendar calculations were updated to use explict integer division.
- Several issues related to MITgcm calendar and time interval support have
been fixed and improved.

- Payu collation jobs launched by an initial payu job would sometimes fail
if the LD_LIBRARY_PATH was not set correctly (usually in a `payu-run`
job). We now prepend the executable's LD_LIBRARY_PATH before collation.

- `runlog: False` was not correctly handled due to changes in the Runlog
object, and has been fixed. Thanks to Nic Hannah for reporting.

- Several OSError checks throughout the module were not being handled
correctly in Python 3, where most of these have been replaced by various
more specific exceptions. We now check for the more generic
EnvironmentError and handle the error code as before. Thanks to Navid
Constantinou for reporting.

- The f90nml parser used by MITgcm was not correctly removing comments from
`data.mnc` config files. We now use the MITgcm-configured parser.

- Userscripts that were not executable but required python may have failed
due to a generic `python` execution. We now use the same executable as
the one used to launch Payu.

- mnctools is now an optional dependency to be installed when MITgcm
collation support is needed. This will also install numpy and netCDF4,
which can be a significant burden if unneeded, and why this is optional.

- globbing support for netCDF files in mppnccombine-fast

- PYTHONPATH is now passed directly to the next job. Previously, we tried
to derive this variable from `sys.path` after removing all the default
paths, but the remainder was virtually identical to PYTHONPATH.

- Runscripts are now constructed by setuptools via `entry_point` rather than
the explicit scripts in the `bin` directory. This was done in part to
accommodate conda environments on NCI. Thanks to Scott Wales for
outlining this method.

- The "reversion" technique of relaunching payu to a fixed version has been
removed, and we now use the version specified at execution.

- Payu is now nominally compatible with Python 2.6, 2.7, and most newer 3.x
releases. Minor issues are expected, and to be fixed in later versions.
2.6 support:
- Implicit string arguments replaced with explicit parameters
- backport of `subprocess.check_output` (public code by Eduardo Felipe)
- Dictionary comprehensions have been removed
3.x support:
- `multiprocessing.dummy` is now replaced with `multiprocessing` in FMS
collation support

Note: Much of the 2.6 support can be reversed when we've officially
stopped using it.

- Payu runtime environment is now strictly defined at execution, and no
longer relies on environment modules for setup. Includes the following:
- Explicit paths to the python executable, rather than relying on `PATH`
- Safer generation of PYTHONPATH to include active version of Payu when
installed outside of default `sys.path`.
- Explicit tracking of payu command line tools
(Note: For now, this assumes a common directory for all tools)
- Python and Payu modules are no longer loaded within payu
(Note: PBS is still loaded for queuing support)
- `LD_LIBRARY_PATH` is constructed and explicitly set for Python runtime
- The `PAYU_MODULENAME` and `PAYU_MODULEPATH` environment variables have
been removed

- Simple test automation using Travis CI. The environment is still
incomplete and only a few command line tests are included, but it includes
a functional Torque (PBS) server for future job submission tests.

- Minor documentation update (thanks to Navid Constantinou for reporting)

Bugfixes

- `runlog: True` no longer causes a runtime crash, due to conflicts with
the complete `runlog` config dictionary tree.
(Thanks to Paul Spence for reporting)

- `payu collate -d $DIR` no longer uses $DIR as `jobname` when this entry is
absent from `config.yaml`. We now default to the directory name itself,
rather than the full path.

We have also added support for a `jobname` record inside the full
`collate:` config dictionary tree.

- `dateutil` is now identified as a Payu dependency

Aidan Heerdegen:
- collate config in `config.yaml` replaces `collate_*` config with
subkeys under `collate:`
- Support for Scott Wales' `mppnccombine-fast` collation
- Multithreaded collate errors are gathered and reported more
thoroughly
- `cli.get_config` was removed and calls replaced with
`fsops.read_config`
Marshall Ward:
- Minor usability improvements to GitHub experiment sync
- MOM (temporarily) produces `log` directories to support OM2 logging

- Reverting the optional_config_files directory chance back to work_input,
since it was breaking MOM6.