Pytroll-collectors

-------------------
- Update changelog. [Panu Lahtinen]
- Bump version: 0.8.1 → 0.8.2. [Panu Lahtinen]
- Merge pull request 28 from pnuu/bugfix-segments-empty-fileset. [Panu
Lahtinen]

Bugfix segment gatherer empty fileset
- Add tests for INI config on dataset without critical files. [Panu
Lahtinen]
- Fix unittests. [Panu Lahtinen]
- Fix mock import for Python 3. [Panu Lahtinen]
- Add a INI config for data having no critical segments. [Panu Lahtinen]
- Add a check that empty channel/segment combination isn't added. [Panu
Lahtinen]
- Merge pull request 27 from pytroll/feature-skip-empty-slots. [Panu
Lahtinen]

Do not publish empty slots from segment gatherer
- Do not publish empty slots from segment gathere. [Martin Raspaud]
- Merge pull request 26 from yufeizhu600/master. [Panu Lahtinen]

add on_moved process to watchdog trigger.
- Fix the _process calling issue. [Yufei Zhu]
- Put common processing part of watchdog trigger into a '_process'
method. [Yufei Zhu]
- Add on_moved process to watchdog processor. [Yufei Zhu]
- Merge pull request 25 from mraspaud/fix-multifile-messages. [Panu
Lahtinen]

Ignore inconvenient messages in segment gatherer
- Ignore inconvenient messages in segment gatherer. [Martin Raspaud]

-------------------
- Update changelog. [Panu Lahtinen]
- Bump version: 0.8.0 → 0.8.1. [Panu Lahtinen]
- Merge pull request 24 from pytroll/bugfix-Pass-single-sensor. [Panu
Lahtinen]

Pass only one instrument to overpass calculations
- Fix a typo: use attribute instead of variable. [Panu Lahtinen]
- Ensure only one instrument is used in Pass() [Panu Lahtinen]
- Merge pull request 23 from pytroll/bugfix-sensors-in-a-list. [Panu
Lahtinen]

Pass only one sensor name to overpass calculations
- Remove duplicate format tags from filename patterns. [Panu Lahtinen]
- Ensure that only one instrument is passed to Pass() calculations.
[Panu Lahtinen]
- Merge pull request 22 from pytroll/feature-faster-image-scaler. [Panu
Lahtinen]

Make scale_image faster
- Clarify log message. [Panu Lahtinen]
- Remove use of .get_data(), replace old with new if mode/shape doesn't
match. [Panu Lahtinen]

-------------------
- Update changelog. [Panu Lahtinen]
- Bump version: 0.7.0 → 0.8.0. [Panu Lahtinen]
- Merge pull request 21 from pytroll/bugfix-pattern-time-name. [Panu
Lahtinen]

Fix adjust_pattern_time_name to use public trollsift functions
- Fix adjust_pattern_time_name to use public trollsift functions. [David
Hoese]
- Merge pull request 19 from pytroll/scale-images-overviews. [Panu
Lahtinen]

Add overview support to scale_images
- Fix save options, add overview settings. [Panu Lahtinen]
- Set PPP_CONFIG_DIR. [Panu Lahtinen]
- Use areas.yaml instead of areas.def. [Panu Lahtinen]
- Merge pull request 20 from pytroll/feature-satpy-independency. [Panu
Lahtinen]

Feature satpy independency
- Bug fix: Use ConfigParser instead of RawConfigParser. [Adam.Dybbroe]
- Add log info. [Adam.Dybbroe]
- Make satpy optional. [Adam.Dybbroe]
- Fix test correct results. [Panu Lahtinen]
- Fix handling of missing/empty segment string. [Panu Lahtinen]
- Merge branch 'master' of github.com:pytroll/pytroll-collectors.
[Adam.Dybbroe]
- Ensure `sensors` is a list. [Panu Lahtinen]
- Fix sensor name collection. [Panu Lahtinen]
- Remove obsolete keyword argument. [Panu Lahtinen]
- Merge pull request 18 from pytroll/bugfix-segment_gatherer. [Panu
Lahtinen]

Fix checks for slot readiness in segment gatherer
- Fix slot readines, fix reading variable tags from config. [Panu
Lahtinen]
- Merge pull request 17 from pytroll/bugfix-fill_value. [Panu Lahtinen]

Fix fill value handling in image scaler
- Adjust image mode to match overlay, if necessary. [Panu Lahtinen]
- Add fill_value to save options, fix value used to check masking. [Panu
Lahtinen]
- Update image scaler unit tests. [Panu Lahtinen]
- Fix copy-paste typo, use single-value fill_value. [Panu Lahtinen]
- Fix testing for NoneType. [Panu Lahtinen]
- Fix typo in variable name. [Panu Lahtinen]
- Get the datatype min/max from the input image. [Panu Lahtinen]
- Fix fill value handling. [Panu Lahtinen]
- Change default fill_value to None, read a single fill_value from
config. [Panu Lahtinen]
- Merge pull request 16 from TAlonglong/develop. [Panu Lahtinen]

Add pyinotify watch to directories created under current watched directory
- Merge branch 'master-fork' into develop-fork. [Trygve Aspenes]
- Possible to more than one directory separated by comma. [Trygve
Aspenes]
- Added feature inotify watch new directories. [Trygve Aspenes]
- Try catch exception when there is an OSError. [Adam.Dybbroe]

-------------------

Fix
~~~
- Bugfix: The destination server should go in the message and not the
host. [Adam.Dybbroe]

Other
~~~~~
- Update changelog. [Panu Lahtinen]
- Bump version: 0.6.0 → 0.7.0. [Panu Lahtinen]
- Merge pull request 15 from pytroll/develop. [Panu Lahtinen]

Add Python 3 support
- Merge pull request 13 from pytroll/feature-python3-support. [Panu
Lahtinen]

Add Python 3 support
- Get the items explicitly as a list so that the checks work. [Panu
Lahtinen]
- Fix handling of empty item strings. [Panu Lahtinen]
- Use RawConfigParser instead of ConfigParser. [Panu Lahtinen]
- Handle empty item strings, use six to import config parser. [Panu
Lahtinen]
- Fix import of queue.Empty. [Panu Lahtinen]
- Fix dictionary usage in iteration and indexing. [Panu Lahtinen]
- Fix ConfigParser to match imported RawConfigParser. [Panu Lahtinen]
- Handle iterators properly in both Py2 and Py3. [Panu Lahtinen]
- Add EPSG:4326 projection. [Panu Lahtinen]
- Use SatPy and Trollimage for I/O. [Panu Lahtinen]
- Fix imports, use RawConfigParser. [Panu Lahtinen]
- Move WorldCompositeDaemon to own test class. [Panu Lahtinen]
- Remove unused arguments. [Panu Lahtinen]
- Remove blending, update tests. [Panu Lahtinen]
- Use size attributes, fix exceptions, fix area attribute handling, fix
blending. [Panu Lahtinen]
- Remove unused argument from read_image() and _get_existing_image()
[Panu Lahtinen]
- Add checks that listener and publisher are there before stopping them.
[Panu Lahtinen]
- Use satpy Scene to read and save images, refactor everything. [Panu
Lahtinen]
- Remove extra space. [Panu Lahtinen]
- Use six and reorder imports. [Panu Lahtinen]
- Use get_area_def from satpy instead of mpop. [Panu Lahtinen]
- Fix lambda syntax to support Python 3. [Panu Lahtinen]
- Use urllib.parse for Python 3. [Panu Lahtinen]
- Import from configparser for Python 3, and from ConfigParser for
Python 2. [Panu Lahtinen]
- Replace print statements with print() [Panu Lahtinen]
- Add sensors to collection metadata. [Panu Lahtinen]
- Fix filename patterns for MSG HRIT files. [Panu Lahtinen]
- Merge pull request 11 from pytroll/feature_multiple_patterns. [Panu
Lahtinen]

Add support to multiple filename patterns for segment gatherer
- Merge branch 'feature_multiple_patterns' of https://github.com/pytroll
/pytroll-collectors into feature_multiple_patterns. [Panu Lahtinen]
- Fix key for delayed files from string 'uid' to variable uid. [Panu
Lahtinen]
- Add more comments to segment gatherer example configs. [Panu Lahtinen]
- Add tests for using .ini config file. [Panu Lahtinen]
- Add PRO and EPI to all_files and wanted_files. [Panu Lahtinen]
- Add more unittests. [Panu Lahtinen]
- Restructure code for easier testing. [Panu Lahtinen]
- Fix patterns, add variable tags. [Panu Lahtinen]
- Add test config for two non-segmented filesets. [Panu Lahtinen]
- Set orig_platform_name as variable tag. [Panu Lahtinen]
- Add all one and two dataset combinations to
test_get_collection_status() [Panu Lahtinen]
- Clarify if-elif-else structure. [Panu Lahtinen]
- Fix behaviour when noncritical set is the only set. [Panu Lahtinen]
- Add check for SLOT_NOT_READY in case when other sets are ready. [Panu
Lahtinen]
- Fix behaviour when slot is ready but wanted files are missing. [Panu
Lahtinen]
- Add unittests for segments.py. [Panu Lahtinen]
- Handle completed slot correctly when timeout has occured. [Panu
Lahtinen]
- Handle missing itm_str inside _compose_filenames() [Panu Lahtinen]
- Add example config for collecting HRPT and PPS files. [Panu Lahtinen]
- Fix handling missing wanted/all segment option. [Panu Lahtinen]
- Fix typo: config -> self._config. [Panu Lahtinen]
- Add absolute path when reading configs in unittests. [Panu Lahtinen]
- Add unittests for __init__ [Panu Lahtinen]
- Add test configs for segment gatherer unit tests. [Panu Lahtinen]
- Add unit test file for segment gatherer. [Panu Lahtinen]
- Move YAML config reading to helper_functions.py. [Panu Lahtinen]
- Set publish topic in setup_messaging() [Panu Lahtinen]
- Move messaging init to a method, move pub/sub to class instances.
[Panu Lahtinen]
- Delete obsolete main() [Panu Lahtinen]
- Move config.ini reading from main() to segments.ini_to_dict() [Panu
Lahtinen]
- Move ini_to_dict from main script to segments.py. [Panu Lahtinen]
- Add example config for collecting multiple sets of files. [Panu
Lahtinen]
- Restructure for multiple collectable sets / filename patterns. [Panu
Lahtinen]
- Change "required" to "is_critical_set" [Panu Lahtinen]
- Stop testing parsers if correct is found. [Panu Lahtinen]
- Add comments, comment out optional settings. [Panu Lahtinen]
- Add example YAML config for MSG/0deg segment gatherer. [Panu Lahtinen]
- Use start_time instead of nominal_time in filename patterns. [Panu
Lahtinen]
- Fix examples for MSG/0deg and MSG/RSS segments. [Panu Lahtinen]
- Set nameservers to None by default. [Panu Lahtinen]
- Fix YAML config loading. [Panu Lahtinen]
- Add separate config reading for .ini and .yaml files. [Panu Lahtinen]
- Split segment gatherer to separate main and library files. [Panu
Lahtinen]
- Rename original segment_gatherer.py. [Panu Lahtinen]
- Fix key for delayed files from string 'uid' to variable uid. [Panu
Lahtinen]
- Add more comments to segment gatherer example configs. [Panu Lahtinen]
- Add tests for using .ini config file. [Panu Lahtinen]
- Add PRO and EPI to all_files and wanted_files. [Panu Lahtinen]
- Add more unittests. [Panu Lahtinen]
- Restructure code for easier testing. [Panu Lahtinen]
- Fix patterns, add variable tags. [Panu Lahtinen]
- Add test config for two non-segmented filesets. [Panu Lahtinen]
- Set orig_platform_name as variable tag. [Panu Lahtinen]
- Add all one and two dataset combinations to
test_get_collection_status() [Panu Lahtinen]
- Clarify if-elif-else structure. [Panu Lahtinen]
- Fix behaviour when noncritical set is the only set. [Panu Lahtinen]
- Add check for SLOT_NOT_READY in case when other sets are ready. [Panu
Lahtinen]
- Fix behaviour when slot is ready but wanted files are missing. [Panu
Lahtinen]
- Add unittests for segments.py. [Panu Lahtinen]
- Handle completed slot correctly when timeout has occured. [Panu
Lahtinen]
- Handle missing itm_str inside _compose_filenames() [Panu Lahtinen]
- Add example config for collecting HRPT and PPS files. [Panu Lahtinen]
- Fix handling missing wanted/all segment option. [Panu Lahtinen]
- Fix typo: config -> self._config. [Panu Lahtinen]
- Add absolute path when reading configs in unittests. [Panu Lahtinen]
- Add unittests for __init__ [Panu Lahtinen]
- Add test configs for segment gatherer unit tests. [Panu Lahtinen]
- Add unit test file for segment gatherer. [Panu Lahtinen]
- Move YAML config reading to helper_functions.py. [Panu Lahtinen]
- Set publish topic in setup_messaging() [Panu Lahtinen]
- Move messaging init to a method, move pub/sub to class instances.
[Panu Lahtinen]
- Delete obsolete main() [Panu Lahtinen]
- Move config.ini reading from main() to segments.ini_to_dict() [Panu
Lahtinen]
- Move ini_to_dict from main script to segments.py. [Panu Lahtinen]
- Add example config for collecting multiple sets of files. [Panu
Lahtinen]
- Restructure for multiple collectable sets / filename patterns. [Panu
Lahtinen]
- Change "required" to "is_critical_set" [Panu Lahtinen]
- Stop testing parsers if correct is found. [Panu Lahtinen]
- Add comments, comment out optional settings. [Panu Lahtinen]
- Add example YAML config for MSG/0deg segment gatherer. [Panu Lahtinen]
- Use start_time instead of nominal_time in filename patterns. [Panu
Lahtinen]
- Fix examples for MSG/0deg and MSG/RSS segments. [Panu Lahtinen]
- Set nameservers to None by default. [Panu Lahtinen]
- Fix YAML config loading. [Panu Lahtinen]
- Add separate config reading for .ini and .yaml files. [Panu Lahtinen]
- Split segment gatherer to separate main and library files. [Panu
Lahtinen]
- Rename original segment_gatherer.py. [Panu Lahtinen]
- Merge pull request 10 from TAlonglong/feature-publish-message-at-
each-update. [Panu Lahtinen]

Added feature to publish the accumulated message after each new segme…
- Merge branch 'develop' into feature-publish-message-at-each-update.
[Panu Lahtinen]
- Make sure that listener and publisher are stopped even after crash.
[Panu Lahtinen]
- Move listener and publisher stop() commands outside the loop. [Panu
Lahtinen]
- Stop also publisher at KeyboardInterrupt. [Panu Lahtinen]
- Add GDAL as optional reader (needed for LA mode images) [Panu
Lahtinen]
- Rename GOES-R to GOES-16. [Panu Lahtinen]
- Update result images. [Panu Lahtinen]
- Prepare to check also LA mode images, check that image shapes match.
[Panu Lahtinen]
- Make it possible to use both RGBA and LA mode images. [Panu Lahtinen]
- Merge branch 'develop' of github.com:pytroll/pytroll-collectors into
develop. [Adam.Dybbroe]
- Merge branch 'develop' of https://github.com/pytroll/pytroll-
collectors into develop. [Panu Lahtinen]
- Try to import get_area_def from satpy.resample, fallback to mpop if
not found. [Panu Lahtinen]
- Fix missing orbit number in NOAA-20 messages. [Adam.Dybbroe]
- Remove redundant check for server. [Adam.Dybbroe]
- Bugfix, remove pdb! [Adam.Dybbroe]
- Bugfix JPSS-1, and handle inconsistent url's from new 2met.
[Adam.Dybbroe]
- Bugfix - messages from new 2met on Merlin. [Adam.Dybbroe]
- Allow url with the ip-adress in addition to host name. [Adam.Dybbroe]
- Add support for the new scisys dispatching messages. [Martin Raspaud]
- Need to check if last file was added to the area. Else message where
sent each time a new file arrived even if the area was not updated.
[Trygve Aspenes]
- Added handeling of service and providing server. [Trygve Aspenes]
- Added feature to publish the accumulated message after each new
segment is received. Eg when collecting EARS ascat bufr data. [Trygve
Aspenes]
- Merge pull request 9 from pytroll/develop. [Panu Lahtinen]

Merge develop to master

-------------------

Fix
~~~
- Bugfix: yaml config reading. [Adam.Dybbroe]

Other
~~~~~
- Update changelog. [Panu Lahtinen]
- Bump version: 0.5.1 → 0.6.0. [Panu Lahtinen]
- Merge pull request 8 from TAlonglong/develop. [Panu Lahtinen]

Develop
- Bin/geo_gatherer.py possible to configure services used with
ListenerContainer. Possible config of providing_server, skipping all
messages not from the providing-server. [Trygve Aspenes]
- Bin/cat.py possible to configure service used with Subscribe. [Trygve
Aspenes]
- Merge remote-tracking branch 'upstream/develop' into develop. [Trygve
Aspenes]
- Pytroll_collectors/trigger.py propagate nameserver into the system.
[Trygve Aspenes]
- Bin/gatherer.py handling nameserver. [Trygve Aspenes]
- Pytroll_collectors/region_collector.py if tle_platform_name in
metadata use this as platform name. Need to introduce this to handle
when TLE platform name differs form given platform name. [Trygve
Aspenes]
- Pytroll_collectors/region_collector.py adding end_time based on
start_time and duration of not given. [Trygve Aspenes]
- Merge pull request 7 from pytroll/feature-listener-port. [Panu
Lahtinen]

Expose listener addresses to segment gatherer
- Set PPP_CONFIG_DIR in setup.py, not in unit test. [Panu Lahtinen]
- Catch NoOptionError for area definition, as it's not required for PIL
images. [Panu Lahtinen]
- Fix areaname. [Panu Lahtinen]
- Add areas.def and mpop.cfg files. [Panu Lahtinen]
- Set PPP_CONFIG_DIR. [Panu Lahtinen]
- Use mpop built-in area. [Panu Lahtinen]
- Fix tests: add milliseconds to start and end times. [Panu Lahtinen]
- Make excluded_satellite_list kwarg. [Panu Lahtinen]
- Fix typo in error message. [Panu Lahtinen]
- Comment out extra listener addresses. [Panu Lahtinen]
- Add required "tcp://"s to example config. [Panu Lahtinen]
- Revert renaming: "nameserver" back to "nameservers" [Panu Lahtinen]
- Expose listener addresses, publish_port and rename nameservers to
nameserver. [Panu Lahtinen]
- Bugfix, take care of tenths of seconds in RDR filename. [Adam.Dybbroe]
- Bugfix list of excluded satellites, default is an empty list.
[Adam.Dybbroe]
- Bugfix. [Adam.Dybbroe]
- Add debug message. [Adam.Dybbroe]
- Prepare for JPSS-1 and add platform exlude list. [Adam.Dybbroe]
- Bugfix. [Adam.Dybbroe]
- Change to use yaml configuration file, and add post-hook (e.g for
nagios monitoring) [Adam.Dybbroe]
- Copy file fisrt to a temporary filename, then move it to the correct
name on the same directory. [Adam.Dybbroe]
- More debug info. [Adam.Dybbroe]
- Add zipcollector runner. [Adam.Dybbroe]
- Check that next_img is not None before trying to use it. [Panu
Lahtinen]
- Retry reading image once after 5 s wait. [Panu Lahtinen]
- Merge pull request 6 from pytroll/feature-wrapping-crop. [Panu
Lahtinen]

Add cropping for areas wrapping around from right edge to left edge
- Add cropping for areas wrapping around from right edge to left edge.
[Panu Lahtinen]
- Merge pull request 5 from pytroll/feature_no_memory_cache. [Panu
Lahtinen]

Do not cache overlays in memory
- Do not cache overlays in memory. [Panu Lahtinen]
- Merge pull request 4 from pytroll/feature_publishercontainer. [Panu
Lahtinen]

Feature publisher
- Add published message to log when file is written. [Panu Lahtinen]
- Use NoisyPublisher directly. [Panu Lahtinen]
- Stop compositor daemon. [Panu Lahtinen]
- Add new message settings. [Panu Lahtinen]
- Remove obsolete and unused file. [Panu Lahtinen]
- Use posttroll.publish.PublisherContainer for sending messages. [Panu
Lahtinen]
- Fix comparison of parsed filename parts. [Panu Lahtinen]

The comparison was made incorrectly against the class attribute, not
local variable

-------------------
- Update changelog. [Panu Lahtinen]
- Bump version: 0.5.0 → 0.5.1. [Panu Lahtinen]
- Add Python3 configparser, try to get log config from a file. [Panu
Lahtinen]
- Handle "ValueError: corrupted page" when reading TIFF images. [Panu
Lahtinen]