Flickr-uploader

Fixes/Features
- Setup via PyPi
- Fixed 73: Weird issue on Synology with python installed from default packages (DO NOT INSTALL IT) . Updated README.
- Fixed 74: Initial logging has an erratic behaviour (Low priority)
- Fixed/Feature: uploadr.cron: Proper use of PREFIX and PATH settings
- Feature: At the end prints the total for (countflickr - countlocal) for easier reading.
- Feature 71: Suggestion: no.delete: New option `--no-delete-from-flickr` (under testing.Use at your own risk for now!)
- Feature 78: Enhancement: Mask sensitive data (like file name and Albums) on Logs. New option: `-m`/`-mask-sensitive`
- Cleaned up debugging of external modules namely multiprocessing logging.
- Testing with python 2.7, 3.6 and 3.7
- Refactored all DB function calls. Protected by error code hangling and lock usage whenm applicable

Setup options:
- (For alpha versions like this one) You can install from pip:
`pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple flickr-uploader==2.8.7a36 --prefix=~/apps/Python/ --no-cache-dir`
- (For regular versions) You can install from pip:
`pip install flickr-uploader --prefix=~/apps/Python`
- You can download and run install with command:
`python3 setup.py install --prefix=~/apps/Python --old-and-unmanageable`
- You can download and run from local folder.
- uploadr.ini: **Note that uploadr.ini definition for path search** is changed:
- Sets base folder with `FOLDER = os.path.abspath(os.getcwd())`
- DB_PATH, LOCK_PATH, TOKEN_CACHE and TOKEN_PATH are now relative to FOLDER, for example: `os.path.join(%(FOLDER)s, "flickrdb")`
- From v2.8.6 onwards it looks for `uploadr.ini` as follows:
1. Use `--config-file` argument option to indicate where to find the file. Example `--config_file uploadr.ini`
1. If not, `os.path.dirname(sys.argv[0])`. Example: ~/apps/Python/bin/uploadr.ini or ./uploadr.ini
1. If not, `os.path.dirname(sys.argv[0]), '..', 'etc', 'uploadr.ini'`. Example: ~/apps/Python/etc/uploadr.ini

Output Messages
- Adjustments on output and logging messages.
- As a reference from version 2.8.6: New set of options related to Rotating Error Logging added to uploadr.ini:


Output logging information into a rotating set of log file(s).
ROTATING_LOGGING to Enable/Disable
ROTATING_LOGGING_PATH location of folder/main logging filename
ROTATING_LOGGING_FILE_SIZE for maximum file size of each log file
ROTATING_LOGGING_FILE_COUNT for maximum count of old log files to keep
ROTATING_LOGGING_LEVEL Level Logging
Check LOGGING_LEVEL setting for options.
Normally set ROTATING_LOGGING_LEVELto lower than LOGGING_LEVEL

ROTATING_LOGGING = True
ROTATING_LOGGING_PATH = os.path.join(os.path.dirname(sys.argv[0]), "logs", "uploadr.err")
ROTATING_LOGGING_FILE_SIZE = 25*1024*1024 25 MBytes
ROTATING_LOGGING_FILE_COUNT = 3
ROTATING_LOGGING_LEVEL = 30


Environment and Coding
- Python 2.7 + 3.6 + 3.7 compatibility: use of " noqa" as applicable
- Development now use coveralls.io to check source code coverage
- autopep8, PEP8, flakes and pylint3 (not all!) adjustments. Several pylint reaching now a rating of 9.45/10.00.
- pytest --flakes & pytest --doctest-modules
- Runs several unittests
- For installation:
- pip install flickr-uploader
- setup.py (**optional use**)
- or manual
- NicePrint:
- use of staticmethods
- New argument logalso
- niceprocessedfiles new argument for adaptable output message
- On upload failure: sleep for(configured on Konstants.py) 20s (instead of 10) to avoid duplicated uploaded files.

Fixes/Features
- Setup via PyPi
- Fixed 73: Weird issue on Synology with python installed from default packages (DO NOT INSTALL IT) . Updated README.
- Fixed 74: Initial logging has an erratic behaviour (Low priority)
- Fixed/Feature: uploadr.cron: Proper use of PREFIX and PATH settings
- Feature: At the end prints the total for (countflickr - countlocal) for easier reading.
- Feature 71: Suggestion: no.delete: New option `--no-delete-from-flickr` (under testing.Use at your own risk for now!)
- Feature 78: Enhancement: Mask sensitive data (like file name and Albums) on Logs. New option: `-m`/`-mask-sensitive`
- Cleaned up debugging of external modules namely multiprocessing logging.
- Testing with python 2.7, 3.6 and 3.7
- Refactored all DB function calls. Protected by error code hangling and lock usage whenm applicable
- NicePrint use of staticmethods

Setup options:
- (For alpha versions like this one) You can install from pip:
`pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple flickr-uploader==2.8.7a36 --prefix=~/apps/Python/ --no-cache-dir`
- (For regular versions) You can install from pip:
`pip install flickr-uploader --prefix=~/apps/Python`
- You can download and run install with command:
`python3 setup.py install --prefix=~/apps/Python --old-and-unmanageable`
- You can download and run from local folder.
- uploadr.ini: **Note that uploadr.ini definition for path search** is changed:
- Sets base folder with `FOLDER = os.path.abspath(os.getcwd())`
- DB_PATH, LOCK_PATH, TOKEN_CACHE and TOKEN_PATH are now relative to FOLDER, for example: `os.path.join(%(FOLDER)s, "flickrdb")`
- From v2.8.6 onwards it looks for `uploadr.ini` as follows:
1. Use `--config-file` argument option to indicate where to find the file. Example `--config_file uploadr.ini`
1. If not, `os.path.dirname(sys.argv[0])`. Example: ~/apps/Python/bin/uploadr.ini or ./uploadr.ini
1. If not, `os.path.dirname(sys.argv[0]), '..', 'etc', 'uploadr.ini'`. Example: ~/apps/Python/etc/uploadr.ini

Output Messages
- Adjustments on output and logging messages.
- As a reference from version 2.8.6: New set of options related to Rotating Error Logging added to uploadr.ini:


Output logging information into a rotating set of log file(s).
ROTATING_LOGGING to Enable/Disable
ROTATING_LOGGING_PATH location of folder/main logging filename
ROTATING_LOGGING_FILE_SIZE for maximum file size of each log file
ROTATING_LOGGING_FILE_COUNT for maximum count of old log files to keep
ROTATING_LOGGING_LEVEL Level Logging
Check LOGGING_LEVEL setting for options.
Normally set ROTATING_LOGGING_LEVELto lower than LOGGING_LEVEL

ROTATING_LOGGING = False
ROTATING_LOGGING_PATH = os.path.join(os.path.dirname(sys.argv[0]), "logs", "uploadr.err")
ROTATING_LOGGING_FILE_SIZE = 25*1024*1024 25 MBytes
ROTATING_LOGGING_FILE_COUNT = 3
ROTATING_LOGGING_LEVEL = 30


Environment and Coding
- Python 2.7 + 3.6 + 3.7 compatibility: use of " noqa" as applicable
- Development now use coveralls.io to check source code coverage
- autopep8, PEP8, flakes and pylint3 (not all!) adjustments. Several pylint reaching now a rating of 9.45/10.00.
- pytest --flakes & pytest --doctest-modules
- Runs several unittests
- For installation:
- pip install flickr-uploader
- setup.py (**optional use**)
- or mannual

Fixes/Features
- Setup via PyPi
- Several under the hood code refactoring, pylint3 check and more.
- Drop unused setting TOKEN_PATH.
- Added verbosity control with argument (-v, -vv, etc...) on NicePrint class.
- Fix 46 Use current working directory to look for INI (uploadr.ini) file
- Now support for multiprocessing for feature: 68 Option to have Rotating Error Log written to files to keep size under control.
- Setup options:
- You can install from pip:
`pip install flickr-uploader --prefix=~/apps/Python`
- You can download and run install with command:
`python3 setup.py install --prefix=~/apps/Python --old-and-unmanageable`
- You can download and run from local folder.
- uploadr.ini: **Note that uploadr.ini definition for path search** is changed:
- Sets base folder with `FOLDER = os.path.abspath(os.getcwd())`
- DB_PATH, LOCK_PATH, TOKEN_CACHE and TOKEN_PATH are now relative to FOLDER, for example: `os.path.join(%(FOLDER)s, "flickrdb")`
- From v2.8.6 onwards it looks for `uploadr.ini` as follows:
1. Use `--config-file` argument option to indicate where to find the file. Example `--config_file uploadr.ini`
1. If not, `os.path.dirname(sys.argv[0])`. Example: ~/apps/Python/bin/uploadr.ini or ./uploadr.ini
1. If not, `os.path.dirname(sys.argv[0]), '..', 'etc', 'uploadr.ini'`. Example: ~/apps/Python/etc/uploadr.ini

Output Messages
- Adjustments on output and logging messages.
- Feature: 68 Option to have Rotating Error Log written to files to keep size under control.
- New set of options related to Rotating Error Logging added to uploadr.ini:


Output logging information into a rotating set of log file(s).
ROTATING_LOGGING to Enable/Disable
ROTATING_LOGGING_PATH location of folder/main logging filename
ROTATING_LOGGING_FILE_SIZE for maximum file size of each log file
ROTATING_LOGGING_FILE_COUNT for maximum count of old log files to keep
ROTATING_LOGGING_LEVEL Level Logging
Check LOGGING_LEVEL setting for options.
Normally set ROTATING_LOGGING_LEVELto lower than LOGGING_LEVEL

ROTATING_LOGGING = False
ROTATING_LOGGING_PATH = os.path.join(os.path.dirname(sys.argv[0]), "logs", "uploadr.err")
ROTATING_LOGGING_FILE_SIZE = 25*1024*1024 25 MBytes
ROTATING_LOGGING_FILE_COUNT = 3
ROTATING_LOGGING_LEVEL = 30


Environment and Coding
- Python 2.7 + 3.6 compatibility: use of " noqa" as applicable
- autopep8, PEP8, flakes and pylint3 (not all!) adjustments
- pytest --flakes & pytest --doctest-modules
- Runs several unittests
- For installation:
- pip install flickr-uploader
- setup.py (**optional use**)
- or mannual

Fixes/Features
- Several under the hood code refactoring, pylint3 check and more.
- Fix: 66 fix Flickr error: "ValueError: invalid literal for int() with base 10"
- Feature: 60 Initial authentication option (-a)
- Internal improvement: 64 Isolate and wrap around retry and try/except sequence to flickrapi function calls
- Feature: 68 Option to have Rotating Error Log written to files to keep size under control.
- Setup (**optional**):
- You can run install with command:
`python3 setup.py install --prefix=~/apps/Python`
- You can run installcfg to copy config files (.ini and .cron) to a designated folder with command:
`python3 setup.py installcfg --folder ~/apps/flickr-uploader`
- From v2.7.6 `uploadr.ini` configuration is searched from:
- `--config-file` argument
- from the sys.argv[0] path (compatible with previous versions)

Questions & Answers
- **Note that uploadr.ini definition for path search** of DB_PATH, LOCK_PATH, TOKEN_CACHE and TOKEN_PATH still uses sys.argv[0] location as a basis.
- (to be) Updated installation notes on Readme.

Output Messages
- Adjustments on output and logging messages.
- Feature: 68 Option to have Rotating Error Log written to files to keep size under control.
- New set of options related to Rotating Error Logging added to uploadr.ini:


Output logging information into a rotating set of log file(s).
ROTATING_LOGGING to Enable/Disable
ROTATING_LOGGING_PATH location of folder/main logging filename
ROTATING_LOGGING_FILE_SIZE for maximum file size of each log file
ROTATING_LOGGING_FILE_COUNT for maximum count of old log files to keep
ROTATING_LOGGING_LEVEL Level Logging
Check LOGGING_LEVEL setting for options.
Normally set ROTATING_LOGGING_LEVELto lower than LOGGING_LEVEL

ROTATING_LOGGING = False
ROTATING_LOGGING_PATH = os.path.join(os.path.dirname(sys.argv[0]), "logs", "uploadr.err")
ROTATING_LOGGING_FILE_SIZE = 25*1024*1024 25 MBytes
ROTATING_LOGGING_FILE_COUNT = 3
ROTATING_LOGGING_LEVEL = 30


Environment and Coding
- Python 2.7 + 3.6 compatibility: use of " noqa" as applicable
- autopep8, PEP8, flakes and pylint3 (not all!) adjustments
- pytest --flakes & pytest --doctest-modules
- Runs several unittests
- setup.py (**optional use**)
- setup.py installcfg (**optional use**)

Fixes/Features
- Several under the hood code refactoring, avoid use of globals, use of (more) modules, pylint3 check, order of imports, remove unused import, docstrs, use logging % instead of .format; and more.
- 61 Use mprocessing function to setup multiprocessing and simplify code.
- Setup (**optional**):
- You can run install with `python3 setup.py install --prefix=~/apps/Python`
- From v2.7.6 `uploadr.ini` configuration is searched from:
- `--config-file` argument
- from the sys.argv[0] path (compatible with previous versions)
- Initial lock control (Linux/Windows compatible) to ensure only one copy is running was corrected.
- Exception handling on (more) sql functions.

Questions & Answers
- **Note that uploadr.ini definition for path search** of DB_PATH, LOCK_PATH, TOKEN_CACHE and TOKEN_PATH still uses sys.argv[0] location as a basis.
- (to be) Updated installation notes on Readme.

Output Messages
- Adjustments on output and logging messages.

Environment and Coding
- Python 2.7 + 3.6 compatibility: use of " noqa" as applicable
- autopep8, PEP8, flakes and pylint3 (not all!) adjustments
- pytest --flakes & pytest --doctest-modules
- Runs several unittests
- setup.py (**optional use**)

Fixes/Features
- 11 Added feature to improve performance on Add Pics to Sets(Albums)
- 57 Used control of closing connection to Database.
- Setup CONVERT_RAW_FILES to **True** on `uploadr.ini` configuration file.
- Uses external tool: [exiftool by Phil Harvey](https://sno.phy.queensu.ca/~phil/exiftool/).
- Dry run option (-n) also works wit convert RAW files.
- Setup (**optional**):
- You can run install with `python3 setup.py install --prefix=~/apps/Python`
- From v2.7.6 `uploadr.ini` configuration is searched from:
- `--config-file` argument
- from the sys.argv[0] path (compatible with previous versions)
- moved niceprocessedfiles function to niceprint module
- created mprocessing function for multiprocessing scenarios
- initial lock control to ensure only one copy is running was corrected
- exception handling on (more) sql functions

Questions & Answers
- **Note that uploadr.ini definition for path search** of DB_PATH, LOCK_PATH, TOKEN_CACHE and TOKEN_PATH still uses sys.argv[0] location as a basis.
- (to be) Updated installation notes on Readme.

Output Messages
- Small adjustments on output messages.

Environment and Coding
- Python 2.7 + 3.6 compatibility: use of " noqa" as applicable
- autopep8, PEP8, flakes adjustments
- pytest --flakes & pytest --doctest-modules
- Runs several unittests
- Created setup.py (**optional use**)