Spidermon

Latest version: v1.21.0

Safety actively analyzes 620872 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 1 of 5

1.21.0

- bug: Fix CI/CD pipelines not working due to changes on Scrapy (`PR426 <https://github.com/scrapinghub/spidermon/pull/426>`_)
- bug: Fix TypeError in ``PeriodicItemCountMonitor`` when item_scraped_count was not defined the first time it was checked (`PR436 <https://github.com/scrapinghub/spidermon/pull/436>`_)
- bug: Fix ``calculate_field_coverage`` incorrect handling of field names ending with _items (`PR437 <https://github.com/scrapinghub/spidermon/pull/437>`_)
- bug: Fix monitor failing when operating offset-naive and offset-aware datetimes (`PR439 <https://github.com/scrapinghub/spidermon/pull/439>`_)
- bug: Fix tests for minimum properties and items in JSON Schema validations (`PR432 <https://github.com/scrapinghub/spidermon/pull/432>`_)
- feature: Add the ``SPIDERMON_DICT_FIELDS_COVERAGE_LEVELS`` setting to limit nested dict stats computation (`PR433 <https://github.com/scrapinghub/spidermon/pull/433>`_)
- feature: Add ``SendSNSNotification`` to Spidermon Notifications (`PR413 <https://github.com/scrapinghub/spidermon/pull/413>`_)
- feature: Change the format of the content of _validation from defaultdict to dict (`PR431 <https://github.com/scrapinghub/spidermon/pull/431>`_)

1.20.0

-------------------

- feature: Add support for nested fields in the ``SPIDERMON_VALIDATION_ERRORS_FIELD`` setting (`PR416 <https://github.com/scrapinghub/spidermon/pull/417>`_)
- bug: Fix ItemValidationPipeline using the ``__setitem__`` method without ItemAdaper (`PR415 <https://github.com/scrapinghub/spidermon/pull/415>`_)
- feature: Improve test coverage for the item validation pipeline (`PR414 <https://github.com/scrapinghub/spidermon/pull/414>`_)
- bug: Fix ``spidermon.utils.zyte.Client`` not following crawler settings (`PR411 <https://github.com/scrapinghub/spidermon/pull/411>`_)
- chore: Migrate ReadTheDocs to config file v2 (`PR408 <https://github.com/scrapinghub/spidermon/pull/408>`_)
- chore: Refactor package requirements (`PR407 <https://github.com/scrapinghub/spidermon/pull/407>`_)

Backward-incompatible changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- The ``SPIDERMON_VALIDATION_ERRORS_FIELD`` setting no longer supports a field
name that contains a dot (``.``). Dots are now be treated as field
separators, to support specifying a nested field.

1.19.0

-------------------

- bug: Fix ``JobTagsAction`` failing due to ``JobMetadata`` not longer supporting ``__setitem__`` method (`PR404 <https://github.com/scrapinghub/spidermon/pull/404>`_)
- feature: Add ``PeriodicItemCountMonitor`` to check for increase in item count. Also included ``PeriodicItemCountMonitorSuite`` suite (`PR402 <https://github.com/scrapinghub/spidermon/pull/402>`_)
- chore: Deprecate support for data validation using ``schematics`` (`PR399 <https://github.com/scrapinghub/spidermon/pull/399>`_)
- chore: Drop support for Python 3.6 and Python 3.7. Added support for Python 3.11 (`PR398 <https://github.com/scrapinghub/spidermon/pull/398>`_)
- feature: Add ability to pass kwargs to Slack APIs (`PR397 <https://github.com/scrapinghub/spidermon/pull/397>`_)

1.18.0

-------------------

- feature: Support setting the ``Return-Path`` for Amazon SES emails (`PR381 <https://github.com/scrapinghub/spidermon/pull/381>`_)
- bug: Fix ``BaseStatMonitor`` failing in Scrapy Cloud when settings were provided as strings (`PR378 <https://github.com/scrapinghub/spidermon/pull/378>`_)
- feature: Add setting ``SPIDERMON_FIELD_COVERAGE_SKIP_IF_NO_ITEM`` to allow skipping ``FieldCoverageMonitor`` if no items were scraped (`PR372 <https://github.com/scrapinghub/spidermon/pull/372>`_)
- feature: Add ``Fallback Actions``. ``Action`` now allows to define ``fallback`` field that may contain an additional ``Action`` to be executed if an exception is raised during the main action (`PR365 <https://github.com/scrapinghub/spidermon/pull/365>`_)
- feature: Use ``ItemAdapter`` when working with items to support the same types of item as Scrapy (`PR358 <https://github.com/scrapinghub/spidermon/pull/358>`_)
- chore: Refactor code from ``spider.contrib.scrapy`` into ``base``, ``monitors`` and ``suites`` subpackages (`PR386 <https://github.com/scrapinghub/spidermon/pull/386>`_)
- chore: Replace ``tox pep8`` functionality with ``pre-commit`` git hooks (`PR387 <https://github.com/scrapinghub/spidermon/pull/387>`_)
- chore: Update contributing guidelines to include reference to ``pre-commit`` tool (`PR392 <https://github.com/scrapinghub/spidermon/pull/392>`_)

1.17.1

-------------------

- bug: Fix Slack dependency name issue (`PR367 <https://github.com/scrapinghub/spidermon/pull/367>`_)
- chore: Change the Ubuntu version on workflow settings (`PR373 <https://github.com/scrapinghub/spidermon/pull/373>`_)
- docs: Adding PeriodicExecutionTimeMonitor to the batteries docs (`PR368 <https://github.com/scrapinghub/spidermon/pull/368>`_)
- feature: Adding the use of ItemAdapter to prevent assumptions of item nature (`PR358 <https://github.com/scrapinghub/spidermon/pull/358>`_)
- misc: Fix compatibility issues with jsonschema>=4 (`PR364 <https://github.com/scrapinghub/spidermon/pull/364>`_)

1.17.0

-------------------

- feature: Updated `DownloaderExceptionMonitor` and `ItemValidationMonitor` to inherit from `BaseStatMonitor` (`PR334 <https://github.com/scrapinghub/spidermon/pull/334>`_, `PR#335 <https://github.com/scrapinghub/spidermon/pull/335>`_)
- feature: Updated Slack action to use `slack-sdk <https://pypi.org/project/slack-sdk/>`_ as library in replacement of deprecated `slackclient <https://pypi.org/project/slackclient/>`_ (`PR#313 <https://github.com/scrapinghub/spidermon/issues/313>`_)
- feature: Added new action to allow to send notification to Discord channels (`PR348 <https://github.com/scrapinghub/spidermon/pull/348>`_)
- feature: Added Python 3.10 support (`PR349 <https://github.com/scrapinghub/spidermon/pull/349>`_)
- feature: Added new action to allow to send email notifications using SMTP server (`PR345 <https://github.com/scrapinghub/spidermon/pull/345>`_)
- misc: small bug fixes and documentation improvements that can be checked in the `milestone summary <https://github.com/scrapinghub/spidermon/milestone/13?closed=1>`_.

Page 1 of 5

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.