Scrapydweb

Latest version: v1.5.0

Safety actively analyzes 619291 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 1 of 3

1.5.0

------------------
- Update dependencies like Flask==2.0.0
- Support Python 3.6 to 3.12

1.4.1

1.4.0

------------------
- New Features
- Add API for sending text or alert via Slack, Telegram, or Email
- Improvements
- UI improvements on sidebar and multinode buttons
- Others
- Update config file to scrapydweb_settings_v10.py


[1.3.0](https://github.com/my8100/scrapydweb/issues?q=is%3Aclosed+milestone%3A1.3.0) (2019-08-04)
------------------
- New Features
- Add new pages Node Reports and Cluster Reports for aggregating jobs stats [(issue 72)](https://github.com/my8100/scrapydweb/issues/72)
- Improvements
- Adapt to [:link: *LogParser*](https://github.com/my8100/logparser) v0.8.2
- Add DATA_PATH option for customizing path to save program data [(issue 40)](https://github.com/my8100/scrapydweb/issues/40)
- Add DATABASE_URL option to support MySQL or PostgreSQL backend [(issue 42)](https://github.com/my8100/scrapydweb/issues/42)
- Support specify the latest version of Scrapy project in the Run Spider page [(issue 4)](https://github.com/my8100/scrapydweb/issues/4#issuecomment-475145676)
- Support specify default values of settings & arguments in the Run Spider page [(issue 55)](https://github.com/my8100/scrapydweb/issues/55)
- Others
- Update config file to scrapydweb_settings_v9.py
- Support continuous integration (CI) on [CircleCI](https://circleci.com/)

1.2.0

------------------
- New Features
- Support :alarm_clock: **Timer Tasks** to schedule a spider run periodically [(issue 4)](https://github.com/my8100/scrapydweb/issues/4)
- Persist jobs information in database [(issue 21)](https://github.com/my8100/scrapydweb/issues/21)
- Improvements
- Adapt to [:link: *LogParser*](https://github.com/my8100/logparser) v0.8.1, show Crawler.stats and Crawler.engine
in the Stats page if available.
- Support backing up stats json files in case the original logfiles are deleted by Scrapyd
- Support setting up EMAIL_USERNAME separately [(issue 28)](https://github.com/my8100/scrapydweb/issues/28)
- Introduce new UI for the Jobs, Logs, and Items page
- Add 'Sync from Servers page' checkbox in the Deploy Project and Run Spider page
- Rename 'Overview' to 'Servers', 'Dashboard' to 'Jobs'
- Others
- Change SCRAPYDWEB_SETTINGS_PY to 'scrapydweb_settings_v8.py'

1.1.0

------------------
- New Features
- **Integrated with [:link: *LogParser*](https://github.com/my8100/logparser)**
- Improvements
- Remove **HTML caching**
- Add 'List Stats' in the Overview page
- Show crawled_pages and scraped_items in the Dashboard page
- Bug Fixes
- Integrated with LogParser to avoid MemoryError when parsing large logfiles [(issue 11)](https://github.com/my8100/scrapydweb/issues/11)
- Support running ScrapydWeb in HTTPS mode [(issue 18)](https://github.com/my8100/scrapydweb/issues/18)
- Others
- Change SCRAPYDWEB_SETTINGS_PY to 'scrapydweb_settings_v7.py'

1.0.0

------------------
- Formal release of v1.0.0 :tada: :cake: :beer:
- Improvements
- Introduce new UI for the Deploy Project page
- Add 500.html for 'Internal Server Error'
- Bug Fixes
- Remove inject_variable() in the base view class **to avoid memory leak** [(issue 14)](https://github.com/my8100/scrapydweb/issues/14)
- Parse project name from scrapy.cfg instead of setting it to the folder name for auto packaging [(issue 15)](https://github.com/my8100/scrapydweb/issues/15)
- Fix the 'CheckAll / UncheckAll' checkbox and the 'Upload file' function in Safari
- Fix the 'go-top' and 'go-bottom' buttons in Firefox
- Fix faulty links in dropdown menu in the cached Log and Stats page of mobile UI
- Change SCRAPYDWEB_SETTINGS_PY to 'scrapydweb_settings_v6.py'

Page 1 of 3

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.