Intelmq

Latest version: v3.3.0

Safety actively analyzes 613603 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 1 of 7

3.3.0

---------------------

Documentation
The documentation is now available at [docs.intelmq.org](https://docs.intelmq.org/). Documentation has been updated and restructured into User, Administrator and Developer Guide. It provides modern look with various quality of life improvements. Big thanks to to gethvi.
We now have a slick, modern mkdocs based documentation. Please do check it out!


Bots
Shadowserver dynamic parser / collector

**Note well**: if you use shadowserver feeds, **please read this section carefully**.

Thanks to shadowserver (elsif2), we have a new dynamic shadowserver reports API integration. What does it do?
It connects to the [Shadowserver API](https://www.shadowserver.org/what-we-do/network-reporting/api-documentation/),
requests a list of all the reports for a specific country and processes the ones that are new.

Motivation for this change:

Shadowserver adds new scans on a nearly weekly basis. IntelMQ's release cycle and the need for a stable release could not keep up with this high intensity of shadowserver parser changes.
We therefore (thanks to eslif2) move the shadowserver reports collector and parser to a new, dynamic system. It can:

- fetch the shadowserver schema from shadowserver (https://interchange.shadowserver.org/intelmq/v1/schema)
- dynamically collect new reports (see also https://docs.intelmq.org/latest/user/bots/?h=shadow#shadowserver-reports-api)
- parse the new reports

**Note well**: if your IntelMQ system runs in an airgapped environment or if it may only reach out to specific IPs/sites, you should read the notes here:
https://docs.intelmq.org/latest/user/bots/#shadowserver.
You will need to download shadowserver-schema.json periodically yourself in this case.

**Note well:**: since dynamic changes are a bit tricky, we defined that there is a schema contract:

> Schema contract
>
> Once set in the schema, the classification.identifier, classification.taxonomy, and classification.type fields will remain static for a specific report.

This makes things deterministic again.

Alienvault OTX

Fix of a bug where a certain condition would have always evaluated to False. (PR2449 by qux-bbb. Thanks)

AMQP
Quite a few changes (thanks to Kamil, gethvi) on AMQP

Obsoleted bots

- `intelmq.bots.parsers.netlab_360.parser`: Removed as the feed is discontinued. (2442 by Filip Pokorný)
- `intelmq.bots.parsers.webinspektor.parser`: Removed as the feed is discontinued. (2442 by Filip Pokorný)
- `intelmq.bots.parsers.sucuri.parser`: Removed as the feed is discontinued. (2442 by Filip Pokorný)

General changes and bug fixes

Digital Trust Center fixed a bug where the config was loaded twice in intelmqctl which created quite some speedups. Thanks!
This speeds up IntelMQ API calls.

Data Format

Shadowserver dynamic parser (see above).

General remarks

The full list of changes can be seen in the CHANGELOG.md file.

3.2.1

---------------------------------

All Bots
Fixes an issue which prevented bots from stopping gracefully after reloading.
As logrotate reloads all bots regularly, this bug affects most IntelMQ installations.

Reverse DNS Expert
Until IntelMQ version 3.2.0, the bot incorrectly cached and re-used results for /24 networks instead of single IP addresses.
If the bot retrieved the PTR for `192.0.43.7`, it was cached for `192.0.43.0/24` and used for all IP addresses in this range, for example for `192.0.43.8`.
IntelMQ version 3.2.1 fixes this issue.

The bugfix will correctly increase the cache sizes and decrease the performance, as less (incorrect) data is re-used.

3.2.0

----------------------------------

No changes are required by administrators.

[IEP007: Running IntelMQ bots as Python Library](https://github.com/certtools/ieps/tree/main/007) is implemented.

The accompanying 3.2.0 release of intelmq-api switches it's backend from the library hug to fastapi.
Deb-packages of intelmq-api 3.2.0 are delayed for some distributions because of necessary changes in packaging.

3.1.0

----------------------------------

Bots
Shadowserver Reports API collector
The misleading `country` parameter has been depreciated and a `reports` parameter has been added.
The backwards-compatibility will be removed in IntelMQ version 4.0.0.
See the [Shadowserver Reports API bot's documentation](https://docs.intelmq.org/latest/user/bots/#shadowserver-reports-api).

GitHub Collector
GitHub removed the basic `Username/Password` Authentication in favor of personal access tokens. So the GitHub Collector uses an Personal Access Token for authentication [GitHub Documentation: Generate a personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)

url expert bot will deprecate url2fqdn
There is a new, more abstract bot (URL expert bot) which will gradually replace the old url2fqdn bot. The new one builds on top of the urlparse lib of python.
If you use the url2fqdn bot, please start migrating.

Feeds
Abuse.ch Feodo Tracker
IntelMQ previously supported two feeds: "Feodo Tracker IPs" (downloaded as CSV file) and "Feodo Tracker Browse" (downloaded as HTML table). These two feeds contain the same data but differ in the additional details. This IntelMQ release replaces both these feeds (and their parsers) with a feed called "Feodo Tracker" (downloaded as JSON file) which contains all the additional details from both feeds. The parser module for the new feed is `intelmq.bots.parsers.abusech.parser_feodotracker`.

Data Format
Field name checks
The field names for all data added to messages must match a pre-defined format.
The check which ensures this, was ineffective prior to this version and is effective again starting with version 3.1.0.
The [Data format documentation](https://docs.intelmq.org/latest/dev/data-format/#rules-for-keys) describes the required format.

Logrotate
The packaged configuration for *logrotate* falsely contained options applying to other programs' log files. This caused wrong ownership of log files.
This issues is corrected, but the ownership of affected log files may need to be changed manually.
To find affected files, you may use:
bash
sudo find /var/log/ -user intelmq ! -path \*intelmq\*


Configuration
Threshold Expert
The parameter `timeout` has been merged into `redis_cache_ttl`.

Postgres databases

The develop branch previously contained a set of unnecessary statements in a pre-release version to update the `classification.identifier`. If these changes were made please revert back to the previous state.

3.0.2

--------------------------------------
Two performance issues were fixed. One affected all collectors which processed high volumes of data and the other issue affected some bots which used threading.
See the changelog for more details.

The section on 3.0.0 in this file now contains more details for the upgrade to 3.0.0 in regards to the configuration.

3.0.1

--------------------------------------

Bots
The malwardomains parser bot was removed. The malwaredomains.com website is offline, therefore the parser can not be used anymore. The `intelmqctl upgrade-config` command warns if you have the feed and the bot in use.
The postgresql output bot was removed. The bot was marked as deprecated in 2019 and announced to be removed in version 3.

Page 1 of 7

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.