Changelogs » Demisto-sdk

PyUp Safety actively tracks 232,000 Python packages for vulnerabilities and notifies you when to upgrade.



* Fixed an issue with the default output file name of the **unify** command when using "." as an output path.
  * **Unify** command now adds contributor details to the display name and description.
  * **Format** command now adds *isFetch* and *incidenttype* fields to integration yml.
  * Removed the *feedIncremental* field from the integration schema.
  * **Format** command now adds *feedBypassExclusionList*, *Fetch indicators*, *feedReputation*, *feedReliability*,
  *feedExpirationPolicy*, *feedExpirationInterval* and *feedFetchInterval* fields to integration yml.
  * Fixed an issue in the playbooks schema.
  * Fixed an issue where generated release notes were out of order.
  * Improved pack dependencies detection.
  * Fixed an issue where test playbooks were mishandled in **validate** command.


* Added a validation for invalid id fields in indicators types files in **validate** command.
  * Added default behavior for **update-release-notes** command.
  * Fixed an error where README files were failing release notes validation.
  * Updated format of generated release notes to be more user friendly.
  * Improved error messages for the **update-release-notes** command.
  * Added support for `Connections`, `Dashboards`, `Widgets`, and `Indicator Types` to **update-release-notes** command.
  * **Validate** now supports scripts under the *TestPlaybooks* directory.
  * Fixed an issue where **validate** did not support powershell files.


* Added a validation for invalid playbookID fields in incidents types files in **validate** command.
  * Added a code formatter for python files.
  * Fixed an issue where new and old classifiers where mixed on validate command.
  * Added *feedIncremental* field to the integration schema.
  * Fixed error in the **upload** command where unified YMLs were not uploaded as expected if the given input was a pack.
  * Fixed an issue where the **secrets** command failed due to a space character in the file name.
  * Ignored RN validation for *NonSupported* pack.
  * You can now ignore IF107, SC100, RP102 error codes in the **validate** command.
  * Fixed an issue where the **download** command was crashing when received as input a JS integration or script.
  * Fixed an issue where **validate** command checked docker image for JS integrations and scripts.
  * **validate** command now checks scheme for reports and connections.
  * Fixed an issue where **validate** command checked docker when running on all files.
  * Fixed an issue where **validate** command did not fail when docker image was not on the latest numeric tag.
  * Fixed an issue where beta integrations were not validated correctly in **validate** command.


* fixed and issue where file types were not recognized correctly in **validate** command.
  * Added better outputs for validate command.


* Fixed an issue where changes to only non-validated files would fail validation.
  * Fixed an issue in **validate** command where moved files were failing validation for new packs.
  * Fixed an issue in **validate** command where added files were failing validation due to wrong file type detection.
  * Added support for new classifiers and mappers in **validate** command.
  * Removed support of old RN format validation.
  * Updated **secrets** command output format.
  * Added support for error ignore on deprecated files in **validate** command.
  * Improved errors outputs in **validate** command.
  * Added support for linting an entire pack.


* Fixed a bug where misleading error was presented when pack name was not found.
  * **Update-release-notes** now detects added files for packs with versions.
  * Readme files are now ignored by **update-release-notes** and validation of release notes.
  * Empty release notes no longer cause an uncaught error during validation.


* Changed the output format of demisto-sdk secrets.
  * Added a validation that checkbox items are not required in integrations.
  * Added pack release notes generation and validation.
  * Improved pack metadata validation.
  * Fixed an issue in **validate** where renamed files caused an error


* Fix the **format** command to update the `id` field to be equal to `details` field in indicator-type files, and to `name` field in incident-type & dashboard files.
  * Fixed a bug in the **validate** command for layout files that had `sortValues` fields.
  * Fixed a bug in the **format** command where `playbookName` field was not always present in the file.
  * Fixed a bug in the **format** command where indicatorField wasn't part of the SDK schemas.
  * Fixed a bug in **upload** command where created unified docker45 yml files were not deleted.
  * Added support for IndicatorTypes directory in packs (for `reputation` files, instead of Misc).
  * Fixed parsing playbook condition names as string instead of boolean in **validate** command
  * Improved image validation in YAML files.
  * Removed validation for else path in playbook condition tasks.


* Fixed a bug in the **format** command where comments were being removed from YAML files.
  * Added output fields: _file_path_ and _kind_ for layouts in the id-set.json created by **create-id-set** command.
  * Fixed a bug in the **create-id-set** command Who returns Duplicate for Layouts with a different kind.
  * Added formatting to **generate-docs** command results replacing all `<br>` tags with `<br/>`.
  * Fixed a bug in the **download** command when custom content contained not supported content entity.
  * Fixed a bug in **format** command in which boolean strings  (e.g. 'yes' or 'no') were converted to boolean values (e.g. 'True' or 'False').
  * **format** command now removes *sourceplaybookid* field from playbook files.
  * Fixed a bug in **generate-docs** command in which integration dependencies were not detected when generating documentation for a playbook.


* Fixed a bug in the **unify** command when output path was provided empty.
  * Improved error message for integration with no tests configured.
  * Improved the error message returned from the **validate** command when an integration is missing or contains malformed fetch incidents related parameters.
  * Fixed a bug in the **create** command where a unified YML with a docker image for 4.5 was copied incorrectly.
  * Missing release notes message are now showing the release notes file path to update.
  * Fixed an issue in the **validate** command in which unified YAML files were not ignored.
  * File format suggestions are now shown in the relevant file format (JSON or YAML).
  * Changed Docker image validation to fail only on non-valid ones.
  * Removed backward compatibility validation when Docker image is updated.


* Improved the *upload* command to support the upload of all the content entities within a pack.
  * The *upload* command now supports the improved pack file structure.
  * Added an interactive option to format integrations, scripts and playbooks with No TestPlaybooks configured.
  * Added an interactive option to configure *conf.json* file with missing test playbooks for integrations, scripts and playbooks
  * Added *download* command to download custom content from Demisto instance to the local content repository.
  * Improved validation failure messages to include a command suggestion, wherever relevant, to fix the raised issue.
  * Improved 'validate' help and documentation description
  * validate - checks that scripts, playbooks, and integrations have the *tests* key.
  * validate - checks that test playbooks are configured in `conf.json`.
  * demisto-sdk lint - Copy dir better handling.
  * demisto-sdk lint - Add error when package missing in docker image.
  * Added *-a , --validate-all* option in *validate* to run all validation on all files.
  * Added *-i , --input* option in *validate* to run validation on a specified pack/file.
  * added *-i, --input* option in *secrets* to run on a specific file.
  * Added an allowed hidden parameter: *longRunning* to the hidden integration parameters validation.
  * Fixed an issue with **format** command when executing with an output path of a folder and not a file path.
  * Bug fixes in generate-docs command given playbook as input.
  * Fixed an issue with lint command in which flake8 was not running on unit test files.


* Added *-c, --command* option in *generate-docs* to generate a specific command from an integration.
  * Fixed an issue when getting README/CHANGELOG files from git and loading them.
  * Removed release notes validation for new content.
  * Fixed secrets validations for files with the same name in a different directory.
  * demisto-sdk lint - parallelization working with specifying the number of workers.
  * demisto-sdk lint - logging levels output, 3 levels.
  * demisto-sdk lint - JSON report, structured error reports in JSON format.
  * demisto-sdk lint - XML JUnit report for unit-tests.
  * demisto-sdk lint - new packages used to accelerate execution time.
  * demisto-sdk secrets - command now respects the generic whitelist, and not only the pack secrets.


[PyPI History][1]


* Fixed an issue in *generate-docs* where Playbooks and Scripts documentation failed.
  * Added a graceful error message when executing the *run" command with a misspelled command.
  * Added more informative errors upon failures of the *upload* command.
  * format command:
  * Added format for json files: IncidentField, IncidentType, IndicatorField, IndicatorType, Layout, Dashboard.
  * Added the *-fv --from-version*, *-nv --no-validation* arguments.
  * Removed the *-t yml_type* argument, the file type will be inferred.
  * Removed the *-g use_git* argument, running format without arguments will run automatically on git diff.
  * Fixed an issue in loading playbooks with '=' character.
  * Fixed an issue in *validate* failed on deleted README files.


* Added the *max* field to the Playbook schema, allowing to define it in tasks loop.
  * Fixed an issue in *validate* where Condition branches checks were case sensitive.


* Added the *slareminder* field to the Playbook schema.
  * Added the *common_server*, *demisto_mock* arguments to the *init* command.
  * Fixed an issue in *generate-docs* where the general section was not being generated correctly.
  * Fixed an issue in *validate* where Incident type validation failed.


* Fixed an issue where the *validate* command did not identify CHANGELOG in packs.
  * Added a new command, *id-set* to create the id set - the content dependency tree by file IDs.


* generate-docs command:
  * Added the *use_cases*, *permissions*, *command_permissions* and *limitations*.
  * Added the *--insecure* argument to support running the script and integration command in Demisto.
  * Removed the *-t yml_type* argument, the file type will be inferred.
  * The *-o --output* argument is no longer mandatory, default value will be the input file directory.
  * Added support for env var: *DEMISTO_SDK_SKIP_VERSION_CHECK*. When set version checks are skipped.
  * Fixed an issue in which the CHANGELOG files did not match our scheme.
  * Added a validator to verify that there are no hidden integration parameters.
  * Fixed an issue where the *validate* command ran on test files.
  * Removed the *env-dir* argument from the demisto-sdk.
  * README files which are html files will now be skipped in the *validate* command.
  * Added support for env var: *DEMISTO_README_VALIDATOR*. When not set the readme validation will not run.


* Added a validator for IncidentTypes (incidenttype-*.json).
  * Fixed an issue where the -p flag in the *validate* command was not working.
  * Added a validator for files.
  * Release notes validator will now run on: incident fields, indicator fields, incident types, dashboard and reputations.
  * Fixed an issue where the validator of reputation(Indicator Type) did not check on the details field.
  * Fixed an issue where the validator attempted validating non-existing files after deletions or name refactoring.
  * Removed the *yml_type* argument in the *split-yml*, *extract-code* commands.
  * Removed the *file_type* argument in the *generate-test-playbook* command.
  * Fixed the *insecure* argument in *upload*.
  * Added the *insecure* argument in *run-playbook*.
  * Standardise the *-i --input*, *-o --output* to demisto-sdk commands.


* Fixed an issue where the incident and indicator field BC check failed.
  * Support for linting and unit testing PowerShell integrations.


* Fixed an issue where validate failed on Windows.
  * Added a validator to verify all branches are handled in conditional task in a playbook.
  * Added a warning message when not running the latest sdk version.
  * Added a validator to check that the root is connected to all tasks in the playbook.
  * Added a validator for Dashboards (dashboard-*.json).
  * Added a validator for Indicator Types (reputation-*.json).
  * Added a BC validation for changing incident field type.
  * Fixed an issue where init command would generate an invalid yml for scripts.
  * Fixed an issue in misleading error message in v2 validation hook.
  * Fixed an issue in v2 hook which now is set only on newly added scripts.
  * Added more indicative message for errors in yaml files.
  * Disabled pykwalify info log prints.


* Added a BC check for incident fields - changing from version is not allowed.
  * Fixed an issue in create-content-artifacts where scripts in Packs in TestPlaybooks dir were copied with a wrong prefix.


* Added a validation that incident field can not be required.
  * Added validation for fetch incident parameters.
  * Added validation for feed integration parameters.
  * Added to the *format* command the deletion of the *sourceplaybookid* field.
  * Fixed an issue where *fieldMapping* in playbook did not pass the scheme validation.
  * Fixed an issue where *create-content-artifacts* did not copy TestPlaybooks in Packs without prefix of *playbook-*.
  * Added a validation the a playbook can not have a rolename set.
  * Added to the image validator the new DBot default image.
  * Added the fields: elasticcommonfields, quiet, quietmode to the Playbook schema.
  * Fixed an issue where *validate* failed on integration commands without outputs.
  * Added a new hook for naming of v2 integrations and scripts.


* Fixed an issue where *create-content-artifact* was not loading the data in the yml correctly.
  * Fixed an issue where *unify* broke long lines in script section causing syntax errors


* Added *generate-docs* command to generate documentation file for integration, playbook or script.
  * Fixed an issue where *unify* created a malformed integration yml.
  * Fixed an issue where demisto-sdk **init** creates unit-test file with invalid import.


* Fixed an issue where demisto-sdk **validate** failed on modified scripts without error message.


* Fixed an issue with docker tag validation for integrations.
  * Restructured repo source code.


* Saved failing unit tests as a file.
  * Fixed an issue where "_test" file for scripts/integrations created using **init** would import the "HelloWorld" templates.
  * Fixed an issue in demisto-sdk **validate** - was failing on backward compatiblity check
  * Fixed an issue in demisto-sdk **secrets** - empty line in .secrets-ignore always made the secrets check to pass
  * Added validation for docker image inside integrations and scripts.
  * Added --use-git flag to **format** command to format all changed files.
  * Fixed an issue where **validate** did not fail on dockerimage changes with bc check.
  * Added new flag **--ignore-entropy** to demisto-sdk **secrets**, this will allow skip entropy secrets check.
  * Added --outfile to **lint** to allow saving failed packages to a file.


* Added backwards compatibility break error message.
  * Added schema for incident types.
  * Added **additionalinfo** field to as an available field for integration configuration.
  * Added pack parameter for **init**.
  * Fixed an issue where error would appear if name parameter is not set in **init**.


* Fixed the handling of classifier files in **validate**.


* Fixed the handling of newly created reputation files in **validate**.
  * Added an option to perform **validate** on a specific file.


* Added support for multi-package **lint** both with parallel and without.
  * Added all parameter in **lint** to run on all packages and packs in content repository.
  * Added **format** for:
  * Scripts
  * Playbooks
  * Integrations
  * Improved user outputs for **secrets** command.
  * Fixed an issue where **lint** would run pytest and pylint only on a single docker per integration.
  * Added auto-complete functionality to demisto-sdk.
  * Added git parameter in **lint** to run only on changed packages.
  * Added the **run-playbook** command
  * Added **run** command which runs a command in the Demisto playground.
  * Added **upload** command which uploads an integration or a script to a Demisto instance.
  * Fixed and issue where **validate** checked if release notes exist for new integrations and scripts.
  * Added **generate-test-playbook** command which generates a basic test playbook for an integration or a script.
  * **validate** now supports indicator fields.
  * Fixed an issue with layouts scheme validation.
  * Adding **init** command.
  * Added **json-to-outputs** command which generates the yaml section for outputs from an API raw response.


* Fixed an issue with locating release notes for beta integrations in **validate**.


* Fixed an issue with locating release notes for beta integrations in **validate**.


* Adding image validation to Beta_Integration and Packs in **validate**.


* Adding Beta_Integration to the structure validation process.
  * Fixing bug where **validate** did checks on TestPlaybooks.
  * Added requirements parameter to **lint**.


* Fixing bug where **lint** did not return exit code 1 on failure.
  * Fixing bug where **validate** did not print error message in case no release notes were give.


* **Validate** now checks that the id and name fields are identical in yml files.
  * Fixed a bug where sdk did not return any exit code.


* Added Release Notes Validator.
  * Fixed the Unifier selection of your python file to use as the code.
  * **Validate** now supports Indicator fields.
  * Fixed a bug where **validate** and **secrets** did not return exit code 1 on failure.
  * **Validate** now runs on newly added scripts.


* Added support for `--version`.
  * Fixed an issue in file_validator when calling `checked_type` method with script regex.


* Restructuring validation to support content packs.
  * Added secrets validation.
  * Added content bundle creation.
  * Added lint and unit test run.


* Added new logic to the unifier.
  * Added detailed README.
  * Some small adjustments and fixes.


  * **Extract** components(code, image, description etc.) from a Demisto YAML file into a directory.
  * **Unify** components(code, image, description etc.) to a single Demisto YAML file.
  * **Validate** Demisto content files.