Pulpcore

==================
REST API
--------

Features
~~~~~~~~

- Added table-of-contents to export and gave import a toc= to find/reassemble pieces on import.
:redmine:`6737`
- Added ability to associate a Remote with a Repository so users no longer have to specify Remote when
syncing.
:redmine:`7015`
- The `/pulp/api/v3/access_policies/` endpoint is available for reading and modifying the AccessPolicy
used for Role Based Access Control for all Pulp endpoints. This allows for complete customization
of the Authorization policies.

NOTE: this endpoint is in tech-preview and may change in backwards incompatible ways in the future.
:redmine:`7160`
- The `/pulp/api/v3/access_policies/` endpoint also includes a `permissions_assignment` section which
customizes the permissions assigned to new objects. This allows for complete customization for how
new objects work with custom define Authorization policies.
:redmine:`7210`
- The `/pulp/api/v3/users/` endpoint is available for reading the Users, Group membership, and
Permissions.

NOTE: this endpoint is in tech-preview and may change in backwards incompatible ways in the future.
:redmine:`7231`
- The `/pulp/api/v3/groups/` endpoint is available for reading the Groups, membership, and
Permissions.

NOTE: this endpoint is in tech-preview and may change in backwards incompatible ways in the future.
:redmine:`7232`
- The `/pulp/api/v3/tasks/` endpoint now provides a user-isolation behavior for non-admin users. This
policy is controllable at the `/pulp/api/v3/access_policies/` endpoint.

NOTE: The user-isolation behavior is in "tech preview" and production systems are recommended to
continue using the build-in ``admin`` user only.
:redmine:`7301`
- Extended endpoint `/pulp/api/v3/groups/:pk/users` to add and remove users from a group.

NOTE: this endpoint is in tech-preview and may change in backwards incompatible ways in the future.
:redmine:`7310`
- Extended endpoints `/pulp/api/v3/groups/:pk/model_permissions` and
`/pulp/api/v3/groups/:pk/object_permissions` to add and remove permissions from a group.

NOTE: this endpoint is in tech-preview and may change in backwards incompatible ways in the future.
:redmine:`7311`


Bugfixes
~~~~~~~~

- WorkerDirectory.delete() no longer recursively trys to delete itself when encountering a permission error
:redmine:`6504`
- Stopped preventing removal of PulpExport/Exporter when last-export existed.
:redmine:`6555`
- First time on demand content requests appear in the access log.
:redmine:`7002`
- Fixed denial of service caused by extra slashes in content urls.
:redmine:`7066`
- Set a default DJANGO_SETTINGS_MODULE env var in content app
:redmine:`7179`
- Added plugin namespace to openapi href identifier.
:redmine:`7209`
- By default, html in field descriptions filtered out in REST API docs unless 'include_html' is set.
:redmine:`7299`
- Fixed plugin filtering in bindings to work independently from "bindings" parameter.
:redmine:`7306`


Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~

- Made password variable consistent with Ansible installer example playbook
:redmine:`7065`
- Fixed various docs bugs in the pulpcore docs.
:redmine:`7090`
- Adds documentation about SSL configuration requirements for reverse proxies.
:redmine:`7285`
- Fixed REST API docs.
:redmine:`7292`


Deprecations and Removals
~~~~~~~~~~~~~~~~~~~~~~~~~

- Removed unnecessary fields from the import/export transfer.
:redmine:`6515`
- Upgrading the api documentation from OpenAPI v2 to OpenAPI v3.
- Methods signatures for bindings may change.
:redmine:`7108`
- Changed default ``download_concurrency`` on Remotes from 20 to 10 to avoid connection problems. Also
updated existing Remotes with ``download_concurrency`` of 20 to 10.
:redmine:`7212`


Misc
~~~~

- :redmine:`6807`, :redmine:`7142`, :redmine:`7196`


Plugin API
----------

Features
~~~~~~~~

- Adding `PulpTemporaryFile` for handling temporary files between the viewset and triggered tasks
:redmine:`6749`
- ``RepositorySyncURLSerializer`` will now check remote on the repository before it raises an
exception if the remote param is not set.
:redmine:`7015`
- Added a hook on ``Repository`` called ``artifacts_for_version()`` that plugins can override to
modify the logic behind ``RepositoryVersion.artifacts``. For now, this is used when exporting
artifacts.
:redmine:`7021`
- Enabling plugin writers to have more control on `HttpDownloader` response codes 400+
by subclassing `HttpDownloader` and overwriting `raise_for_status` method
:redmine:`7117`
- `BaseModel` now inherits from `LifecycleModel` provided by `django-lifecycle` allowing any subclass
to also use it instead of signals.
:redmine:`7151`
- A new `pulpcore.plugin.models.AutoDeleteObjPermsMixin` object can be added to models to
automatically delete all user and group permissions for an object just before the object is deleted.
This provides an easy cleanup mechanism and can be added to models as a mixin. Note that your model
must support `django-lifecycle` to use this mixin.
:redmine:`7157`
- A new model `pulpcore.plugin.models.AccessPolicy` is available to store AccessPolicy statements in
the database. The model's `statements` field stores the list of policy statements as a JSON field.
The `name` field stores the name of the Viewset the `AccessPolicy` is protecting.

Additionally, the `pulpcore.plugin.access_policy.AccessPolicyFromDB` is a drf-access-policy which
viewsets can use to protect their viewsets with. See the :ref:`viewset_enforcement` for more
information on this.
:redmine:`7158`
- Adds the `TaskViewSet` and `TaskGroupViewSet` objects to the plugin api.
:redmine:`7187`
- Enabled plugin writers to create immutable repository ViewSets
:redmine:`7191`
- A new `pulpcore.plugin.models.AutoAddObjPermsMixin` object can be added to models to automatically
add permissions for an object just after the object is created. This is controlled by data saved in
the `permissions_assignment` attribute of the `pulpcore.plugin.models.AccessPolicy` allowing users
to control what permissions are created. Note that your model must support `django-lifecycle` to use
this mixin.
:redmine:`7210`
- Added ability for plugin writers to set a ``content_mapping`` property on content resources to
provide a custom mapping of content to repositories.
:redmine:`7252`
- Automatically excluding ``pulp_id``, ``pulp_created``, and ``pulp_last_updated`` for
``QueryModelResources``.
:redmine:`7277`
- Viewsets that subclass ``pulpcore.plugin.viewsets.NamedModelViewSet` can declare the
``queryset_filtering_required_permission`` class attribute naming the permission required to view
an object. See the :ref:`queryset_scoping` documentation for more information.
:redmine:`7300`


Bugfixes
~~~~~~~~

- Making operation_id unique
:redmine:`7233`
- Making ReDoc OpenAPI summary human readable
:redmine:`7237`
- OpenAPI schema generation from CLI
:redmine:`7258`
- Allow `pulpcore.plugin.models.AutoAddObjPermsMixin.add_for_object_creator` to skip assignment of
permissions if there is no known user. This allows endpoints that do not use authorization but still
create objects in the DB to execute without error.
:redmine:`7312`


Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~

- Omit a view/viewset from the OpenAPI schema
:redmine:`7133`
- Added plugin writer docs for ``BaseContentResource``.
:redmine:`7296`


Deprecations and Removals
~~~~~~~~~~~~~~~~~~~~~~~~~

- Newlines in certificate string (ca_cert, client_cert, client_key) on Remotes are not required to be escaped.
:redmine:`6735`
- Replaced drf-yasg with drf-spectacular.
- This updates the api documentation to openapi v3.
- Plugins may require changes.
- Methods signatures for bindings may change.
:redmine:`7108`
- Moving containers from pulpcore to pulp-operator
:redmine:`7171`

==================
REST API
--------

Features
~~~~~~~~

- Added start_versions= to export to allow for arbitrary incremental exports.
:redmine:`6763`
- Added GroupProgressReport to track progress in a TaskGroup.
:redmine:`6858`
- Provide a user agent string with all aiohttp requests by default.
:redmine:`6954`


Bugfixes
~~~~~~~~

- Fixed 'integer out of range' error during sync by changing RemoteArtifact size field to BigIntegerField.
:redmine:`6717`
- Added a more descriptive error message that is shown when CONTENT_ORIGIN is not properly configured
:redmine:`6771`
- Including requirements.txt on MANIFEST.in
:redmine:`6888`
- Corrected a number of filters to be django-filter-2.3.0-compliant.
:redmine:`6915`
- Locked Content table to prevent import-deadlock.
:redmine:`7073`


Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~

- Updating installation docs
:redmine:`6836`
- Fixed a number of typos in the import/export workflow docs.
:redmine:`6919`
- Fixed docs which claim that admin user has a default password.
:redmine:`6992`
- Fixed broken link to content plugins web page
:redmine:`7017`


Deprecations and Removals
~~~~~~~~~~~~~~~~~~~~~~~~~

- Removes the Write models from the OpenAPI schema.
Brings back the models that were accidentally removed from the OpenAPI schema in 3.4.0 release.
:redmine:`7087`


Misc
~~~~

- :redmine:`6483`, :redmine:`6925`


Plugin API
----------

Features
~~~~~~~~

- Views can specify the tag name with `pulp_tag_name`
:redmine:`6832`
- Added GroupProgressReport to track progress in a TaskGroup.
:redmine:`6858`
- Exported the symbols `serializers.SingleContentArtifactField` and `files.PulpTemporaryUploadedFile`.
:redmine:`7088`


----

==================
REST API
--------

Bugfixes
~~~~~~~~

- Including requirements.txt on MANIFEST.in
:redmine:`6888`


Plugin API
----------

No significant changes.


----

==================
REST API
--------

Features
~~~~~~~~

- Implemented incremental-exporting for PulpExport.
:redmine:`6136`
- Added support for S3 and other non-filesystem storage options to pulp import/export functionality.
:redmine:`6456`
- Optimized imports by having repository versions processed using child tasks.
:redmine:`6484`
- Added repository type check during Pulp imports.
:redmine:`6532`
- Added version checking to import process.
:redmine:`6558`
- Taught PulpExport to export by RepositoryVersions if specified.
:redmine:`6566`
- Task groups now have an 'all_tasks_dispatched' field which denotes that no more tasks will spawn
as part of this group.
:redmine:`6591`
- Taught export how to split export-file into chunk_size bytes.
:redmine:`6736`


Bugfixes
~~~~~~~~

- Remote fields `username` and `password` show up in:
REST docs, API responses, and are available in the bindings.
:redmine:`6346`
- Fixed a bug, where the attempt to cancel a completed task lead to a strange response.
:redmine:`6465`
- Fixed KeyError during OpenAPI schema generation.
:redmine:`6468`
- Added a missing trailing slash to distribution's base_url
:redmine:`6507`
- Fixed a bug where the wrong kind of error was being raised for href parameters of mismatched types.
:redmine:`6521`
- containers: Fix pulp_rpm 3.3.0 install by replacing the python3-createrepo_c RPM with its build-dependencies, so createrep_c gets installed & built from PyPI
:redmine:`6523`
- Fixed OpenAPI schema for importer and export APIs.
:redmine:`6556`
- Normalized export-file-path for PulpExports.
:redmine:`6564`
- Changed repository viewset to use the general_update and general_delete tasks.
This fixes a bug where updating specialized fields of a repository was impossible due to using the wrong serializer.
:redmine:`6569`
- Only uses multipart OpenAPI Schema when dealing with `file` fields
:redmine:`6702`
- Fixed a bug that prevented write_only fields from being present in the API docs and bindings
:redmine:`6775`
- Added proper headers for index.html pages served by content app.
:redmine:`6802`
- Removed Content-Encoding header from pulpcore-content responses.
:redmine:`6831`


Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~

- Adding docs for importing and exporting from Pulp to Pulp.
:redmine:`6364`
- Add some documentation around TaskGroups.
:redmine:`6641`
- Introduced a brief explanation about `pulp_installer`
:redmine:`6674`
- Added a warning that the REST API is not safe for multi-user use until RBAC is implemented.
:redmine:`6692`
- Updated the required roles names
:redmine:`6758`


Deprecations and Removals
~~~~~~~~~~~~~~~~~~~~~~~~~

- Changed repositories field on ``/pulp/api/v3/exporters/core/pulp/`` from UUIDs to hrefs.
:redmine:`6457`
- Imports now spawn child tasks which can be fetched via the ``child_tasks`` field of the import task.
:redmine:`6484`
- Content of ssl certificates and keys changed to be return their full value instead of sha256 through REST API.
:redmine:`6691`
- Replaced PulpExport filename/sha256 fields, with output_info_file, a '<filename>': '<hash>' dictionary.
:redmine:`6736`


Misc
~~~~

- :redmine:`5020`, :redmine:`6421`, :redmine:`6477`, :redmine:`6539`, :redmine:`6542`, :redmine:`6544`, :redmine:`6572`, :redmine:`6583`, :redmine:`6695`, :redmine:`6803`, :redmine:`6804`


Plugin API
----------

Features
~~~~~~~~

- Added new NoArtifactContentUploadSerializer and NoArtifactContentUploadViewSet to enable plugin
writers to upload content without storing an Artifact
:redmine:`6281`
- Added view_name_pattern to DetailRelatedField and DetailIdentityField to properly identify wrong resource types.
:redmine:`6521`
- Added support for Distributions to provide non-Artifact content via a content_handler.
:redmine:`6570`
- Added constants to the plugin API at ``pulpcore.plugin.constants``.
:redmine:`6579`
- TaskGroups now have an 'all_tasks_dispatched' field that can be used to notify systems that no
further tasks will be dispatched for a TaskGroup. Plugin writers should call ".finish()" on all
TaskGroups created once they are done using them to set this field.
:redmine:`6591`


Bugfixes
~~~~~~~~

- Added ``RemoteFilter`` to the plugin API as it was missing but used by plugin_template.
:redmine:`6563`


Deprecations and Removals
~~~~~~~~~~~~~~~~~~~~~~~~~

- Fields: `username` and `password` will be returned to the rest API user requesting a `Remote`
:redmine:`6346`
- Rehomed QueryModelResource to pulpcore.plugin.importexport.
:redmine:`6514`
- The :meth:`pulpcore.content.handler.Handler.list_directory` function now returns a set of strings where it returned a string of HTML before.
:redmine:`6570`


----

==================
REST API
--------

Bugfixes
~~~~~~~~

- Fixed partial and general update calls for SecretCharField on the Remote.
:redmine:`6565`
- Fixed bug where ``TaskGroup`` was showing up as null for ``created_resources`` in tasks.
:redmine:`6573`


Plugin API
----------

Features
~~~~~~~~

- Add TaskGroup to the plugin API.
:redmine:`6603`


----

==================
REST API
--------

Features
~~~~~~~~

- Added support for repairing a RepositoryVersion by redownloading corrupted artifact files.
Sending a POST request to
``/pulp/api/v3/repositories/<plugin>/<type>/<repository-uuid>/versions/<version-number>/repair/``
will trigger a task that scans all associated artfacts and attempts to fetch missing or corrupted ones again.
:redmine:`5613`
- Added support for exporting pulp-repo-versions. POSTing to an exporter using the
``/pulp/api/v3/exporters/core/pulp/<exporter-uuid>/exports/`` API will instantiate a
PulpExport entity, which will generate an export-tar.gz file at
``<exporter.path>/export-<export-uuid>-YYYYMMDD_hhMM.tar.gz``
:redmine:`6135`
- Added API for importing Pulp Exports at ``POST /importers/core/pulp/<uuid>/imports/``.
:redmine:`6137`
- Added the new setting CHUNKED_UPLOAD_DIR for configuring a default directory used for uploads
:redmine:`6253`
- Exported SigningService in plugin api
:redmine:`6256`
- Added name filter for SigningService
:redmine:`6257`
- Relationships between tasks that spawn other tasks will be shown in the Task API.
:redmine:`6282`
- Added a new APIs for PulpExporters and Exports at ``/exporters/core/pulp/`` and
``/exporters/core/pulp/<uuid>/exports/``.
:redmine:`6328`
- Added PulpImporter API at ``/pulp/api/v3/importers/core/pulp/``. PulpImporters are used for
importing exports from Pulp.
:redmine:`6329`
- Added an ``ALLOWED_EXPORT_PATHS`` setting with list of filesystem locations that exporters can export to.
:redmine:`6335`
- Indroduced `ordering` keyword, which orders the results by specified field.
Pulp objects will by default be ordered by pulp_created if that field exists.
:redmine:`6347`
- Task Groups added -- Plugin writers can spawn tasks as part of a "task group",
which facilitates easier monitoring of related tasks.
:redmine:`6414`


Bugfixes
~~~~~~~~

- Improved the overall performance while syncing very large repositories
:redmine:`6121`
- Made chunked uploads to be stored in a local file system instead of a default file storage
:redmine:`6253`
- Fixed 500 error when calling modify on nonexistent repo.
:redmine:`6284`
- Fixed bug where user could delete repository version 0 but not recreate it by preventing users from
deleting repo version 0.
:redmine:`6308`
- Fixed non unique content units on content list
:redmine:`6347`
- Properly sort endpoints during generation of the OpenAPI schema.
:redmine:`6372`
- Improved resync performance by up to 2x with a change to the content stages.
:redmine:`6373`
- Fixed bug where 'secret' fields would be set to the sha256 checksum of the original value.
:redmine:`6402`
- Fixed pulp containers not allowing commands to be run via absolute path.
:redmine:`6420`


Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~

- Documented bindings installation for a dev environment
:redmine:`6221`
- Added documentation for how to write changelog messages.
:redmine:`6336`
- Cleared up a line in the database settings documentation that was ambiguous.
:redmine:`6384`
- Updated docs to reflect that S3/Azure are supported and no longer tech preview.
:redmine:`6443`
- Added tech preview note to docs for importers/exporters.
:redmine:`6454`
- Renamed ansible-pulp to pulp_installer (to avoid confusion with pulp-ansible)
:redmine:`6461`
- Fixed missing terms in documentation.
:redmine:`6485`


Deprecations and Removals
~~~~~~~~~~~~~~~~~~~~~~~~~

- Changing STATIC_URL from `/static/` to `/assets/` for avoiding conflicts
:redmine:`6128`
- Exporting now requires the configuration of the ``ALLOWED_EXPORT_PATHS`` setting. Without this
configuration, Pulp will not export content to the filesystem.
:redmine:`6335`


Misc
~~~~

- :redmine:`5826`, :redmine:`6155`, :redmine:`6357`, :redmine:`6450`, :redmine:`6451`, :redmine:`6481`, :redmine:`6482`


Plugin API
----------

Features
~~~~~~~~

- Tasks can now be spawned from inside other tasks, and these relationships can be explored
via the "parent_task" field and "child_tasks" related name on the Task model.
:redmine:`6282`
- Added a new Export model, serializer, and viewset.
:redmine:`6328`
- Added models Import and Importer (as well as serializers and viewsets) that can be used for
importing data into Pulp.
:redmine:`6329`
- `NamedModelViewSet` uses a default ordering of `-pulp_created` using the `StableOrderingFilter`.
Users using the `ordering` keyword will be the primary ordering used when specified.
:redmine:`6347`
- Added two new repo validation methods (validate_repo_version and validate_duplicate_content).
:redmine:`6362`
- enqueue_with_reservation() provides a new optional argument for "task_group".
:redmine:`6414`


Bugfixes
~~~~~~~~

- Fixed bug where RepositoryVersion.artifacts returns None.
:redmine:`6439`


Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~

- Add plugin writer docs on adding MANIFEST.in entry to include ``webserver_snippets`` in the Python
package.
:redmine:`6249`
- Updated the metadata signing plugin writers documentation.
:redmine:`6342`


Deprecations and Removals
~~~~~~~~~~~~~~~~~~~~~~~~~

- Changed master model from FileSystemExporter to Exporter. Plugins will still need to extend
FileSystemExporter but the master table is now core_exporter. This will require that plugins drop
and recreate their filesystem exporter tables.
:redmine:`6328`
- RepositoryVersion add_content no longer checks for duplicate content.
:redmine:`6362`


Misc
~~~~

- :redmine:`6342`


----