Pulpcore

===================
REST API
--------

Features
~~~~~~~~

- Add support for automatic publishing and distributing.
:redmine:`7626`
- Add a warning at startup time if there are remote artifacts with checksums but no allowed checksums.
:redmine:`7985`
- Added support in content app for properly handling unknown or forbidden digest errors.
:redmine:`7989`
- Added sync check that raises error when only forbidden checksums are found for on-demand content.
:redmine:`8423`
- Added ability for users to delete repo version 0 as long as they still have at least one repo
version for their repo.
:redmine:`8454`


Bugfixes
~~~~~~~~

- Added asynchronous tasking to the Update and Delete endpoints of PulpExporter to provide proper locking on resources.
:redmine:`7438`
- Fixed a scenario where canceled tasks could be marked failed.
:redmine:`7980`
- Taught ``PulpImport`` correct way to find and import ``RepositoryVersions``. Previous
implementation only worked for ``RepositoryVersions`` that were the 'current' version
of the exported ``Repository``.
:redmine:`8116`
- Fixed a race condition that sometimes surfaced during handling of reserved resources.
:redmine:`8352`
- Made digest and size sync erros more helpful by logging url of the requested files.
:redmine:`8357`
- Fixed artifact-stage to handle an edge-case when multiple multi-artifact content, from different remotes, is in a single batch.
:redmine:`8377`
- Fixed Azure artifacts download.
:redmine:`8427`
- Fixed bug during sync where a unique constraint violation for ``Content`` would raise an "X matching
query does not exist" error.
:redmine:`8430`
- Fix artifact checksum check to not check on-demand content.
:redmine:`8445`
- Fixed a bug where the existence of PublishedMetadata caused ``LookupError`` when querying ``/pulp/api/v3/content/``
:redmine:`8447`
- Distributions are now viewable again at the base url of the content app
:redmine:`8475`
- Fixed a path in artifact_stages that could lead to sync-failures in pulp_container.
:redmine:`8489`


Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~

- Update docs with guide how to change 'ALLOWED_CONTENT_CHECKSUMS' setting using 'pulpcore-manager handle-artifact-checksums --report' if needed.
:redmine:`8325`


Removals
~~~~~~~~

- The Update and Delete endpoints of Exporters changed to now return 202 with tasks.
:redmine:`7438`
- Deprecation warnings are now being logged by default if the log level includes WARNING. This can be
disabled by adjusting the log level of ``pulpcore.deprecation``. See the deprecation docs for more
information.
:redmine:`8499`


Misc
~~~~

- :redmine:`8450`


Plugin API
----------

Features
~~~~~~~~

- Added a new callback method to ``Repository`` named ``on_new_version()``, which runs when a new repository version has been created. This can be used for e.g. automatically publishing or distributing a new repository version after it has been created.
:redmine:`7626`
- Added url as optional argument to ``DigestValidationError`` and ``SizeValidationError`` exceptions to log urls in the exception message.
:redmine:`8357`
- Added the following new objects related to a new ``Distribution`` MasterModel:
* ``pulpcore.plugin.models.Distribution`` - A new MasterModel ``Distribution`` which replaces the
``pulpcore.plugin.models.BaseDistribution``. This now contains the ``repository``,
``repository_version``, and ``publication`` fields on the MasterModel instead of on the detail
models as was done with ``pulpcore.plugin.models.BaseDistribution``.
* ``pulpcore.plugin.serializer.DistributionSerializer`` - A serializer plugin writers should use
with the new ``pulpcore.plugin.models.Distribution``.
* ``pulpcore.plugin.viewset.DistributionViewSet`` - The viewset that replaces the deprecated
``pulpcore.plugin.viewset.BaseDistributionViewSet``.
* ``pulpcore.plugin.viewset.NewDistributionFilter`` - The filter that pairs with the
``Distribution`` model.
:redmine:`8384`
- Added checksum type enforcement to ``pulpcore.plugin.download.BaseDownloader``.
:redmine:`8435`
- Adds the ``pulpcore.plugin.tasking.dispatch`` interface which replaces the
``pulpcore.plugin.tasking.enqueue_with_reservation`` interface. It is the same except:
* It returns a ``pulpcore.plugin.models.Task`` instead of an RQ object
* It does not support the ``options`` keyword argument

Additionally the ``pulpcore.plugin.viewsets.OperationPostponedResponse`` was updated to support both
the ``dispatch`` and ``enqueue_with_reservation`` interfaces.
:redmine:`8496`


Bugfixes
~~~~~~~~

- Allow plugins to unset the ``queryset_filtering_required_permission`` attribute in ``NamedModelViewSet``.
:redmine:`8438`


Removals
~~~~~~~~

- Removed checksum type filtering from ``pulpcore.plugin.models.Remote.get_downloader`` and ``pulpcore.plugin.stages.DeclarativeArtifact.download``.
:redmine:`8435`


Deprecations
~~~~~~~~~~~~

- The following objects were deprecated:
* ``pulpcore.plugin.models.BaseDistribution`` -- Instead use
``pulpcore.plugin.models.Distribution``.
* ``pulpcore.plugin.viewset.BaseDistributionViewSet`` -- Instead use
``pulpcore.plugin.viewset.DistributionViewSet``.
* ``pulpcore.plugin.serializer.BaseDistributionSerializer`` -- Instead use
``pulpcore.plugin.serializer.DistributionSerializer``.
* ``pulpcore.plugin.serializer.PublicationDistributionSerializer`` -- Instead use define the
``publication`` field directly on your detail distribution object. See the docstring for
``pulpcore.plugin.serializer.DistributionSerializer`` for an example.
* ``pulpcore.plugin.serializer.RepositoryVersionDistributionSerializer`` -- Instead use define the
``repository_version`` field directly on your detail distribution object. See the docstring for
``pulpcore.plugin.serializer.DistributionSerializer`` for an example.
* ``pulpcore.plugin.viewset.DistributionFilter`` -- Instead use
``pulpcore.plugin.viewset.NewDistributionFilter``.

.. note::

You will have to define a migration to move your data from
``pulpcore.plugin.models.BaseDistribution`` to ``pulpcore.plugin.models.Distribution``. See the
pulp_file migration 0009 as a reference example.

:redmine:`8385`
- Deprecated the ``pulpcore.plugin.tasking.enqueue_with_reservation``. Instead use the
``pulpcore.plugin.tasking.dispatch`` interface.
:redmine:`8496`
- The usage of non-JSON serializable types of ``args`` and ``kwargs`` to tasks is deprecated. Future
releases of pulpcore may discontinue accepting complex argument types. Note, UUID objects are not
JSON serializable. A deprecated warning is logged if a non-JSON serializable is used.
:redmine:`8505`

===================
REST API
--------

Bugfixes
~~~~~~~~

- Skip allowed content checks on collectstatic
(backported from 8711)
:redmine:`8712`
- Fixed cleanup of UploadChunks when their corresponding Upload is deleted.
(backported from 7316)
:redmine:`8757`
- Fixed compatibility with Django 2.2 LTS. Pulp now requires Django~=2.2.23
(backported from 8691)
:redmine:`8758`
- Pinned click~=7.1.2 to ensure RQ is compatible with it.
:redmine:`8767`


Plugin API
----------

No significant changes.

===================
REST API
--------

Bugfixes
~~~~~~~~

- Fixed a race condition that sometimes surfaced during handling of reserved resources.
:redmine:`8632`
- Handled a tasking race condition where cleaning up resource reservations sometimes raised an IntegrityError.
:redmine:`8648`


Plugin API
----------

Bugfixes
~~~~~~~~

- Allow plugins to unset the ``queryset_filtering_required_permission`` attribute in ``NamedModelViewSet``.
:redmine:`8444`

===================
REST API
--------

Features
~~~~~~~~

- Raise error when syncing content with a checksum not included in ``ALLOWED_CONTENT_CHECKSUMS``.
:redmine:`7854`
- User can evaluate how many content units are affected with checksum type change with 'pulpcore-manager handle-artifact-checksums --report'.
:redmine:`7986`
- The fields ``proxy_username`` and ``proxy_password`` have been added to remotes.
Credentials can no longer be specified as part of the ``proxy_url``.
A data migration will move the proxy auth information on existing remotes to the new fields.
:redmine:`8167`
- Added the ``WORKER_TTL`` setting, that specifies the interval a worker is considered missing after its last heartbeat.
:redmine:`8291`
- Due to the removal of ``md5`` and ``sha1`` from the ``ALLOWED_CONTENT_CHECKSUMS`` setting, every
system that had any Artifacts synced in in prior to 3.11 will have to run the ``pulpcore-manager
handle-content-checksums`` command. A data migration is provided with 3.11 that will run this
automatically as part of the ``pulpcore-manager migrate`` command all upgrades must run anyway.
:redmine:`8322`


Bugfixes
~~~~~~~~

- Fixed a bug experienced by the migration plugin where all content objects are assumed to have a
remote associated with them.
:redmine:`7876`
- Restored inadvertently-changed content-guards API to its correct endpoint.

In the process of adding generic list-endpoints, the /pulp/api/v3/contentguards
API was inadvertently rehomed to /pulp/api/v3/content_guards. This change restores
it to its published value.
:redmine:`8283`
- Added headers field to the list of fields in the ``RemoteSerializer`` base class and marked it optional to make it accessible via the REST api.
:redmine:`8330`
- Fixed AccessPolicy AttributeError.
:redmine:`8395`


Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~

- Removed correlation id feature from tech preview.
:redmine:`7927`
- Removed 'tech preview' label from ``handle-artifact-checksums`` command.

``handle-artifact-checksums`` is now a fully-supported part of Pulp3.
:redmine:`7928`
- Added a warning banner to the ``ALLOWED_CONTENT_CHECKSUMS`` setting section indicating the setting
is not fully enforcing in ``pulpcore`` code and various plugins.
:redmine:`8342`


Removals
~~~~~~~~

- The ``component`` field of the ``versions`` section of the status API /pulp/api/v3/status/`` now
lists the Django app name, not the Python package name. Similarly the OpenAPI schema at
``/pulp/api/v3`` does also.
:redmine:`8198`
- Removed sensitive fields ``username``, ``password``, and ``client_key`` from Remote responses. These
fields can still be set and updated but will no longer be readable.
:redmine:`8202`
- Adjusted the ``ALLOWED_CONTENT_CHECKSUMS`` setting to remove ``md5`` and ``sha1`` since they are
insecure. Now, by default, the ``ALLOWED_CONTENT_CHECKSUMS`` contain ``sha224``, ``sha256``,
``sha384``, and ``sha512``.
:redmine:`8246`


Misc
~~~~

- :redmine:`7797`, :redmine:`7984`, :redmine:`8315`


Plugin API
----------

Features
~~~~~~~~

- Allow developers to use more than one WorkingDirectory() within a task, including nested calls. Tasks will also now use a temporary working directory by default.
:redmine:`7815`
- Added the ``pulpcore.app.pulp_hashlib`` module which provides the ``new`` function and ensures only
allowed hashers listed in ``ALLOWED_CONTENT_CHECKSUMS`` can be instantiated. Plugin writers should
use this instead of ``hashlib.new`` to generate checksum hashers.
:redmine:`7984`
- Add a ``get_content`` method to ``pulpcore.plugin.models.RepositoryVersion`` that accepts a
queryset and returns a list of content in that repository using the given queryset.
This allows for specific content type to be returned by executing
``repo_version.get_content(content_qs=MyContentType.objects)``.
:redmine:`8375`


Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~

- Added docs identifying plugin writers to use the ``pulpcore.app.pulp_hashlib`` module which provides
the ``new`` function and ensures only allowed hashers can be instantiated. This should be used in
place of ``hashlib.new``.
:redmine:`7984`
- The use of ``tempdir.TemporaryDirectory`` in tasks has been documented.
:redmine:`8231`


Removals
~~~~~~~~

- Adjusted the ``ALLOWED_CONTENT_CHECKSUMS`` setting to remove ``md5`` and ``sha1`` since they are
insecure. Now, by default, the ``ALLOWED_CONTENT_CHECKSUMS`` contain ``sha224``, ``sha256``,
``sha384``, and ``sha512``.
:redmine:`8246`
- Removed unused `get_plugin_storage_path` method.
:redmine:`8343`
- It is not longer possible to address AccessPolicy via the viewset's classname. Viewset's urlpattern should be used instead.
:redmine:`8397`
- Removed deprecated `key` field returned by the signing service.
Plugin writers must now refer directly to the `public_key` field on the signing service object.
:redmine:`8398`


Deprecations
~~~~~~~~~~~~

- ``pulpcore.plugin.tasking.WorkingDirectory`` has been deprecated.
:redmine:`8231`

===================
REST API
--------

Features
~~~~~~~~

- Change the default deployment layout

This changes the default deployment layout. The main change is that MEDIA_ROOT gets its own
directory. This allows limiting the file permissions in a shared Pulp 2 + Pulp 3 deployment and the
SELinux file contexts. Another benefit is compatibility with django_extensions' unreferenced_files
command which lists all files in MEDIA_ROOT that are not in the database.

Other paths are kept on the same absolute paths. The documentation is updated to show the latest
best practices.
:redmine:`7178`
- Added general endpoints to list ``Content``, ``ContentGuards``, and ``Repositories``.
:redmine:`7204`
- Added /importers/core/pulp/import-check/ to validate import-parameters.
:redmine:`7549`
- Added a new field called public_key to SigningService. This field preserves the value of the public
key. In addition to that, the field fingerprint was introduced as well. This field identifies the
public key.
:redmine:`7700`
- Added possibility to filter users and groups by various fields.
:redmine:`7975`
- Added pulp_labels to allow users to add key/value data to objects.
:redmine:`8065`
- Added ``pulp_label_select`` filter to allow users to filter by labels.
:redmine:`8067`
- Added optional headers field to the aiohttp ClientSession.
:redmine:`8083`
- Allow querying names on the api using name__icontains, name__contains and name__startswith query parameters.
:redmine:`8094`
- Added RBAC to the endpoint for managing groups.
:redmine:`8159`
- Added RBAC to the endpoint for managing group users.
:redmine:`8160`
- Added the ``AccessPolicy.customized`` field which if ``True`` indicates a user has modified the
default AccessPolicy.
:redmine:`8182`
- Added filtering for access policies.
:redmine:`8189`
- As an authenticated user I can create and view artifacts.
:redmine:`8193`


Bugfixes
~~~~~~~~

- Fixed bug where duplicate artifact error message was nondeterministic in displaying different error
messages with different checksum types. Also, updated duplicate artifact error message to be more
descriptive.
:redmine:`3387`
- Fixed Pulp import/export bug that occurs when sha384 or sha512 is not in ``ALLOWED_CONTENT_CHECKSUMS``.
:redmine:`7836`
- X-CSRFToken is not sent through ajax requests (PUT) in api.html. Fixed by setting the right value in
the JS code.
:redmine:`7888`
- Provide a mechanism to automatically resolve issues and prevent deadlocks when Redis experiences data loss (such as a restart).
:redmine:`7912`
- Silence unnecessary log messages from django_guid which were spamming up the logs.
:redmine:`7982`
- Changed the default permission class to ``IsAdminUser`` to protect endpoints not yet guarded by an access policy from users without permission.
:redmine:`8018`
- Fixed apidoc bug, where model and object permissions on groups overlapped.
:redmine:`8033`
- Fixed the viewset_name used by access policy for the cases when parent_viewset is involved.
:redmine:`8152`
- Made the viewset_name property of access policies read only.
:redmine:`8185`


Improved Documentation
~~~~~~~~~~~~~~~~~~~~~~

- Added a description of the common filesystem layout in the deployment section.
:redmine:`7750`
- Updated the reference to the new location of pulplift at the installer repository in the development section.
:redmine:`7878`
- Add links to plugin docs into docs.pulpproject.org.
:redmine:`8131`
- Added documentation for labels.
:redmine:`8157`


Misc
~~~~

- :redmine:`8203`


Plugin API
----------

Features
~~~~~~~~

- Add ``rate_limit`` option to ``Remote``
:redmine:`7965`
- Made DistributionFilter accessible to plugin writers.
:redmine:`8059`
- Adding ``Label`` and ``LabelSerializer`` to the plugin api.
:redmine:`8065`
- Added ``LabelSelectFilter`` to filter resources by labels.
:redmine:`8067`
- Added ReadOnlyRepositoryViewset to the plugin API.
:redmine:`8103`
- Added NAME_FILTER_OPTIONS to the plugin API to gain more consistency across plugins when filter by name or similar CharFields.
:redmine:`8117`
- Added `has_repo_attr_obj_perms` and `has_repo_attr_model_or_obj_perms` to the global access checks available to all plugins to use.
:redmine:`8161`


Removals
~~~~~~~~

- Plugins are required to define a ``version`` attribute on their subclass of
``PulpPluginAppConfig``. Starting with pulpcore==3.10, if undefined while Pulp loads, Pulp will
refuse to start.
:redmine:`7930`
- Changed the default permission class to from ``IsAuthenticated`` to ``IsAdminUser``.
Any endpoints that should be accessible by all known to the system users need to specify the permission_classes accordingly.
:redmine:`8018`
- ``pulpcore.plugin.models.UnsupportedDigestValidationError`` has been removed. Plugins should
look for this at ``pulpcore.plugin.exceptions.UnsupportedDigestValidationError`` instead.
:redmine:`8169`


Deprecations
~~~~~~~~~~~~

- Access to the path of the public key of a signing service was deprecated. The value of the public
key is now expected to be saved in the model instance as ``SigningService.public_key``.
:redmine:`7700`
- The ``pulpcore.plugin.storage.get_plugin_storage_path()`` method has been deprecated.
:redmine:`7935`

==================
REST API
--------

Removals
~~~~~~~~

- CHUNKED_UPLOAD_DIR was converted to a relative path inside MEDIA_ROOT.
:redmine:`8099`

Plugin API
----------

No significant changes.