Changelogs >

Django

PyUp actively tracks 419,855 Python packages for vulnerabilities to keep your Python environments secure.

Scan your dependencies

4.0.4

==========================

*April 11, 2022*

Django 4.0.4 fixes two security issues with severity "high" and two bugs in
4.0.3.

CVE-2022-28346: Potential SQL injection in ``QuerySet.annotate()``, ``aggregate()``, and ``extra()``
====================================================================================================

:meth:`.QuerySet.annotate`, :meth:`~.QuerySet.aggregate`, and
:meth:`~.QuerySet.extra` methods were subject to SQL injection in column
aliases, using a suitably crafted dictionary, with dictionary expansion, as the
``**kwargs`` passed to these methods.

CVE-2022-28347: Potential SQL injection via ``QuerySet.explain(**options)`` on PostgreSQL
=========================================================================================

:meth:`.QuerySet.explain` method was subject to SQL injection in option names,
using a suitably crafted dictionary, with dictionary expansion, as the
``**options`` argument.

Bugfixes
========

* Fixed a regression in Django 4.0 that caused ignoring multiple
``FilteredRelation()`` relationships to the same field (:ticket:`33598`).

* Fixed a regression in Django 3.2.4 that caused the auto-reloader to no longer
detect changes when the ``DIRS`` option of the ``TEMPLATES`` setting
contained an empty string (:ticket:`33628`).


==========================

4.0.3

Not secure
==========================

*March 1, 2022*

Django 4.0.3 fixes several bugs in 4.0.2. Also, all Python code in Django is
reformatted with `black`_.

.. _black: https://pypi.org/project/black/

Bugfixes
========

* Prevented, following a regression in Django 4.0.1, :djadmin:`makemigrations`
from generating infinite migrations for a model with ``ManyToManyField`` to
a lowercased swappable model such as ``'auth.user'`` (:ticket:`33515`).

* Fixed a regression in Django 4.0 that caused a crash when rendering invalid
inlines with :attr:`~django.contrib.admin.ModelAdmin.readonly_fields` in the
admin (:ticket:`33547`).


==========================

4.0.2

Not secure
==========================

*February 1, 2022*

Django 4.0.2 fixes two security issues with severity "medium" and several bugs
in 4.0.1. Also, the latest string translations from Transifex are incorporated,
with a special mention for Bulgarian (fully translated).

CVE-2022-22818: Possible XSS via ``{% debug %}`` template tag
=============================================================

The ``{% debug %}`` template tag didn't properly encode the current context,
posing an XSS attack vector.

In order to avoid this vulnerability, ``{% debug %}`` no longer outputs
information when the ``DEBUG`` setting is ``False``, and it ensures all context
variables are correctly escaped when the ``DEBUG`` setting is ``True``.

CVE-2022-23833: Denial-of-service possibility in file uploads
=============================================================

Passing certain inputs to multipart forms could result in an infinite loop when
parsing files.

Bugfixes
========

* Fixed a bug in Django 4.0 where ``TestCase.captureOnCommitCallbacks()`` could
execute callbacks multiple times (:ticket:`33410`).

* Fixed a regression in Django 4.0 where ``help_text`` was HTML-escaped in
automatically-generated forms (:ticket:`33419`).

* Fixed a regression in Django 4.0 that caused displaying an incorrect name for
class-based views on the technical 404 debug page (:ticket:`33425`).

* Fixed a regression in Django 4.0 that caused an incorrect ``repr`` of
``ResolverMatch`` for class-based views (:ticket:`33426`).

* Fixed a regression in Django 4.0 that caused a crash of ``makemigrations`` on
models without ``Meta.order_with_respect_to`` but with a field named
``_order`` (:ticket:`33449`).

* Fixed a regression in Django 4.0 that caused incorrect
:attr:`.ModelAdmin.radio_fields` layout in the admin (:ticket:`33407`).

* Fixed a duplicate operation regression in Django 4.0 that caused a migration
crash when altering a primary key type for a concrete parent model referenced
by a foreign key (:ticket:`33462`).

* Fixed a bug in Django 4.0 that caused a crash of ``QuerySet.aggregate()``
after ``annotate()`` on an aggregate function with a
:ref:`default <aggregate-default>` (:ticket:`33468`).

* Fixed a regression in Django 4.0 that caused a crash of ``makemigrations``
when renaming a field of a renamed model (:ticket:`33480`).


==========================

4.0.1

Not secure
==========================

*January 4, 2022*

Django 4.0.1 fixes one security issue with severity "medium", two security
issues with severity "low", and several bugs in 4.0.

CVE-2021-45115: Denial-of-service possibility in ``UserAttributeSimilarityValidator``
=====================================================================================

:class:`.UserAttributeSimilarityValidator` incurred significant overhead
evaluating submitted password that were artificially large in relative to the
comparison values. On the assumption that access to user registration was
unrestricted this provided a potential vector for a denial-of-service attack.

In order to mitigate this issue, relatively long values are now ignored by
``UserAttributeSimilarityValidator``.

This issue has severity "medium" according to the :ref:`Django security policy
<security-disclosure>`.

CVE-2021-45116: Potential information disclosure in ``dictsort`` template filter
================================================================================

Due to leveraging the Django Template Language's variable resolution logic, the
:tfilter:`dictsort` template filter was potentially vulnerable to information
disclosure or unintended method calls, if passed a suitably crafted key.

In order to avoid this possibility, ``dictsort`` now works with a restricted
resolution logic, that will not call methods, nor allow indexing on
dictionaries.

As a reminder, all untrusted user input should be validated before use.

This issue has severity "low" according to the :ref:`Django security policy
<security-disclosure>`.

CVE-2021-45452: Potential directory-traversal via ``Storage.save()``
====================================================================

``Storage.save()`` allowed directory-traversal if directly passed suitably
crafted file names.

This issue has severity "low" according to the :ref:`Django security policy
<security-disclosure>`.

Bugfixes
========

* Fixed a regression in Django 4.0 that caused a crash of
:meth:`~django.test.SimpleTestCase.assertFormsetError` on a formset named
``form`` (:ticket:`33346`).

* Fixed a bug in Django 4.0 that caused a crash on booleans with the
``RedisCache`` backend (:ticket:`33361`).

* Relaxed the check added in Django 4.0 to reallow use of a duck-typed
``HttpRequest`` in ``django.views.decorators.cache.cache_control()`` and
``never_cache()`` decorators (:ticket:`33350`).

* Fixed a regression in Django 4.0 that caused creating bogus migrations for
models that reference swappable models such as ``auth.User``
(:ticket:`33366`).

* Fixed a long standing bug in :ref:`geos-geometry-collections` and
:class:`~django.contrib.gis.geos.Polygon` that caused a crash on some
platforms (reported on macOS based on the ``ARM64`` architecture)
(:ticket:`32600`).


========================

4.0

Not secure
========================

*December 7, 2021*

Welcome to Django 4.0!

These release notes cover the :ref:`new features <whats-new-4.0>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-4.0>` you'll
want to be aware of when upgrading from Django 3.2 or earlier. We've
:ref:`begun the deprecation process for some features
<deprecated-features-4.0>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Python compatibility
====================

Django 4.0 supports Python 3.8, 3.9, and 3.10. We **highly recommend** and only
officially support the latest release of each series.

The Django 3.2.x series is the last to support Python 3.6 and 3.7.

.. _whats-new-4.0:

What's new in Django 4.0
========================

``zoneinfo`` default timezone implementation
--------------------------------------------

The Python standard library's :mod:`zoneinfo` is now the default timezone
implementation in Django.

This is the next step in the migration from using ``pytz`` to using
:mod:`zoneinfo`. Django 3.2 allowed the use of non-``pytz`` time zones. Django
4.0 makes ``zoneinfo`` the default implementation. Support for ``pytz`` is now
deprecated and will be removed in Django 5.0.

:mod:`zoneinfo` is part of the Python standard library from Python 3.9. The
``backports.zoneinfo`` package is automatically installed alongside Django if
you are using Python 3.8.

The move to ``zoneinfo`` should be largely transparent. Selection of the
current timezone, conversion of datetime instances to the current timezone in
forms and templates, as well as operations on aware datetimes in UTC are
unaffected.

However, if you are working with non-UTC time zones, and using the ``pytz``
``normalize()`` and ``localize()`` APIs, possibly with the :setting:`TIME_ZONE
<DATABASE-TIME_ZONE>` setting, you will need to audit your code, since ``pytz``
and ``zoneinfo`` are not entirely equivalent.

To give time for such an audit, the transitional :setting:`USE_DEPRECATED_PYTZ`
setting allows continued use of ``pytz`` during the 4.x release cycle. This
setting will be removed in Django 5.0.

In addition, a `pytz_deprecation_shim`_ package, created by the ``zoneinfo``
author, can be used to assist with the migration from ``pytz``. This package
provides shims to help you safely remove ``pytz``, and has a detailed
`migration guide`_ showing how to move to the new ``zoneinfo`` APIs.

Using `pytz_deprecation_shim`_ and the :setting:`USE_DEPRECATED_PYTZ`
transitional setting is recommended if you need a gradual update path.

.. _pytz_deprecation_shim: https://pytz-deprecation-shim.readthedocs.io/en/latest/index.html
.. _migration guide: https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html

Functional unique constraints
-----------------------------

The new :attr:`*expressions <django.db.models.UniqueConstraint.expressions>`
positional argument of
:class:`UniqueConstraint() <django.db.models.UniqueConstraint>` enables
creating functional unique constraints on expressions and database functions.
For example::

from django.db import models
from django.db.models import UniqueConstraint
from django.db.models.functions import Lower


class MyModel(models.Model):
first_name = models.CharField(max_length=255)
last_name = models.CharField(max_length=255)

class Meta:
constraints = [
UniqueConstraint(
Lower('first_name'),
Lower('last_name').desc(),
name='first_last_name_unique',
),
]

Functional unique constraints are added to models using the
:attr:`Meta.constraints <django.db.models.Options.constraints>` option.

``scrypt`` password hasher
--------------------------

The new :ref:`scrypt password hasher <scrypt-usage>` is more secure and
recommended over PBKDF2. However, it's not the default as it requires OpenSSL
1.1+ and more memory.

Redis cache backend
-------------------

The new ``django.core.cache.backends.redis.RedisCache`` cache backend provides
built-in support for caching with Redis. `redis-py`_ 3.0.0 or higher is
required. For more details, see the :ref:`documentation on caching with Redis
in Django <redis>`.

.. _`redis-py`: https://pypi.org/project/redis/

Template based form rendering
-----------------------------

:class:`Forms <django.forms.Form>`, :doc:`Formsets </topics/forms/formsets>`,
and :class:`~django.forms.ErrorList` are now rendered using the template engine
to enhance customization. See the new :meth:`~django.forms.Form.render`,
:meth:`~django.forms.Form.get_context`, and
:attr:`~django.forms.Form.template_name` for ``Form`` and
:ref:`formset rendering <formset-rendering>` for ``Formset``.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The ``admin/base.html`` template now has a new block ``header`` which
contains the admin site header.

* The new :meth:`.ModelAdmin.get_formset_kwargs` method allows customizing the
keyword arguments passed to the constructor of a formset.

* The navigation sidebar now has a quick filter toolbar.

* The new context variable ``model`` which contains the model class for each
model is added to the :meth:`.AdminSite.each_context` method.

* The new :attr:`.ModelAdmin.search_help_text` attribute allows specifying a
descriptive text for the search box.

* The :attr:`.InlineModelAdmin.verbose_name_plural` attribute now fallbacks to
the :attr:`.InlineModelAdmin.verbose_name` + ``'s'``.

* jQuery is upgraded from version 3.5.1 to 3.6.0.

:mod:`django.contrib.admindocs`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The admindocs now allows esoteric setups where :setting:`ROOT_URLCONF` is not
a string.

* The model section of the ``admindocs`` now shows cached properties.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* The default iteration count for the PBKDF2 password hasher is increased from
260,000 to 320,000.

* The new
:attr:`LoginView.next_page <django.contrib.auth.views.LoginView.next_page>`
attribute and
:meth:`~django.contrib.auth.views.LoginView.get_default_redirect_url` method
allow customizing the redirect after login.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* Added support for SpatiaLite 5.

* :class:`~django.contrib.gis.gdal.GDALRaster` now allows creating rasters in
any GDAL virtual filesystem.

* The new :class:`~django.contrib.gis.admin.GISModelAdmin` class allows
customizing the widget used for ``GeometryField``. This is encouraged instead
of deprecated ``GeoModelAdmin`` and ``OSMGeoAdmin``.

:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The PostgreSQL backend now supports connecting by a service name. See
:ref:`postgresql-connection-settings` for more details.

* The new :class:`~django.contrib.postgres.operations.AddConstraintNotValid`
operation allows creating check constraints on PostgreSQL without verifying
that all existing rows satisfy the new constraint.

* The new :class:`~django.contrib.postgres.operations.ValidateConstraint`
operation allows validating check constraints which were created using
:class:`~django.contrib.postgres.operations.AddConstraintNotValid` on
PostgreSQL.

* The new
:class:`ArraySubquery() <django.contrib.postgres.expressions.ArraySubquery>`
expression allows using subqueries to construct lists of values on
PostgreSQL.

* The new :lookup:`trigram_word_similar` lookup, and the
:class:`TrigramWordDistance()
<django.contrib.postgres.search.TrigramWordDistance>` and
:class:`TrigramWordSimilarity()
<django.contrib.postgres.search.TrigramWordSimilarity>` expressions allow
using trigram word similarity.

:mod:`django.contrib.staticfiles`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* :class:`~django.contrib.staticfiles.storage.ManifestStaticFilesStorage` now
replaces paths to JavaScript source map references with their hashed
counterparts.

* The new ``manifest_storage`` argument of
:class:`~django.contrib.staticfiles.storage.ManifestFilesMixin` and
:class:`~django.contrib.staticfiles.storage.ManifestStaticFilesStorage`
allows customizing the manifest file storage.

Cache
~~~~~

* The new async API for ``django.core.cache.backends.base.BaseCache`` begins
the process of making cache backends async-compatible. The new async methods
all have ``a`` prefixed names, e.g. ``aadd()``, ``aget()``, ``aset()``,
``aget_or_set()``, or ``adelete_many()``.

Going forward, the ``a`` prefix will be used for async variants of methods
generally.

CSRF
~~~~

* CSRF protection now consults the ``Origin`` header, if present. To facilitate
this, :ref:`some changes <csrf-trusted-origins-changes-4.0>` to the
:setting:`CSRF_TRUSTED_ORIGINS` setting are required.

Forms
~~~~~

* :class:`~django.forms.ModelChoiceField` now includes the provided value in
the ``params`` argument of a raised
:exc:`~django.core.exceptions.ValidationError` for the ``invalid_choice``
error message. This allows custom error messages to use the ``%(value)s``
placeholder.

* :class:`~django.forms.formsets.BaseFormSet` now renders non-form errors with
an additional class of ``nonform`` to help distinguish them from
form-specific errors.

* :class:`~django.forms.formsets.BaseFormSet` now allows customizing the widget
used when deleting forms via
:attr:`~django.forms.formsets.BaseFormSet.can_delete` by setting the
:attr:`~django.forms.formsets.BaseFormSet.deletion_widget` attribute or
overriding :meth:`~django.forms.formsets.BaseFormSet.get_deletion_widget`
method.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* Added support and translations for the Malay language.

Generic Views
~~~~~~~~~~~~~

* :class:`~django.views.generic.edit.DeleteView` now uses
:class:`~django.views.generic.edit.FormMixin`, allowing you to provide a
:class:`~django.forms.Form` subclass, with a checkbox for example, to confirm
deletion. In addition, this allows ``DeleteView`` to function with
:class:`django.contrib.messages.views.SuccessMessageMixin`.

In accordance with ``FormMixin``, object deletion for POST requests is
handled in ``form_valid()``. Custom delete logic in ``delete()`` handlers
should be moved to ``form_valid()``, or a shared helper method, as needed.

Logging
~~~~~~~

* The alias of the database used in an SQL call is now passed as extra context
along with each message to the :ref:`django-db-logger` logger.

Management Commands
~~~~~~~~~~~~~~~~~~~

* The :djadmin:`runserver` management command now supports the
:option:`--skip-checks` option.

* On PostgreSQL, :djadmin:`dbshell` now supports specifying a password file.

* The :djadmin:`shell` command now respects :py:data:`sys.__interactivehook__`
at startup. This allows loading shell history between interactive sessions.
As a consequence, ``readline`` is no longer loaded if running in *isolated*
mode.

* The new :attr:`BaseCommand.suppressed_base_arguments
<django.core.management.BaseCommand.suppressed_base_arguments>` attribute
allows suppressing unsupported default command options in the help output.

* The new :option:`startapp --exclude` and :option:`startproject --exclude`
options allow excluding directories from the template.

Models
~~~~~~

* New :meth:`QuerySet.contains(obj) <.QuerySet.contains>` method returns
whether the queryset contains the given object. This tries to perform the
query in the simplest and fastest way possible.

* The new ``precision`` argument of the
:class:`Round() <django.db.models.functions.Round>` database function allows
specifying the number of decimal places after rounding.

* :meth:`.QuerySet.bulk_create` now sets the primary key on objects when using
SQLite 3.35+.

* :class:`~django.db.models.DurationField` now supports multiplying and
dividing by scalar values on SQLite.

* :meth:`.QuerySet.bulk_update` now returns the number of objects updated.

* The new :attr:`.Expression.empty_result_set_value` attribute allows
specifying a value to return when the function is used over an empty result
set.

* The ``skip_locked`` argument of :meth:`.QuerySet.select_for_update()` is now
allowed on MariaDB 10.6+.

* :class:`~django.db.models.Lookup` expressions may now be used in ``QuerySet``
annotations, aggregations, and directly in filters.

* The new :ref:`default <aggregate-default>` argument for built-in aggregates
allows specifying a value to be returned when the queryset (or grouping)
contains no entries, rather than ``None``.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* The :class:`~django.middleware.security.SecurityMiddleware` now adds the
:ref:`Cross-Origin Opener Policy <cross-origin-opener-policy>` header with a
value of ``'same-origin'`` to prevent cross-origin popups from sharing the
same browsing context. You can prevent this header from being added by
setting the :setting:`SECURE_CROSS_ORIGIN_OPENER_POLICY` setting to ``None``.

Signals
~~~~~~~

* The new ``stdout`` argument for :func:`~django.db.models.signals.pre_migrate`
and :func:`~django.db.models.signals.post_migrate` signals allows redirecting
output to a stream-like object. It should be preferred over
:py:data:`sys.stdout` and :py:func:`print` when emitting verbose output in
order to allow proper capture when testing.

Templates
~~~~~~~~~

* :tfilter:`floatformat` template filter now allows using the ``u`` suffix to
force disabling localization.

Tests
~~~~~

* The new ``serialized_aliases`` argument of
:func:`django.test.utils.setup_databases` determines which
:setting:`DATABASES` aliases test databases should have their state
serialized to allow usage of the
:ref:`serialized_rollback <test-case-serialized-rollback>` feature.

* Django test runner now supports a :option:`--buffer <test --buffer>` option
with parallel tests.

* The new ``logger`` argument to :class:`~django.test.runner.DiscoverRunner`
allows a Python :py:ref:`logger <logger>` to be used for logging.

* The new :meth:`.DiscoverRunner.log` method provides a way to log messages
that uses the ``DiscoverRunner.logger``, or prints to the console if not set.

* Django test runner now supports a :option:`--shuffle <test --shuffle>` option
to execute tests in a random order.

* The :option:`test --parallel` option now supports the value ``auto`` to run
one test process for each processor core.

* :meth:`.TestCase.captureOnCommitCallbacks` now captures new callbacks added
while executing :func:`.transaction.on_commit` callbacks.

.. _backwards-incompatible-4.0:

Backwards incompatible changes in 4.0
=====================================

Database backend API
--------------------

This section describes changes that may be needed in third-party database
backends.

* ``DatabaseOperations.year_lookup_bounds_for_date_field()`` and
``year_lookup_bounds_for_datetime_field()`` methods now take the optional
``iso_year`` argument in order to support bounds for ISO-8601 week-numbering
years.

* The second argument of ``DatabaseSchemaEditor._unique_sql()`` and
``_create_unique_sql()`` methods is now ``fields`` instead of ``columns``.

:mod:`django.contrib.gis`
-------------------------

* Support for PostGIS 2.3 is removed.

* Support for GDAL 2.0 and GEOS 3.5 is removed.

Dropped support for PostgreSQL 9.6
----------------------------------

Upstream support for PostgreSQL 9.6 ends in November 2021. Django 4.0 supports
PostgreSQL 10 and higher.

Also, the minimum supported version of ``psycopg2`` is increased from 2.5.4 to
2.8.4, as ``psycopg2`` 2.8.4 is the first release to support Python 3.8.

Dropped support for Oracle 12.2 and 18c
---------------------------------------

Upstream support for Oracle 12.2 ends in March 2022 and for Oracle 18c it ends
in June 2021. Django 3.2 will be supported until April 2024. Django 4.0
officially supports Oracle 19c.

.. _csrf-trusted-origins-changes-4.0:

``CSRF_TRUSTED_ORIGINS`` changes
--------------------------------

Format change
~~~~~~~~~~~~~

Values in the :setting:`CSRF_TRUSTED_ORIGINS` setting must include the scheme
(e.g. ``'http://'`` or ``'https://'``) instead of only the hostname.

Also, values that started with a dot, must now also include an asterisk before
the dot. For example, change ``'.example.com'`` to ``'https://*.example.com'``.

A system check detects any required changes.

Configuring it may now be required
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

As CSRF protection now consults the ``Origin`` header, you may need to set
:setting:`CSRF_TRUSTED_ORIGINS`, particularly if you allow requests from
subdomains by setting :setting:`CSRF_COOKIE_DOMAIN` (or
:setting:`SESSION_COOKIE_DOMAIN` if :setting:`CSRF_USE_SESSIONS` is enabled) to
a value starting with a dot.

``SecurityMiddleware`` no longer sets the ``X-XSS-Protection`` header
---------------------------------------------------------------------

The :class:`~django.middleware.security.SecurityMiddleware` no longer sets the
``X-XSS-Protection`` header if the ``SECURE_BROWSER_XSS_FILTER`` setting is
``True``. The setting is removed.

Most modern browsers don't honor the ``X-XSS-Protection`` HTTP header. You can
use Content-Security-Policy_ without allowing ``'unsafe-inline'`` scripts
instead.

If you want to support legacy browsers and set the header, use this line in a
custom middleware::

response.headers.setdefault('X-XSS-Protection', '1; mode=block')

.. _Content-Security-Policy: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy

Migrations autodetector changes
-------------------------------

The migrations autodetector now uses model states instead of model classes.
Also, migration operations for ``ForeignKey`` and ``ManyToManyField`` fields no
longer specify attributes which were not passed to the fields during
initialization.

As a side-effect, running ``makemigrations`` might generate no-op
``AlterField`` operations for ``ManyToManyField`` and ``ForeignKey`` fields in
some cases.

``DeleteView`` changes
----------------------

:class:`~django.views.generic.edit.DeleteView` now uses
:class:`~django.views.generic.edit.FormMixin` to handle POST requests. As a
consequence, any custom deletion logic in ``delete()`` handlers should be
moved to ``form_valid()``, or a shared helper method, if required.

Miscellaneous
-------------

* Support for ``cx_Oracle`` < 7.0 is removed.

* To allow serving a Django site on a subpath without changing the value of
:setting:`STATIC_URL`, the leading slash is removed from that setting (now
``'static/'``) in the default :djadmin:`startproject` template.

* The :class:`~django.contrib.admin.AdminSite` method for the admin ``index``
view is no longer decorated with ``never_cache`` when accessed directly,
rather than via the recommended ``AdminSite.urls`` property, or
``AdminSite.get_urls()`` method.

* Unsupported operations on a sliced queryset now raise ``TypeError`` instead
of ``AssertionError``.

* The undocumented ``django.test.runner.reorder_suite()`` function is renamed
to ``reorder_tests()``. It now accepts an iterable of tests rather than a
test suite, and returns an iterator of tests.

* Calling ``FileSystemStorage.delete()`` with an empty ``name`` now raises
``ValueError`` instead of ``AssertionError``.

* Calling ``EmailMultiAlternatives.attach_alternative()`` or
``EmailMessage.attach()`` with an invalid ``content`` or ``mimetype``
arguments now raise ``ValueError`` instead of ``AssertionError``.

* :meth:`~django.test.SimpleTestCase.assertHTMLEqual` no longer considers a
non-boolean attribute without a value equal to an attribute with the same
name and value.

* Tests that fail to load, for example due to syntax errors, now always match
when using :option:`test --tag`.

* The undocumented ``django.contrib.admin.utils.lookup_needs_distinct()``
function is renamed to ``lookup_spawns_duplicates()``.

* The undocumented ``HttpRequest.get_raw_uri()`` method is removed. The
:meth:`.HttpRequest.build_absolute_uri` method may be a suitable alternative.

* The ``object`` argument of undocumented ``ModelAdmin.log_addition()``,
``log_change()``, and ``log_deletion()`` methods is renamed to ``obj``.

* :class:`~django.utils.feedgenerator.RssFeed`,
:class:`~django.utils.feedgenerator.Atom1Feed`, and their subclasses now emit
elements with no content as self-closing tags.

* ``NodeList.render()`` no longer casts the output of ``render()`` method for
individual nodes to a string. ``Node.render()`` should always return a string
as documented.

* The ``where_class`` property of ``django.db.models.sql.query.Query`` and the
``where_class`` argument to the private ``get_extra_restriction()`` method of
``ForeignObject`` and ``ForeignObjectRel`` are removed. If needed, initialize
``django.db.models.sql.where.WhereNode`` instead.

* The ``filter_clause`` argument of the undocumented ``Query.add_filter()``
method is replaced by two positional arguments ``filter_lhs`` and
``filter_rhs``.

* :class:`~django.middleware.csrf.CsrfViewMiddleware` now uses
``request.META['CSRF_COOKIE_NEEDS_UPDATE']`` in place of
``request.META['CSRF_COOKIE_USED']``, ``request.csrf_cookie_needs_reset``,
and ``response.csrf_cookie_set`` to track whether the CSRF cookie should be
sent. This is an undocumented, private API.

* The undocumented ``TRANSLATOR_COMMENT_MARK`` constant is moved from
``django.template.base`` to ``django.utils.translation.template``.

* The ``real_apps`` argument of the undocumented
``django.db.migrations.state.ProjectState.__init__()`` method must now be a
set if provided.

* :class:`~django.forms.RadioSelect` and
:class:`~django.forms.CheckboxSelectMultiple` widgets are now rendered in
``<div>`` tags so they are announced more concisely by screen readers. If you
need the previous behavior, :ref:`override the widget template
<overriding-built-in-widget-templates>` with the appropriate template from
Django 3.2.

* The :tfilter:`floatformat` template filter no longer depends on the
``USE_L10N`` setting and always returns localized output. Use the ``u``
suffix to disable localization.

* The default value of the ``USE_L10N`` setting is changed to ``True``. See the
:ref:`Localization section <use_l10n_deprecation>` above for more details.

* As part of the :ref:`move to zoneinfo <whats-new-4.0>`,
:attr:`django.utils.timezone.utc` is changed to alias
:attr:`datetime.timezone.utc`.

* The minimum supported version of ``asgiref`` is increased from 3.3.2 to
3.4.1.

.. _deprecated-features-4.0:

Features deprecated in 4.0
==========================

Use of ``pytz`` time zones
--------------------------

As part of the :ref:`move to zoneinfo <whats-new-4.0>`, use of ``pytz`` time
zones is deprecated.

Accordingly, the ``is_dst`` arguments to the following are also deprecated:

* :meth:`django.db.models.query.QuerySet.datetimes`
* :func:`django.db.models.functions.Trunc`
* :func:`django.db.models.functions.TruncSecond`
* :func:`django.db.models.functions.TruncMinute`
* :func:`django.db.models.functions.TruncHour`
* :func:`django.db.models.functions.TruncDay`
* :func:`django.db.models.functions.TruncWeek`
* :func:`django.db.models.functions.TruncMonth`
* :func:`django.db.models.functions.TruncQuarter`
* :func:`django.db.models.functions.TruncYear`
* :func:`django.utils.timezone.make_aware`

Support for use of ``pytz`` will be removed in Django 5.0.

Time zone support
-----------------

In order to follow good practice, the default value of the :setting:`USE_TZ`
setting will change from ``False`` to ``True``, and time zone support will be
enabled by default, in Django 5.0.

Note that the default :file:`settings.py` file created by
:djadmin:`django-admin startproject <startproject>` includes
:setting:`USE_TZ = True <USE_TZ>` since Django 1.4.

You can set ``USE_TZ`` to ``False`` in your project settings before then to
opt-out.

.. _use_l10n_deprecation:

Localization
------------

In order to follow good practice, the default value of the ``USE_L10N`` setting
is changed from ``False`` to ``True``.

Moreover ``USE_L10N`` is deprecated as of this release. Starting with Django
5.0, by default, any date or number displayed by Django will be localized.

The :ttag:`{% localize %} <localize>` tag and the :tfilter:`localize`/
:tfilter:`unlocalize` filters will still be honored by Django.

Miscellaneous
-------------

* ``SERIALIZE`` test setting is deprecated as it can be inferred from the
:attr:`~django.test.TestCase.databases` with the
:ref:`serialized_rollback <test-case-serialized-rollback>` option enabled.

* The undocumented ``django.utils.baseconv`` module is deprecated.

* The undocumented ``django.utils.datetime_safe`` module is deprecated.

* The default sitemap protocol for sitemaps built outside the context of a
request will change from ``'http'`` to ``'https'`` in Django 5.0.

* The ``extra_tests`` argument for :meth:`.DiscoverRunner.build_suite` and
:meth:`.DiscoverRunner.run_tests` is deprecated.

* The :class:`~django.contrib.postgres.aggregates.ArrayAgg`,
:class:`~django.contrib.postgres.aggregates.JSONBAgg`, and
:class:`~django.contrib.postgres.aggregates.StringAgg` aggregates will return
``None`` when there are no rows instead of ``[]``, ``[]``, and ``''``
respectively in Django 5.0. If you need the previous behavior, explicitly set
``default`` to ``Value([])``, ``Value('[]')``, or ``Value('')``.

* The ``django.contrib.gis.admin.GeoModelAdmin`` and ``OSMGeoAdmin`` classes
are deprecated. Use :class:`~django.contrib.admin.ModelAdmin` and
:class:`~django.contrib.gis.admin.GISModelAdmin` instead.

* Since form rendering now uses the template engine, the undocumented
``BaseForm._html_output()`` helper method is deprecated.

* The ability to return a ``str`` from ``ErrorList`` and ``ErrorDict`` is
deprecated. It is expected these methods return a ``SafeString``.

Features removed in 4.0
=======================

These features have reached the end of their deprecation cycle and are removed
in Django 4.0.

See :ref:`deprecated-features-3.0` for details on these changes, including how
to remove usage of these features.

* ``django.utils.http.urlquote()``, ``urlquote_plus()``, ``urlunquote()``, and
``urlunquote_plus()`` are removed.

* ``django.utils.encoding.force_text()`` and ``smart_text()`` are removed.

* ``django.utils.translation.ugettext()``, ``ugettext_lazy()``,
``ugettext_noop()``, ``ungettext()``, and ``ungettext_lazy()`` are removed.

* ``django.views.i18n.set_language()`` doesn't set the user language in
``request.session`` (key ``_language``).

* ``alias=None`` is required in the signature of
``django.db.models.Expression.get_group_by_cols()`` subclasses.

* ``django.utils.text.unescape_entities()`` is removed.

* ``django.utils.http.is_safe_url()`` is removed.

See :ref:`deprecated-features-3.1` for details on these changes, including how
to remove usage of these features.

* The ``PASSWORD_RESET_TIMEOUT_DAYS`` setting is removed.

* The :lookup:`isnull` lookup no longer allows using non-boolean values as the
right-hand side.

* The ``django.db.models.query_utils.InvalidQuery`` exception class is removed.

* The ``django-admin.py`` entry point is removed.

* The ``HttpRequest.is_ajax()`` method is removed.

* Support for the pre-Django 3.1 encoding format of cookies values used by
``django.contrib.messages.storage.cookie.CookieStorage`` is removed.

* Support for the pre-Django 3.1 password reset tokens in the admin site (that
use the SHA-1 hashing algorithm) is removed.

* Support for the pre-Django 3.1 encoding format of sessions is removed.

* Support for the pre-Django 3.1 ``django.core.signing.Signer`` signatures
(encoded with the SHA-1 algorithm) is removed.

* Support for the pre-Django 3.1 ``django.core.signing.dumps()`` signatures
(encoded with the SHA-1 algorithm) in ``django.core.signing.loads()`` is
removed.

* Support for the pre-Django 3.1 user sessions (that use the SHA-1 algorithm)
is removed.

* The ``get_response`` argument for
``django.utils.deprecation.MiddlewareMixin.__init__()`` is required and
doesn't accept ``None``.

* The ``providing_args`` argument for ``django.dispatch.Signal`` is removed.

* The ``length`` argument for ``django.utils.crypto.get_random_string()`` is
required.

* The ``list`` message for ``ModelMultipleChoiceField`` is removed.

* Support for passing raw column aliases to ``QuerySet.order_by()`` is removed.

* The ``NullBooleanField`` model field is removed, except for support in
historical migrations.

* ``django.conf.urls.url()`` is removed.

* The ``django.contrib.postgres.fields.JSONField`` model field is removed,
except for support in historical migrations.

* ``django.contrib.postgres.fields.jsonb.KeyTransform`` and
``django.contrib.postgres.fields.jsonb.KeyTextTransform`` are removed.

* ``django.contrib.postgres.forms.JSONField`` is removed.

* The ``{% ifequal %}`` and ``{% ifnotequal %}`` template tags are removed.

* The ``DEFAULT_HASHING_ALGORITHM`` transitional setting is removed.


===========================

3.2.13

===========================

*April 11, 2022*

Django 3.2.13 fixes two security issues with severity "high" in
3.2.12 and a regression in 3.2.4.

CVE-2022-28346: Potential SQL injection in ``QuerySet.annotate()``, ``aggregate()``, and ``extra()``
====================================================================================================

:meth:`.QuerySet.annotate`, :meth:`~.QuerySet.aggregate`, and
:meth:`~.QuerySet.extra` methods were subject to SQL injection in column
aliases, using a suitably crafted dictionary, with dictionary expansion, as the
``**kwargs`` passed to these methods.

CVE-2022-28347: Potential SQL injection via ``QuerySet.explain(**options)`` on PostgreSQL
=========================================================================================

:meth:`.QuerySet.explain` method was subject to SQL injection in option names,
using a suitably crafted dictionary, with dictionary expansion, as the
``**options`` argument.

Bugfixes
========

* Fixed a regression in Django 3.2.4 that caused the auto-reloader to no longer
detect changes when the ``DIRS`` option of the ``TEMPLATES`` setting
contained an empty string (:ticket:`33628`).


===========================

3.2.12

Not secure
===========================

*February 1, 2022*

Django 3.2.12 fixes two security issues with severity "medium" in 3.2.11.

CVE-2022-22818: Possible XSS via ``{% debug %}`` template tag
=============================================================

The ``{% debug %}`` template tag didn't properly encode the current context,
posing an XSS attack vector.

In order to avoid this vulnerability, ``{% debug %}`` no longer outputs
information when the ``DEBUG`` setting is ``False``, and it ensures all context
variables are correctly escaped when the ``DEBUG`` setting is ``True``.

CVE-2022-23833: Denial-of-service possibility in file uploads
=============================================================

Passing certain inputs to multipart forms could result in an infinite loop when
parsing files.


===========================

3.2.11

Not secure
===========================

*January 4, 2022*

Django 3.2.11 fixes one security issue with severity "medium" and two security
issues with severity "low" in 3.2.10.

CVE-2021-45115: Denial-of-service possibility in ``UserAttributeSimilarityValidator``
=====================================================================================

:class:`.UserAttributeSimilarityValidator` incurred significant overhead
evaluating submitted password that were artificially large in relative to the
comparison values. On the assumption that access to user registration was
unrestricted this provided a potential vector for a denial-of-service attack.

In order to mitigate this issue, relatively long values are now ignored by
``UserAttributeSimilarityValidator``.

This issue has severity "medium" according to the :ref:`Django security policy
<security-disclosure>`.

CVE-2021-45116: Potential information disclosure in ``dictsort`` template filter
================================================================================

Due to leveraging the Django Template Language's variable resolution logic, the
:tfilter:`dictsort` template filter was potentially vulnerable to information
disclosure or unintended method calls, if passed a suitably crafted key.

In order to avoid this possibility, ``dictsort`` now works with a restricted
resolution logic, that will not call methods, nor allow indexing on
dictionaries.

As a reminder, all untrusted user input should be validated before use.

This issue has severity "low" according to the :ref:`Django security policy
<security-disclosure>`.

CVE-2021-45452: Potential directory-traversal via ``Storage.save()``
====================================================================

``Storage.save()`` allowed directory-traversal if directly passed suitably
crafted file names.

This issue has severity "low" according to the :ref:`Django security policy
<security-disclosure>`.


===========================

3.2.10

Not secure
===========================

*December 7, 2021*

Django 3.2.10 fixes a security issue with severity "low" and a bug in 3.2.9.

CVE-2021-44420: Potential bypass of an upstream access control based on URL paths
=================================================================================

HTTP requests for URLs with trailing newlines could bypass an upstream access
control based on URL paths.

Bugfixes
========

* Fixed a regression in Django 3.2 that caused a crash of ``setUpTestData()``
with ``BinaryField`` on PostgreSQL, which is ``memoryview``-backed
(:ticket:`33333`).


==========================

3.2.9

Not secure
==========================

*November 1, 2021*

Django 3.2.9 fixes a bug in 3.2.8 and adds compatibility with Python 3.10.

Bugfixes
========

* Fixed a bug in Django 3.2 that caused a migration crash on SQLite when
altering a field with a functional index (:ticket:`33194`).


==========================

3.2.8

Not secure
==========================

*October 5, 2021*

Django 3.2.8 fixes two bugs in 3.2.7.

Bugfixes
========

* Fixed a bug in Django 3.2 that caused incorrect links on read-only fields in
the admin (:ticket:`33077`).

* Fixed a regression in Django 3.2 that caused incorrect selection of items
across all pages when actions were placed both on the top and bottom of the
admin change-list view (:ticket:`33083`).


==========================

3.2.7

Not secure
==========================

*September 1, 2021*

Django 3.2.7 fixes a bug in 3.2.6.

Bugfixes
========

* Fixed a regression in Django 3.2 that caused the incorrect offset extraction
from fixed offset timezones (:ticket:`32992`).


==========================

3.2.6

Not secure
==========================

*August 2, 2021*

Django 3.2.6 fixes several bugs in 3.2.5.

Bugfixes
========

* Fixed a regression in Django 3.2 that caused a crash validating ``"NaN"``
input with a ``forms.DecimalField`` when additional constraints, e.g.
``max_value``, were specified (:ticket:`32949`).

* Fixed a bug in Django 3.2 where a system check would crash on a model with a
reverse many-to-many relation inherited from a parent class
(:ticket:`32947`).


==========================

3.2.5

Not secure
==========================

*July 1, 2021*

Django 3.2.5 fixes a security issue with severity "high" and several bugs in
3.2.4. Also, the latest string translations from Transifex are incorporated.

CVE-2021-35042: Potential SQL injection via unsanitized ``QuerySet.order_by()`` input
=====================================================================================

Unsanitized user input passed to ``QuerySet.order_by()`` could bypass intended
column reference validation in path marked for deprecation resulting in a
potential SQL injection even if a deprecation warning is emitted.

As a mitigation the strict column reference validation was restored for the
duration of the deprecation period. This regression appeared in 3.1 as a side
effect of fixing :ticket:`31426`.

The issue is not present in the main branch as the deprecated path has been
removed.

Bugfixes
========

* Fixed a regression in Django 3.2 that caused a crash of
``QuerySet.values_list(…, named=True)`` after ``prefetch_related()``
(:ticket:`32812`).

* Fixed a bug in Django 3.2 that caused a migration crash on MySQL 8.0.13+ when
altering ``BinaryField``, ``JSONField``, or ``TextField`` to non-nullable
(:ticket:`32503`).

* Fixed a regression in Django 3.2 that caused a migration crash on MySQL
8.0.13+ when adding nullable ``BinaryField``, ``JSONField``, or ``TextField``
with a default value (:ticket:`32832`).

* Fixed a bug in Django 3.2 where a system check would crash on a model with an
invalid ``app_label`` (:ticket:`32863`).


==========================

3.2.4

Not secure
==========================

*June 2, 2021*

Django 3.2.4 fixes two security issues and several bugs in 3.2.3.

CVE-2021-33203: Potential directory traversal via ``admindocs``
===============================================================

Staff members could use the :mod:`~django.contrib.admindocs`
``TemplateDetailView`` view to check the existence of arbitrary files.
Additionally, if (and only if) the default admindocs templates have been
customized by the developers to also expose the file contents, then not only
the existence but also the file contents would have been exposed.

As a mitigation, path sanitation is now applied and only files within the
template root directories can be loaded.

CVE-2021-33571: Possible indeterminate SSRF, RFI, and LFI attacks since validators accepted leading zeros in IPv4 addresses
===========================================================================================================================

:class:`~django.core.validators.URLValidator`,
:func:`~django.core.validators.validate_ipv4_address`, and
:func:`~django.core.validators.validate_ipv46_address` didn't prohibit leading
zeros in octal literals. If you used such values you could suffer from
indeterminate SSRF, RFI, and LFI attacks.

:func:`~django.core.validators.validate_ipv4_address` and
:func:`~django.core.validators.validate_ipv46_address` validators were not
affected on Python 3.9.5+.

Bugfixes
========

* Fixed a bug in Django 3.2 where a final catch-all view in the admin didn't
respect the server-provided value of ``SCRIPT_NAME`` when redirecting
unauthenticated users to the login page (:ticket:`32754`).

* Fixed a bug in Django 3.2 where a system check would crash on an abstract
model (:ticket:`32733`).

* Prevented unnecessary initialization of unused caches following a regression
in Django 3.2 (:ticket:`32747`).

* Fixed a crash in Django 3.2 that could occur when running ``mod_wsgi`` with
the recommended settings while the Windows ``colorama`` library was installed
(:ticket:`32740`).

* Fixed a bug in Django 3.2 that would trigger the auto-reloader for template
changes when directory paths were specified with strings (:ticket:`32744`).

* Fixed a regression in Django 3.2 that caused a crash of auto-reloader with
``AttributeError``, e.g. inside a ``Conda`` environment (:ticket:`32783`).

* Fixed a regression in Django 3.2 that caused a loss of precision for
operations with ``DecimalField`` on MySQL (:ticket:`32793`).


==========================

3.2.3

Not secure
==========================

*May 13, 2021*

Django 3.2.3 fixes several bugs in 3.2.2.

Bugfixes
========

* Prepared for ``mysqlclient`` > 2.0.3 support (:ticket:`32732`).

* Fixed a regression in Django 3.2 that caused the incorrect filtering of
querysets combined with the ``|`` operator (:ticket:`32717`).

* Fixed a regression in Django 3.2.1 where saving ``FileField`` would raise a
``SuspiciousFileOperation`` even when a custom
:attr:`~django.db.models.FileField.upload_to` returns a valid file path
(:ticket:`32718`).


==========================

3.2.2

Not secure
==========================

*May 6, 2021*

Django 3.2.2 fixes a security issue and a bug in 3.2.1.

CVE-2021-32052: Header injection possibility since ``URLValidator`` accepted newlines in input on Python 3.9.5+
===============================================================================================================

On Python 3.9.5+, :class:`~django.core.validators.URLValidator` didn't prohibit
newlines and tabs. If you used values with newlines in HTTP response, you could
suffer from header injection attacks. Django itself wasn't vulnerable because
:class:`~django.http.HttpResponse` prohibits newlines in HTTP headers.

Moreover, the ``URLField`` form field which uses ``URLValidator`` silently
removes newlines and tabs on Python 3.9.5+, so the possibility of newlines
entering your data only existed if you are using this validator outside of the
form fields.

This issue was introduced by the :bpo:`43882` fix.

Bugfixes
========

* Prevented, following a regression in Django 3.2.1, :djadmin:`makemigrations`
from generating infinite migrations for a model with ``Meta.ordering``
contained ``OrderBy`` expressions (:ticket:`32714`).


==========================

3.2.1

Not secure
==========================

*May 4, 2021*

Django 3.2.1 fixes a security issue and several bugs in 3.2.

CVE-2021-31542: Potential directory-traversal via uploaded files
================================================================

``MultiPartParser``, ``UploadedFile``, and ``FieldFile`` allowed
directory-traversal via uploaded files with suitably crafted file names.

In order to mitigate this risk, stricter basename and path sanitation is now
applied.

Bugfixes
========

* Corrected detection of GDAL 3.2 on Windows (:ticket:`32544`).

* Fixed a bug in Django 3.2 where subclasses of ``BigAutoField`` and
``SmallAutoField`` were not allowed for the :setting:`DEFAULT_AUTO_FIELD`
setting (:ticket:`32620`).

* Fixed a regression in Django 3.2 that caused a crash of
``QuerySet.values()/values_list()`` after ``QuerySet.union()``,
``intersection()``, and ``difference()`` when it was ordered by an
unannotated field (:ticket:`32627`).

* Restored, following a regression in Django 3.2, displaying an exception
message on the technical 404 debug page (:ticket:`32637`).

* Fixed a bug in Django 3.2 where a system check would crash on a reverse
one-to-one relationships in ``CheckConstraint.check`` or
``UniqueConstraint.condition`` (:ticket:`32635`).

* Fixed a regression in Django 3.2 that caused a crash of
:attr:`.ModelAdmin.search_fields` when searching against phrases with
unbalanced quotes (:ticket:`32649`).

* Fixed a bug in Django 3.2 where variable lookup errors were logged rendering
the sitemap template if alternates were not defined (:ticket:`32648`).

* Fixed a regression in Django 3.2 that caused a crash when combining ``Q()``
objects which contains boolean expressions (:ticket:`32548`).

* Fixed a regression in Django 3.2 that caused a crash of ``QuerySet.update()``
on a queryset ordered by inherited or joined fields on MySQL and MariaDB
(:ticket:`32645`).

* Fixed a regression in Django 3.2 that caused a crash when decoding a cookie
value, used by ``django.contrib.messages.storage.cookie.CookieStorage``, in
the pre-Django 3.2 format (:ticket:`32643`).

* Fixed a regression in Django 3.2 that stopped the shift-key modifier
selecting multiple rows in the admin changelist (:ticket:`32647`).

* Fixed a bug in Django 3.2 where a system check would crash on the
:setting:`STATICFILES_DIRS` setting with a list of 2-tuples of
``(prefix, path)`` (:ticket:`32665`).

* Fixed a long standing bug involving queryset bitwise combination when used
with subqueries that began manifesting in Django 3.2, due to a separate fix
using ``Exists`` to ``exclude()`` multi-valued relationships
(:ticket:`32650`).

* Fixed a bug in Django 3.2 where variable lookup errors were logged when
rendering some admin templates (:ticket:`32681`).

* Fixed a bug in Django 3.2 where an admin changelist would crash when deleting
objects filtered against multi-valued relationships (:ticket:`32682`). The
admin changelist now uses ``Exists()`` instead ``QuerySet.distinct()``
because calling ``delete()`` after ``distinct()`` is not allowed in Django
3.2 to address a data loss possibility.

* Fixed a regression in Django 3.2 where the calling process environment would
not be passed to the ``dbshell`` command on PostgreSQL (:ticket:`32687`).

* Fixed a performance regression in Django 3.2 when building complex filters
with subqueries (:ticket:`32632`). As a side-effect the private API to check
``django.db.sql.query.Query`` equality is removed.


========================

3.2

Not secure
========================

*April 6, 2021*

Welcome to Django 3.2!

These release notes cover the :ref:`new features <whats-new-3.2>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-3.2>` you'll
want to be aware of when upgrading from Django 3.1 or earlier. We've
:ref:`begun the deprecation process for some features
<deprecated-features-3.2>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Django 3.2 is designated as a :term:`long-term support release
<Long-term support release>`. It will receive security updates for at least
three years after its release. Support for the previous LTS, Django 2.2, will
end in April 2022.

Python compatibility
====================

Django 3.2 supports Python 3.6, 3.7, 3.8, 3.9, and 3.10 (as of 3.2.9). We
**highly recommend** and only officially support the latest release of each
series.

.. _whats-new-3.2:

What's new in Django 3.2
========================

Automatic :class:`~django.apps.AppConfig` discovery
---------------------------------------------------

Most pluggable applications define an :class:`~django.apps.AppConfig` subclass
in an ``apps.py`` submodule. Many define a ``default_app_config`` variable
pointing to this class in their ``__init__.py``.

When the ``apps.py`` submodule exists and defines a single
:class:`~django.apps.AppConfig` subclass, Django now uses that configuration
automatically, so you can remove ``default_app_config``.

``default_app_config`` made it possible to declare only the application's path
in :setting:`INSTALLED_APPS` (e.g. ``'django.contrib.admin'``) rather than the
app config's path (e.g. ``'django.contrib.admin.apps.AdminConfig'``). It was
introduced for backwards-compatibility with the former style, with the intent
to switch the ecosystem to the latter, but the switch didn't happen.

With automatic ``AppConfig`` discovery, ``default_app_config`` is no longer
needed. As a consequence, it's deprecated.

See :ref:`configuring-applications-ref` for full details.

Customizing type of auto-created primary keys
---------------------------------------------

When defining a model, if no field in a model is defined with
:attr:`primary_key=True <django.db.models.Field.primary_key>` an implicit
primary key is added. The type of this implicit primary key can now be
controlled via the :setting:`DEFAULT_AUTO_FIELD` setting and
:attr:`AppConfig.default_auto_field <django.apps.AppConfig.default_auto_field>`
attribute. No more needing to override primary keys in all models.

Maintaining the historical behavior, the default value for
:setting:`DEFAULT_AUTO_FIELD` is :class:`~django.db.models.AutoField`. Starting
with 3.2 new projects are generated with :setting:`DEFAULT_AUTO_FIELD` set to
:class:`~django.db.models.BigAutoField`. Also, new apps are generated with
:attr:`AppConfig.default_auto_field <django.apps.AppConfig.default_auto_field>`
set to :class:`~django.db.models.BigAutoField`. In a future Django release the
default value of :setting:`DEFAULT_AUTO_FIELD` will be changed to
:class:`~django.db.models.BigAutoField`.

To avoid unwanted migrations in the future, either explicitly set
:setting:`DEFAULT_AUTO_FIELD` to :class:`~django.db.models.AutoField`::

DEFAULT_AUTO_FIELD = 'django.db.models.AutoField'

or configure it on a per-app basis::

from django.apps import AppConfig

class MyAppConfig(AppConfig):
default_auto_field = 'django.db.models.AutoField'
name = 'my_app'

or on a per-model basis::

from django.db import models

class MyModel(models.Model):
id = models.AutoField(primary_key=True)

In anticipation of the changing default, a system check will provide a warning
if you do not have an explicit setting for :setting:`DEFAULT_AUTO_FIELD`.

When changing the value of :setting:`DEFAULT_AUTO_FIELD`, migrations for the
primary key of existing auto-created through tables cannot be generated
currently. See the :setting:`DEFAULT_AUTO_FIELD` docs for details on migrating
such tables.

.. _new_functional_indexes:

Functional indexes
------------------

The new :attr:`*expressions <django.db.models.Index.expressions>` positional
argument of :class:`Index() <django.db.models.Index>` enables creating
functional indexes on expressions and database functions. For example::

from django.db import models
from django.db.models import F, Index, Value
from django.db.models.functions import Lower, Upper


class MyModel(models.Model):
first_name = models.CharField(max_length=255)
last_name = models.CharField(max_length=255)
height = models.IntegerField()
weight = models.IntegerField()

class Meta:
indexes = [
Index(
Lower('first_name'),
Upper('last_name').desc(),
name='first_last_name_idx',
),
Index(
F('height') / (F('weight') + Value(5)),
name='calc_idx',
),
]

Functional indexes are added to models using the
:attr:`Meta.indexes <django.db.models.Options.indexes>` option.

``pymemcache`` support
----------------------

The new ``django.core.cache.backends.memcached.PyMemcacheCache`` cache backend
allows using the pymemcache_ library for memcached. ``pymemcache`` 3.4.0 or
higher is required. For more details, see the :doc:`documentation on caching in
Django </topics/cache>`.

.. _pymemcache: https://pypi.org/project/pymemcache/

New decorators for the admin site
---------------------------------

The new :func:`~django.contrib.admin.display` decorator allows for easily
adding options to custom display functions that can be used with
:attr:`~django.contrib.admin.ModelAdmin.list_display` or
:attr:`~django.contrib.admin.ModelAdmin.readonly_fields`.

Likewise, the new :func:`~django.contrib.admin.action` decorator allows for
easily adding options to action functions that can be used with
:attr:`~django.contrib.admin.ModelAdmin.actions`.

Using the ``display`` decorator has the advantage that it is now
possible to use the ``property`` decorator when needing to specify attributes
on the custom method. Prior to this it was necessary to use the ``property()``
function instead after assigning the required attributes to the method.

Using decorators has the advantage that these options are more discoverable as
they can be suggested by completion utilities in code editors. They are merely
a convenience and still set the same attributes on the functions under the
hood.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* :attr:`.ModelAdmin.search_fields` now allows searching against quoted phrases
with spaces.

* Read-only related fields are now rendered as navigable links if target models
are registered in the admin.

* The admin now supports theming, and includes a dark theme that is enabled
according to browser settings. See :ref:`admin-theming` for more details.

* :attr:`.ModelAdmin.autocomplete_fields` now respects
:attr:`ForeignKey.to_field <django.db.models.ForeignKey.to_field>` and
:attr:`ForeignKey.limit_choices_to
<django.db.models.ForeignKey.limit_choices_to>` when searching a related
model.

* The admin now installs a final catch-all view that redirects unauthenticated
users to the login page, regardless of whether the URL is otherwise valid.
This protects against a potential model enumeration privacy issue.

Although not recommended, you may set the new
:attr:`.AdminSite.final_catch_all_view` to ``False`` to disable the
catch-all view.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* The default iteration count for the PBKDF2 password hasher is increased from
216,000 to 260,000.

* The default variant for the Argon2 password hasher is changed to Argon2id.
``memory_cost`` and ``parallelism`` are increased to 102,400 and 8
respectively to match the ``argon2-cffi`` defaults.

Increasing the ``memory_cost`` pushes the required memory from 512 KB to 100
MB. This is still rather conservative but can lead to problems in memory
constrained environments. If this is the case, the existing hasher can be
subclassed to override the defaults.

* The default salt entropy for the Argon2, MD5, PBKDF2, SHA-1 password hashers
is increased from 71 to 128 bits.

:mod:`django.contrib.contenttypes`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new ``absolute_max`` argument for
:func:`~django.contrib.contenttypes.forms.generic_inlineformset_factory`
allows customizing the maximum number of forms that can be instantiated when
supplying ``POST`` data. See :ref:`formsets-absolute-max` for more details.

* The new ``can_delete_extra`` argument for
:func:`~django.contrib.contenttypes.forms.generic_inlineformset_factory`
allows removal of the option to delete extra forms. See
:attr:`~.BaseFormSet.can_delete_extra` for more information.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* The :meth:`.GDALRaster.transform` method now supports
:class:`~django.contrib.gis.gdal.SpatialReference`.

* The :class:`~django.contrib.gis.gdal.DataSource` class now supports
:class:`pathlib.Path`.

* The :class:`~django.contrib.gis.utils.LayerMapping` class now supports
:class:`pathlib.Path`.

:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :attr:`.ExclusionConstraint.include` attribute allows creating
covering exclusion constraints on PostgreSQL 12+.

* The new :attr:`.ExclusionConstraint.opclasses` attribute allows setting
PostgreSQL operator classes.

* The new :attr:`.JSONBAgg.ordering` attribute determines the ordering of the
aggregated elements.

* The new :attr:`.JSONBAgg.distinct` attribute determines if aggregated values
will be distinct.

* The :class:`~django.contrib.postgres.operations.CreateExtension` operation
now checks that the extension already exists in the database and skips the
migration if so.

* The new :class:`~django.contrib.postgres.operations.CreateCollation` and
:class:`~django.contrib.postgres.operations.RemoveCollation` operations
allow creating and dropping collations on PostgreSQL. See
:ref:`manage-postgresql-collations` for more details.

* Lookups for :class:`~django.contrib.postgres.fields.ArrayField` now allow
(non-nested) arrays containing expressions as right-hand sides.

* The new :class:`OpClass() <django.contrib.postgres.indexes.OpClass>`
expression allows creating functional indexes on expressions with a custom
operator class. See :ref:`new_functional_indexes` for more details.

:mod:`django.contrib.sitemaps`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :class:`~django.contrib.sitemaps.Sitemap` attributes
:attr:`~django.contrib.sitemaps.Sitemap.alternates`,
:attr:`~django.contrib.sitemaps.Sitemap.languages` and
:attr:`~django.contrib.sitemaps.Sitemap.x_default` allow
generating sitemap *alternates* to localized versions of your pages.

:mod:`django.contrib.syndication`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new ``item_comments`` hook allows specifying a comments URL per feed
item.

Database backends
~~~~~~~~~~~~~~~~~

* Third-party database backends can now skip or mark as expected failures
tests in Django's test suite using the new
``DatabaseFeatures.django_test_skips`` and
``django_test_expected_failures`` attributes.

Decorators
~~~~~~~~~~

* The new :func:`~django.views.decorators.common.no_append_slash` decorator
allows individual views to be excluded from :setting:`APPEND_SLASH` URL
normalization.

Error Reporting
~~~~~~~~~~~~~~~

* Custom :class:`~django.views.debug.ExceptionReporter` subclasses can now
define the :attr:`~django.views.debug.ExceptionReporter.html_template_path`
and :attr:`~django.views.debug.ExceptionReporter.text_template_path`
properties to override the templates used to render exception reports.

File Uploads
~~~~~~~~~~~~

* The new :meth:`FileUploadHandler.upload_interrupted()
<django.core.files.uploadhandler.FileUploadHandler.upload_interrupted>`
callback allows handling interrupted uploads.

Forms
~~~~~

* The new ``absolute_max`` argument for :func:`.formset_factory`,
:func:`.inlineformset_factory`, and :func:`.modelformset_factory` allows
customizing the maximum number of forms that can be instantiated when
supplying ``POST`` data. See :ref:`formsets-absolute-max` for more details.

* The new ``can_delete_extra`` argument for :func:`.formset_factory`,
:func:`.inlineformset_factory`, and :func:`.modelformset_factory` allows
removal of the option to delete extra forms. See
:attr:`~.BaseFormSet.can_delete_extra` for more information.

* :class:`~django.forms.formsets.BaseFormSet` now reports a user facing error,
rather than raising an exception, when the management form is missing or has
been tampered with. To customize this error message, pass the
``error_messages`` argument with the key ``'missing_management_form'`` when
instantiating the formset.

Generic Views
~~~~~~~~~~~~~

* The ``week_format`` attributes of
:class:`~django.views.generic.dates.WeekMixin` and
:class:`~django.views.generic.dates.WeekArchiveView` now support the
``'%V'`` ISO 8601 week format.

Management Commands
~~~~~~~~~~~~~~~~~~~

* :djadmin:`loaddata` now supports fixtures stored in XZ archives (``.xz``) and
LZMA archives (``.lzma``).

* :djadmin:`dumpdata` now can compress data in the ``bz2``, ``gz``, ``lzma``,
or ``xz`` formats.

* :djadmin:`makemigrations` can now be called without an active database
connection. In that case, check for a consistent migration history is
skipped.

* :attr:`.BaseCommand.requires_system_checks` now supports specifying a list of
tags. System checks registered in the chosen tags will be checked for errors
prior to executing the command. In previous versions, either all or none
of the system checks were performed.

* Support for colored terminal output on Windows is updated. Various modern
terminal environments are automatically detected, and the options for
enabling support in other cases are improved. See :ref:`syntax-coloring` for
more details.

Migrations
~~~~~~~~~~

* The new ``Operation.migration_name_fragment`` property allows providing a
filename fragment that will be used to name a migration containing only that
operation.

* Migrations now support serialization of pure and concrete path objects from
:mod:`pathlib`, and :class:`os.PathLike` instances.

Models
~~~~~~

* The new ``no_key`` parameter for :meth:`.QuerySet.select_for_update()`,
supported on PostgreSQL, allows acquiring weaker locks that don't block the
creation of rows that reference locked rows through a foreign key.

* :class:`When() <django.db.models.expressions.When>` expression now allows
using the ``condition`` argument with ``lookups``.

* The new :attr:`.Index.include` and :attr:`.UniqueConstraint.include`
attributes allow creating covering indexes and covering unique constraints on
PostgreSQL 11+.

* The new :attr:`.UniqueConstraint.opclasses` attribute allows setting
PostgreSQL operator classes.

* The :meth:`.QuerySet.update` method now respects the ``order_by()`` clause on
MySQL and MariaDB.

* :class:`FilteredRelation() <django.db.models.FilteredRelation>` now supports
nested relations.

* The ``of`` argument of :meth:`.QuerySet.select_for_update()` is now allowed
on MySQL 8.0.1+.

* :class:`Value() <django.db.models.Value>` expression now
automatically resolves its ``output_field`` to the appropriate
:class:`Field <django.db.models.Field>` subclass based on the type of
its provided ``value`` for :py:class:`bool`, :py:class:`bytes`,
:py:class:`float`, :py:class:`int`, :py:class:`str`,
:py:class:`datetime.date`, :py:class:`datetime.datetime`,
:py:class:`datetime.time`, :py:class:`datetime.timedelta`,
:py:class:`decimal.Decimal`, and :py:class:`uuid.UUID` instances. As a
consequence, resolving an ``output_field`` for database functions and
combined expressions may now crash with mixed types when using ``Value()``.
You will need to explicitly set the ``output_field`` in such cases.

* The new :meth:`.QuerySet.alias` method allows creating reusable aliases for
expressions that don't need to be selected but are used for filtering,
ordering, or as a part of complex expressions.

* The new :class:`~django.db.models.functions.Collate` function allows
filtering and ordering by specified database collations.

* The ``field_name`` argument of :meth:`.QuerySet.in_bulk()` now accepts
distinct fields if there's only one field specified in
:meth:`.QuerySet.distinct`.

* The new ``tzinfo`` parameter of the
:class:`~django.db.models.functions.TruncDate` and
:class:`~django.db.models.functions.TruncTime` database functions allows
truncating datetimes in a specific timezone.

* The new ``db_collation`` argument for
:attr:`CharField <django.db.models.CharField.db_collation>` and
:attr:`TextField <django.db.models.TextField.db_collation>` allows setting a
database collation for the field.

* Added the :class:`~django.db.models.functions.Random` database function.

* :ref:`aggregation-functions`, :class:`F() <django.db.models.F>`,
:class:`OuterRef() <django.db.models.OuterRef>`, and other expressions now
allow using transforms. See :ref:`using-transforms-in-expressions` for
details.

* The new ``durable`` argument for :func:`~django.db.transaction.atomic`
guarantees that changes made in the atomic block will be committed if the
block exits without errors. A nested atomic block marked as durable will
raise a ``RuntimeError``.

* Added the :class:`~django.db.models.functions.JSONObject` database function.

Pagination
~~~~~~~~~~

* The new :meth:`django.core.paginator.Paginator.get_elided_page_range` method
allows generating a page range with some of the values elided. If there are a
large number of pages, this can be helpful for generating a reasonable number
of page links in a template.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* Response headers are now stored in :attr:`.HttpResponse.headers`. This can be
used instead of the original dict-like interface of ``HttpResponse`` objects.
Both interfaces will continue to be supported. See
:ref:`setting-header-fields` for details.

* The new ``headers`` parameter of :class:`~django.http.HttpResponse`,
:class:`~django.template.response.SimpleTemplateResponse`, and
:class:`~django.template.response.TemplateResponse` allows setting response
:attr:`~django.http.HttpResponse.headers` on instantiation.

Security
~~~~~~~~

* The :setting:`SECRET_KEY` setting is now checked for a valid value upon first
access, rather than when settings are first loaded. This enables running
management commands that do not rely on the ``SECRET_KEY`` without needing to
provide a value. As a consequence of this, calling
:func:`~django.conf.settings.configure` without providing a valid
``SECRET_KEY``, and then going on to access ``settings.SECRET_KEY`` will now
raise an :exc:`~django.core.exceptions.ImproperlyConfigured` exception.

* The new ``Signer.sign_object()`` and ``Signer.unsign_object()`` methods allow
signing complex data structures. See :ref:`signing-complex-data` for more
details.

Also, :func:`signing.dumps() <django.core.signing.dumps>` and
:func:`~django.core.signing.loads` become shortcuts for
:meth:`.TimestampSigner.sign_object` and
:meth:`~.TimestampSigner.unsign_object`.

Serialization
~~~~~~~~~~~~~

* The new :ref:`JSONL <serialization-formats-jsonl>` serializer allows using
the JSON Lines format with :djadmin:`dumpdata` and :djadmin:`loaddata`. This
can be useful for populating large databases because data is loaded line by
line into memory, rather than being loaded all at once.

Signals
~~~~~~~

* :meth:`Signal.send_robust() <django.dispatch.Signal.send_robust>` now logs
exceptions.

Templates
~~~~~~~~~

* :tfilter:`floatformat` template filter now allows using the ``g`` suffix to
force grouping by the :setting:`THOUSAND_SEPARATOR` for the active locale.

* Templates cached with :ref:`Cached template loaders<template-loaders>` are
now correctly reloaded in development.

Tests
~~~~~

* Objects assigned to class attributes in :meth:`.TestCase.setUpTestData` are
now isolated for each test method. Such objects are now required to support
creating deep copies with :py:func:`copy.deepcopy`. Assigning objects which
don't support ``deepcopy()`` is deprecated and will be removed in Django 4.1.

* :class:`~django.test.runner.DiscoverRunner` now enables
:py:mod:`faulthandler` by default. This can be disabled by using the
:option:`test --no-faulthandler` option.

* :class:`~django.test.runner.DiscoverRunner` and the
:djadmin:`test` management command can now track timings, including database
setup and total run time. This can be enabled by using the :option:`test
--timing` option.

* :class:`~django.test.Client` now preserves the request query string when
following 307 and 308 redirects.

* The new :meth:`.TestCase.captureOnCommitCallbacks` method captures callback
functions passed to :func:`transaction.on_commit()
<django.db.transaction.on_commit>` in a list. This allows you to test such
callbacks without using the slower :class:`.TransactionTestCase`.

* :meth:`.TransactionTestCase.assertQuerysetEqual` now supports direct
comparison against another queryset rather than being restricted to
comparison against a list of string representations of objects when using the
default value for the ``transform`` argument.

Utilities
~~~~~~~~~

* The new ``depth`` parameter of ``django.utils.timesince.timesince()`` and
``django.utils.timesince.timeuntil()`` functions allows specifying the number
of adjacent time units to return.

Validators
~~~~~~~~~~

* Built-in validators now include the provided value in the ``params`` argument
of a raised :exc:`~django.core.exceptions.ValidationError`. This allows
custom error messages to use the ``%(value)s`` placeholder.

* The :class:`.ValidationError` equality operator now ignores ``messages`` and
``params`` ordering.

.. _backwards-incompatible-3.2:

Backwards incompatible changes in 3.2
=====================================

Database backend API
--------------------

This section describes changes that may be needed in third-party database
backends.

* The new ``DatabaseFeatures.introspected_field_types`` property replaces these
features:

* ``can_introspect_autofield``
* ``can_introspect_big_integer_field``
* ``can_introspect_binary_field``
* ``can_introspect_decimal_field``
* ``can_introspect_duration_field``
* ``can_introspect_ip_address_field``
* ``can_introspect_positive_integer_field``
* ``can_introspect_small_integer_field``
* ``can_introspect_time_field``
* ``introspected_big_auto_field_type``
* ``introspected_small_auto_field_type``
* ``introspected_boolean_field_type``

* To enable support for covering indexes (:attr:`.Index.include`) and covering
unique constraints (:attr:`.UniqueConstraint.include`), set
``DatabaseFeatures.supports_covering_indexes`` to ``True``.

* Third-party database backends must implement support for column database
collations on ``CharField``\s and ``TextField``\s or set
``DatabaseFeatures.supports_collation_on_charfield`` and
``DatabaseFeatures.supports_collation_on_textfield`` to ``False``. If
non-deterministic collations are not supported, set
``supports_non_deterministic_collations`` to ``False``.

* ``DatabaseOperations.random_function_sql()`` is removed in favor of the new
:class:`~django.db.models.functions.Random` database function.

* ``DatabaseOperations.date_trunc_sql()`` and
``DatabaseOperations.time_trunc_sql()`` now take the optional ``tzname``
argument in order to truncate in a specific timezone.

* ``DatabaseClient.runshell()`` now gets arguments and an optional dictionary
with environment variables to the underlying command-line client from
``DatabaseClient.settings_to_cmd_args_env()`` method. Third-party database
backends must implement ``DatabaseClient.settings_to_cmd_args_env()`` or
override ``DatabaseClient.runshell()``.

* Third-party database backends must implement support for functional indexes
(:attr:`.Index.expressions`) or set
``DatabaseFeatures.supports_expression_indexes`` to ``False``. If ``COLLATE``
is not a part of the ``CREATE INDEX`` statement, set
``DatabaseFeatures.collate_as_index_expression`` to ``True``.

:mod:`django.contrib.admin`
---------------------------

* Pagination links in the admin are now 1-indexed instead of 0-indexed, i.e.
the query string for the first page is ``?p=1`` instead of ``?p=0``.

* The new admin catch-all view will break URL patterns routed after the admin
URLs and matching the admin URL prefix. You can either adjust your URL
ordering or, if necessary, set :attr:`AdminSite.final_catch_all_view
<django.contrib.admin.AdminSite.final_catch_all_view>` to ``False``,
disabling the catch-all view. See :ref:`whats-new-3.2` for more details.

* Minified JavaScript files are no longer included with the admin. If you
require these files to be minified, consider using a third party app or
external build tool. The minified vendored JavaScript files packaged with the
admin (e.g. :ref:`jquery.min.js <contrib-admin-jquery>`) are still included.

* :attr:`.ModelAdmin.prepopulated_fields` no longer strips English stop words,
such as ``'a'`` or ``'an'``.

:mod:`django.contrib.gis`
-------------------------

* Support for PostGIS 2.2 is removed.

* The Oracle backend now clones polygons (and geometry collections containing
polygons) before reorienting them and saving them to the database. They are
no longer mutated in place. You might notice this if you use the polygons
after a model is saved.

Dropped support for PostgreSQL 9.5
----------------------------------

Upstream support for PostgreSQL 9.5 ends in February 2021. Django 3.2 supports
PostgreSQL 9.6 and higher.

Dropped support for MySQL 5.6
-----------------------------

The end of upstream support for MySQL 5.6 is April 2021. Django 3.2 supports
MySQL 5.7 and higher.

Miscellaneous
-------------

* Django now supports non-``pytz`` time zones, such as Python 3.9+'s
:mod:`zoneinfo` module and its backport.

* The undocumented ``SpatiaLiteOperations.proj4_version()`` method is renamed
to ``proj_version()``.

* :func:`~django.utils.text.slugify` now removes leading and trailing dashes
and underscores.

* The :tfilter:`intcomma` and :tfilter:`intword` template filters no longer
depend on the ``USE_L10N`` setting.

* Support for ``argon2-cffi`` < 19.1.0 is removed.

* The cache keys no longer includes the language when internationalization is
disabled (``USE_I18N = False``) and localization is enabled
(``USE_L10N = True``). After upgrading to Django 3.2 in such configurations,
the first request to any previously cached value will be a cache miss.

* ``ForeignKey.validate()`` now uses
:attr:`~django.db.models.Model._base_manager` rather than
:attr:`~django.db.models.Model._default_manager` to check that related
instances exist.

* When an application defines an :class:`~django.apps.AppConfig` subclass in
an ``apps.py`` submodule, Django now uses this configuration automatically,
even if it isn't enabled with ``default_app_config``. Set ``default = False``
in the :class:`~django.apps.AppConfig` subclass if you need to prevent this
behavior. See :ref:`whats-new-3.2` for more details.

* Instantiating an abstract model now raises ``TypeError``.

* Keyword arguments to :func:`~django.test.utils.setup_databases` are now
keyword-only.

* The undocumented ``django.utils.http.limited_parse_qsl()`` function is
removed. Please use :func:`urllib.parse.parse_qsl` instead.

* ``django.test.utils.TestContextDecorator`` now uses
:py:meth:`~unittest.TestCase.addCleanup` so that cleanups registered in the
:py:meth:`~unittest.TestCase.setUp` method are called before
``TestContextDecorator.disable()``.

* ``SessionMiddleware`` now raises a
:exc:`~django.contrib.sessions.exceptions.SessionInterrupted` exception
instead of :exc:`~django.core.exceptions.SuspiciousOperation` when a session
is destroyed in a concurrent request.

* The :class:`django.db.models.Field` equality operator now correctly
distinguishes inherited field instances across models. Additionally, the
ordering of such fields is now defined.

* The undocumented ``django.core.files.locks.lock()`` function now returns
``False`` if the file cannot be locked, instead of raising
:exc:`BlockingIOError`.

* The password reset mechanism now invalidates tokens when the user email is
changed.

* :djadmin:`makemessages` command no longer processes invalid locales specified
using :option:`makemessages --locale` option, when they contain hyphens
(``'-'``).

* The ``django.contrib.auth.forms.ReadOnlyPasswordHashField`` form field is now
:attr:`~django.forms.Field.disabled` by default. Therefore
``UserChangeForm.clean_password()`` is no longer required to return the
initial value.

* The ``cache.get_many()``, ``get_or_set()``, ``has_key()``, ``incr()``,
``decr()``, ``incr_version()``, and ``decr_version()`` cache operations now
correctly handle ``None`` stored in the cache, in the same way as any other
value, instead of behaving as though the key didn't exist.

Due to a ``python-memcached`` limitation, the previous behavior is kept for
the deprecated ``MemcachedCache`` backend.

* The minimum supported version of SQLite is increased from 3.8.3 to 3.9.0.

* :class:`~django.contrib.messages.storage.cookie.CookieStorage` now stores
messages in the :rfc:`6265` compliant format. Support for cookies that use
the old format remains until Django 4.1.

* The minimum supported version of ``asgiref`` is increased from 3.2.10 to
3.3.2.

.. _deprecated-features-3.2:

Features deprecated in 3.2
==========================

Miscellaneous
-------------

* Assigning objects which don't support creating deep copies with
:py:func:`copy.deepcopy` to class attributes in
:meth:`.TestCase.setUpTestData` is deprecated.

* Using a boolean value in :attr:`.BaseCommand.requires_system_checks` is
deprecated. Use ``'__all__'`` instead of ``True``, and ``[]`` (an empty list)
instead of ``False``.

* The ``whitelist`` argument and ``domain_whitelist`` attribute of
:class:`~django.core.validators.EmailValidator` are deprecated. Use
``allowlist`` instead of ``whitelist``, and ``domain_allowlist`` instead of
``domain_whitelist``. You may need to rename ``whitelist`` in existing
migrations.

* The ``default_app_config`` application configuration variable is deprecated,
due to the now automatic ``AppConfig`` discovery. See :ref:`whats-new-3.2`
for more details.

* Automatically calling ``repr()`` on a queryset in
``TransactionTestCase.assertQuerysetEqual()``, when compared to string
values, is deprecated. If you need the previous behavior, explicitly set
``transform`` to ``repr``.

* The ``django.core.cache.backends.memcached.MemcachedCache`` backend is
deprecated as ``python-memcached`` has some problems and seems to be
unmaintained. Use ``django.core.cache.backends.memcached.PyMemcacheCache``
or ``django.core.cache.backends.memcached.PyLibMCCache`` instead.

* The format of messages used by
``django.contrib.messages.storage.cookie.CookieStorage`` is different from
the format generated by older versions of Django. Support for the old format
remains until Django 4.1.


===========================

3.1.14

Not secure
===========================

*December 7, 2021*

Django 3.1.14 fixes a security issue with severity "low" in 3.1.13.

CVE-2021-44420: Potential bypass of an upstream access control based on URL paths
=================================================================================

HTTP requests for URLs with trailing newlines could bypass an upstream access
control based on URL paths.


===========================

3.1.13

Not secure
===========================

*July 1, 2021*

Django 3.1.13 fixes a security issue with severity "high" in 3.1.12.

CVE-2021-35042: Potential SQL injection via unsanitized ``QuerySet.order_by()`` input
=====================================================================================

Unsanitized user input passed to ``QuerySet.order_by()`` could bypass intended
column reference validation in path marked for deprecation resulting in a
potential SQL injection even if a deprecation warning is emitted.

As a mitigation the strict column reference validation was restored for the
duration of the deprecation period. This regression appeared in 3.1 as a side
effect of fixing :ticket:`31426`.

The issue is not present in the main branch as the deprecated path has been
removed.


===========================

3.1.12

Not secure
===========================

*June 2, 2021*

Django 3.1.12 fixes two security issues in 3.1.11.

CVE-2021-33203: Potential directory traversal via ``admindocs``
===============================================================

Staff members could use the :mod:`~django.contrib.admindocs`
``TemplateDetailView`` view to check the existence of arbitrary files.
Additionally, if (and only if) the default admindocs templates have been
customized by the developers to also expose the file contents, then not only
the existence but also the file contents would have been exposed.

As a mitigation, path sanitation is now applied and only files within the
template root directories can be loaded.

CVE-2021-33571: Possible indeterminate SSRF, RFI, and LFI attacks since validators accepted leading zeros in IPv4 addresses
===========================================================================================================================

:class:`~django.core.validators.URLValidator`,
:func:`~django.core.validators.validate_ipv4_address`, and
:func:`~django.core.validators.validate_ipv46_address` didn't prohibit leading
zeros in octal literals. If you used such values you could suffer from
indeterminate SSRF, RFI, and LFI attacks.

:func:`~django.core.validators.validate_ipv4_address` and
:func:`~django.core.validators.validate_ipv46_address` validators were not
affected on Python 3.9.5+.


===========================

3.1.11

Not secure
===========================

*May 13, 2021*

Django 3.1.11 fixes a regression in 3.1.9.

Bugfixes
========

* Fixed a regression in Django 3.1.9 where saving ``FileField`` would raise a
``SuspiciousFileOperation`` even when a custom
:attr:`~django.db.models.FileField.upload_to` returns a valid file path
(:ticket:`32718`).


===========================

3.1.10

Not secure
===========================

*May 6, 2021*

Django 3.1.10 fixes a security issue in 3.1.9.

CVE-2021-32052: Header injection possibility since ``URLValidator`` accepted newlines in input on Python 3.9.5+
===============================================================================================================

On Python 3.9.5+, :class:`~django.core.validators.URLValidator` didn't prohibit
newlines and tabs. If you used values with newlines in HTTP response, you could
suffer from header injection attacks. Django itself wasn't vulnerable because
:class:`~django.http.HttpResponse` prohibits newlines in HTTP headers.

Moreover, the ``URLField`` form field which uses ``URLValidator`` silently
removes newlines and tabs on Python 3.9.5+, so the possibility of newlines
entering your data only existed if you are using this validator outside of the
form fields.

This issue was introduced by the :bpo:`43882` fix.


==========================

3.1.9

Not secure
==========================

*May 4, 2021*

Django 3.1.9 fixes a security issue in 3.1.8.

CVE-2021-31542: Potential directory-traversal via uploaded files
================================================================

``MultiPartParser``, ``UploadedFile``, and ``FieldFile`` allowed
directory-traversal via uploaded files with suitably crafted file names.

In order to mitigate this risk, stricter basename and path sanitation is now
applied.


==========================

3.1.8

Not secure
==========================

*April 6, 2021*

Django 3.1.8 fixes a security issue with severity "low" and a bug in 3.1.7.

CVE-2021-28658: Potential directory-traversal via uploaded files
================================================================

``MultiPartParser`` allowed directory-traversal via uploaded files with
suitably crafted file names.

Built-in upload handlers were not affected by this vulnerability.

Bugfixes
========

* Fixed a bug in Django 3.1 where the output was hidden on a test error or
failure when using :option:`test --pdb` with the
:option:`--buffer <test --buffer>` option (:ticket:`32560`).


==========================

3.1.7

Not secure
==========================

*February 19, 2021*

Django 3.1.7 fixes a security issue and a bug in 3.1.6.

CVE-2021-23336: Web cache poisoning via ``django.utils.http.limited_parse_qsl()``
=================================================================================

Django contains a copy of :func:`urllib.parse.parse_qsl` which was added to
backport some security fixes. A further security fix has been issued recently
such that ``parse_qsl()`` no longer allows using ``;`` as a query parameter
separator by default. Django now includes this fix. See :bpo:`42967` for
further details.

Bugfixes
========

* Fixed a regression in Django 3.1 that caused ``RuntimeError`` instead of
connection errors when using only the ``'postgres'`` database
(:ticket:`32403`).


==========================

3.1.6

Not secure
==========================

*February 1, 2021*

Django 3.1.6 fixes a security issue with severity "low" and a bug in 3.1.5.

CVE-2021-3281: Potential directory-traversal via ``archive.extract()``
======================================================================

The ``django.utils.archive.extract()`` function, used by
:option:`startapp --template` and :option:`startproject --template`, allowed
directory-traversal via an archive with absolute paths or relative paths with
dot segments.

Bugfixes
========

* Fixed an admin layout issue in Django 3.1 where changelist filter controls
would become squashed (:ticket:`32391`).


==========================

3.1.5

Not secure
==========================

*January 4, 2021*

Django 3.1.5 fixes several bugs in 3.1.4.

Bugfixes
========

* Fixed ``__isnull=True`` lookup on key transforms for
:class:`~django.db.models.JSONField` with Oracle and SQLite
(:ticket:`32252`).

* Fixed a bug in Django 3.1 that caused a crash when processing middlewares in
an async context with a middleware that raises a ``MiddlewareNotUsed``
exception (:ticket:`32299`).

* Fixed a regression in Django 3.1 that caused the incorrect prefixing of
``STATIC_URL`` and ``MEDIA_URL`` settings, by the server-provided value of
``SCRIPT_NAME`` (or ``/`` if not set), when set to a URL specifying the
protocol but without a top-level domain, e.g. ``http://myhost/``
(:ticket:`32304`).


==========================

3.1.4

Not secure
==========================

*December 1, 2020*

Django 3.1.4 fixes several bugs in 3.1.3.

Bugfixes
========

* Fixed setting the ``Content-Length`` HTTP header in ``AsyncRequestFactory``
(:ticket:`32162`).

* Fixed passing extra HTTP headers to ``AsyncRequestFactory`` request methods
(:ticket:`32159`).

* Fixed crash of key transforms for :class:`~django.db.models.JSONField` on
PostgreSQL when using on a ``Subquery()`` annotation (:ticket:`32182`).

* Fixed a regression in Django 3.1 that caused a crash of auto-reloader for
certain invocations of ``runserver`` on Windows with Python 3.7 and below
(:ticket:`32202`).

* Fixed a regression in Django 3.1 that caused the incorrect grouping by a
``Q`` object annotation (:ticket:`32200`).

* Fixed a regression in Django 3.1 that caused suppressing connection errors
when :class:`~django.db.models.JSONField` is used on SQLite
(:ticket:`32224`).

* Fixed a crash on SQLite, when ``QuerySet.values()/values_list()`` contained
key transforms for :class:`~django.db.models.JSONField` returning non-string
primitive values (:ticket:`32203`).


==========================

3.1.3

Not secure
==========================

*November 2, 2020*

Django 3.1.3 fixes several bugs in 3.1.2 and adds compatibility with Python
3.9.

Bugfixes
========

* Fixed a regression in Django 3.1.2 that caused the incorrect height of the
admin changelist search bar (:ticket:`32072`).

* Fixed a regression in Django 3.1.2 that caused the incorrect width of the
admin changelist search bar on a filtered page (:ticket:`32091`).

* Fixed displaying Unicode characters in
:class:`forms.JSONField <django.forms.JSONField>` and read-only
:class:`models.JSONField <django.db.models.JSONField>` values in the admin
(:ticket:`32080`).

* Fixed a regression in Django 3.1 that caused a crash of
:class:`~django.contrib.postgres.aggregates.ArrayAgg` and
:class:`~django.contrib.postgres.aggregates.StringAgg` with ``ordering``
on key transforms for :class:`~django.db.models.JSONField` (:ticket:`32096`).

* Fixed a regression in Django 3.1 that caused a crash of ``__in`` lookup when
using key transforms for :class:`~django.db.models.JSONField` in the lookup
value (:ticket:`32096`).

* Fixed a regression in Django 3.1 that caused a crash of
:class:`~django.db.models.ExpressionWrapper` with key transforms for
:class:`~django.db.models.JSONField` (:ticket:`32096`).

* Fixed a regression in Django 3.1 that caused a migrations crash on PostgreSQL
when adding an
:class:`~django.contrib.postgres.constraints.ExclusionConstraint` with key
transforms for :class:`~django.db.models.JSONField` in ``expressions``
(:ticket:`32096`).

* Fixed a regression in Django 3.1 where
:exc:`ProtectedError.protected_objects <django.db.models.ProtectedError>` and
:exc:`RestrictedError.restricted_objects <django.db.models.RestrictedError>`
attributes returned iterators instead of :py:class:`set` of objects
(:ticket:`32107`).

* Fixed a regression in Django 3.1.2 that caused incorrect form input layout on
small screens in the admin change form view (:ticket:`32069`).

* Fixed a regression in Django 3.1 that invalidated pre-Django 3.1 password
reset tokens (:ticket:`32130`).

* Added support for ``asgiref`` 3.3 (:ticket:`32128`).

* Fixed a regression in Django 3.1 that caused incorrect textarea layout on
medium-sized screens in the admin change form view with the sidebar open
(:ticket:`32127`).

* Fixed a regression in Django 3.0.7 that didn't use ``Subquery()`` aliases in
the ``GROUP BY`` clause (:ticket:`32152`).


==========================

3.1.2

Not secure
==========================

*October 1, 2020*

Django 3.1.2 fixes several bugs in 3.1.1.

Bugfixes
========

* Fixed a bug in Django 3.1 where ``FileField`` instances with a callable
storage were not correctly deconstructed (:ticket:`31941`).

* Fixed a regression in Django 3.1 where the :attr:`.QuerySet.ordered`
attribute returned incorrectly ``True`` for ``GROUP BY`` queries (e.g.
``.annotate().values()``) on models with ``Meta.ordering``. A model's
``Meta.ordering`` doesn't affect such queries (:ticket:`31990`).

* Fixed a regression in Django 3.1 where a queryset would crash if it contained
an aggregation and a ``Q`` object annotation (:ticket:`32007`).

* Fixed a bug in Django 3.1 where a test database was not synced during
creation when using the :setting:`MIGRATE <TEST_MIGRATE>` test database
setting (:ticket:`32012`).

* Fixed a ``django.contrib.admin.EmptyFieldListFilter`` crash when using on a
``GenericRelation`` (:ticket:`32038`).

* Fixed a regression in Django 3.1.1 where the admin changelist filter sidebar
would not scroll for a long list of available filters (:ticket:`31986`).


==========================

3.1.1

Not secure
==========================

*September 1, 2020*

Django 3.1.1 fixes two security issues and several bugs in 3.1.

CVE-2020-24583: Incorrect permissions on intermediate-level directories on Python 3.7+
======================================================================================

On Python 3.7+, :setting:`FILE_UPLOAD_DIRECTORY_PERMISSIONS` mode was not
applied to intermediate-level directories created in the process of uploading
files and to intermediate-level collected static directories when using the
:djadmin:`collectstatic` management command.

You should review and manually fix permissions on existing intermediate-level
directories.

CVE-2020-24584: Permission escalation in intermediate-level directories of the file system cache on Python 3.7+
===============================================================================================================

On Python 3.7+, the intermediate-level directories of the file system cache had
the system's standard umask rather than ``0o077`` (no group or others
permissions).

Bugfixes
========

* Fixed wrapping of translated action labels in the admin's navigation sidebar
for East Asian languages (:ticket:`31853`).

* Fixed wrapping of long model names in the admin's navigation sidebar
(:ticket:`31854`).

* Fixed encoding session data while upgrading multiple instances of the same
project to Django 3.1 (:ticket:`31864`).

* Adjusted admin's navigation sidebar template to reduce debug logging when
rendering (:ticket:`31865`).

* Fixed a data loss possibility in the
:meth:`~django.db.models.query.QuerySet.select_for_update()`. When using
related fields pointing to a proxy model in the ``of`` argument, the
corresponding model was not locked (:ticket:`31866`).

* Fixed a data loss possibility, following a regression in Django 2.0, when
copying model instances with a cached fields value (:ticket:`31863`).

* Fixed a regression in Django 3.1 that caused a crash when decoding an invalid
session data (:ticket:`31895`).

* Reverted a deprecation in Django 3.1 that caused a crash when passing
deprecated keyword arguments to a queryset in
``TemplateView.get_context_data()`` (:ticket:`31877`).

* Enforced thread sensitivity of the :class:`MiddlewareMixin.process_request()
<django.utils.deprecation.MiddlewareMixin>` and ``process_response()`` hooks
when in an async context (:ticket:`31905`).

* Fixed ``__in`` lookup on key transforms for
:class:`~django.db.models.JSONField` with MariaDB, MySQL, Oracle, and SQLite
(:ticket:`31936`).

* Fixed a regression in Django 3.1 that caused permission errors in
``CommonPasswordValidator`` and ``settings.py`` generated by the
:djadmin:`startproject` command, when user didn't have permissions to all
intermediate directories in a Django installation path (:ticket:`31912`).

* Fixed detecting an async ``get_response`` callable in various builtin
middlewares (:ticket:`31928`).

* Fixed a ``QuerySet.order_by()`` crash on PostgreSQL when ordering and
grouping by :class:`~django.db.models.JSONField` with a custom
:attr:`~django.db.models.JSONField.decoder` (:ticket:`31956`). As a
consequence, fetching a ``JSONField`` with raw SQL now returns a string
instead of preloaded data. You will need to explicitly call ``json.loads()``
in such cases.

* Fixed a ``QuerySet.delete()`` crash on MySQL, following a performance
regression in Django 3.1 on MariaDB 10.3.2+, when filtering against an
aggregate function (:ticket:`31965`).

* Fixed a ``django.contrib.admin.EmptyFieldListFilter`` crash when using on
reverse relations (:ticket:`31952`).

* Prevented content overflowing in the admin changelist view when the
navigation sidebar is enabled (:ticket:`31901`).


========================

3.1

Not secure
========================

*August 4, 2020*

Welcome to Django 3.1!

These release notes cover the :ref:`new features <whats-new-3.1>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-3.1>` you'll
want to be aware of when upgrading from Django 3.0 or earlier. We've
:ref:`dropped some features<removed-features-3.1>` that have reached the end of
their deprecation cycle, and we've :ref:`begun the deprecation process for
some features <deprecated-features-3.1>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Python compatibility
====================

Django 3.1 supports Python 3.6, 3.7, 3.8, and 3.9 (as of 3.1.3). We **highly
recommend** and only officially support the latest release of each series.

.. _whats-new-3.1:

What's new in Django 3.1
========================

Asynchronous views and middleware support
-----------------------------------------

Django now supports a fully asynchronous request path, including:

* :ref:`Asynchronous views <async-views>`
* :ref:`Asynchronous middleware <async-middleware>`
* :ref:`Asynchronous tests and test client <async-tests>`

To get started with async views, you need to declare a view using
``async def``::

async def my_view(request):
await asyncio.sleep(0.5)
return HttpResponse('Hello, async world!')

All asynchronous features are supported whether you are running under WSGI or
ASGI mode. However, there will be performance penalties using async code in
WSGI mode. You can read more about the specifics in :doc:`/topics/async`
documentation.

You are free to mix async and sync views, middleware, and tests as much as you
want. Django will ensure that you always end up with the right execution
context. We expect most projects will keep the majority of their views
synchronous, and only have a select few running in async mode - but it is
entirely your choice.

Django's ORM, cache layer, and other pieces of code that do long-running
network calls do not yet support async access. We expect to add support for
them in upcoming releases. Async views are ideal, however, if you are doing a
lot of API or HTTP calls inside your view, you can now natively do all those
HTTP calls in parallel to considerably speed up your view's execution.

Asynchronous support should be entirely backwards-compatible and we have tried
to ensure that it has no speed regressions for your existing, synchronous code.
It should have no noticeable effect on any existing Django projects.

JSONField for all supported database backends
---------------------------------------------

Django now includes :class:`.models.JSONField` and
:class:`forms.JSONField <django.forms.JSONField>` that can be used on all
supported database backends. Both fields support the use of custom JSON
encoders and decoders. The model field supports the introspection,
:ref:`lookups, and transforms <querying-jsonfield>` that were previously
PostgreSQL-only::

from django.db import models

class ContactInfo(models.Model):
data = models.JSONField()

ContactInfo.objects.create(data={
'name': 'John',
'cities': ['London', 'Cambridge'],
'pets': {'dogs': ['Rufus', 'Meg']},
})
ContactInfo.objects.filter(
data__name='John',
data__pets__has_key='dogs',
data__cities__contains='London',
).delete()

If your project uses ``django.contrib.postgres.fields.JSONField``, plus the
related form field and transforms, you should adjust to use the new fields,
and generate and apply a database migration. For now, the old fields and
transforms are left as a reference to the new ones and are :ref:`deprecated as
of this release <deprecated-jsonfield>`.

.. _default-hashing-algorithm-usage:

``DEFAULT_HASHING_ALGORITHM`` settings
--------------------------------------

The new ``DEFAULT_HASHING_ALGORITHM`` transitional setting allows specifying
the default hashing algorithm to use for encoding cookies, password reset
tokens in the admin site, user sessions, and signatures created by
:class:`django.core.signing.Signer` and :meth:`django.core.signing.dumps`.

Support for SHA-256 was added in Django 3.1. If you are upgrading multiple
instances of the same project to Django 3.1, you should set
``DEFAULT_HASHING_ALGORITHM`` to ``'sha1'`` during the transition, in order to
allow compatibility with the older versions of Django. Note that this requires
Django 3.1.1+. Once the transition to 3.1 is complete you can stop overriding
``DEFAULT_HASHING_ALGORITHM``.

This setting is deprecated as of this release, because support for tokens,
cookies, sessions, and signatures that use SHA-1 algorithm will be removed in
Django 4.0.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new ``django.contrib.admin.EmptyFieldListFilter`` for
:attr:`.ModelAdmin.list_filter` allows filtering on empty values (empty
strings and nulls) in the admin changelist view.

* Filters in the right sidebar of the admin changelist view now contain a link
to clear all filters.

* The admin now has a sidebar on larger screens for easier navigation. It is
enabled by default but can be disabled by using a custom ``AdminSite`` and
setting :attr:`.AdminSite.enable_nav_sidebar` to ``False``.

Rendering the sidebar requires access to the current request in order to set
CSS and ARIA role affordances. This requires using
``'django.template.context_processors.request'`` in the
``'context_processors'`` option of :setting:`OPTIONS <TEMPLATES-OPTIONS>`.

* Initially empty ``extra`` inlines can now be removed, in the same way as
dynamically created ones.

* ``XRegExp`` is upgraded from version 2.0.0 to 3.2.0.

* jQuery is upgraded from version 3.4.1 to 3.5.1.

* Select2 library is upgraded from version 4.0.7 to 4.0.13.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* The default iteration count for the PBKDF2 password hasher is increased from
180,000 to 216,000.

* The new :setting:`PASSWORD_RESET_TIMEOUT` setting allows defining the number
of seconds a password reset link is valid for. This is encouraged instead of
the deprecated ``PASSWORD_RESET_TIMEOUT_DAYS`` setting, which will be removed
in Django 4.0.

* The password reset mechanism now uses the SHA-256 hashing algorithm. Support
for tokens that use the old hashing algorithm remains until Django 4.0.

* :meth:`.AbstractBaseUser.get_session_auth_hash` now uses the SHA-256 hashing
algorithm. Support for user sessions that use the old hashing algorithm
remains until Django 4.0.

:mod:`django.contrib.contenttypes`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :option:`remove_stale_contenttypes --include-stale-apps` option
allows removing stale content types from previously installed apps that have
been removed from :setting:`INSTALLED_APPS`.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* :lookup:`relate` lookup is now supported on MariaDB.

* Added the :attr:`.LinearRing.is_counterclockwise` property.

* :class:`~django.contrib.gis.db.models.functions.AsGeoJSON` is now supported
on Oracle.

* Added the :class:`~django.contrib.gis.db.models.functions.AsWKB` and
:class:`~django.contrib.gis.db.models.functions.AsWKT` functions.

* Added support for PostGIS 3 and GDAL 3.

:mod:`django.contrib.humanize`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* :tfilter:`intword` template filter now supports negative integers.

:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :class:`~django.contrib.postgres.indexes.BloomIndex` class allows
creating ``bloom`` indexes in the database. The new
:class:`~django.contrib.postgres.operations.BloomExtension` migration
operation installs the ``bloom`` extension to add support for this index.

* :meth:`~django.db.models.Model.get_FOO_display` now supports
:class:`~django.contrib.postgres.fields.ArrayField` and
:class:`~django.contrib.postgres.fields.RangeField`.

* The new :lookup:`rangefield.lower_inc`, :lookup:`rangefield.lower_inf`,
:lookup:`rangefield.upper_inc`, and :lookup:`rangefield.upper_inf` lookups
allow querying :class:`~django.contrib.postgres.fields.RangeField` by a bound
type.

* :lookup:`rangefield.contained_by` now supports
:class:`~django.db.models.SmallAutoField`,
:class:`~django.db.models.AutoField`,
:class:`~django.db.models.BigAutoField`,
:class:`~django.db.models.SmallIntegerField`, and
:class:`~django.db.models.DecimalField`.

* :class:`~django.contrib.postgres.search.SearchQuery` now supports
``'websearch'`` search type on PostgreSQL 11+.

* :class:`SearchQuery.value <django.contrib.postgres.search.SearchQuery>` now
supports query expressions.

* The new :class:`~django.contrib.postgres.search.SearchHeadline` class allows
highlighting search results.

* :lookup:`search` lookup now supports query expressions.

* The new ``cover_density`` parameter of
:class:`~django.contrib.postgres.search.SearchRank` allows ranking by cover
density.

* The new ``normalization`` parameter of
:class:`~django.contrib.postgres.search.SearchRank` allows rank
normalization.

* The new :attr:`.ExclusionConstraint.deferrable` attribute allows creating
deferrable exclusion constraints.

:mod:`django.contrib.sessions`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The :setting:`SESSION_COOKIE_SAMESITE` setting now allows ``'None'`` (string)
value to explicitly state that the cookie is sent with all same-site and
cross-site requests.

:mod:`django.contrib.staticfiles`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The :setting:`STATICFILES_DIRS` setting now supports :class:`pathlib.Path`.

Cache
~~~~~

* The :func:`~django.views.decorators.cache.cache_control` decorator and
:func:`~django.utils.cache.patch_cache_control` method now support multiple
field names in the ``no-cache`` directive for the ``Cache-Control`` header,
according to :rfc:`7234section-5.2.2.2`.

* :meth:`~django.core.caches.cache.delete` now returns ``True`` if the key was
successfully deleted, ``False`` otherwise.

CSRF
~~~~

* The :setting:`CSRF_COOKIE_SAMESITE` setting now allows ``'None'`` (string)
value to explicitly state that the cookie is sent with all same-site and
cross-site requests.

Email
~~~~~

* The :setting:`EMAIL_FILE_PATH` setting, used by the :ref:`file email backend
<topic-email-file-backend>`, now supports :class:`pathlib.Path`.

Error Reporting
~~~~~~~~~~~~~~~

* :class:`django.views.debug.SafeExceptionReporterFilter` now filters sensitive
values from ``request.META`` in exception reports.

* The new :attr:`.SafeExceptionReporterFilter.cleansed_substitute` and
:attr:`.SafeExceptionReporterFilter.hidden_settings` attributes allow
customization of sensitive settings and ``request.META`` filtering in
exception reports.

* The technical 404 debug view now respects
:setting:`DEFAULT_EXCEPTION_REPORTER_FILTER` when applying settings
filtering.

* The new :setting:`DEFAULT_EXCEPTION_REPORTER` allows providing a
:class:`django.views.debug.ExceptionReporter` subclass to customize exception
report generation. See :ref:`custom-error-reports` for details.

File Storage
~~~~~~~~~~~~

* ``FileSystemStorage.save()`` method now supports :class:`pathlib.Path`.

* :class:`~django.db.models.FileField` and
:class:`~django.db.models.ImageField` now accept a callable for ``storage``.
This allows you to modify the used storage at runtime, selecting different
storages for different environments, for example.

Forms
~~~~~

* :class:`~django.forms.ModelChoiceIterator`, used by
:class:`~django.forms.ModelChoiceField` and
:class:`~django.forms.ModelMultipleChoiceField`, now uses
:class:`~django.forms.ModelChoiceIteratorValue` that can be used by widgets
to access model instances. See :ref:`iterating-relationship-choices` for
details.

* :class:`django.forms.DateTimeField` now accepts dates in a subset of ISO 8601
datetime formats, including optional timezone, e.g. ``2019-10-10T06:47``,
``2019-10-10T06:47:23+04:00``, or ``2019-10-10T06:47:23Z``. The timezone will
always be retained if provided, with timezone-aware datetimes being returned
even when :setting:`USE_TZ` is ``False``.

Additionally, ``DateTimeField`` now uses ``DATE_INPUT_FORMATS`` in addition
to ``DATETIME_INPUT_FORMATS`` when converting a field input to a ``datetime``
value.

* :attr:`.MultiWidget.widgets` now accepts a dictionary which allows
customizing subwidget ``name`` attributes.

* The new :attr:`.BoundField.widget_type` property can be used to dynamically
adjust form rendering based upon the widget type.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* The :setting:`LANGUAGE_COOKIE_SAMESITE` setting now allows ``'None'``
(string) value to explicitly state that the cookie is sent with all same-site
and cross-site requests.

* Added support and translations for the Algerian Arabic, Igbo, Kyrgyz, Tajik,
and Turkmen languages.

Management Commands
~~~~~~~~~~~~~~~~~~~

* The new :option:`check --database` option allows specifying database aliases
for running the ``database`` system checks. Previously these checks were
enabled for all configured :setting:`DATABASES` by passing the ``database``
tag to the command.

* The new :option:`migrate --check` option makes the command exit with a
non-zero status when unapplied migrations are detected.

* The new ``returncode`` argument for
:attr:`~django.core.management.CommandError` allows customizing the exit
status for management commands.

* The new :option:`dbshell -- ARGUMENTS <dbshell -->` option allows passing
extra arguments to the command-line client for the database.

* The :djadmin:`flush` and :djadmin:`sqlflush` commands now include SQL to
reset sequences on SQLite.

Models
~~~~~~

* The new :class:`~django.db.models.functions.ExtractIsoWeekDay` function
extracts ISO-8601 week days from :class:`~django.db.models.DateField` and
:class:`~django.db.models.DateTimeField`, and the new :lookup:`iso_week_day`
lookup allows querying by an ISO-8601 day of week.

* :meth:`.QuerySet.explain` now supports:

* ``TREE`` format on MySQL 8.0.16+,
* ``analyze`` option on MySQL 8.0.18+ and MariaDB.

* Added :class:`~django.db.models.PositiveBigIntegerField` which acts much like
a :class:`~django.db.models.PositiveIntegerField` except that it only allows
values under a certain (database-dependent) limit. Values from ``0`` to
``9223372036854775807`` are safe in all databases supported by Django.

* The new :class:`~django.db.models.RESTRICT` option for
:attr:`~django.db.models.ForeignKey.on_delete` argument of ``ForeignKey`` and
``OneToOneField`` emulates the behavior of the SQL constraint ``ON DELETE
RESTRICT``.

* :attr:`.CheckConstraint.check` now supports boolean expressions.

* The :meth:`.RelatedManager.add`, :meth:`~.RelatedManager.create`, and
:meth:`~.RelatedManager.set` methods now accept callables as values in the
``through_defaults`` argument.

* The new ``is_dst`` parameter of the :meth:`.QuerySet.datetimes` determines
the treatment of nonexistent and ambiguous datetimes.

* The new :class:`~django.db.models.F` expression ``bitxor()`` method allows
:ref:`bitwise XOR operation <using-f-expressions-in-filters>`.

* :meth:`.QuerySet.bulk_create` now sets the primary key on objects when using
MariaDB 10.5+.

* The ``DatabaseOperations.sql_flush()`` method now generates more efficient
SQL on MySQL by using ``DELETE`` instead of ``TRUNCATE`` statements for
tables which don't require resetting sequences.

* SQLite functions are now marked as :py:meth:`deterministic
<sqlite3.Connection.create_function>` on Python 3.8+. This allows using them
in check constraints and partial indexes.

* The new :attr:`.UniqueConstraint.deferrable` attribute allows creating
deferrable unique constraints.

Pagination
~~~~~~~~~~

* :class:`~django.core.paginator.Paginator` can now be iterated over to yield
its pages.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* If :setting:`ALLOWED_HOSTS` is empty and ``DEBUG=True``, subdomains of
localhost are now allowed in the ``Host`` header, e.g. ``static.localhost``.

* :meth:`.HttpResponse.set_cookie` and :meth:`.HttpResponse.set_signed_cookie`
now allow using ``samesite='None'`` (string) to explicitly state that the
cookie is sent with all same-site and cross-site requests.

* The new :meth:`.HttpRequest.accepts` method returns whether the request
accepts the given MIME type according to the ``Accept`` HTTP header.

.. _whats-new-security-3.1:

Security
~~~~~~~~

* The :setting:`SECURE_REFERRER_POLICY` setting now defaults to
``'same-origin'``. With this configured,
:class:`~django.middleware.security.SecurityMiddleware` sets the
:ref:`referrer-policy` header to ``same-origin`` on all responses that do not
already have it. This prevents the ``Referer`` header being sent to other
origins. If you need the previous behavior, explicitly set
:setting:`SECURE_REFERRER_POLICY` to ``None``.

* The default algorithm of :class:`django.core.signing.Signer`,
:meth:`django.core.signing.loads`, and :meth:`django.core.signing.dumps` is
changed to the SHA-256. Support for signatures made with the old SHA-1
algorithm remains until Django 4.0.

Also, the new ``algorithm`` parameter of the
:class:`~django.core.signing.Signer` allows customizing the hashing
algorithm.

Templates
~~~~~~~~~

* The renamed :ttag:`translate` and :ttag:`blocktranslate` template tags are
introduced for internationalization in template code. The older :ttag:`trans`
and :ttag:`blocktrans` template tags aliases continue to work, and will be
retained for the foreseeable future.

* The :ttag:`include` template tag now accepts iterables of template names.

Tests
~~~~~

* :class:`~django.test.SimpleTestCase` now implements the ``debug()`` method to
allow running a test without collecting the result and catching exceptions.
This can be used to support running tests under a debugger.

* The new :setting:`MIGRATE <TEST_MIGRATE>` test database setting allows
disabling of migrations during a test database creation.

* Django test runner now supports a :option:`test --buffer` option to discard
output for passing tests.

* :class:`~django.test.runner.DiscoverRunner` now skips running the system
checks on databases not :ref:`referenced by tests<testing-multi-db>`.

* :class:`~django.test.TransactionTestCase` teardown is now faster on MySQL
due to :djadmin:`flush` command improvements. As a side effect the latter
doesn't automatically reset sequences on teardown anymore. Enable
:attr:`.TransactionTestCase.reset_sequences` if your tests require this
feature.

URLs
~~~~

* :ref:`Path converters <registering-custom-path-converters>` can now raise
``ValueError`` in ``to_url()`` to indicate no match when reversing URLs.

Utilities
~~~~~~~~~

* :func:`~django.utils.encoding.filepath_to_uri` now supports
:class:`pathlib.Path`.

* :func:`~django.utils.dateparse.parse_duration` now supports comma separators
for decimal fractions in the ISO 8601 format.

* :func:`~django.utils.dateparse.parse_datetime`,
:func:`~django.utils.dateparse.parse_duration`, and
:func:`~django.utils.dateparse.parse_time` now support comma separators for
milliseconds.

Miscellaneous
~~~~~~~~~~~~~

* The SQLite backend now supports :class:`pathlib.Path` for the ``NAME``
setting.

* The ``settings.py`` generated by the :djadmin:`startproject` command now uses
:class:`pathlib.Path` instead of :mod:`os.path` for building filesystem
paths.

* The :setting:`TIME_ZONE <DATABASE-TIME_ZONE>` setting is now allowed on
databases that support time zones.

.. _backwards-incompatible-3.1:

Backwards incompatible changes in 3.1
=====================================

Database backend API
--------------------

This section describes changes that may be needed in third-party database
backends.

* ``DatabaseOperations.fetch_returned_insert_columns()`` now requires an
additional ``returning_params`` argument.

* ``connection.timezone`` property is now ``'UTC'`` by default, or the
:setting:`TIME_ZONE <DATABASE-TIME_ZONE>` when :setting:`USE_TZ` is ``True``
on databases that support time zones. Previously, it was ``None`` on
databases that support time zones.

* ``connection._nodb_connection`` property is changed to the
``connection._nodb_cursor()`` method and now returns a context manager that
yields a cursor and automatically closes the cursor and connection upon
exiting the ``with`` statement.

* ``DatabaseClient.runshell()`` now requires an additional ``parameters``
argument as a list of extra arguments to pass on to the command-line client.

* The ``sequences`` positional argument of ``DatabaseOperations.sql_flush()``
is replaced by the boolean keyword-only argument ``reset_sequences``. If
``True``, the sequences of the truncated tables will be reset.

* The ``allow_cascade`` argument of ``DatabaseOperations.sql_flush()`` is now a
keyword-only argument.

* The ``using`` positional argument of
``DatabaseOperations.execute_sql_flush()`` is removed. The method now uses
the database of the called instance.

* Third-party database backends must implement support for ``JSONField`` or set
``DatabaseFeatures.supports_json_field`` to ``False``. If storing primitives
is not supported, set ``DatabaseFeatures.supports_primitives_in_json_field``
to ``False``. If there is a true datatype for JSON, set
``DatabaseFeatures.has_native_json_field`` to ``True``. If
:lookup:`jsonfield.contains` and :lookup:`jsonfield.contained_by` are not
supported, set ``DatabaseFeatures.supports_json_field_contains`` to
``False``.

* Third party database backends must implement introspection for ``JSONField``
or set ``can_introspect_json_field`` to ``False``.

Dropped support for MariaDB 10.1
--------------------------------

Upstream support for MariaDB 10.1 ends in October 2020. Django 3.1 supports
MariaDB 10.2 and higher.

``contrib.admin`` browser support
---------------------------------

The admin no longer supports the legacy Internet Explorer browser. See
:ref:`the admin FAQ <admin-browser-support>` for details on supported browsers.

:attr:`AbstractUser.first_name <django.contrib.auth.models.User.first_name>` ``max_length`` increased to 150
------------------------------------------------------------------------------------------------------------

A migration for :attr:`django.contrib.auth.models.User.first_name` is included.
If you have a custom user model inheriting from ``AbstractUser``, you'll need
to generate and apply a database migration for your user model.

If you want to preserve the 30 character limit for first names, use a custom
form::

from django import forms
from django.contrib.auth.forms import UserChangeForm

class MyUserChangeForm(UserChangeForm):
first_name = forms.CharField(max_length=30, required=False)

If you wish to keep this restriction in the admin when editing users, set
``UserAdmin.form`` to use this form::

from django.contrib.auth.admin import UserAdmin
from django.contrib.auth.models import User

class MyUserAdmin(UserAdmin):
form = MyUserChangeForm

admin.site.unregister(User)
admin.site.register(User, MyUserAdmin)

Miscellaneous
-------------

* The cache keys used by :ttag:`cache` and generated by
:func:`~django.core.cache.utils.make_template_fragment_key` are different
from the keys generated by older versions of Django. After upgrading to
Django 3.1, the first request to any previously cached template fragment will
be a cache miss.

* The logic behind the decision to return a redirection fallback or a 204 HTTP
response from the :func:`~django.views.i18n.set_language` view is now based
on the ``Accept`` HTTP header instead of the ``X-Requested-With`` HTTP header
presence.

* The compatibility imports of ``django.core.exceptions.EmptyResultSet`` in
``django.db.models.query``, ``django.db.models.sql``, and
``django.db.models.sql.datastructures`` are removed.

* The compatibility import of ``django.core.exceptions.FieldDoesNotExist`` in
``django.db.models.fields`` is removed.

* The compatibility imports of ``django.forms.utils.pretty_name()`` and
``django.forms.boundfield.BoundField`` in ``django.forms.forms`` are removed.

* The compatibility imports of ``Context``, ``ContextPopException``, and
``RequestContext`` in ``django.template.base`` are removed.

* The compatibility import of
``django.contrib.admin.helpers.ACTION_CHECKBOX_NAME`` in
``django.contrib.admin`` is removed.

* The :setting:`STATIC_URL` and :setting:`MEDIA_URL` settings set to relative
paths are now prefixed by the server-provided value of ``SCRIPT_NAME`` (or
``/`` if not set). This change should not affect settings set to valid URLs
or absolute paths.

* :class:`~django.middleware.http.ConditionalGetMiddleware` no longer adds the
``ETag`` header to responses with an empty
:attr:`~django.http.HttpResponse.content`.

* ``django.utils.decorators.classproperty()`` decorator is made public and
moved to :class:`django.utils.functional.classproperty()`.

* :tfilter:`floatformat` template filter now outputs (positive) ``0`` for
negative numbers which round to zero.

* :attr:`Meta.ordering <django.db.models.Options.ordering>` and
:attr:`Meta.unique_together <django.db.models.Options.unique_together>`
options on models in ``django.contrib`` modules that were formerly tuples are
now lists.

* The admin calendar widget now handles two-digit years according to the Open
Group Specification, i.e. values between 69 and 99 are mapped to the previous
century, and values between 0 and 68 are mapped to the current century.

* Date-only formats are removed from the default list for
:setting:`DATETIME_INPUT_FORMATS`.

* The :class:`~django.forms.FileInput` widget no longer renders with the
``required`` HTML attribute when initial data exists.

* The undocumented ``django.views.debug.ExceptionReporterFilter`` class is
removed. As per the :ref:`custom-error-reports` documentation, classes to be
used with :setting:`DEFAULT_EXCEPTION_REPORTER_FILTER` need to inherit from
:class:`django.views.debug.SafeExceptionReporterFilter`.

* The cache timeout set by :func:`~django.views.decorators.cache.cache_page`
decorator now takes precedence over the ``max-age`` directive from the
``Cache-Control`` header.

* Providing a non-local remote field in the :attr:`.ForeignKey.to_field`
argument now raises :class:`~django.core.exceptions.FieldError`.

* :setting:`SECURE_REFERRER_POLICY` now defaults to ``'same-origin'``. See the
*What's New* :ref:`Security section <whats-new-security-3.1>` above for more
details.

* :djadmin:`check` management command now runs the ``database`` system checks
only for database aliases specified using :option:`check --database` option.

* :djadmin:`migrate` management command now runs the ``database`` system checks
only for a database to migrate.

* The admin CSS classes ``row1`` and ``row2`` are removed in favor of
``:nth-child(odd)`` and ``:nth-child(even)`` pseudo-classes.

* The :func:`~django.contrib.auth.hashers.make_password` function now requires
its argument to be a string or bytes. Other types should be explicitly cast
to one of these.

* The undocumented ``version`` parameter to the
:class:`~django.contrib.gis.db.models.functions.AsKML` function is removed.

* :ref:`JSON and YAML serializers <serialization-formats>`, used by
:djadmin:`dumpdata`, now dump all data with Unicode by default. If you need
the previous behavior, pass ``ensure_ascii=True`` to JSON serializer, or
``allow_unicode=False`` to YAML serializer.

* The auto-reloader no longer monitors changes in built-in Django translation
files.

* The minimum supported version of ``mysqlclient`` is increased from 1.3.13 to
1.4.0.

* The undocumented ``django.contrib.postgres.forms.InvalidJSONInput`` and
``django.contrib.postgres.forms.JSONString`` are moved to
``django.forms.fields``.

* The undocumented ``django.contrib.postgres.fields.jsonb.JsonAdapter`` class
is removed.

* The :ttag:`{% localize off %} <localize>` tag and :tfilter:`unlocalize`
filter no longer respect :setting:`DECIMAL_SEPARATOR` setting.

* The minimum supported version of ``asgiref`` is increased from 3.2 to
3.2.10.

* The :doc:`Media </topics/forms/media>` class now renders ``<script>`` tags
without the ``type`` attribute to follow `WHATWG recommendations
<https://html.spec.whatwg.org/multipage/scripting.html#the-script-element>`_.

* :class:`~django.forms.ModelChoiceIterator`, used by
:class:`~django.forms.ModelChoiceField` and
:class:`~django.forms.ModelMultipleChoiceField`, now yields 2-tuple choices
containing :class:`~django.forms.ModelChoiceIteratorValue` instances as the
first ``value`` element in each choice. In most cases this proxies
transparently, but if you need the ``field`` value itself, use the
:attr:`.ModelChoiceIteratorValue.value` attribute instead.

.. _deprecated-features-3.1:

Features deprecated in 3.1
==========================

.. _deprecated-jsonfield:

PostgreSQL ``JSONField``
------------------------

``django.contrib.postgres.fields.JSONField`` and
``django.contrib.postgres.forms.JSONField`` are deprecated in favor of
:class:`.models.JSONField` and
:class:`forms.JSONField <django.forms.JSONField>`.

The undocumented ``django.contrib.postgres.fields.jsonb.KeyTransform`` and
``django.contrib.postgres.fields.jsonb.KeyTextTransform`` are also deprecated
in favor of the transforms in ``django.db.models.fields.json``.

The new ``JSONField``\s, ``KeyTransform``, and ``KeyTextTransform`` can be used
on all supported database backends.

Miscellaneous
-------------

* ``PASSWORD_RESET_TIMEOUT_DAYS`` setting is deprecated in favor of
:setting:`PASSWORD_RESET_TIMEOUT`.

* The undocumented usage of the :lookup:`isnull` lookup with non-boolean values
as the right-hand side is deprecated, use ``True`` or ``False`` instead.

* The barely documented ``django.db.models.query_utils.InvalidQuery`` exception
class is deprecated in favor of
:class:`~django.core.exceptions.FieldDoesNotExist` and
:class:`~django.core.exceptions.FieldError`.

* The ``django-admin.py`` entry point is deprecated in favor of
``django-admin``.

* The ``HttpRequest.is_ajax()`` method is deprecated as it relied on a
jQuery-specific way of signifying AJAX calls, while current usage tends to
use the JavaScript `Fetch API
<https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API>`_. Depending on
your use case, you can either write your own AJAX detection method, or use
the new :meth:`.HttpRequest.accepts` method if your code depends on the
client ``Accept`` HTTP header.

If you are writing your own AJAX detection method, ``request.is_ajax()`` can
be reproduced exactly as
``request.headers.get('x-requested-with') == 'XMLHttpRequest'``.

* Passing ``None`` as the first argument to
``django.utils.deprecation.MiddlewareMixin.__init__()`` is deprecated.

* The encoding format of cookies values used by
:class:`~django.contrib.messages.storage.cookie.CookieStorage` is different
from the format generated by older versions of Django. Support for the old
format remains until Django 4.0.

* The encoding format of sessions is different from the format generated by
older versions of Django. Support for the old format remains until Django
4.0.

* The purely documentational ``providing_args`` argument for
:class:`~django.dispatch.Signal` is deprecated. If you rely on this
argument as documentation, you can move the text to a code comment or
docstring.

* Calling ``django.utils.crypto.get_random_string()`` without a ``length``
argument is deprecated.

* The ``list`` message for :class:`~django.forms.ModelMultipleChoiceField` is
deprecated in favor of ``invalid_list``.

* Passing raw column aliases to :meth:`.QuerySet.order_by` is deprecated. The
same result can be achieved by passing aliases in a
:class:`~django.db.models.expressions.RawSQL` instead beforehand.

* The ``NullBooleanField`` model field is deprecated in favor of
``BooleanField(null=True)``.

* ``django.conf.urls.url()`` alias of :func:`django.urls.re_path` is
deprecated.

* The ``{% ifequal %}`` and ``{% ifnotequal %}`` template tags are deprecated
in favor of :ttag:`{% if %}<if>`. ``{% if %}`` covers all use cases, but if
you need to continue using these tags, they can be extracted from Django to a
module and included as a built-in tag in the :class:`'builtins'
<django.template.backends.django.DjangoTemplates>` option in
:setting:`OPTIONS <TEMPLATES-OPTIONS>`.

* ``DEFAULT_HASHING_ALGORITHM`` transitional setting is deprecated.

.. _removed-features-3.1:

Features removed in 3.1
=======================

These features have reached the end of their deprecation cycle and are removed
in Django 3.1.

See :ref:`deprecated-features-2.2` for details on these changes, including how
to remove usage of these features.

* ``django.utils.timezone.FixedOffset`` is removed.

* ``django.core.paginator.QuerySetPaginator`` is removed.

* A model's ``Meta.ordering`` doesn't affect ``GROUP BY`` queries.

* ``django.contrib.postgres.fields.FloatRangeField`` and
``django.contrib.postgres.forms.FloatRangeField`` are removed.

* The ``FILE_CHARSET`` setting is removed.

* ``django.contrib.staticfiles.storage.CachedStaticFilesStorage`` is removed.

* The ``RemoteUserBackend.configure_user()`` method requires ``request`` as the
first positional argument.

* Support for ``SimpleTestCase.allow_database_queries`` and
``TransactionTestCase.multi_db`` is removed.


===========================

3.0.14

Not secure
===========================

*April 6, 2021*

Django 3.0.14 fixes a security issue with severity "low" in 3.0.13.

CVE-2021-28658: Potential directory-traversal via uploaded files
================================================================

``MultiPartParser`` allowed directory-traversal via uploaded files with
suitably crafted file names.

Built-in upload handlers were not affected by this vulnerability.


===========================

3.0.13

Not secure
===========================

*February 19, 2021*

Django 3.0.13 fixes a security issue in 3.0.12.

CVE-2021-23336: Web cache poisoning via ``django.utils.http.limited_parse_qsl()``
=================================================================================

Django contains a copy of :func:`urllib.parse.parse_qsl` which was added to
backport some security fixes. A further security fix has been issued recently
such that ``parse_qsl()`` no longer allows using ``;`` as a query parameter
separator by default. Django now includes this fix. See :bpo:`42967` for
further details.


===========================

3.0.12

Not secure
===========================

*February 1, 2021*

Django 3.0.12 fixes a security issue with severity "low" in 3.0.11.

CVE-2021-3281: Potential directory-traversal via ``archive.extract()``
======================================================================

The ``django.utils.archive.extract()`` function, used by
:option:`startapp --template` and :option:`startproject --template`, allowed
directory-traversal via an archive with absolute paths or relative paths with
dot segments.


===========================

3.0.11

Not secure
===========================

*November 2, 2020*

Django 3.0.11 fixes a regression in 3.0.7 and adds compatibility with Python
3.9.

Bugfixes
========

* Fixed a regression in Django 3.0.7 that didn't use ``Subquery()`` aliases in
the ``GROUP BY`` clause (:ticket:`32152`).


===========================

3.0.10

Not secure
===========================

*September 1, 2020*

Django 3.0.10 fixes two security issues and two data loss bugs in 3.0.9.

CVE-2020-24583: Incorrect permissions on intermediate-level directories on Python 3.7+
======================================================================================

On Python 3.7+, :setting:`FILE_UPLOAD_DIRECTORY_PERMISSIONS` mode was not
applied to intermediate-level directories created in the process of uploading
files and to intermediate-level collected static directories when using the
:djadmin:`collectstatic` management command.

You should review and manually fix permissions on existing intermediate-level
directories.

CVE-2020-24584: Permission escalation in intermediate-level directories of the file system cache on Python 3.7+
===============================================================================================================

On Python 3.7+, the intermediate-level directories of the file system cache had
the system's standard umask rather than ``0o077`` (no group or others
permissions).

Bugfixes
========

* Fixed a data loss possibility in the
:meth:`~django.db.models.query.QuerySet.select_for_update()`. When using
related fields pointing to a proxy model in the ``of`` argument, the
corresponding model was not locked (:ticket:`31866`).

* Fixed a data loss possibility, following a regression in Django 2.0, when
copying model instances with a cached fields value (:ticket:`31863`).


==========================

3.0.9

Not secure
==========================

*August 3, 2020*

Django 3.0.9 fixes several bugs in 3.0.8.

Bugfixes
========

* Allowed setting the ``SameSite`` cookie flag in
:meth:`.HttpResponse.delete_cookie` (:ticket:`31790`).

* Fixed crash when sending emails to addresses with display names longer than
75 chars on Python 3.6.11+, 3.7.8+, and 3.8.4+ (:ticket:`31784`).


==========================

3.0.8

Not secure
==========================

*July 1, 2020*

Django 3.0.8 fixes several bugs in 3.0.7.

Bugfixes
========

* Fixed messages of ``InvalidCacheKey`` exceptions and ``CacheKeyWarning``
warnings raised by cache key validation (:ticket:`31654`).

* Fixed a regression in Django 3.0.7 that caused a queryset crash when grouping
by a many-to-one relationship (:ticket:`31660`).

* Reallowed, following a regression in Django 3.0, non-expressions having a
``filterable`` attribute to be used as the right-hand side in queryset
filters (:ticket:`31664`).

* Fixed a regression in Django 3.0.2 that caused a migration crash on
PostgreSQL when adding a foreign key to a model with a namespaced
``db_table`` (:ticket:`31735`).

* Added compatibility for ``cx_Oracle`` 8 (:ticket:`31751`).


==========================

3.0.7

Not secure
==========================

*June 3, 2020*

Django 3.0.7 fixes two security issues and several bugs in 3.0.6.

CVE-2020-13254: Potential data leakage via malformed memcached keys
===================================================================

In cases where a memcached backend does not perform key validation, passing
malformed cache keys could result in a key collision, and potential data
leakage. In order to avoid this vulnerability, key validation is added to the
memcached cache backends.

CVE-2020-13596: Possible XSS via admin ``ForeignKeyRawIdWidget``
================================================================

Query parameters for the admin ``ForeignKeyRawIdWidget`` were not properly URL
encoded, posing an XSS attack vector. ``ForeignKeyRawIdWidget`` now
ensures query parameters are correctly URL encoded.

Bugfixes
========

* Fixed a regression in Django 3.0 by restoring the ability to use field
lookups in ``Meta.ordering`` (:ticket:`31538`).

* Fixed a regression in Django 3.0 where ``QuerySet.values()`` and
``values_list()`` crashed if a queryset contained an aggregation and a
subquery annotation (:ticket:`31566`).

* Fixed a regression in Django 3.0 where aggregates used wrong annotations when
a queryset has multiple subqueries annotations (:ticket:`31568`).

* Fixed a regression in Django 3.0 where ``QuerySet.values()`` and
``values_list()`` crashed if a queryset contained an aggregation and an
``Exists()`` annotation on Oracle (:ticket:`31584`).

* Fixed a regression in Django 3.0 where all resolved ``Subquery()``
expressions were considered equal (:ticket:`31607`).

* Fixed a regression in Django 3.0.5 that affected translation loading for apps
providing translations for territorial language variants as well as a generic
language, where the project has different plural equations for the language
(:ticket:`31570`).

* Tracking a jQuery security release, upgraded the version of jQuery used by
the admin from 3.4.1 to 3.5.1.


==========================

3.0.6

Not secure
==========================

*May 4, 2020*

Django 3.0.6 fixes a bug in 3.0.5.

Bugfixes
========

* Fixed a regression in Django 3.0 that caused a crash when filtering a
``Subquery()`` annotation of a queryset containing a single related field
against a ``SimpleLazyObject`` (:ticket:`31420`).


==========================

3.0.5

Not secure
==========================

*April 1, 2020*

Django 3.0.5 fixes several bugs in 3.0.4.

Bugfixes
========

* Added the ability to handle ``.po`` files containing different plural
equations for the same language (:ticket:`30439`).

* Fixed a regression in Django 3.0 where ``QuerySet.values()`` and
``values_list()`` crashed if a queryset contained an aggregation and
``Subquery()`` annotation that collides with a field name (:ticket:`31377`).


==========================

3.0.4

Not secure
==========================

*March 4, 2020*

Django 3.0.4 fixes a security issue and several bugs in 3.0.3.

CVE-2020-9402: Potential SQL injection via ``tolerance`` parameter in GIS functions and aggregates on Oracle
============================================================================================================

GIS functions and aggregates on Oracle were subject to SQL injection,
using a suitably crafted ``tolerance``.

Bugfixes
========

* Fixed a data loss possibility when using caching from async code
(:ticket:`31253`).

* Fixed a regression in Django 3.0 that caused a file response using a
temporary file to be closed incorrectly (:ticket:`31240`).

* Fixed a data loss possibility in the
:meth:`~django.db.models.query.QuerySet.select_for_update`. When using
related fields or parent link fields with :ref:`multi-table-inheritance` in
the ``of`` argument, the corresponding models were not locked
(:ticket:`31246`).

* Fixed a regression in Django 3.0 that caused misplacing parameters in logged
SQL queries on Oracle (:ticket:`31271`).

* Fixed a regression in Django 3.0.3 that caused misplacing parameters of SQL
queries when subtracting ``DateField`` or ``DateTimeField`` expressions on
MySQL (:ticket:`31312`).

* Fixed a regression in Django 3.0 that didn't include subqueries spanning
multivalued relations in the ``GROUP BY`` clause (:ticket:`31150`).


==========================

3.0.3

Not secure
==========================

*February 3, 2020*

Django 3.0.3 fixes a security issue and several bugs in 3.0.2.

CVE-2020-7471: Potential SQL injection via ``StringAgg(delimiter)``
===================================================================

:class:`~django.contrib.postgres.aggregates.StringAgg` aggregation function was
subject to SQL injection, using a suitably crafted ``delimiter``.

Bugfixes
========

* Fixed a regression in Django 3.0 that caused a crash when subtracting
``DateField``, ``DateTimeField``, or ``TimeField`` from a ``Subquery()``
annotation (:ticket:`31133`).

* Fixed a regression in Django 3.0 where ``QuerySet.values()`` and
``values_list()`` crashed if a queryset contained an aggregation and
``Exists()`` annotation (:ticket:`31136`).

* Relaxed the system check added in Django 3.0 to reallow use of a sublanguage
in the :setting:`LANGUAGE_CODE` setting, when a base language is available in
Django but the sublanguage is not (:ticket:`31141`).

* Added support for using enumeration types ``TextChoices``,
``IntegerChoices``, and ``Choices`` in templates (:ticket:`31154`).

* Fixed a system check to ensure the ``max_length`` attribute fits the longest
choice, when a named group contains only non-string values (:ticket:`31155`).

* Fixed a regression in Django 2.2 that caused a crash of
:class:`~django.contrib.postgres.aggregates.ArrayAgg` and
:class:`~django.contrib.postgres.aggregates.StringAgg` with ``filter``
argument when used in a ``Subquery`` (:ticket:`31097`).

* Fixed a regression in Django 2.2.7 that caused
:meth:`~django.db.models.Model.get_FOO_display` to work incorrectly when
overriding inherited choices (:ticket:`31124`).

* Fixed a regression in Django 3.0 that caused a crash of
``QuerySet.prefetch_related()`` for ``GenericForeignKey`` with a custom
``ContentType`` foreign key (:ticket:`31190`).


==========================

3.0.2

Not secure
==========================

*January 2, 2020*

Django 3.0.2 fixes several bugs in 3.0.1.

Bugfixes
========

* Fixed a regression in Django 3.0 that didn't include columns referenced by a
``Subquery()`` in the ``GROUP BY`` clause (:ticket:`31094`).

* Fixed a regression in Django 3.0 where ``QuerySet.exists()`` crashed if a
queryset contained an aggregation over a ``Subquery()`` (:ticket:`31109`).

* Fixed a regression in Django 3.0 that caused a migration crash on PostgreSQL
10+ when adding a foreign key and changing data in the same migration
(:ticket:`31106`).

* Fixed a regression in Django 3.0 where loading fixtures crashed for models
defining a :attr:`~django.db.models.Field.default` for the primary key
(:ticket:`31071`).


==========================

3.0.1

Not secure
==========================

*December 18, 2019*

Django 3.0.1 fixes a security issue and several bugs in 3.0.

CVE-2019-19844: Potential account hijack via password reset form
================================================================

By submitting a suitably crafted email address making use of Unicode
characters, that compared equal to an existing user email when lower-cased for
comparison, an attacker could be sent a password reset token for the matched
account.

In order to avoid this vulnerability, password reset requests now compare the
submitted email using the stricter, recommended algorithm for case-insensitive
comparison of two identifiers from `Unicode Technical Report 36, section
2.11.2(B)(2)`__. Upon a match, the email containing the reset token will be
sent to the email address on record rather than the submitted address.

.. __: https://www.unicode.org/reports/tr36/#Recommendations_General

Bugfixes
========

* Fixed a regression in Django 3.0 by restoring the ability to use Django
inside Jupyter and other environments that force an async context, by adding
an option to disable :ref:`async-safety` mechanism with
:envvar:`DJANGO_ALLOW_ASYNC_UNSAFE` environment variable (:ticket:`31056`).

* Fixed a regression in Django 3.0 where ``RegexPattern``, used by
:func:`~django.urls.re_path`, returned positional arguments to be passed to
the view when all optional named groups were missing (:ticket:`31061`).

* Reallowed, following a regression in Django 3.0,
:class:`~django.db.models.expressions.Window` expressions to be used in
conditions outside of queryset filters, e.g. in
:class:`~django.db.models.expressions.When` conditions (:ticket:`31060`).

* Fixed a data loss possibility in
:class:`~django.contrib.postgres.forms.SplitArrayField`. When using with
``ArrayField(BooleanField())``, all values after the first ``True`` value
were marked as checked instead of preserving passed values (:ticket:`31073`).


========================

3.0

Not secure
========================

*December 2, 2019*

Welcome to Django 3.0!

These release notes cover the :ref:`new features <whats-new-3.0>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-3.0>` you'll
want to be aware of when upgrading from Django 2.2 or earlier. We've
:ref:`dropped some features<removed-features-3.0>` that have reached the end of
their deprecation cycle, and we've :ref:`begun the deprecation process for
some features <deprecated-features-3.0>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Python compatibility
====================

Django 3.0 supports Python 3.6, 3.7, 3.8, and 3.9 (as of 3.0.11). We **highly
recommend** and only officially support the latest release of each series.

The Django 2.2.x series is the last to support Python 3.5.

Third-party library support for older version of Django
=======================================================

Following the release of Django 3.0, we suggest that third-party app authors
drop support for all versions of Django prior to 2.2. At that time, you should
be able to run your package's tests using ``python -Wd`` so that deprecation
warnings appear. After making the deprecation warning fixes, your app should be
compatible with Django 3.0.

.. _whats-new-3.0:

What's new in Django 3.0
========================

MariaDB support
---------------

Django now officially supports `MariaDB <https://mariadb.org/>`_ 10.1 and
higher. See :ref:`MariaDB notes <mariadb-notes>` for more details.

ASGI support
------------

Django 3.0 begins our journey to making Django fully async-capable by providing
support for running as an `ASGI <https://asgi.readthedocs.io/>`_ application.

This is in addition to our existing WSGI support. Django intends to support
both for the foreseeable future. Async features will only be available to
applications that run under ASGI, however.

At this stage async support only applies to the outer ASGI application.
Internally everything remains synchronous. Asynchronous middleware, views, etc.
are not yet supported. You can, however, use ASGI middleware around Django's
application, allowing you to combine Django with other ASGI frameworks.

There is no need to switch your applications over unless you want to start
experimenting with asynchronous code, but we have
:doc:`documentation on deploying with ASGI </howto/deployment/asgi/index>` if
you want to learn more.

Note that as a side-effect of this change, Django is now aware of asynchronous
event loops and will block you calling code marked as "async unsafe" - such as
ORM operations - from an asynchronous context. If you were using Django from
async code before, this may trigger if you were doing it incorrectly. If you
see a ``SynchronousOnlyOperation`` error, then closely examine your code and
move any database operations to be in a synchronous child thread.

Exclusion constraints on PostgreSQL
-----------------------------------

The new :class:`~django.contrib.postgres.constraints.ExclusionConstraint` class
enable adding exclusion constraints on PostgreSQL. Constraints are added to
models using the
:attr:`Meta.constraints <django.db.models.Options.constraints>` option.

Filter expressions
------------------

Expressions that output :class:`~django.db.models.BooleanField` may now be
used directly in ``QuerySet`` filters, without having to first annotate and
then filter against the annotation.

Enumerations for model field choices
------------------------------------

Custom enumeration types ``TextChoices``, ``IntegerChoices``, and ``Choices``
are now available as a way to define :attr:`.Field.choices`. ``TextChoices``
and ``IntegerChoices`` types are provided for text and integer fields. The
``Choices`` class allows defining a compatible enumeration for other concrete
data types. These custom enumeration types support human-readable labels that
can be translated and accessed via a property on the enumeration or its
members. See :ref:`Enumeration types <field-choices-enum-types>` for more
details and examples.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Added support for the ``admin_order_field`` attribute on properties in
:attr:`.ModelAdmin.list_display`.

* The new :meth:`ModelAdmin.get_inlines()
<django.contrib.admin.ModelAdmin.get_inlines>` method allows specifying the
inlines based on the request or model instance.

* Select2 library is upgraded from version 4.0.3 to 4.0.7.

* jQuery is upgraded from version 3.3.1 to 3.4.1.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new ``reset_url_token`` attribute in
:class:`~django.contrib.auth.views.PasswordResetConfirmView` allows
specifying a token parameter displayed as a component of password reset
URLs.

* Added :class:`~django.contrib.auth.backends.BaseBackend` class to ease
customization of authentication backends.

* Added :meth:`~django.contrib.auth.models.User.get_user_permissions()` method
to mirror the existing
:meth:`~django.contrib.auth.models.User.get_group_permissions()` method.

* Added HTML ``autocomplete`` attribute to widgets of username, email, and
password fields in :mod:`django.contrib.auth.forms` for better interaction
with browser password managers.

* :djadmin:`createsuperuser` now falls back to environment variables for
password and required fields, when a corresponding command line argument
isn't provided in non-interactive mode.

* :attr:`~django.contrib.auth.models.CustomUser.REQUIRED_FIELDS` now supports
:class:`~django.db.models.ManyToManyField`\s.

* The new :meth:`.UserManager.with_perm` method returns users that have the
specified permission.

* The default iteration count for the PBKDF2 password hasher is increased from
150,000 to 180,000.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* Allowed MySQL spatial lookup functions to operate on real geometries.
Previous support was limited to bounding boxes.

* Added the :class:`~django.contrib.gis.db.models.functions.GeometryDistance`
function, supported on PostGIS.

* Added support for the ``furlong`` unit in
:class:`~django.contrib.gis.measure.Distance`.

* The :setting:`GEOIP_PATH` setting now supports :class:`pathlib.Path`.

* The :class:`~django.contrib.gis.geoip2.GeoIP2` class now accepts
:class:`pathlib.Path` ``path``.

:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :class:`~django.contrib.postgres.fields.RangeOperators` helps to
avoid typos in SQL operators that can be used together with
:class:`~django.contrib.postgres.fields.RangeField`.

* The new :class:`~django.contrib.postgres.fields.RangeBoundary` expression
represents the range boundaries.

* The new :class:`~django.contrib.postgres.operations.AddIndexConcurrently`
and :class:`~django.contrib.postgres.operations.RemoveIndexConcurrently`
classes allow creating and dropping indexes ``CONCURRENTLY`` on PostgreSQL.

:mod:`django.contrib.sessions`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new
:meth:`~django.contrib.sessions.backends.base.SessionBase.get_session_cookie_age()`
method allows dynamically specifying the session cookie age.

:mod:`django.contrib.syndication`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Added the ``language`` class attribute to the
:class:`django.contrib.syndication.views.Feed` to customize a feed language.
The default value is :func:`~django.utils.translation.get_language()` instead
of :setting:`LANGUAGE_CODE`.

Cache
~~~~~

* :func:`~django.utils.cache.add_never_cache_headers` and
:func:`~django.views.decorators.cache.never_cache` now add the ``private``
directive to ``Cache-Control`` headers.

File Storage
~~~~~~~~~~~~

* The new :meth:`.Storage.get_alternative_name` method allows customizing the
algorithm for generating filenames if a file with the uploaded name already
exists.

Forms
~~~~~

* Formsets may control the widget used when ordering forms via
:attr:`~django.forms.formsets.BaseFormSet.can_order` by setting the
:attr:`~django.forms.formsets.BaseFormSet.ordering_widget` attribute or
overriding :attr:`~django.forms.formsets.BaseFormSet.get_ordering_widget()`.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* Added the :setting:`LANGUAGE_COOKIE_HTTPONLY`,
:setting:`LANGUAGE_COOKIE_SAMESITE`, and :setting:`LANGUAGE_COOKIE_SECURE`
settings to set the ``HttpOnly``, ``SameSite``, and ``Secure`` flags on
language cookies. The default values of these settings preserve the previous
behavior.

* Added support and translations for the Uzbek language.

Logging
~~~~~~~

* The new ``reporter_class`` parameter of
:class:`~django.utils.log.AdminEmailHandler` allows providing an
``django.views.debug.ExceptionReporter`` subclass to customize the traceback
text sent to site :setting:`ADMINS` when :setting:`DEBUG` is ``False``.

Management Commands
~~~~~~~~~~~~~~~~~~~

* The new :option:`compilemessages --ignore` option allows ignoring specific
directories when searching for ``.po`` files to compile.

* :option:`showmigrations --list` now shows the applied datetimes when
``--verbosity`` is 2 and above.

* On PostgreSQL, :djadmin:`dbshell` now supports client-side TLS certificates.

* :djadmin:`inspectdb` now introspects :class:`~django.db.models.OneToOneField`
when a foreign key has a unique or primary key constraint.

* The new :option:`--skip-checks` option skips running system checks prior to
running the command.

* The :option:`startapp --template` and :option:`startproject --template`
options now support templates stored in XZ archives (``.tar.xz``, ``.txz``)
and LZMA archives (``.tar.lzma``, ``.tlz``).

Models
~~~~~~

* Added hash database functions :class:`~django.db.models.functions.MD5`,
:class:`~django.db.models.functions.SHA1`,
:class:`~django.db.models.functions.SHA224`,
:class:`~django.db.models.functions.SHA256`,
:class:`~django.db.models.functions.SHA384`, and
:class:`~django.db.models.functions.SHA512`.

* Added the :class:`~django.db.models.functions.Sign` database function.

* The new ``is_dst`` parameter of the
:class:`~django.db.models.functions.Trunc` database functions determines the
treatment of nonexistent and ambiguous datetimes.

* ``connection.queries`` now shows ``COPY … TO`` statements on PostgreSQL.

* :class:`~django.db.models.FilePathField` now accepts a callable for ``path``.

* Allowed symmetrical intermediate table for self-referential
:class:`~django.db.models.ManyToManyField`.

* The ``name`` attributes of :class:`~django.db.models.CheckConstraint`,
:class:`~django.db.models.UniqueConstraint`, and
:class:`~django.db.models.Index` now support app label and class
interpolation using the ``'%(app_label)s'`` and ``'%(class)s'`` placeholders.

* The new :attr:`.Field.descriptor_class` attribute allows model fields to
customize the get and set behavior by overriding their
:py:ref:`descriptors <descriptors>`.

* :class:`~django.db.models.Avg` and :class:`~django.db.models.Sum` now support
the ``distinct`` argument.

* Added :class:`~django.db.models.SmallAutoField` which acts much like an
:class:`~django.db.models.AutoField` except that it only allows values under
a certain (database-dependent) limit. Values from ``1`` to ``32767`` are safe
in all databases supported by Django.

* :class:`~django.db.models.AutoField`,
:class:`~django.db.models.BigAutoField`, and
:class:`~django.db.models.SmallAutoField` now inherit from
``IntegerField``, ``BigIntegerField`` and ``SmallIntegerField`` respectively.
System checks and validators are now also properly inherited.

* :attr:`.FileField.upload_to` now supports :class:`pathlib.Path`.

* :class:`~django.db.models.CheckConstraint` is now supported on MySQL 8.0.16+.

* The new ``allows_group_by_selected_pks_on_model()`` method of
``django.db.backends.base.BaseDatabaseFeatures`` allows optimization of
``GROUP BY`` clauses to require only the selected models' primary keys. By
default, it's supported only for managed models on PostgreSQL.

To enable the ``GROUP BY`` primary key-only optimization for unmanaged
models, you have to subclass the PostgreSQL database engine, overriding the
features class ``allows_group_by_selected_pks_on_model()`` method as you
require. See :ref:`Subclassing the built-in database backends
<subclassing-database-backends>` for an example.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* Allowed :class:`~django.http.HttpResponse` to be initialized with
:class:`memoryview` content.

* For use in, for example, Django templates, :attr:`.HttpRequest.headers` now
allows lookups using underscores (e.g. ``user_agent``) in place of hyphens.

.. _whats-new-security-3.0:

Security
~~~~~~~~

* :setting:`X_FRAME_OPTIONS` now defaults to ``'DENY'``. In older versions, the
:setting:`X_FRAME_OPTIONS` setting defaults to ``'SAMEORIGIN'``. If your site
uses frames of itself, you will need to explicitly set ``X_FRAME_OPTIONS =
'SAMEORIGIN'`` for them to continue working.

* :setting:`SECURE_CONTENT_TYPE_NOSNIFF` now defaults to ``True``. With this
enabled, :class:`~django.middleware.security.SecurityMiddleware` sets the
:ref:`x-content-type-options` header on all responses that do not already
have it.

* :class:`~django.middleware.security.SecurityMiddleware` can now send the
:ref:`Referrer-Policy <referrer-policy>` header.

Tests
~~~~~

* The new test :class:`~django.test.Client` argument
``raise_request_exception`` allows controlling whether or not exceptions
raised during the request should also be raised in the test. The value
defaults to ``True`` for backwards compatibility. If it is ``False`` and an
exception occurs, the test client will return a 500 response with the
attribute :attr:`~django.test.Response.exc_info`, a tuple providing
information of the exception that occurred.

* Tests and test cases to run can be selected by test name pattern using the
new :option:`test -k` option.

* HTML comparison, as used by
:meth:`~django.test.SimpleTestCase.assertHTMLEqual`, now treats text, character
references, and entity references that refer to the same character as
equivalent.

* Django test runner now supports headless mode for selenium tests on supported
browsers. Add the ``--headless`` option to enable this mode.

* Django test runner now supports ``--start-at`` and ``--start-after`` options
to run tests starting from a specific top-level module.

* Django test runner now supports a ``--pdb`` option to spawn a debugger at
each error or failure.

.. _backwards-incompatible-3.0:

Backwards incompatible changes in 3.0
=====================================

``Model.save()`` when providing a default for the primary key
-------------------------------------------------------------

:meth:`.Model.save` no longer attempts to find a row when saving a new
``Model`` instance and a default value for the primary key is provided, and
always performs a single ``INSERT`` query. In older Django versions,
``Model.save()`` performed either an ``INSERT`` or an ``UPDATE`` based on
whether or not the row exists.

This makes calling ``Model.save()`` while providing a default primary key value
equivalent to passing :ref:`force_insert=True <ref-models-force-insert>` to
model's ``save()``. Attempts to use a new ``Model`` instance to update an
existing row will result in an ``IntegrityError``.

In order to update an existing model for a specific primary key value, use the
:meth:`~django.db.models.query.QuerySet.update_or_create` method or
``QuerySet.filter(pk=…).update(…)`` instead. For example::

>>> MyModel.objects.update_or_create(pk=existing_pk, defaults={'name': 'new name'})
>>> MyModel.objects.filter(pk=existing_pk).update(name='new name')

Database backend API
--------------------

This section describes changes that may be needed in third-party database
backends.

* The second argument of ``DatabaseIntrospection.get_geometry_type()`` is now
the row description instead of the column name.

* ``DatabaseIntrospection.get_field_type()`` may no longer return tuples.

* If the database can create foreign keys in the same SQL statement that adds a
field, add ``SchemaEditor.sql_create_column_inline_fk`` with the appropriate
SQL; otherwise, set ``DatabaseFeatures.can_create_inline_fk = False``.

* ``DatabaseFeatures.can_return_id_from_insert`` and
``can_return_ids_from_bulk_insert`` are renamed to
``can_return_columns_from_insert`` and ``can_return_rows_from_bulk_insert``.

* Database functions now handle :class:`datetime.timezone` formats when created
using :class:`datetime.timedelta` instances (e.g.
``timezone(timedelta(hours=5))``, which would output ``'UTC+05:00'``).
Third-party backends should handle this format when preparing
:class:`~django.db.models.DateTimeField` in ``datetime_cast_date_sql()``,
``datetime_extract_sql()``, etc.

* Entries for ``AutoField``, ``BigAutoField``, and ``SmallAutoField`` are added
to ``DatabaseOperations.integer_field_ranges`` to support the integer range
validators on these field types. Third-party backends may need to customize
the default entries.

* ``DatabaseOperations.fetch_returned_insert_id()`` is replaced by
``fetch_returned_insert_columns()`` which returns a list of values returned
by the ``INSERT … RETURNING`` statement, instead of a single value.

* ``DatabaseOperations.return_insert_id()`` is replaced by
``return_insert_columns()`` that accepts a ``fields``
argument, which is an iterable of fields to be returned after insert. Usually
this is only the auto-generated primary key.

:mod:`django.contrib.admin`
---------------------------

* Admin's model history change messages now prefers more readable field labels
instead of field names.

:mod:`django.contrib.gis`
-------------------------

* Support for PostGIS 2.1 is removed.

* Support for SpatiaLite 4.1 and 4.2 is removed.

* Support for GDAL 1.11 and GEOS 3.4 is removed.

Dropped support for PostgreSQL 9.4
----------------------------------

Upstream support for PostgreSQL 9.4 ends in December 2019. Django 3.0 supports
PostgreSQL 9.5 and higher.

Dropped support for Oracle 12.1
-------------------------------

Upstream support for Oracle 12.1 ends in July 2021. Django 2.2 will be
supported until April 2022. Django 3.0 officially supports Oracle 12.2 and 18c.

Removed private Python 2 compatibility APIs
-------------------------------------------

While Python 2 support was removed in Django 2.0, some private APIs weren't
removed from Django so that third party apps could continue using them until
the Python 2 end-of-life.

Since we expect apps to drop Python 2 compatibility when adding support for
Django 3.0, we're removing these APIs at this time.

* ``django.test.utils.str_prefix()`` - Strings don't have 'u' prefixes in
Python 3.

* ``django.test.utils.patch_logger()`` - Use
:meth:`unittest.TestCase.assertLogs` instead.

* ``django.utils.lru_cache.lru_cache()`` - Alias of
:func:`functools.lru_cache`.

* ``django.utils.decorators.available_attrs()`` - This function returns
``functools.WRAPPER_ASSIGNMENTS``.

* ``django.utils.decorators.ContextDecorator`` - Alias of
:class:`contextlib.ContextDecorator`.

* ``django.utils._os.abspathu()`` - Alias of :func:`os.path.abspath`.

* ``django.utils._os.upath()`` and ``npath()`` - These functions do nothing on
Python 3.

* ``django.utils.six`` - Remove usage of this vendored library or switch to
`six <https://pypi.org/project/six/>`_.

* ``django.utils.encoding.python_2_unicode_compatible()`` - Alias of
``six.python_2_unicode_compatible()``.

* ``django.utils.functional.curry()`` - Use :func:`functools.partial` or
:class:`functools.partialmethod`. See
:commit:`5b1c389603a353625ae1603ba345147356336afb`.

* ``django.utils.safestring.SafeBytes`` - Unused since Django 2.0.

New default value for the ``FILE_UPLOAD_PERMISSIONS`` setting
-------------------------------------------------------------

In older versions, the :setting:`FILE_UPLOAD_PERMISSIONS` setting defaults to
``None``. With the default :setting:`FILE_UPLOAD_HANDLERS`, this results in
uploaded files having different permissions depending on their size and which
upload handler is used.

``FILE_UPLOAD_PERMISSIONS`` now defaults to ``0o644`` to avoid this
inconsistency.

New default values for security settings
----------------------------------------

To make Django projects more secure by default, some security settings now have
more secure default values:

* :setting:`X_FRAME_OPTIONS` now defaults to ``'DENY'``.

* :setting:`SECURE_CONTENT_TYPE_NOSNIFF` now defaults to ``True``.

See the *What's New* :ref:`Security section <whats-new-security-3.0>` above for
more details on these changes.

Miscellaneous
-------------

* ``ContentType.__str__()`` now includes the model's ``app_label`` to
disambiguate models with the same name in different apps.

* Because accessing the language in the session rather than in the cookie is
deprecated, ``LocaleMiddleware`` no longer looks for the user's language in
the session and :func:`django.contrib.auth.logout` no longer preserves the
session's language after logout.

* :func:`django.utils.html.escape` now uses :func:`html.escape` to escape HTML.
This converts ``'`` to ``&x27;`` instead of the previous equivalent decimal
code ``&39;``.

* The ``django-admin test -k`` option now works as the :option:`unittest
-k<unittest.-k>` option rather than as a shortcut for ``--keepdb``.

* Support for ``pywatchman`` < 1.2.0 is removed.

* :func:`~django.utils.http.urlencode` now encodes iterable values as they are
when ``doseq=False``, rather than iterating them, bringing it into line with
the standard library :func:`urllib.parse.urlencode` function.

* ``intword`` template filter now translates ``1.0`` as a singular phrase and
all other numeric values as plural. This may be incorrect for some languages.

* Assigning a value to a model's :class:`~django.db.models.ForeignKey` or
:class:`~django.db.models.OneToOneField` ``'_id'`` attribute now unsets the
corresponding field. Accessing the field afterward will result in a query.

* :func:`~django.utils.cache.patch_vary_headers` now handles an asterisk
``'*'`` according to :rfc:`7231section-7.1.4`, i.e. if a list of header
field names contains an asterisk, then the ``Vary`` header will consist of a
single asterisk ``'*'``.

* On MySQL 8.0.16+, ``PositiveIntegerField`` and ``PositiveSmallIntegerField``
now include a check constraint to prevent negative values in the database.

* ``alias=None`` is added to the signature of
:meth:`.Expression.get_group_by_cols`.

* ``RegexPattern``, used by :func:`~django.urls.re_path`, no longer returns
keyword arguments with ``None`` values to be passed to the view for the
optional named groups that are missing.

.. _deprecated-features-3.0:

Features deprecated in 3.0
==========================

``django.utils.encoding.force_text()`` and ``smart_text()``
-----------------------------------------------------------

The ``smart_text()`` and ``force_text()`` aliases (since Django 2.0) of
``smart_str()`` and ``force_str()`` are deprecated. Ignore this deprecation if
your code supports Python 2 as the behavior of ``smart_str()`` and
``force_str()`` is different there.

Miscellaneous
-------------

* ``django.utils.http.urlquote()``, ``urlquote_plus()``, ``urlunquote()``, and
``urlunquote_plus()`` are deprecated in favor of the functions that they're
aliases for: :func:`urllib.parse.quote`, :func:`~urllib.parse.quote_plus`,
:func:`~urllib.parse.unquote`, and :func:`~urllib.parse.unquote_plus`.

* ``django.utils.translation.ugettext()``, ``ugettext_lazy()``,
``ugettext_noop()``, ``ungettext()``, and ``ungettext_lazy()`` are deprecated
in favor of the functions that they're aliases for:
:func:`django.utils.translation.gettext`,
:func:`~django.utils.translation.gettext_lazy`,
:func:`~django.utils.translation.gettext_noop`,
:func:`~django.utils.translation.ngettext`, and
:func:`~django.utils.translation.ngettext_lazy`.

* To limit creation of sessions and hence favor some caching strategies,
:func:`django.views.i18n.set_language` will stop setting the user's language
in the session in Django 4.0. Since Django 2.1, the language is always stored
in the :setting:`LANGUAGE_COOKIE_NAME` cookie.

* ``django.utils.text.unescape_entities()`` is deprecated in favor of
:func:`html.unescape`. Note that unlike ``unescape_entities()``,
``html.unescape()`` evaluates lazy strings immediately.

* To avoid possible confusion as to effective scope, the private internal
utility ``is_safe_url()`` is renamed to
``url_has_allowed_host_and_scheme()``. That a URL has an allowed host and
scheme doesn't in general imply that it's "safe". It may still be quoted
incorrectly, for example. Ensure to also use
:func:`~django.utils.encoding.iri_to_uri` on the path component of untrusted
URLs.

.. _removed-features-3.0:

Features removed in 3.0
=======================

These features have reached the end of their deprecation cycle and are removed
in Django 3.0.

See :ref:`deprecated-features-2.0` for details on these changes, including how
to remove usage of these features.

* The ``django.db.backends.postgresql_psycopg2`` module is removed.

* ``django.shortcuts.render_to_response()`` is removed.

* The ``DEFAULT_CONTENT_TYPE`` setting is removed.

* ``HttpRequest.xreadlines()`` is removed.

* Support for the ``context`` argument of ``Field.from_db_value()`` and
``Expression.convert_value()`` is removed.

* The ``field_name`` keyword argument of ``QuerySet.earliest()`` and
``latest()`` is removed.

See :ref:`deprecated-features-2.1` for details on these changes, including how
to remove usage of these features.

* The ``ForceRHR`` GIS function is removed.

* ``django.utils.http.cookie_date()`` is removed.

* The ``staticfiles`` and ``admin_static`` template tag libraries are removed.

* ``django.contrib.staticfiles.templatetags.staticfiles.static()`` is removed.


===========================

2.2.28

===========================

*April 11, 2022*

Django 2.2.28 fixes two security issues with severity "high" in 2.2.27.

CVE-2022-28346: Potential SQL injection in ``QuerySet.annotate()``, ``aggregate()``, and ``extra()``
====================================================================================================

:meth:`.QuerySet.annotate`, :meth:`~.QuerySet.aggregate`, and
:meth:`~.QuerySet.extra` methods were subject to SQL injection in column
aliases, using a suitably crafted dictionary, with dictionary expansion, as the
``**kwargs`` passed to these methods.

CVE-2022-28347: Potential SQL injection via ``QuerySet.explain(**options)`` on PostgreSQL
=========================================================================================

:meth:`.QuerySet.explain` method was subject to SQL injection in option names,
using a suitably crafted dictionary, with dictionary expansion, as the
``**options`` argument.


===========================

2.2.27

Not secure
===========================

*February 1, 2022*

Django 2.2.27 fixes two security issues with severity "medium" in 2.2.26.

CVE-2022-22818: Possible XSS via ``{% debug %}`` template tag
=============================================================

The ``{% debug %}`` template tag didn't properly encode the current context,
posing an XSS attack vector.

In order to avoid this vulnerability, ``{% debug %}`` no longer outputs
information when the ``DEBUG`` setting is ``False``, and it ensures all context
variables are correctly escaped when the ``DEBUG`` setting is ``True``.

CVE-2022-23833: Denial-of-service possibility in file uploads
=============================================================

Passing certain inputs to multipart forms could result in an infinite loop when
parsing files.


===========================

2.2.26

Not secure
===========================

*January 4, 2022*

Django 2.2.26 fixes one security issue with severity "medium" and two security
issues with severity "low" in 2.2.25.

CVE-2021-45115: Denial-of-service possibility in ``UserAttributeSimilarityValidator``
=====================================================================================

:class:`.UserAttributeSimilarityValidator` incurred significant overhead
evaluating submitted password that were artificially large in relative to the
comparison values. On the assumption that access to user registration was
unrestricted this provided a potential vector for a denial-of-service attack.

In order to mitigate this issue, relatively long values are now ignored by
``UserAttributeSimilarityValidator``.

This issue has severity "medium" according to the :ref:`Django security policy
<security-disclosure>`.

CVE-2021-45116: Potential information disclosure in ``dictsort`` template filter
================================================================================

Due to leveraging the Django Template Language's variable resolution logic, the
:tfilter:`dictsort` template filter was potentially vulnerable to information
disclosure or unintended method calls, if passed a suitably crafted key.

In order to avoid this possibility, ``dictsort`` now works with a restricted
resolution logic, that will not call methods, nor allow indexing on
dictionaries.

As a reminder, all untrusted user input should be validated before use.

This issue has severity "low" according to the :ref:`Django security policy
<security-disclosure>`.

CVE-2021-45452: Potential directory-traversal via ``Storage.save()``
====================================================================

``Storage.save()`` allowed directory-traversal if directly passed suitably
crafted file names.

This issue has severity "low" according to the :ref:`Django security policy
<security-disclosure>`.


===========================

2.2.25

Not secure
===========================

*December 7, 2021*

Django 2.2.25 fixes a security issue with severity "low" in 2.2.24.

CVE-2021-44420: Potential bypass of an upstream access control based on URL paths
=================================================================================

HTTP requests for URLs with trailing newlines could bypass an upstream access
control based on URL paths.


===========================

2.2.24

Not secure
===========================

*June 2, 2021*

Django 2.2.24 fixes two security issues in 2.2.23.

CVE-2021-33203: Potential directory traversal via ``admindocs``
===============================================================

Staff members could use the :mod:`~django.contrib.admindocs`
``TemplateDetailView`` view to check the existence of arbitrary files.
Additionally, if (and only if) the default admindocs templates have been
customized by the developers to also expose the file contents, then not only
the existence but also the file contents would have been exposed.

As a mitigation, path sanitation is now applied and only files within the
template root directories can be loaded.

CVE-2021-33571: Possible indeterminate SSRF, RFI, and LFI attacks since validators accepted leading zeros in IPv4 addresses
===========================================================================================================================

:class:`~django.core.validators.URLValidator`,
:func:`~django.core.validators.validate_ipv4_address`, and
:func:`~django.core.validators.validate_ipv46_address` didn't prohibit leading
zeros in octal literals. If you used such values you could suffer from
indeterminate SSRF, RFI, and LFI attacks.

:func:`~django.core.validators.validate_ipv4_address` and
:func:`~django.core.validators.validate_ipv46_address` validators were not
affected on Python 3.9.5+.


===========================

2.2.23

Not secure
===========================

*May 13, 2021*

Django 2.2.23 fixes a regression in 2.2.21.

Bugfixes
========

* Fixed a regression in Django 2.2.21 where saving ``FileField`` would raise a
``SuspiciousFileOperation`` even when a custom
:attr:`~django.db.models.FileField.upload_to` returns a valid file path
(:ticket:`32718`).


===========================

2.2.22

Not secure
===========================

*May 6, 2021*

Django 2.2.22 fixes a security issue in 2.2.21.

CVE-2021-32052: Header injection possibility since ``URLValidator`` accepted newlines in input on Python 3.9.5+
===============================================================================================================

On Python 3.9.5+, :class:`~django.core.validators.URLValidator` didn't prohibit
newlines and tabs. If you used values with newlines in HTTP response, you could
suffer from header injection attacks. Django itself wasn't vulnerable because
:class:`~django.http.HttpResponse` prohibits newlines in HTTP headers.

Moreover, the ``URLField`` form field which uses ``URLValidator`` silently
removes newlines and tabs on Python 3.9.5+, so the possibility of newlines
entering your data only existed if you are using this validator outside of the
form fields.

This issue was introduced by the :bpo:`43882` fix.


===========================

2.2.21

Not secure
===========================

*May 4, 2021*

Django 2.2.21 fixes a security issue in 2.2.20.

CVE-2021-31542: Potential directory-traversal via uploaded files
================================================================

``MultiPartParser``, ``UploadedFile``, and ``FieldFile`` allowed
directory-traversal via uploaded files with suitably crafted file names.

In order to mitigate this risk, stricter basename and path sanitation is now
applied.


===========================

2.2.20

Not secure
===========================

*April 6, 2021*

Django 2.2.20 fixes a security issue with severity "low" in 2.2.19.

CVE-2021-28658: Potential directory-traversal via uploaded files
================================================================

``MultiPartParser`` allowed directory-traversal via uploaded files with
suitably crafted file names.

Built-in upload handlers were not affected by this vulnerability.


===========================

2.2.19

Not secure
===========================

*February 19, 2021*

Django 2.2.19 fixes a security issue in 2.2.18.

CVE-2021-23336: Web cache poisoning via ``django.utils.http.limited_parse_qsl()``
=================================================================================

Django contains a copy of :func:`urllib.parse.parse_qsl` which was added to
backport some security fixes. A further security fix has been issued recently
such that ``parse_qsl()`` no longer allows using ``;`` as a query parameter
separator by default. Django now includes this fix. See :bpo:`42967` for
further details.


===========================

2.2.18

Not secure
===========================

*February 1, 2021*

Django 2.2.18 fixes a security issue with severity "low" in 2.2.17.

CVE-2021-3281: Potential directory-traversal via ``archive.extract()``
======================================================================

The ``django.utils.archive.extract()`` function, used by
:option:`startapp --template` and :option:`startproject --template`, allowed
directory-traversal via an archive with absolute paths or relative paths with
dot segments.


===========================

2.2.17

Not secure
===========================

*November 2, 2020*

Django 2.2.17 adds compatibility with Python 3.9.


===========================

2.2.16

Not secure
===========================

*September 1, 2020*

Django 2.2.16 fixes two security issues and two data loss bugs in 2.2.15.

CVE-2020-24583: Incorrect permissions on intermediate-level directories on Python 3.7+
======================================================================================

On Python 3.7+, :setting:`FILE_UPLOAD_DIRECTORY_PERMISSIONS` mode was not
applied to intermediate-level directories created in the process of uploading
files and to intermediate-level collected static directories when using the
:djadmin:`collectstatic` management command.

You should review and manually fix permissions on existing intermediate-level
directories.

CVE-2020-24584: Permission escalation in intermediate-level directories of the file system cache on Python 3.7+
===============================================================================================================

On Python 3.7+, the intermediate-level directories of the file system cache had
the system's standard umask rather than ``0o077`` (no group or others
permissions).

Bugfixes
========

* Fixed a data loss possibility in the
:meth:`~django.db.models.query.QuerySet.select_for_update()`. When using
related fields pointing to a proxy model in the ``of`` argument, the
corresponding model was not locked (:ticket:`31866`).

* Fixed a data loss possibility, following a regression in Django 2.0, when
copying model instances with a cached fields value (:ticket:`31863`).


===========================

2.2.15

Not secure
===========================

*August 3, 2020*

Django 2.2.15 fixes two bugs in 2.2.14.

Bugfixes
========

* Allowed setting the ``SameSite`` cookie flag in
:meth:`.HttpResponse.delete_cookie` (:ticket:`31790`).

* Fixed crash when sending emails to addresses with display names longer than
75 chars on Python 3.6.11+, 3.7.8+, and 3.8.4+ (:ticket:`31784`).


===========================

2.2.14

Not secure
===========================

*July 1, 2020*

Django 2.2.14 fixes a bug in 2.2.13.

Bugfixes
========

* Fixed messages of ``InvalidCacheKey`` exceptions and ``CacheKeyWarning``
warnings raised by cache key validation (:ticket:`31654`).


===========================

2.2.13

Not secure
===========================

*June 3, 2020*

Django 2.2.13 fixes two security issues and a regression in 2.2.12.

CVE-2020-13254: Potential data leakage via malformed memcached keys
===================================================================

In cases where a memcached backend does not perform key validation, passing
malformed cache keys could result in a key collision, and potential data
leakage. In order to avoid this vulnerability, key validation is added to the
memcached cache backends.

CVE-2020-13596: Possible XSS via admin ``ForeignKeyRawIdWidget``
================================================================

Query parameters for the admin ``ForeignKeyRawIdWidget`` were not properly URL
encoded, posing an XSS attack vector. ``ForeignKeyRawIdWidget`` now
ensures query parameters are correctly URL encoded.

Bugfixes
========

* Fixed a regression in Django 2.2.12 that affected translation loading for
apps providing translations for territorial language variants as well as a
generic language, where the project has different plural equations for the
language (:ticket:`31570`).

* Tracking a jQuery security release, upgraded the version of jQuery used by
the admin from 3.3.1 to 3.5.1.


===========================

2.2.12

Not secure
===========================

*April 1, 2020*

Django 2.2.12 fixes a bug in 2.2.11.

Bugfixes
========

* Added the ability to handle ``.po`` files containing different plural
equations for the same language (:ticket:`30439`).


===========================

2.2.11

Not secure
===========================

*March 4, 2020*

Django 2.2.11 fixes a security issue and a data loss bug in 2.2.10.

CVE-2020-9402: Potential SQL injection via ``tolerance`` parameter in GIS functions and aggregates on Oracle
============================================================================================================

GIS functions and aggregates on Oracle were subject to SQL injection,
using a suitably crafted ``tolerance``.

Bugfixes
========

* Fixed a data loss possibility in the
:meth:`~django.db.models.query.QuerySet.select_for_update`. When using
related fields or parent link fields with :ref:`multi-table-inheritance` in
the ``of`` argument, the corresponding models were not locked
(:ticket:`31246`).


===========================

2.2.10

Not secure
===========================

*February 3, 2020*

Django 2.2.10 fixes a security issue in 2.2.9.

CVE-2020-7471: Potential SQL injection via ``StringAgg(delimiter)``
===================================================================

:class:`~django.contrib.postgres.aggregates.StringAgg` aggregation function was
subject to SQL injection, using a suitably crafted ``delimiter``.


==========================

2.2.9

Not secure
==========================

*December 18, 2019*

Django 2.2.9 fixes a security issue and a data loss bug in 2.2.8.

CVE-2019-19844: Potential account hijack via password reset form
================================================================

By submitting a suitably crafted email address making use of Unicode
characters, that compared equal to an existing user email when lower-cased for
comparison, an attacker could be sent a password reset token for the matched
account.

In order to avoid this vulnerability, password reset requests now compare the
submitted email using the stricter, recommended algorithm for case-insensitive
comparison of two identifiers from `Unicode Technical Report 36, section
2.11.2(B)(2)`__. Upon a match, the email containing the reset token will be
sent to the email address on record rather than the submitted address.

.. __: https://www.unicode.org/reports/tr36/#Recommendations_General

Bugfixes
========

* Fixed a data loss possibility in
:class:`~django.contrib.postgres.forms.SplitArrayField`. When using with
``ArrayField(BooleanField())``, all values after the first ``True`` value
were marked as checked instead of preserving passed values (:ticket:`31073`).


==========================

2.2.8

Not secure
==========================

*December 2, 2019*

Django 2.2.8 fixes a security issue, several bugs in 2.2.7, and adds
compatibility with Python 3.8.

CVE-2019-19118: Privilege escalation in the Django admin.
=========================================================

Since Django 2.1, a Django model admin displaying a parent model with related
model inlines, where the user has view-only permissions to a parent model but
edit permissions to the inline model, would display a read-only view of the
parent model but editable forms for the inline.

Submitting these forms would not allow direct edits to the parent model, but
would trigger the parent model's ``save()`` method, and cause pre and post-save
signal handlers to be invoked. This is a privilege escalation as a user who
lacks permission to edit a model should not be able to trigger its save-related
signals.

To resolve this issue, the permission handling code of the Django admin
interface has been changed. Now, if a user has only the "view" permission for a
parent model, the entire displayed form will not be editable, even if the user
has permission to edit models included in inlines.

This is a backwards-incompatible change, and the Django security team is aware
that some users of Django were depending on the ability to allow editing of
inlines in the admin form of an otherwise view-only parent model.

Given the complexity of the Django admin, and in-particular the permissions
related checks, it is the view of the Django security team that this change was
necessary: that it is not currently feasible to maintain the existing behavior
while escaping the potential privilege escalation in a way that would avoid a
recurrence of similar issues in the future, and that would be compatible with
Django's *safe by default* philosophy.

For the time being, developers whose applications are affected by this change
should replace the use of inlines in read-only parents with custom forms and
views that explicitly implement the desired functionality. In the longer term,
adding a documented, supported, and properly-tested mechanism for
partially-editable multi-model forms to the admin interface may occur in Django
itself.

Bugfixes
========

* Fixed a data loss possibility in the admin changelist view when a custom
:ref:`formset's prefix <formset-prefix>` contains regular expression special
characters, e.g. ``'$'`` (:ticket:`31031`).

* Fixed a regression in Django 2.2.1 that caused a crash when migrating
permissions for proxy models with a multiple database setup if the
``default`` entry was empty (:ticket:`31021`).

* Fixed a data loss possibility in the
:meth:`~django.db.models.query.QuerySet.select_for_update()`. When using
``'self'`` in the ``of`` argument with :ref:`multi-table inheritance
<multi-table-inheritance>`, a parent model was locked instead of the
queryset's model (:ticket:`30953`).


==========================

2.2.7

Not secure
==========================

*November 4, 2019*

Django 2.2.7 fixes several bugs in 2.2.6.

Bugfixes
========

* Fixed a crash when using a ``contains``, ``contained_by``, ``has_key``,
``has_keys``, or ``has_any_keys`` lookup on
``django.contrib.postgres.fields.JSONField``, if the right or left hand
side of an expression is a key transform (:ticket:`30826`).

* Prevented :option:`migrate --plan` from showing that ``RunPython`` operations
are irreversible when ``reverse_code`` callables don't have docstrings or
when showing a forward migration plan (:ticket:`30870`).

* Fixed migrations crash on PostgreSQL when adding an
:class:`~django.db.models.Index` with fields ordering and
:attr:`~.Index.opclasses` (:ticket:`30903`).

* Restored the ability to override
:meth:`~django.db.models.Model.get_FOO_display` (:ticket:`30931`).


==========================

2.2.6

Not secure
==========================

*October 1, 2019*

Django 2.2.6 fixes several bugs in 2.2.5.

Bugfixes
========

* Fixed migrations crash on SQLite when altering a model containing partial
indexes (:ticket:`30754`).

* Fixed a regression in Django 2.2.4 that caused a crash when filtering with a
``Subquery()`` annotation of a queryset containing
``django.contrib.postgres.fields.JSONField`` or
:class:`~django.contrib.postgres.fields.HStoreField` (:ticket:`30769`).


==========================

2.2.5

Not secure
==========================

*September 2, 2019*

Django 2.2.5 fixes several bugs in 2.2.4.

Bugfixes
========

* Relaxed the system check added in Django 2.2 for models to reallow use of the
same ``db_table`` by multiple models when database routers are installed
(:ticket:`30673`).

* Fixed crash of ``KeyTransform()`` for
``django.contrib.postgres.fields.JSONField`` and
:class:`~django.contrib.postgres.fields.HStoreField` when using on
expressions with params (:ticket:`30672`).

* Fixed a regression in Django 2.2 where
:attr:`ModelAdmin.list_filter <django.contrib.admin.ModelAdmin.list_filter>`
choices to foreign objects don't respect a model's ``Meta.ordering``
(:ticket:`30449`).


==========================

2.2.4

Not secure
==========================

*August 1, 2019*

Django 2.2.4 fixes security issues and several bugs in 2.2.3.

CVE-2019-14232: Denial-of-service possibility in ``django.utils.text.Truncator``
================================================================================

If ``django.utils.text.Truncator``'s ``chars()`` and ``words()`` methods
were passed the ``html=True`` argument, they were extremely slow to evaluate
certain inputs due to a catastrophic backtracking vulnerability in a regular
expression. The ``chars()`` and ``words()`` methods are used to implement the
:tfilter:`truncatechars_html` and :tfilter:`truncatewords_html` template
filters, which were thus vulnerable.

The regular expressions used by ``Truncator`` have been simplified in order to
avoid potential backtracking issues. As a consequence, trailing punctuation may
now at times be included in the truncated output.

CVE-2019-14233: Denial-of-service possibility in ``strip_tags()``
=================================================================

Due to the behavior of the underlying ``HTMLParser``,
:func:`django.utils.html.strip_tags` would be extremely slow to evaluate
certain inputs containing large sequences of nested incomplete HTML entities.
The ``strip_tags()`` method is used to implement the corresponding
:tfilter:`striptags` template filter, which was thus also vulnerable.

``strip_tags()`` now avoids recursive calls to ``HTMLParser`` when progress
removing tags, but necessarily incomplete HTML entities, stops being made.

Remember that absolutely NO guarantee is provided about the results of
``strip_tags()`` being HTML safe. So NEVER mark safe the result of a
``strip_tags()`` call without escaping it first, for example with
:func:`django.utils.html.escape`.

CVE-2019-14234: SQL injection possibility in key and index lookups for ``JSONField``/``HStoreField``
====================================================================================================

:lookup:`Key and index lookups <jsonfield.key>` for
``django.contrib.postgres.fields.JSONField`` and :lookup:`key lookups
<hstorefield.key>` for :class:`~django.contrib.postgres.fields.HStoreField`
were subject to SQL injection, using a suitably crafted dictionary, with
dictionary expansion, as the ``**kwargs`` passed to ``QuerySet.filter()``.

CVE-2019-14235: Potential memory exhaustion in ``django.utils.encoding.uri_to_iri()``
=====================================================================================

If passed certain inputs, :func:`django.utils.encoding.uri_to_iri` could lead
to significant memory usage due to excessive recursion when re-percent-encoding
invalid UTF-8 octet sequences.

``uri_to_iri()`` now avoids recursion when re-percent-encoding invalid UTF-8
octet sequences.

Bugfixes
========

* Fixed a regression in Django 2.2 when ordering a ``QuerySet.union()``,
``intersection()``, or ``difference()`` by a field type present more than
once results in the wrong ordering being used (:ticket:`30628`).

* Fixed a migration crash on PostgreSQL when adding a check constraint
with a ``contains`` lookup on
:class:`~django.contrib.postgres.fields.DateRangeField` or
:class:`~django.contrib.postgres.fields.DateTimeRangeField`, if the right
hand side of an expression is the same type (:ticket:`30621`).

* Fixed a regression in Django 2.2 where auto-reloader crashes if a file path
contains nulls characters (``'\x00'``) (:ticket:`30506`).

* Fixed a regression in Django 2.2 where auto-reloader crashes if a translation
directory cannot be resolved (:ticket:`30647`).


==========================

2.2.3

Not secure
==========================

*July 1, 2019*

Django 2.2.3 fixes a security issue and several bugs in 2.2.2. Also, the latest
string translations from Transifex are incorporated.

CVE-2019-12781: Incorrect HTTP detection with reverse-proxy connecting via HTTPS
--------------------------------------------------------------------------------

When deployed behind a reverse-proxy connecting to Django via HTTPS,
:attr:`django.http.HttpRequest.scheme` would incorrectly detect client
requests made via HTTP as using HTTPS. This entails incorrect results for
:meth:`~django.http.HttpRequest.is_secure`, and
:meth:`~django.http.HttpRequest.build_absolute_uri`, and that HTTP
requests would not be redirected to HTTPS in accordance with
:setting:`SECURE_SSL_REDIRECT`.

``HttpRequest.scheme`` now respects :setting:`SECURE_PROXY_SSL_HEADER`, if it is
configured, and the appropriate header is set on the request, for both HTTP and
HTTPS requests.

If you deploy Django behind a reverse-proxy that forwards HTTP requests, and
that connects to Django via HTTPS, be sure to verify that your application
correctly handles code paths relying on ``scheme``, ``is_secure()``,
``build_absolute_uri()``, and ``SECURE_SSL_REDIRECT``.

Bugfixes
========

* Fixed a regression in Django 2.2 where :class:`~django.db.models.Avg`,
:class:`~django.db.models.StdDev`, and :class:`~django.db.models.Variance`
crash with ``filter`` argument (:ticket:`30542`).

* Fixed a regression in Django 2.2.2 where auto-reloader crashes with
``AttributeError``, e.g. when using ``ipdb`` (:ticket:`30588`).


==========================

2.2.2

Not secure
==========================

*June 3, 2019*

Django 2.2.2 fixes security issues and several bugs in 2.2.1.

CVE-2019-12308: AdminURLFieldWidget XSS
---------------------------------------

The clickable "Current URL" link generated by ``AdminURLFieldWidget`` displayed
the provided value without validating it as a safe URL. Thus, an unvalidated
value stored in the database, or a value provided as a URL query parameter
payload, could result in an clickable JavaScript link.

``AdminURLFieldWidget`` now validates the provided value using
:class:`~django.core.validators.URLValidator` before displaying the clickable
link. You may customize the validator by passing a ``validator_class`` kwarg to
``AdminURLFieldWidget.__init__()``, e.g. when using
:attr:`~django.contrib.admin.ModelAdmin.formfield_overrides`.

Patched bundled jQuery for CVE-2019-11358: Prototype pollution
--------------------------------------------------------------

jQuery before 3.4.0, mishandles ``jQuery.extend(true, {}, ...)`` because of
``Object.prototype`` pollution. If an unsanitized source object contained an
enumerable ``__proto__`` property, it could extend the native
``Object.prototype``.

The bundled version of jQuery used by the Django admin has been patched to
allow for the ``select2`` library's use of ``jQuery.extend()``.

Bugfixes
========

* Fixed a regression in Django 2.2 that stopped Show/Hide toggles working on
dynamically added admin inlines (:ticket:`30459`).

* Fixed a regression in Django 2.2 where deprecation message crashes if
``Meta.ordering`` contains an expression (:ticket:`30463`).

* Fixed a regression in Django 2.2.1 where
:class:`~django.contrib.postgres.search.SearchVector` generates SQL with a
redundant ``Coalesce`` call (:ticket:`30488`).

* Fixed a regression in Django 2.2 where auto-reloader doesn't detect changes
in ``manage.py`` file when using ``StatReloader`` (:ticket:`30479`).

* Fixed crash of :class:`~django.contrib.postgres.aggregates.ArrayAgg` and
:class:`~django.contrib.postgres.aggregates.StringAgg` with ``ordering``
argument when used in a ``Subquery`` (:ticket:`30315`).

* Fixed a regression in Django 2.2 that caused a crash of auto-reloader when
an exception with custom signature is raised (:ticket:`30516`).

* Fixed a regression in Django 2.2.1 where auto-reloader unnecessarily reloads
translation files multiple times when using ``StatReloader``
(:ticket:`30523`).


==========================

2.2.1

Not secure
==========================

*May 1, 2019*

Django 2.2.1 fixes several bugs in 2.2.

Bugfixes
========

* Fixed a regression in Django 2.1 that caused the incorrect quoting of
database user password when using :djadmin:`dbshell` on Oracle
(:ticket:`30307`).

* Added compatibility for ``psycopg2`` 2.8 (:ticket:`30331`).

* Fixed a regression in Django 2.2 that caused a crash when loading the
template for the technical 500 debug page (:ticket:`30324`).

* Fixed crash of ``ordering`` argument in
:class:`~django.contrib.postgres.aggregates.ArrayAgg` and
:class:`~django.contrib.postgres.aggregates.StringAgg` when it contains an
expression with params (:ticket:`30332`).

* Fixed a regression in Django 2.2 that caused a single instance fast-delete
to not set the primary key to ``None`` (:ticket:`30330`).

* Prevented :djadmin:`makemigrations` from generating infinite migrations for
check constraints and partial indexes when ``condition`` contains
a :class:`~python:range` object (:ticket:`30350`).

* Reverted an optimization in Django 2.2 (:ticket:`29725`) that caused the
inconsistent behavior of ``count()`` and ``exists()`` on a reverse
many-to-many relationship with a custom manager (:ticket:`30325`).

* Fixed a regression in Django 2.2 where
:class:`~django.core.paginator.Paginator` crashes if ``object_list`` is
a queryset ordered or aggregated over a nested ``JSONField`` key transform
(:ticket:`30335`).

* Fixed a regression in Django 2.2 where ``IntegerField`` validation of
database limits crashes if ``limit_value`` attribute in a custom validator is
callable (:ticket:`30328`).

* Fixed a regression in Django 2.2 where
:class:`~django.contrib.postgres.search.SearchVector` generates SQL that is
not indexable (:ticket:`30385`).

* Fixed a regression in Django 2.2 that caused an exception to be raised when
a custom error handler could not be imported (:ticket:`30318`).

* Relaxed the system check added in Django 2.2 for the admin app's dependencies
to reallow use of
:class:`~django.contrib.sessions.middleware.SessionMiddleware` subclasses,
rather than requiring :mod:`django.contrib.sessions` to be in
:setting:`INSTALLED_APPS` (:ticket:`30312`).

* Increased the default timeout when using ``Watchman`` to 5 seconds to prevent
falling back to ``StatReloader`` on larger projects and made it customizable
via the :envvar:`DJANGO_WATCHMAN_TIMEOUT` environment variable
(:ticket:`30361`).

* Fixed a regression in Django 2.2 that caused a crash when migrating
permissions for proxy models if the target permissions already existed. For
example, when a permission had been created manually or a model had been
migrated from concrete to proxy (:ticket:`30351`).

* Fixed a regression in Django 2.2 that caused a crash of :djadmin:`runserver`
when URLConf modules raised exceptions (:ticket:`30323`).

* Fixed a regression in Django 2.2 where changes were not reliably detected by
auto-reloader when using ``StatReloader`` (:ticket:`30323`).

* Fixed a migration crash on Oracle and PostgreSQL when adding a check
constraint with a ``contains``, ``startswith``, or ``endswith`` lookup (or
their case-insensitive variant) (:ticket:`30408`).

* Fixed a migration crash on Oracle and SQLite when adding a check constraint
with ``condition`` contains ``|`` (``OR``) operator (:ticket:`30412`).


========================

2.2

Not secure
========================

*April 1, 2019*

Welcome to Django 2.2!

These release notes cover the :ref:`new features <whats-new-2.2>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-2.2>` you'll
want to be aware of when upgrading from Django 2.1 or earlier. We've
:ref:`begun the deprecation process for some features
<deprecated-features-2.2>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Django 2.2 is designated as a :term:`long-term support release
<Long-term support release>`. It will receive security updates for at least
three years after its release. Support for the previous LTS, Django 1.11, will
end in April 2020.

Python compatibility
====================

Django 2.2 supports Python 3.5, 3.6, 3.7, 3.8 (as of 2.2.8), and 3.9 (as of
2.2.17). We **highly recommend** and only officially support the latest release
of each series.

.. _whats-new-2.2:

What's new in Django 2.2
========================

Constraints
-----------

The new :class:`~django.db.models.CheckConstraint` and
:class:`~django.db.models.UniqueConstraint` classes enable adding custom
database constraints. Constraints are added to models using the
:attr:`Meta.constraints <django.db.models.Options.constraints>` option.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Added a CSS class to the column headers of
:class:`~django.contrib.admin.TabularInline`.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* The ``HttpRequest`` is now passed as the first positional argument to
:meth:`.RemoteUserBackend.configure_user`, if it accepts it.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* Added Oracle support for the
:class:`~django.contrib.gis.db.models.functions.Envelope` function.

* Added SpatiaLite support for the :lookup:`coveredby` and :lookup:`covers`
lookups.

:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new ``ordering`` argument for
:class:`~django.contrib.postgres.aggregates.ArrayAgg` and
:class:`~django.contrib.postgres.aggregates.StringAgg` determines the
ordering of the aggregated elements.

* The new :class:`~django.contrib.postgres.indexes.BTreeIndex`,
:class:`~django.contrib.postgres.indexes.HashIndex` and
:class:`~django.contrib.postgres.indexes.SpGistIndex` classes allow
creating ``B-Tree``, ``hash``, and ``SP-GiST`` indexes in the database.

* :class:`~django.contrib.postgres.indexes.BrinIndex` now has the
``autosummarize`` parameter.

* The new ``search_type`` parameter of
:class:`~django.contrib.postgres.search.SearchQuery` allows searching for
a phrase or raw expression.

:mod:`django.contrib.staticfiles`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Added path matching to the :option:`collectstatic --ignore` option so that
patterns like ``/vendor/*.js`` can be used.

Database backends
~~~~~~~~~~~~~~~~~

* Added result streaming for :meth:`.QuerySet.iterator` on SQLite.

Generic Views
~~~~~~~~~~~~~

* The new :meth:`View.setup <django.views.generic.base.View.setup>` hook
initializes view attributes before calling
:meth:`~django.views.generic.base.View.dispatch`. It allows mixins to set up
instance attributes for reuse in child classes.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* Added support and translations for the Armenian language.

Management Commands
~~~~~~~~~~~~~~~~~~~

* The new :option:`--force-color` option forces colorization of the command
output.

* :djadmin:`inspectdb` now creates models for foreign tables on PostgreSQL.

* :option:`inspectdb --include-views` now creates models for materialized views
on Oracle and PostgreSQL.

* The new :option:`inspectdb --include-partitions` option allows creating
models for partition tables on PostgreSQL. In older versions, models are
created child tables instead the parent.

* :djadmin:`inspectdb` now introspects :class:`~django.db.models.DurationField`
for Oracle and PostgreSQL, and :class:`~django.db.models.AutoField` for
SQLite.

* On Oracle, :djadmin:`dbshell` is wrapped with ``rlwrap``, if available.
``rlwrap`` provides a command history and editing of keyboard input.

* The new :option:`makemigrations --no-header` option avoids writing header
comments in generated migration file(s). This option is also available for
:djadmin:`squashmigrations`.

* :djadmin:`runserver` can now use `Watchman
<https://facebook.github.io/watchman/>`_ to improve the performance of
watching a large number of files for changes.

Migrations
~~~~~~~~~~

* The new :option:`migrate --plan` option prints the list of migration
operations that will be performed.

* ``NoneType`` can now be serialized in migrations.

* You can now :ref:`register custom serializers <custom-migration-serializers>`
for migrations.

Models
~~~~~~

* Added support for PostgreSQL operator classes (:attr:`.Index.opclasses`).

* Added support for partial indexes (:attr:`.Index.condition`).

* Added the :class:`~django.db.models.functions.NullIf` and
:class:`~django.db.models.functions.Reverse` database functions, as well as
many :ref:`math database functions <math-functions>`.

* Setting the new ``ignore_conflicts`` parameter of
:meth:`.QuerySet.bulk_create` to ``True`` tells the database to ignore
failure to insert rows that fail uniqueness constraints or other checks.

* The new :class:`~django.db.models.functions.ExtractIsoYear` function extracts
ISO-8601 week-numbering years from :class:`~django.db.models.DateField` and
:class:`~django.db.models.DateTimeField`, and the new :lookup:`iso_year`
lookup allows querying by an ISO-8601 week-numbering year.

* The new :meth:`.QuerySet.bulk_update` method allows efficiently updating
specific fields on multiple model instances.

* Django no longer always starts a transaction when a single query is being
performed, such as ``Model.save()``, ``QuerySet.update()``, and
``Model.delete()``. This improves the performance of autocommit by reducing
the number of database round trips.

* Added SQLite support for the :class:`~django.db.models.StdDev` and
:class:`~django.db.models.Variance` functions.

* The handling of ``DISTINCT`` aggregation is added to the
:class:`~django.db.models.Aggregate` class. Adding :attr:`allow_distinct =
True <django.db.models.Aggregate.allow_distinct>` as a class attribute on
``Aggregate`` subclasses allows a ``distinct`` keyword argument to be
specified on initialization to ensure that the aggregate function is only
called for each distinct value of ``expressions``.

* The :meth:`.RelatedManager.add`, :meth:`~.RelatedManager.create`,
:meth:`~.RelatedManager.remove`, :meth:`~.RelatedManager.set`,
``get_or_create()``, and ``update_or_create()`` methods are now allowed on
many-to-many relationships with intermediate models. The new
``through_defaults`` argument is used to specify values for new intermediate
model instance(s).

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* Added :attr:`.HttpRequest.headers` to allow simple access to a request's
headers.

Serialization
~~~~~~~~~~~~~

* You can now deserialize data using natural keys containing :ref:`forward
references <natural-keys-and-forward-references>` by passing
``handle_forward_references=True`` to ``serializers.deserialize()``.
Additionally, :djadmin:`loaddata` handles forward references automatically.

Tests
~~~~~

* The new :meth:`.SimpleTestCase.assertURLEqual` assertion checks for a given
URL, ignoring the ordering of the query string.
:meth:`~.SimpleTestCase.assertRedirects` uses the new assertion.

* The test :class:`~.django.test.Client` now supports automatic JSON
serialization of list and tuple ``data`` when
``content_type='application/json'``.

* The new :setting:`ORACLE_MANAGED_FILES <TEST_ORACLE_MANAGED_FILES>` test
database setting allows using Oracle Managed Files (OMF) tablespaces.

* Deferrable database constraints are now checked at the end of each
:class:`~django.test.TestCase` test on SQLite 3.20+, just like on other
backends that support deferrable constraints. These checks aren't implemented
for older versions of SQLite because they would require expensive table
introspection there.

* :class:`~django.test.runner.DiscoverRunner` now skips the setup of databases
not :ref:`referenced by tests<testing-multi-db>`.

URLs
~~~~

* The new :attr:`.ResolverMatch.route` attribute stores the route of the
matching URL pattern.

Validators
~~~~~~~~~~

* :class:`.MaxValueValidator`, :class:`.MinValueValidator`,
:class:`.MinLengthValidator`, and :class:`.MaxLengthValidator` now accept
a callable ``limit_value``.

.. _backwards-incompatible-2.2:

Backwards incompatible changes in 2.2
=====================================

Database backend API
--------------------

This section describes changes that may be needed in third-party database
backends.

* Third-party database backends must implement support for table check
constraints or set ``DatabaseFeatures.supports_table_check_constraints`` to
``False``.

* Third party database backends must implement support for ignoring
constraints or uniqueness errors while inserting or set
``DatabaseFeatures.supports_ignore_conflicts`` to ``False``.

* Third party database backends must implement introspection for
``DurationField`` or set ``DatabaseFeatures.can_introspect_duration_field``
to ``False``.

* ``DatabaseFeatures.uses_savepoints`` now defaults to ``True``.

* Third party database backends must implement support for partial indexes or
set ``DatabaseFeatures.supports_partial_indexes`` to ``False``.

* ``DatabaseIntrospection.table_name_converter()`` and
``column_name_converter()`` are removed. Third party database backends may
need to instead implement ``DatabaseIntrospection.identifier_converter()``.
In that case, the constraint names that
``DatabaseIntrospection.get_constraints()`` returns must be normalized by
``identifier_converter()``.

* SQL generation for indexes is moved from :class:`~django.db.models.Index` to
``SchemaEditor`` and these ``SchemaEditor`` methods are added:

* ``_create_primary_key_sql()`` and ``_delete_primary_key_sql()``
* ``_delete_index_sql()`` (to pair with ``_create_index_sql()``)
* ``_delete_unique_sql`` (to pair with ``_create_unique_sql()``)
* ``_delete_fk_sql()`` (to pair with ``_create_fk_sql()``)
* ``_create_check_sql()`` and ``_delete_check_sql()``

* The third argument of ``DatabaseWrapper.__init__()``,
``allow_thread_sharing``, is removed.

Admin actions are no longer collected from base ``ModelAdmin`` classes
----------------------------------------------------------------------

For example, in older versions of Django::

from django.contrib import admin

class BaseAdmin(admin.ModelAdmin):
actions = ['a']

class SubAdmin(BaseAdmin):
actions = ['b']

``SubAdmin`` would have actions ``'a'`` and ``'b'``.

Now ``actions`` follows standard Python inheritance. To get the same result as
before::

class SubAdmin(BaseAdmin):
actions = BaseAdmin.actions + ['b']

:mod:`django.contrib.gis`
-------------------------

* Support for GDAL 1.9 and 1.10 is dropped.

``TransactionTestCase`` serialized data loading
-----------------------------------------------

Initial data migrations are now loaded in
:class:`~django.test.TransactionTestCase` at the end of the test, after the
database flush. In older versions, this data was loaded at the beginning of the
test, but this prevents the :option:`test --keepdb` option from working
properly (the database was empty at the end of the whole test suite). This
change shouldn't have an impact on your tests unless you've customized
:class:`~django.test.TransactionTestCase`'s internals.

``sqlparse`` is required dependency
-----------------------------------

To simplify a few parts of Django's database handling, `sqlparse 0.2.2+
<https://pypi.org/project/sqlparse/>`_ is now a required dependency. It's
automatically installed along with Django.

``cached_property`` aliases
---------------------------

In usage like::

from django.utils.functional import cached_property

class A:

cached_property
def base(self):
return ...

alias = base

``alias`` is not cached. Where the problem can be detected (Python 3.6 and
later), such usage now raises ``TypeError: Cannot assign the same
cached_property to two different names ('base' and 'alias').``

Use this instead::

import operator

class A:

...

alias = property(operator.attrgetter('base'))

Permissions for proxy models
----------------------------

:ref:`Permissions for proxy models <proxy-models-permissions-topic>` are now
created using the content type of the proxy model rather than the content type
of the concrete model. A migration will update existing permissions when you
run :djadmin:`migrate`.

In the admin, the change is transparent for proxy models having the same
``app_label`` as their concrete model. However, in older versions, users with
permissions for a proxy model with a *different* ``app_label`` than its
concrete model couldn't access the model in the admin. That's now fixed, but
you might want to audit the permissions assignments for such proxy models
(``[add|view|change|delete]_myproxy``) prior to upgrading to ensure the new
access is appropriate.

Finally, proxy model permission strings must be updated to use their own
``app_label``. For example, for ``app.MyProxyModel`` inheriting from
``other_app.ConcreteModel``, update
``user.has_perm('other_app.add_myproxymodel')`` to
``user.has_perm('app.add_myproxymodel')``.

Merging of form ``Media`` assets
--------------------------------

Form ``Media`` assets are now merged using a topological sort algorithm, as the
old pairwise merging algorithm is insufficient for some cases. CSS and
JavaScript files which don't include their dependencies may now be sorted
incorrectly (where the old algorithm produced results correctly by
coincidence).

Audit all ``Media`` classes for any missing dependencies. For example,
widgets depending on ``django.jQuery`` must specify
``js=['admin/js/jquery.init.js', ...]`` when :ref:`declaring form media assets
<assets-as-a-static-definition>`.

Miscellaneous
-------------

* To improve readability, the ``UUIDField`` form field now displays values with
dashes, e.g. ``550e8400-e29b-41d4-a716-446655440000`` instead of
``550e8400e29b41d4a716446655440000``.

* On SQLite, ``PositiveIntegerField`` and ``PositiveSmallIntegerField`` now
include a check constraint to prevent negative values in the database. If you
have existing invalid data and run a migration that recreates a table, you'll
see ``CHECK constraint failed``.

* For consistency with WSGI servers, the test client now sets the
``Content-Length`` header to a string rather than an integer.

* The return value of :func:`django.utils.text.slugify` is no longer marked as
HTML safe.

* The default truncation character used by the :tfilter:`urlizetrunc`,
:tfilter:`truncatechars`, :tfilter:`truncatechars_html`,
:tfilter:`truncatewords`, and :tfilter:`truncatewords_html` template filters
is now the real ellipsis character (``…``) instead of 3 dots. You may have to
adapt some test output comparisons.

* Support for bytestring paths in the template filesystem loader is removed.

* :func:`django.utils.http.urlsafe_base64_encode` now returns a string instead
of a bytestring, and :func:`django.utils.http.urlsafe_base64_decode` may no
longer be passed a bytestring.

* Support for ``cx_Oracle`` < 6.0 is removed.

* The minimum supported version of ``mysqlclient`` is increased from 1.3.7 to
1.3.13.

* The minimum supported version of SQLite is increased from 3.7.15 to 3.8.3.

* In an attempt to provide more semantic query data, ``NullBooleanSelect`` now
renders ``<option>`` values of ``unknown``, ``true``, and ``false`` instead
of ``1``, ``2``, and ``3``. For backwards compatibility, the old values are
still accepted as data.

* :attr:`Group.name <django.contrib.auth.models.Group.name>` ``max_length``
is increased from 80 to 150 characters.

* Tests that violate deferrable database constraints now error when run on
SQLite 3.20+, just like on other backends that support such constraints.

* To catch usage mistakes, the test :class:`~django.test.Client` and
:func:`django.utils.http.urlencode` now raise ``TypeError`` if ``None`` is
passed as a value to encode because ``None`` can't be encoded in GET and POST
data. Either pass an empty string or omit the value.

* The :djadmin:`ping_google` management command now defaults to ``https``
instead of ``http`` for the sitemap's URL. If your site uses http, use the
new :option:`ping_google --sitemap-uses-http` option. If you use the
:func:`~django.contrib.sitemaps.ping_google` function, set the new
``sitemap_uses_https`` argument to ``False``.

* :djadmin:`runserver` no longer supports ``pyinotify`` (replaced by Watchman).

* The :class:`~django.db.models.Avg`, :class:`~django.db.models.StdDev`, and
:class:`~django.db.models.Variance` aggregate functions now return a
``Decimal`` instead of a ``float`` when the input is ``Decimal``.

* Tests will fail on SQLite if apps without migrations have relations to apps
with migrations. This has been a documented restriction since migrations were
added in Django 1.7, but it fails more reliably now. You'll see tests failing
with errors like ``no such table: <app_label>_<model>``. This was observed
with several third-party apps that had models in tests without migrations.
You must add migrations for such models.

* Providing an integer in the ``key`` argument of the :meth:`.cache.delete` or
:meth:`.cache.get` now raises :exc:`ValueError`.

* Plural equations for some languages are changed, because the latest versions
from Transifex are incorporated.

.. note::

The ability to handle ``.po`` files containing different plural equations
for the same language was added in Django 2.2.12.

.. _deprecated-features-2.2:

Features deprecated in 2.2
==========================

Model ``Meta.ordering`` will no longer affect ``GROUP BY`` queries
------------------------------------------------------------------

A model's ``Meta.ordering`` affecting ``GROUP BY`` queries (such as
``.annotate().values()``) is a common source of confusion. Such queries now
issue a deprecation warning with the advice to add an ``order_by()`` to retain
the current query. ``Meta.ordering`` will be ignored in such queries starting
in Django 3.1.

Miscellaneous
-------------

* ``django.utils.timezone.FixedOffset`` is deprecated in favor of
:class:`datetime.timezone`.

* The undocumented ``QuerySetPaginator`` alias of
``django.core.paginator.Paginator`` is deprecated.

* The ``FloatRangeField`` model and form fields in ``django.contrib.postgres``
are deprecated in favor of a new name, ``DecimalRangeField``, to match the
underlying ``numrange`` data type used in the database.

* The ``FILE_CHARSET`` setting is deprecated. Starting with Django 3.1, files
read from disk must be UTF-8 encoded.

* ``django.contrib.staticfiles.storage.CachedStaticFilesStorage`` is
deprecated due to the intractable problems that it has. Use
:class:`.ManifestStaticFilesStorage` or a third-party cloud storage instead.

* :meth:`.RemoteUserBackend.configure_user` is now passed ``request`` as the
first positional argument, if it accepts it. Support for overrides that don't
accept it will be removed in Django 3.1.

* The ``SimpleTestCase.allow_database_queries``,
``TransactionTestCase.multi_db``, and ``TestCase.multi_db``
attributes are deprecated in favor of :attr:`.SimpleTestCase.databases`,
:attr:`.TransactionTestCase.databases`, and :attr:`.TestCase.databases`.
These new attributes allow databases dependencies to be declared in order to
prevent unexpected queries against non-default databases to leak state
between tests. The previous behavior of ``allow_database_queries=True`` and
``multi_db=True`` can be achieved by setting ``databases='__all__'``.


===========================

2.1.15

Not secure
===========================

*December 2, 2019*

Django 2.1.15 fixes a security issue and a data loss bug in 2.1.14.

CVE-2019-19118: Privilege escalation in the Django admin.
=========================================================

Since Django 2.1, a Django model admin displaying a parent model with related
model inlines, where the user has view-only permissions to a parent model but
edit permissions to the inline model, would display a read-only view of the
parent model but editable forms for the inline.

Submitting these forms would not allow direct edits to the parent model, but
would trigger the parent model's ``save()`` method, and cause pre and post-save
signal handlers to be invoked. This is a privilege escalation as a user who
lacks permission to edit a model should not be able to trigger its save-related
signals.

To resolve this issue, the permission handling code of the Django admin
interface has been changed. Now, if a user has only the "view" permission for a
parent model, the entire displayed form will not be editable, even if the user
has permission to edit models included in inlines.

This is a backwards-incompatible change, and the Django security team is aware
that some users of Django were depending on the ability to allow editing of
inlines in the admin form of an otherwise view-only parent model.

Given the complexity of the Django admin, and in-particular the permissions
related checks, it is the view of the Django security team that this change was
necessary: that it is not currently feasible to maintain the existing behavior
while escaping the potential privilege escalation in a way that would avoid a
recurrence of similar issues in the future, and that would be compatible with
Django's *safe by default* philosophy.

For the time being, developers whose applications are affected by this change
should replace the use of inlines in read-only parents with custom forms and
views that explicitly implement the desired functionality. In the longer term,
adding a documented, supported, and properly-tested mechanism for
partially-editable multi-model forms to the admin interface may occur in Django
itself.

Bugfixes
========

* Fixed a data loss possibility in the
:meth:`~django.db.models.query.QuerySet.select_for_update()`. When using
``'self'`` in the ``of`` argument with :ref:`multi-table inheritance
<multi-table-inheritance>`, a parent model was locked instead of the
queryset's model (:ticket:`30953`).


===========================

2.1.14

Not secure
===========================

*November 4, 2019*

Django 2.1.14 fixes a regression in 2.1.13.

Bugfixes
========

* Fixed a crash when using a ``contains``, ``contained_by``, ``has_key``,
``has_keys``, or ``has_any_keys`` lookup on
``django.contrib.postgres.fields.JSONField``, if the right or left hand
side of an expression is a key transform (:ticket:`30826`).


===========================

2.1.13

Not secure
===========================

*October 1, 2019*

Django 2.1.13 fixes a regression in 2.1.11.

Bugfixes
========

* Fixed a crash when filtering with a ``Subquery()`` annotation of a queryset
containing ``django.contrib.postgres.fields.JSONField`` or
:class:`~django.contrib.postgres.fields.HStoreField` (:ticket:`30769`).


===========================

2.1.12

Not secure
===========================

*September 2, 2019*

Django 2.1.12 fixes a regression in 2.1.11.

Bugfixes
========

* Fixed crash of ``KeyTransform()`` for
``django.contrib.postgres.fields.JSONField`` and
:class:`~django.contrib.postgres.fields.HStoreField` when using on
expressions with params (:ticket:`30672`).


===========================

2.1.11

Not secure
===========================

*August 1, 2019*

Django 2.1.11 fixes security issues in 2.1.10.

CVE-2019-14232: Denial-of-service possibility in ``django.utils.text.Truncator``
================================================================================

If ``django.utils.text.Truncator``'s ``chars()`` and ``words()`` methods
were passed the ``html=True`` argument, they were extremely slow to evaluate
certain inputs due to a catastrophic backtracking vulnerability in a regular
expression. The ``chars()`` and ``words()`` methods are used to implement the
:tfilter:`truncatechars_html` and :tfilter:`truncatewords_html` template
filters, which were thus vulnerable.

The regular expressions used by ``Truncator`` have been simplified in order to
avoid potential backtracking issues. As a consequence, trailing punctuation may
now at times be included in the truncated output.

CVE-2019-14233: Denial-of-service possibility in ``strip_tags()``
=================================================================

Due to the behavior of the underlying ``HTMLParser``,
:func:`django.utils.html.strip_tags` would be extremely slow to evaluate
certain inputs containing large sequences of nested incomplete HTML entities.
The ``strip_tags()`` method is used to implement the corresponding
:tfilter:`striptags` template filter, which was thus also vulnerable.

``strip_tags()`` now avoids recursive calls to ``HTMLParser`` when progress
removing tags, but necessarily incomplete HTML entities, stops being made.

Remember that absolutely NO guarantee is provided about the results of
``strip_tags()`` being HTML safe. So NEVER mark safe the result of a
``strip_tags()`` call without escaping it first, for example with
:func:`django.utils.html.escape`.

CVE-2019-14234: SQL injection possibility in key and index lookups for ``JSONField``/``HStoreField``
====================================================================================================

:lookup:`Key and index lookups <jsonfield.key>` for
``django.contrib.postgres.fields.JSONField`` and :lookup:`key lookups
<hstorefield.key>` for :class:`~django.contrib.postgres.fields.HStoreField`
were subject to SQL injection, using a suitably crafted dictionary, with
dictionary expansion, as the ``**kwargs`` passed to ``QuerySet.filter()``.

CVE-2019-14235: Potential memory exhaustion in ``django.utils.encoding.uri_to_iri()``
=====================================================================================

If passed certain inputs, :func:`django.utils.encoding.uri_to_iri` could lead
to significant memory usage due to excessive recursion when re-percent-encoding
invalid UTF-8 octet sequences.

``uri_to_iri()`` now avoids recursion when re-percent-encoding invalid UTF-8
octet sequences.


===========================

2.1.10

Not secure
===========================

*July 1, 2019*

Django 2.1.10 fixes a security issue in 2.1.9.

CVE-2019-12781: Incorrect HTTP detection with reverse-proxy connecting via HTTPS
--------------------------------------------------------------------------------

When deployed behind a reverse-proxy connecting to Django via HTTPS,
:attr:`django.http.HttpRequest.scheme` would incorrectly detect client
requests made via HTTP as using HTTPS. This entails incorrect results for
:meth:`~django.http.HttpRequest.is_secure`, and
:meth:`~django.http.HttpRequest.build_absolute_uri`, and that HTTP
requests would not be redirected to HTTPS in accordance with
:setting:`SECURE_SSL_REDIRECT`.

``HttpRequest.scheme`` now respects :setting:`SECURE_PROXY_SSL_HEADER`, if it
is configured, and the appropriate header is set on the request, for both HTTP
and HTTPS requests.

If you deploy Django behind a reverse-proxy that forwards HTTP requests, and
that connects to Django via HTTPS, be sure to verify that your application
correctly handles code paths relying on ``scheme``, ``is_secure()``,
``build_absolute_uri()``, and ``SECURE_SSL_REDIRECT``.


============================

2.1.9

Not secure
============================

*June 3, 2019*

Django 2.1.9 fixes security issues in 2.1.8.

CVE-2019-12308: AdminURLFieldWidget XSS
---------------------------------------

The clickable "Current URL" link generated by ``AdminURLFieldWidget`` displayed
the provided value without validating it as a safe URL. Thus, an unvalidated
value stored in the database, or a value provided as a URL query parameter
payload, could result in an clickable JavaScript link.

``AdminURLFieldWidget`` now validates the provided value using
:class:`~django.core.validators.URLValidator` before displaying the clickable
link. You may customize the validator by passing a ``validator_class`` kwarg to
``AdminURLFieldWidget.__init__()``, e.g. when using
:attr:`~django.contrib.admin.ModelAdmin.formfield_overrides`.

Patched bundled jQuery for CVE-2019-11358: Prototype pollution
--------------------------------------------------------------

jQuery before 3.4.0, mishandles ``jQuery.extend(true, {}, ...)`` because of
``Object.prototype`` pollution. If an unsanitized source object contained an
enumerable ``__proto__`` property, it could extend the native
``Object.prototype``.

The bundled version of jQuery used by the Django admin has been patched to
allow for the ``select2`` library's use of ``jQuery.extend()``.


==========================

2.1.8

Not secure
==========================

*April 1, 2019*

Django 2.1.8 fixes a bug in 2.1.7.

Bugfixes
========

* Prevented admin inlines for a ``ManyToManyField``\'s implicit through model
from being editable if the user only has the view permission
(:ticket:`30289`).


==========================

2.1.7

Not secure
==========================

*February 11, 2019*

Django 2.1.7 fixes a packaging error in 2.1.6.

Bugfixes
========

* Corrected packaging error from 2.1.6 (:ticket:`30175`).


==========================

2.1.5

Not secure
==========================

*January 4, 2019*

Django 2.1.5 fixes a security issue and several bugs in 2.1.4.

CVE-2019-3498: Content spoofing possibility in the default 404 page
-------------------------------------------------------------------

An attacker could craft a malicious URL that could make spoofed content appear
on the default page generated by the ``django.views.defaults.page_not_found()``
view.

The URL path is no longer displayed in the default 404 template and the
``request_path`` context variable is now quoted to fix the issue for custom
templates that use the path.

Bugfixes
========

* Fixed compatibility with mysqlclient 1.3.14 (:ticket:`30013`).

* Fixed a schema corruption issue on SQLite 3.26+. You might have to drop and
rebuild your SQLite database if you applied a migration while using an older
version of Django with SQLite 3.26 or later (:ticket:`29182`).

* Prevented SQLite schema alterations while foreign key checks are enabled to
avoid the possibility of schema corruption (:ticket:`30023`).

* Fixed a regression in Django 2.1.4 (which enabled keep-alive connections)
where request body data isn't properly consumed for such connections
(:ticket:`30015`).

* Fixed a regression in Django 2.1.4 where
``InlineModelAdmin.has_change_permission()`` is incorrectly called with a
non-``None`` ``obj`` argument during an object add (:ticket:`30050`).


==========================

2.1.4

Not secure
==========================

*December 3, 2018*

Django 2.1.4 fixes several bugs in 2.1.3.

Bugfixes
========

* Corrected the default password list that ``CommonPasswordValidator`` uses by
lowercasing all passwords to match the format expected by the validator
(:ticket:`29952`).

* Prevented repetitive calls to ``geos_version_tuple()`` in the ``WKBWriter``
class in an attempt to fix a random crash involving ``LooseVersion``
(:ticket:`29959`).

* Fixed keep-alive support in ``runserver`` after it was disabled to fix
another issue in Django 2.0 (:ticket:`29849`).

* Fixed admin view-only change form crash when using
``ModelAdmin.prepopulated_fields`` (:ticket:`29929`).

* Fixed "Please correct the errors below" error message when editing an object
in the admin if the user only has the "view" permission on inlines
(:ticket:`29930`).


==========================

2.1.3

Not secure
==========================

*November 1, 2018*

Django 2.1.3 fixes several bugs in 2.1.2.

Bugfixes
========

* Fixed a regression in Django 2.0 where combining ``Q`` objects with ``__in``
lookups and lists crashed (:ticket:`29838`).

* Fixed a regression in Django 1.11 where ``django-admin shell`` may hang
on startup (:ticket:`29774`).

* Fixed a regression in Django 2.0 where test databases aren't reused with
``manage.py test --keepdb`` on MySQL (:ticket:`29827`).

* Fixed a regression where cached foreign keys that use ``to_field`` were
incorrectly cleared in ``Model.save()`` (:ticket:`29896`).

* Fixed a regression in Django 2.0 where ``FileSystemStorage`` crashes with
``FileExistsError`` if concurrent saves try to create the same directory
(:ticket:`29890`).


==========================

2.1.2

Not secure
==========================

*October 1, 2018*

Django 2.1.2 fixes a security issue and several bugs in 2.1.1. Also, the latest
string translations from Transifex are incorporated.

CVE-2018-16984: Password hash disclosure to "view only" admin users
===================================================================

If an admin user has the change permission to the user model, only part of the
password hash is displayed in the change form. Admin users with the view (but
not change) permission to the user model were displayed the entire hash. While
it's typically infeasible to reverse a strong password hash, if your site uses
weaker password hashing algorithms such as MD5 or SHA1, it could be a problem.

Bugfixes
========

* Fixed a regression where nonexistent joins in ``F()`` no longer raised
``FieldError`` (:ticket:`29727`).

* Fixed a regression where files starting with a tilde or underscore weren't
ignored by the migrations loader (:ticket:`29749`).

* Made migrations detect changes to ``Meta.default_related_name``
(:ticket:`29755`).

* Added compatibility for ``cx_Oracle`` 7 (:ticket:`29759`).

* Fixed a regression in Django 2.0 where unique index names weren't quoted
(:ticket:`29778`).

* Fixed a regression where sliced queries with multiple columns with the same
name crashed on Oracle 12.1 (:ticket:`29630`).

* Fixed a crash when a user with the view (but not change) permission made a
POST request to an admin user change form (:ticket:`29809`).


==========================

2.1.1

Not secure
==========================

*August 31, 2018*

Django 2.1.1 fixes several bugs in 2.1.

Bugfixes
========

* Fixed a race condition in ``QuerySet.update_or_create()`` that could result
in data loss (:ticket:`29499`).

* Fixed a regression where ``QueryDict.urlencode()`` crashed if the dictionary
contains a non-string value (:ticket:`29627`).

* Fixed a regression in Django 2.0 where using ``manage.py test --keepdb``
fails on PostgreSQL if the database exists and the user doesn't have
permission to create databases (:ticket:`29613`).

* Fixed a regression in Django 2.0 where combining ``Q`` objects with ``__in``
lookups and lists crashed (:ticket:`29643`).

* Fixed translation failure of ``DurationField``'s "overflow" error message
(:ticket:`29623`).

* Fixed a regression where the admin change form crashed if the user doesn't
have the 'add' permission to a model that uses ``TabularInline``
(:ticket:`29637`).

* Fixed a regression where a ``related_query_name`` reverse accessor wasn't set
up when a ``GenericRelation`` is declared on an abstract base model
(:ticket:`29653`).

* Fixed the test client's JSON serialization of a request data dictionary for
structured content type suffixes (:ticket:`29662`).

* Made the admin change view redirect to the changelist view after a POST if
the user has the 'view' permission (:ticket:`29663`).

* Fixed admin change view crash for view-only users if the form has an extra
form field (:ticket:`29682`).

* Fixed a regression in Django 2.0.5 where ``QuerySet.values()`` or
``values_list()`` after combining querysets with ``extra()`` with
``union()``, ``difference()``, or ``intersection()`` crashed due to
mismatching columns (:ticket:`29694`).

* Fixed crash if ``InlineModelAdmin.has_add_permission()`` doesn't accept the
``obj`` argument (:ticket:`29723`).


========================

2.1

Not secure
========================

*August 1, 2018*

Welcome to Django 2.1!

These release notes cover the :ref:`new features <whats-new-2.1>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-2.1>` you'll
want to be aware of when upgrading from Django 2.0 or earlier. We've
:ref:`dropped some features<removed-features-2.1>` that have reached the end of
their deprecation cycle, and we've :ref:`begun the deprecation process for some
features <deprecated-features-2.1>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Python compatibility
====================

Django 2.1 supports Python 3.5, 3.6, and 3.7. Django 2.0 is the last version to
support Python 3.4. We **highly recommend** and only officially support the
latest release of each series.

.. _whats-new-2.1:

What's new in Django 2.1
========================

Model "view" permission
-----------------------

A "view" permission is added to the model :attr:`Meta.default_permissions
<django.db.models.Options.default_permissions>`. The new permissions will be
created automatically when running :djadmin:`migrate`.

This allows giving users read-only access to models in the admin.
:meth:`.ModelAdmin.has_view_permission` is new. The implementation is backwards
compatible in that there isn't a need to assign the "view" permission to allow
users who have the "change" permission to edit objects.

There are a couple of :ref:`backwards incompatible considerations
<view_permission_backwards_incompatible>`.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* :attr:`.ModelAdmin.search_fields` now accepts any lookup such as
``field__exact``.

* jQuery is upgraded from version 2.2.3 to 3.3.1.

* The new :meth:`.ModelAdmin.delete_queryset` method allows customizing the
deletion process of the "delete selected objects" action.

* You can now :ref:`override the default admin site
<overriding-default-admin-site>`.

* The new :attr:`.ModelAdmin.sortable_by` attribute and
:meth:`.ModelAdmin.get_sortable_by` method allow limiting the columns that
can be sorted in the change list page.

* The ``admin_order_field`` attribute for elements in
:attr:`.ModelAdmin.list_display` may now be a query expression.

* The new :meth:`.ModelAdmin.get_deleted_objects()` method allows customizing
the deletion process of the delete view and the "delete selected" action.

* The ``actions.html``, ``change_list_results.html``, ``date_hierarchy.html``,
``pagination.html``, ``prepopulated_fields_js.html``, ``search_form.html``,
and ``submit_line.html`` templates can now be :ref:`overridden per app or
per model <admin-templates-overridden-per-app-or-model>` (besides overridden
globally).

* The admin change list and change form object tools can now be :ref:`overridden
per app, per model, or globally <admin-templates-overridden-per-app-or-model>`
with ``change_list_object_tools.html`` and
``change_form_object_tools.html`` templates.

* :meth:`.InlineModelAdmin.has_add_permission` is now passed the parent object
as the second positional argument, ``obj``.

* Admin actions may now :ref:`specify permissions <admin-action-permissions>`
to limit their availability to certain users.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* :djadmin:`createsuperuser` now gives a prompt to allow bypassing the
:setting:`AUTH_PASSWORD_VALIDATORS` checks.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :meth:`.GEOSGeometry.buffer_with_style` method is a version of
:meth:`~.GEOSGeometry.buffer` that allows customizing the style of the
buffer.

* :class:`~django.contrib.gis.forms.widgets.OpenLayersWidget` is now based on
OpenLayers 4.6.5 (previously 3.20.1).

:mod:`django.contrib.sessions`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Added the :setting:`SESSION_COOKIE_SAMESITE` setting to set the ``SameSite``
cookie flag on session cookies.

Cache
~~~~~

* The :ref:`local-memory cache backend <local-memory-caching>` now uses a
least-recently-used (LRU) culling strategy rather than a pseudo-random one.

* The new :meth:`~django.core.caches.cache.touch` method of the :ref:`low-level
cache API <low-level-cache-api>` updates the timeout of cache keys.

CSRF
~~~~

* Added the :setting:`CSRF_COOKIE_SAMESITE` setting to set the ``SameSite``
cookie flag on CSRF cookies.

Forms
~~~~~

* The widget for ``ImageField`` now renders with the HTML attribute
``accept="image/*"``.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* Added the :meth:`~django.utils.translation.get_supported_language_variant`
function.

* Untranslated strings for territorial language variants now use the
translations of the generic language. For example, untranslated ``pt_BR``
strings use ``pt`` translations.

Management Commands
~~~~~~~~~~~~~~~~~~~

* The new :option:`inspectdb --include-views` option allows creating models
for database views.

* The :class:`~django.core.management.BaseCommand` class now uses a custom help
formatter so that the standard options like ``--verbosity`` or ``--settings``
appear last in the help output, giving a more prominent position to subclassed
command's options.

Migrations
~~~~~~~~~~

* Added support for serialization of ``functools.partialmethod`` objects.

* To support frozen environments, migrations may be loaded from ``.pyc`` files.

Models
~~~~~~

* Models can now use ``__init_subclass__()`` from :pep:`487`.

* A ``BinaryField`` may now be set to ``editable=True`` if you wish to include
it in model forms.

* A number of new text database functions are added:
:class:`~django.db.models.functions.Chr`,
:class:`~django.db.models.functions.Left`,
:class:`~django.db.models.functions.LPad`,
:class:`~django.db.models.functions.LTrim`,
:class:`~django.db.models.functions.Ord`,
:class:`~django.db.models.functions.Repeat`,
:class:`~django.db.models.functions.Replace`,
:class:`~django.db.models.functions.Right`,
:class:`~django.db.models.functions.RPad`,
:class:`~django.db.models.functions.RTrim`, and
:class:`~django.db.models.functions.Trim`.

* The new :class:`~django.db.models.functions.TruncWeek` function truncates
:class:`~django.db.models.DateField` and
:class:`~django.db.models.DateTimeField` to the Monday of a week.

* Query expressions can now be negated using a minus sign.

* :meth:`.QuerySet.order_by` and :meth:`distinct(*fields) <.QuerySet.distinct>`
now support using field transforms.

* :class:`~django.db.models.BooleanField` can now be ``null=True``. This is
encouraged instead of ``NullBooleanField``, which will likely be deprecated
in the future.

* The new :meth:`.QuerySet.explain` method displays the database's execution
plan of a queryset's query.

* :meth:`.QuerySet.raw` now supports :meth:`~.QuerySet.prefetch_related`.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* Added :meth:`.HttpRequest.get_full_path_info`.

* Added the ``samesite`` argument to :meth:`.HttpResponse.set_cookie` to allow
setting the ``SameSite`` cookie flag.

* The new ``as_attachment`` argument for :class:`~django.http.FileResponse`
sets the ``Content-Disposition`` header to make the browser ask if the user
wants to download the file. ``FileResponse`` also tries to set the
``Content-Type`` and ``Content-Length`` headers where appropriate.

Templates
~~~~~~~~~

* The new :tfilter:`json_script` filter safely outputs a Python object as JSON,
wrapped in a ``<script>`` tag, ready for use with JavaScript.

Tests
~~~~~

* Added test :class:`~django.test.Client` support for 307 and 308 redirects.

* The test :class:`~django.test.Client` now serializes a request data
dictionary as JSON if ``content_type='application/json'``. You can customize
the JSON encoder with test client's ``json_encoder`` parameter.

* The new :meth:`.SimpleTestCase.assertWarnsMessage` method is a simpler
version of :meth:`~unittest.TestCase.assertWarnsRegex`.

.. _backwards-incompatible-2.1:

Backwards incompatible changes in 2.1
=====================================

Database backend API
--------------------

This section describes changes that may be needed in third-party database
backends.

* To adhere to :pep:`249`, exceptions where a database doesn't support a
feature are changed from :exc:`NotImplementedError` to
:exc:`django.db.NotSupportedError`.

* Renamed the ``allow_sliced_subqueries`` database feature flag to
``allow_sliced_subqueries_with_in``.

* ``DatabaseOperations.distinct_sql()`` now requires an additional ``params``
argument and returns a tuple of SQL and parameters instead of an SQL string.

* ``DatabaseFeatures.introspected_boolean_field_type`` is changed from a method
to a property.

:mod:`django.contrib.gis`
-------------------------

* Support for SpatiaLite 4.0 is removed.

Dropped support for MySQL 5.5
-----------------------------

The end of upstream support for MySQL 5.5 is December 2018. Django 2.1 supports
MySQL 5.6 and higher.

Dropped support for PostgreSQL 9.3
----------------------------------

The end of upstream support for PostgreSQL 9.3 is September 2018. Django 2.1
supports PostgreSQL 9.4 and higher.

Removed ``BCryptPasswordHasher`` from the default ``PASSWORD_HASHERS`` setting
------------------------------------------------------------------------------

If you used bcrypt with Django 1.4 or 1.5 (before ``BCryptSHA256PasswordHasher``
was added in Django 1.6), you might have some passwords that use the
``BCryptPasswordHasher`` hasher.

You can check if that's the case like this::

from django.contrib.auth import get_user_model
User = get_user_model()
User.objects.filter(password__startswith='bcrypt$$')

If you want to continue to allow those passwords to be used, you'll
have to define the :setting:`PASSWORD_HASHERS` setting (if you don't already)
and include ``'django.contrib.auth.hashers.BCryptPasswordHasher'``.

Moved ``wrap_label`` widget template context variable
-----------------------------------------------------

To fix the lack of ``<label>`` when using ``RadioSelect`` and
``CheckboxSelectMultiple`` with ``MultiWidget``, the ``wrap_label`` context
variable now appears as an attribute of each option. For example, in a custom
``input_option.html`` template, change ``{% if wrap_label %}`` to
``{% if widget.wrap_label %}``.

``SameSite`` cookies
--------------------

The cookies used for ``django.contrib.sessions``, ``django.contrib.messages``,
and Django's CSRF protection now set the ``SameSite`` flag to ``Lax`` by
default. Browsers that respect this flag won't send these cookies on
cross-origin requests. If you rely on the old behavior, set the
:setting:`SESSION_COOKIE_SAMESITE` and/or :setting:`CSRF_COOKIE_SAMESITE`
setting to ``None``.

.. _view_permission_backwards_incompatible:

Considerations for the new model "view" permission
--------------------------------------------------

Custom admin forms need to take the view-only case into account
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

With the new "view" permission, existing custom admin forms may raise errors
when a user doesn't have the change permission because the form might access
nonexistent fields. Fix this by overriding :meth:`.ModelAdmin.get_form` and
checking if the user has the "change" permissions and returning the default
form if not::

class MyAdmin(admin.ModelAdmin):
def get_form(self, request, obj=None, **kwargs):
if not self.has_change_permission(request, obj):
return super().get_form(request, obj, **kwargs)
return CustomForm

New default view permission could allow unwanted access to admin views
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you have a custom permission with a codename of the form
``view_<modelname>``, the new view permission handling in the admin will allow
view access to the changelist and detail pages for those models. If this is
unwanted, you must change your custom permission codename.

Miscellaneous
-------------

* The minimum supported version of ``mysqlclient`` is increased from 1.3.3 to
1.3.7.

* Support for SQLite < 3.7.15 is removed.

* The date format of ``Set-Cookie``'s ``Expires`` directive is changed to
follow :rfc:`7231section-7.1.1.1` instead of Netscape's cookie standard.
Hyphens present in dates like ``Tue, 25-Dec-2018 22:26:13 GMT`` are removed.
This change should be merely cosmetic except perhaps for antiquated browsers
that don't parse the new format.

* ``allowed_hosts`` is now a required argument of private API
``django.utils.http.is_safe_url()``.

* The ``multiple`` attribute rendered by the
:class:`~django.forms.SelectMultiple` widget now uses HTML5 boolean syntax
rather than XHTML's ``multiple="multiple"``.

* HTML rendered by form widgets no longer includes a closing slash on void
elements, e.g. ``<br>``. This is incompatible within XHTML, although some
widgets already used aspects of HTML5 such as boolean attributes.

* The value of :class:`~django.forms.SelectDateWidget`'s empty options is
changed from 0 to an empty string, which mainly may require some adjustments
in tests that compare HTML.

* :meth:`.User.has_usable_password` and the
:func:`~django.contrib.auth.hashers.is_password_usable` function no longer
return ``False`` if the password is ``None`` or an empty string, or if the
password uses a hasher that's not in the :setting:`PASSWORD_HASHERS` setting.
This undocumented behavior was a regression in Django 1.6 and prevented users
with such passwords from requesting a password reset. Audit your code to
confirm that your usage of these APIs don't rely on the old behavior.

* Since migrations are now loaded from ``.pyc`` files, you might need to delete
them if you're working in a mixed Python 2 and Python 3 environment.

* Using ``None`` as a ``django.contrib.postgres.fields.JSONField`` lookup
value now matches objects that have the specified key and a null value rather
than objects that don't have the key.

* The admin CSS class ``field-box`` is renamed to ``fieldBox`` to prevent
conflicts with the class given to model fields named "box".

* Since the admin's ``actions.html``, ``change_list_results.html``,
``date_hierarchy.html``, ``pagination.html``, ``prepopulated_fields_js.html``,
``search_form.html``, and ``submit_line.html`` templates can now be
overridden per app or per model, you may need to rename existing templates
with those names that were written for a different purpose.

* ``QuerySet.raw()`` now caches its results like regular querysets. Use
``iterator()`` if you don't want caching.

* The database router :meth:`allow_relation` method is called in more cases.
Improperly written routers may need to be updated accordingly.

* Translations are no longer deactivated before running management commands.
If your custom command requires translations to be deactivated (for example,
to insert untranslated content into the database), use the new
:ref:`no_translations decorator <management-commands-and-locales>`.

* Management commands no longer allow the abbreviated forms of the
``--settings`` and ``--pythonpath`` arguments.

* The private ``django.db.models.sql.constants.QUERY_TERMS`` constant is
removed. The :meth:`~.RegisterLookupMixin.get_lookup`
and :meth:`~.RegisterLookupMixin.get_lookups` methods
of the :ref:`Lookup Registration API <lookup-registration-api>` may be
suitable alternatives. Compared to the ``QUERY_TERMS`` constant, they allow
your code to also account for any custom lookups that have been registered.

* Compatibility with ``py-bcrypt`` is removed as it's unmaintained. Use `bcrypt
<https://pypi.org/project/bcrypt/>`_ instead.

.. _deprecated-features-2.1:

Features deprecated in 2.1
==========================

Miscellaneous
-------------

* The ``ForceRHR`` GIS function is deprecated in favor of the new
:class:`~django.contrib.gis.db.models.functions.ForcePolygonCW` function.

* ``django.utils.http.cookie_date()`` is deprecated in favor of
:func:`~django.utils.http.http_date`, which follows the format of the latest
RFC.

* ``{% load staticfiles %}`` and ``{% load admin_static %}`` are deprecated
in favor of ``{% load static %}``, which works the same.

* ``django.contrib.staticfiles.templatetags.static()`` is deprecated in favor
of ``django.templatetags.static.static()``.

* Support for :meth:`.InlineModelAdmin.has_add_permission` methods that don't
accept ``obj`` as the second positional argument will be removed in Django
3.0.

.. _removed-features-2.1:

Features removed in 2.1
=======================

These features have reached the end of their deprecation cycle and are removed
in Django 2.1. See :ref:`deprecated-features-1.11` for details, including how
to remove usage of these features.

* ``contrib.auth.views.login()``, ``logout()``, ``password_change()``,
``password_change_done()``, ``password_reset()``, ``password_reset_done()``,
``password_reset_confirm()``, and ``password_reset_complete()`` are removed.

* The ``extra_context`` parameter of ``contrib.auth.views.logout_then_login()``
is removed.

* ``django.test.runner.setup_databases()`` is removed.

* ``django.utils.translation.string_concat()`` is removed.

* ``django.core.cache.backends.memcached.PyLibMCCache`` no longer supports
passing ``pylibmc`` behavior settings as top-level attributes of ``OPTIONS``.

* The ``host`` parameter of ``django.utils.http.is_safe_url()`` is removed.

* Silencing of exceptions raised while rendering the ``{% include %}`` template
tag is removed.

* ``DatabaseIntrospection.get_indexes()`` is removed.

* The ``authenticate()`` method of authentication backends requires ``request``
as the first positional argument.

* The ``django.db.models.permalink()`` decorator is removed.

* The ``USE_ETAGS`` setting is removed. ``CommonMiddleware`` and
``django.utils.cache.patch_response_headers()`` no longer set ETags.

* The ``Model._meta.has_auto_field`` attribute is removed.

* ``url()``'s support for inline flags in regular expression groups (``(?i)``,
``(?L)``, ``(?m)``, ``(?s)``, and ``(?u)``) is removed.

* Support for ``Widget.render()`` methods without the ``renderer`` argument
is removed.


===========================

2.0.13

Not secure
===========================

*February 12, 2019*

Django 2.0.13 fixes a regression in 2.0.12/2.0.11.

Bugfixes
========

* Fixed crash in ``django.utils.numberformat.format_number()`` when the number
has over 200 digits (:ticket:`30177`).


===========================

2.0.12

Not secure
===========================

*February 11, 2019*

Django 2.0.12 fixes a packaging error in 2.0.11.

Bugfixes
========

* Corrected packaging error from 2.0.11 (:ticket:`30175`).


===========================

2.0.10

Not secure
===========================

*January 4, 2019*

Django 2.0.10 fixes a security issue and several bugs in 2.0.9.

CVE-2019-3498: Content spoofing possibility in the default 404 page
-------------------------------------------------------------------

An attacker could craft a malicious URL that could make spoofed content appear
on the default page generated by the ``django.views.defaults.page_not_found()``
view.

The URL path is no longer displayed in the default 404 template and the
``request_path`` context variable is now quoted to fix the issue for custom
templates that use the path.

Bugfixes
========

* Prevented repetitive calls to ``geos_version_tuple()`` in the ``WKBWriter``
class in an attempt to fix a random crash involving ``LooseVersion`` since
Django 2.0.6 (:ticket:`29959`).

* Fixed a schema corruption issue on SQLite 3.26+. You might have to drop and
rebuild your SQLite database if you applied a migration while using an older
version of Django with SQLite 3.26 or later (:ticket:`29182`).

* Prevented SQLite schema alterations while foreign key checks are enabled to
avoid the possibility of schema corruption (:ticket:`30023`).


==========================

2.0.9

Not secure
==========================

*October 1, 2018*

Django 2.0.9 fixes a data loss bug in 2.0.8.

Bugfixes
========

* Fixed a race condition in ``QuerySet.update_or_create()`` that could result
in data loss (:ticket:`29499`).


==========================

2.0.8

Not secure
==========================

*August 1, 2018*

Django 2.0.8 fixes a security issue and several bugs in 2.0.7.

CVE-2018-14574: Open redirect possibility in ``CommonMiddleware``
=================================================================

If the :class:`~django.middleware.common.CommonMiddleware` and the
:setting:`APPEND_SLASH` setting are both enabled, and if the project has a
URL pattern that accepts any path ending in a slash (many content management
systems have such a pattern), then a request to a maliciously crafted URL of
that site could lead to a redirect to another site, enabling phishing and other
attacks.

``CommonMiddleware`` now escapes leading slashes to prevent redirects to other
domains.

Bugfixes
========

* Fixed a regression in Django 2.0.7 that broke the ``regex`` lookup on MariaDB
(even though MariaDB isn't officially supported) (:ticket:`29544`).

* Fixed a regression where ``django.template.Template`` crashed if the
``template_string`` argument is lazy (:ticket:`29617`).


==========================

2.0.7

Not secure
==========================

*July 2, 2018*

Django 2.0.7 fixes several bugs in 2.0.6.

Bugfixes
========

* Fixed admin changelist crash when using a query expression without ``asc()``
or ``desc()`` in the page's ordering (:ticket:`29428`).

* Fixed admin check crash when using a query expression in
``ModelAdmin.ordering`` (:ticket:`29428`).

* Fixed ``__regex`` and ``__iregex`` lookups with MySQL 8 (:ticket:`29451`).

* Fixed migrations crash with namespace packages on Python 3.7
(:ticket:`28814`).


==========================

2.0.6

Not secure
==========================

*June 1, 2018*

Django 2.0.6 fixes several bugs in 2.0.5.

Bugfixes
========

* Fixed a regression that broke custom template filters that use decorators
(:ticket:`29400`).

* Fixed detection of custom URL converters in included patterns
(:ticket:`29415`).

* Fixed a regression that added an unnecessary subquery to the ``GROUP BY``
clause on MySQL when using a ``RawSQL`` annotation (:ticket:`29416`).

* Fixed ``WKBWriter.write()`` and ``write_hex()`` for empty polygons on
GEOS 3.6.1+ (:ticket:`29460`).

* Fixed a regression in Django 1.10 that could result in large memory usage
when making edits using ``ModelAdmin.list_editable`` (:ticket:`28462`).


==========================

2.0.5

Not secure
==========================

*May 1, 2018*

Django 2.0.5 fixes several bugs in 2.0.4.

Bugfixes
========

* Corrected the import paths that ``inspectdb`` generates for
``django.contrib.postgres`` fields (:ticket:`29307`).

* Fixed a regression in Django 1.11.8 where altering a field with a unique
constraint may drop and rebuild more foreign keys than necessary
(:ticket:`29193`).

* Fixed crashes in ``django.contrib.admindocs`` when a view is a callable
object, such as ``django.contrib.syndication.views.Feed`` (:ticket:`29296`).

* Fixed a regression in Django 2.0.4 where ``QuerySet.values()`` or
``values_list()`` after combining an annotated and unannotated queryset with
``union()``, ``difference()``, or ``intersection()`` crashed due to mismatching
columns (:ticket:`29286`).


==========================

2.0.4

Not secure
==========================

*April 2, 2018*

Django 2.0.4 fixes several bugs in 2.0.3.

Bugfixes
========

* Fixed a crash when filtering with an ``Exists()`` annotation of a queryset
containing a single field (:ticket:`29195`).

* Fixed admin autocomplete widget's translations for ``zh-hans`` and
``zh-hant`` languages (:ticket:`29213`).

* Corrected admin's autocomplete widget to add a space after custom classes
(:ticket:`29221`).

* Fixed ``PasswordResetConfirmView`` crash when using a user model with a
``UUIDField`` primary key and the reset URL contains an encoded primary key
value that decodes to an invalid UUID (:ticket:`29206`).

* Fixed a regression in Django 1.11.8 where combining two annotated
``values_list()`` querysets with ``union()``, ``difference()``, or
``intersection()`` crashed due to mismatching columns (:ticket:`29229`).

* Fixed a regression in Django 1.11 where an empty choice could be initially
selected for the ``SelectMultiple`` and ``CheckboxSelectMultiple`` widgets
(:ticket:`29273`).

* Fixed a regression in Django 2.0 where ``OpenLayersWidget`` deserialization
ignored the widget map's SRID and assumed 4326 (WGS84) (:ticket:`29116`).


==========================

2.0.3

Not secure
==========================

*March 6, 2018*

Django 2.0.3 fixes two security issues and several bugs in 2.0.2. Also, the
latest string translations from Transifex are incorporated.

CVE-2018-7536: Denial-of-service possibility in ``urlize`` and ``urlizetrunc`` template filters
===============================================================================================

The ``django.utils.html.urlize()`` function was extremely slow to evaluate
certain inputs due to catastrophic backtracking vulnerabilities in two regular
expressions. The ``urlize()`` function is used to implement the ``urlize`` and
``urlizetrunc`` template filters, which were thus vulnerable.

The problematic regular expressions are replaced with parsing logic that
behaves similarly.

CVE-2018-7537: Denial-of-service possibility in ``truncatechars_html`` and ``truncatewords_html`` template filters
==================================================================================================================

If ``django.utils.text.Truncator``'s ``chars()`` and ``words()`` methods were
passed the ``html=True`` argument, they were extremely slow to evaluate certain
inputs due to a catastrophic backtracking vulnerability in a regular
expression. The ``chars()`` and ``words()`` methods are used to implement the
``truncatechars_html`` and ``truncatewords_html`` template filters, which were
thus vulnerable.

The backtracking problem in the regular expression is fixed.

Bugfixes
========

* Fixed a regression that caused sliced ``QuerySet.distinct().order_by()``
followed by ``count()`` to crash (:ticket:`29108`).

* Prioritized the datetime and time input formats without ``%f`` for the Thai
locale to fix the admin time picker widget displaying "undefined"
(:ticket:`29109`).

* Fixed crash with ``QuerySet.order_by(Exists(...))`` (:ticket:`29118`).

* Made ``Q.deconstruct()`` deterministic with multiple keyword arguments
(:ticket:`29125`). You may need to modify ``Q``'s in existing migrations, or
accept an autogenerated migration.

* Fixed a regression where a ``When()`` expression with a list argument crashes
(:ticket:`29166`).

* Fixed crash when using a ``Window()`` expression in a subquery
(:ticket:`29172`).

* Fixed ``AbstractBaseUser.normalize_username()`` crash if the ``username``
argument isn't a string (:ticket:`29176`).


==========================

2.0.2

Not secure
==========================

*February 1, 2018*

Django 2.0.2 fixes a security issue and several bugs in 2.0.1.

CVE-2018-6188: Information leakage in ``AuthenticationForm``
============================================================

A regression in Django 1.11.8 made
:class:`~django.contrib.auth.forms.AuthenticationForm` run its
``confirm_login_allowed()`` method even if an incorrect password is entered.
This can leak information about a user, depending on what messages
``confirm_login_allowed()`` raises. If ``confirm_login_allowed()`` isn't
overridden, an attacker enter an arbitrary username and see if that user has
been set to ``is_active=False``. If ``confirm_login_allowed()`` is overridden,
more sensitive details could be leaked.

This issue is fixed with the caveat that ``AuthenticationForm`` can no longer
raise the "This account is inactive." error if the authentication backend
rejects inactive users (the default authentication backend, ``ModelBackend``,
has done that since Django 1.10). This issue will be revisited for Django 2.1
as a fix to address the caveat will likely be too invasive for inclusion in
older versions.

Bugfixes
========

* Fixed hidden content at the bottom of the "The install worked successfully!"
page for some languages (:ticket:`28885`).

* Fixed incorrect foreign key nullification if a model has two foreign keys to
the same model and a target model is deleted (:ticket:`29016`).

* Fixed regression in the use of ``QuerySet.values_list(..., flat=True)``
followed by ``annotate()`` (:ticket:`29067`).

* Fixed a regression where a queryset that annotates with geometry objects
crashes (:ticket:`29054`).

* Fixed a regression where ``contrib.auth.authenticate()`` crashes if an
authentication backend doesn't accept ``request`` and a later one does
(:ticket:`29071`).

* Fixed a regression where ``makemigrations`` crashes if a migrations directory
doesn't have an ``__init__.py`` file (:ticket:`29091`).

* Fixed crash when entering an invalid uuid in ``ModelAdmin.raw_id_fields``
(:ticket:`29094`).


==========================

2.0.1

Not secure
==========================

*January 1, 2018*

Django 2.0.1 fixes several bugs in 2.0.

Bugfixes
========

* Fixed a regression in Django 1.11 that added newlines between ``MultiWidget``'s
subwidgets (:ticket:`28890`).

* Fixed incorrect class-based model index name generation for models with
quoted ``db_table`` (:ticket:`28876`).

* Fixed incorrect foreign key constraint name for models with quoted
``db_table`` (:ticket:`28876`).

* Fixed a regression in caching of a ``GenericForeignKey`` when the referenced
model instance uses more than one level of multi-table inheritance
(:ticket:`28856`).

* Reallowed filtering a queryset with ``GeometryField=None`` (:ticket:`28896`).

* Corrected admin check to allow a ``OneToOneField`` in
``ModelAdmin.autocomplete_fields`` (:ticket:`28898`).

* Fixed a regression on SQLite where ``DecimalField`` returned a result with
trailing zeros in the fractional part truncated (:ticket:`28915`).

* Fixed crash in the ``testserver`` command startup (:ticket:`28941`).

* Fixed crash when coercing a translatable URL pattern to ``str``
(:ticket:`28947`).

* Fixed crash on SQLite when renaming a field in a model referenced by a
``ManyToManyField`` (:ticket:`28884`).

* Fixed a crash when chaining ``values()`` or ``values_list()`` after
``QuerySet.select_for_update(of=(...))`` (:ticket:`28944`).

* Fixed admin changelist crash when using a query expression in the page's
ordering (:ticket:`28958`).


========================

2.0

Not secure
========================

*December 2, 2017*

Welcome to Django 2.0!

These release notes cover the :ref:`new features <whats-new-2.0>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-2.0>` you'll
want to be aware of when upgrading from Django 1.11 or earlier. We've
:ref:`dropped some features<removed-features-2.0>` that have reached the end of
their deprecation cycle, and we've :ref:`begun the deprecation process for some
features <deprecated-features-2.0>`.

This release starts Django's use of a :ref:`loose form of semantic versioning
<internal-release-cadence>`, but there aren't any major backwards incompatible
changes that might be expected of a 2.0 release. Upgrading should be a similar
amount of effort as past feature releases.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Python compatibility
====================

Django 2.0 supports Python 3.4, 3.5, 3.6, and 3.7. We **highly recommend** and
only officially support the latest release of each series.

The Django 1.11.x series is the last to support Python 2.7.

Django 2.0 will be the last release series to support Python 3.4. If you plan
a deployment of Python 3.4 beyond the end-of-life for Django 2.0 (April 2019),
stick with Django 1.11 LTS (supported until April 2020) instead. Note, however,
that the end-of-life for Python 3.4 is March 2019.

Third-party library support for older version of Django
=======================================================

Following the release of Django 2.0, we suggest that third-party app authors
drop support for all versions of Django prior to 1.11. At that time, you should
be able to run your package's tests using ``python -Wd`` so that deprecation
warnings do appear. After making the deprecation warning fixes, your app should
be compatible with Django 2.0.

.. _whats-new-2.0:

What's new in Django 2.0
========================

Simplified URL routing syntax
-----------------------------

The new :func:`django.urls.path()` function allows a simpler, more readable URL
routing syntax. For example, this example from previous Django releases::

url(r'^articles/(?P<year>[0-9]{4})/$', views.year_archive),

could be written as::

path('articles/<int:year>/', views.year_archive),

The new syntax supports type coercion of URL parameters. In the example, the
view will receive the ``year`` keyword argument as an integer rather than as
a string. Also, the URLs that will match are slightly less constrained in the
rewritten example. For example, the year 10000 will now match since the year
integers aren't constrained to be exactly four digits long as they are in the
regular expression.

The ``django.conf.urls.url()`` function from previous versions is now available
as :func:`django.urls.re_path`. The old location remains for backwards
compatibility, without an imminent deprecation. The old
``django.conf.urls.include()`` function is now importable from ``django.urls``
so you can use ``from django.urls import include, path, re_path`` in your
URLconfs.

The :doc:`/topics/http/urls` document is rewritten to feature the new syntax
and provide more details.

Mobile-friendly ``contrib.admin``
---------------------------------

The admin is now responsive and supports all major mobile devices. Older
browsers may experience varying levels of graceful degradation.

Window expressions
------------------

The new :class:`~django.db.models.expressions.Window` expression allows
adding an ``OVER`` clause to querysets. You can use :ref:`window functions
<window-functions>` and :ref:`aggregate functions <aggregation-functions>` in
the expression.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :attr:`.ModelAdmin.autocomplete_fields` attribute and
:meth:`.ModelAdmin.get_autocomplete_fields` method allow using a
`Select2 <https://select2.org/>`_ search widget for ``ForeignKey`` and
``ManyToManyField``.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* The default iteration count for the PBKDF2 password hasher is increased from
36,000 to 100,000.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* Added MySQL support for the
:class:`~django.contrib.gis.db.models.functions.AsGeoJSON` function,
:class:`~django.contrib.gis.db.models.functions.GeoHash` function,
:class:`~django.contrib.gis.db.models.functions.IsValid` function,
:lookup:`isvalid` lookup, and :ref:`distance lookups <distance-lookups>`.

* Added the :class:`~django.contrib.gis.db.models.functions.Azimuth` and
:class:`~django.contrib.gis.db.models.functions.LineLocatePoint` functions,
supported on PostGIS and SpatiaLite.

* Any :class:`~django.contrib.gis.geos.GEOSGeometry` imported from GeoJSON now
has its SRID set.

* Added the :attr:`.OSMWidget.default_zoom` attribute to customize the map's
default zoom level.

* Made metadata readable and editable on rasters through the
:attr:`~django.contrib.gis.gdal.GDALRaster.metadata`,
:attr:`~django.contrib.gis.gdal.GDALRaster.info`, and
:attr:`~django.contrib.gis.gdal.GDALBand.metadata` attributes.

* Allowed passing driver-specific creation options to
:class:`~django.contrib.gis.gdal.GDALRaster` objects using ``papsz_options``.

* Allowed creating :class:`~django.contrib.gis.gdal.GDALRaster` objects in
GDAL's internal virtual filesystem. Rasters can now be :ref:`created from and
converted to binary data <gdal-raster-vsimem>` in-memory.

* The new :meth:`GDALBand.color_interp()
<django.contrib.gis.gdal.GDALBand.color_interp>` method returns the color
interpretation for the band.

:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new ``distinct`` argument for
:class:`~django.contrib.postgres.aggregates.ArrayAgg` determines if
concatenated values will be distinct.

* The new :class:`~django.contrib.postgres.functions.RandomUUID` database
function returns a version 4 UUID. It requires use of PostgreSQL's
``pgcrypto`` extension which can be activated using the new
:class:`~django.contrib.postgres.operations.CryptoExtension` migration
operation.

* :class:`django.contrib.postgres.indexes.GinIndex` now supports the
``fastupdate`` and ``gin_pending_list_limit`` parameters.

* The new :class:`~django.contrib.postgres.indexes.GistIndex` class allows
creating ``GiST`` indexes in the database. The new
:class:`~django.contrib.postgres.operations.BtreeGistExtension` migration
operation installs the ``btree_gist`` extension to add support for operator
classes that aren't built-in.

* :djadmin:`inspectdb` can now introspect ``JSONField`` and various
``RangeField``\s (``django.contrib.postgres`` must be in ``INSTALLED_APPS``).

:mod:`django.contrib.sitemaps`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Added the ``protocol`` keyword argument to the
:class:`~django.contrib.sitemaps.GenericSitemap` constructor.

Cache
~~~~~

* ``cache.set_many()`` now returns a list of keys that failed to be inserted.
For the built-in backends, failed inserts can only happen on memcached.

File Storage
~~~~~~~~~~~~

* :meth:`File.open() <django.core.files.File.open>` can be used as a context
manager, e.g. ``with file.open() as f:``.

Forms
~~~~~

* The new ``date_attrs`` and ``time_attrs`` arguments for
:class:`~django.forms.SplitDateTimeWidget` and
:class:`~django.forms.SplitHiddenDateTimeWidget` allow specifying different
HTML attributes for the ``DateInput`` and ``TimeInput`` (or hidden)
subwidgets.

* The new :meth:`Form.errors.get_json_data()
<django.forms.Form.errors.get_json_data>` method returns form errors as
a dictionary suitable for including in a JSON response.

Generic Views
~~~~~~~~~~~~~

* The new :attr:`.ContextMixin.extra_context` attribute allows adding context
in ``View.as_view()``.

Management Commands
~~~~~~~~~~~~~~~~~~~

* :djadmin:`inspectdb` now translates MySQL's unsigned integer columns to
``PositiveIntegerField`` or ``PositiveSmallIntegerField``.

* The new :option:`makemessages --add-location` option controls the comment
format in ``.po`` files.

* :djadmin:`loaddata` can now :ref:`read from stdin <loading-fixtures-stdin>`.

* The new :option:`diffsettings --output` option allows formatting the output
in a unified diff format.

* On Oracle, :djadmin:`inspectdb` can now introspect ``AutoField`` if the
column is created as an identity column.

* On MySQL, :djadmin:`dbshell` now supports client-side TLS certificates.

Migrations
~~~~~~~~~~

* The new :option:`squashmigrations --squashed-name` option allows naming the
squashed migration.

Models
~~~~~~

* The new :class:`~django.db.models.functions.StrIndex` database function
finds the starting index of a string inside another string.

* On Oracle, ``AutoField`` and ``BigAutoField`` are now created as `identity
columns <https://docs.oracle.com/en/database/oracle/oracle-database/21/drdaa/
intro-to-tools-and-products-supporting-migraiton.htmlGUID-14E4817B-F3BC-4D99
-B471-A0ACDC303CC9>`__.

* The new ``chunk_size`` parameter of :meth:`.QuerySet.iterator` controls the
number of rows fetched by the Python database client when streaming results
from the database. For databases that don't support server-side cursors, it
controls the number of results Django fetches from the database adapter.

* :meth:`.QuerySet.earliest`, :meth:`.QuerySet.latest`, and
:attr:`Meta.get_latest_by <django.db.models.Options.get_latest_by>` now
allow ordering by several fields.

* Added the :class:`~django.db.models.functions.ExtractQuarter` function to
extract the quarter from :class:`~django.db.models.DateField` and
:class:`~django.db.models.DateTimeField`, and exposed it through the
:lookup:`quarter` lookup.

* Added the :class:`~django.db.models.functions.TruncQuarter` function to
truncate :class:`~django.db.models.DateField` and
:class:`~django.db.models.DateTimeField` to the first day of a quarter.

* Added the :attr:`~django.db.models.Index.db_tablespace` parameter to
class-based indexes.

* If the database supports a native duration field (Oracle and PostgreSQL),
:class:`~django.db.models.functions.Extract` now works with
:class:`~django.db.models.DurationField`.

* Added the ``of`` argument to :meth:`.QuerySet.select_for_update()`, supported
on PostgreSQL and Oracle, to lock only rows from specific tables rather than
all selected tables. It may be helpful particularly when
:meth:`~.QuerySet.select_for_update()` is used in conjunction with
:meth:`~.QuerySet.select_related()`.

* The new ``field_name`` parameter of :meth:`.QuerySet.in_bulk` allows fetching
results based on any unique model field.

* :meth:`.CursorWrapper.callproc()` now takes an optional dictionary of keyword
parameters, if the backend supports this feature. Of Django's built-in
backends, only Oracle supports it.

* The new :meth:`connection.execute_wrapper()
<django.db.backends.base.DatabaseWrapper.execute_wrapper>` method allows
:doc:`installing wrappers around execution of database queries
</topics/db/instrumentation>`.

* The new ``filter`` argument for built-in aggregates allows :ref:`adding
different conditionals <conditional-aggregation>` to multiple aggregations
over the same fields or relations.

* Added support for expressions in :attr:`Meta.ordering
<django.db.models.Options.ordering>`.

* The new ``named`` parameter of :meth:`.QuerySet.values_list` allows fetching
results as named tuples.

* The new :class:`.FilteredRelation` class allows adding an ``ON`` clause to
querysets.

Pagination
~~~~~~~~~~

* Added :meth:`Paginator.get_page() <django.core.paginator.Paginator.get_page>`
to provide the documented pattern of handling invalid page numbers.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* The :djadmin:`runserver` web server supports HTTP 1.1.

Templates
~~~~~~~~~

* To increase the usefulness of :meth:`.Engine.get_default` in third-party
apps, it now returns the first engine if multiple ``DjangoTemplates`` engines
are configured in ``TEMPLATES`` rather than raising ``ImproperlyConfigured``.

* Custom template tags may now accept keyword-only arguments.

Tests
~~~~~

* Added threading support to :class:`~django.test.LiveServerTestCase`.

* Added settings that allow customizing the test tablespace parameters for
Oracle: :setting:`DATAFILE_SIZE`, :setting:`DATAFILE_TMP_SIZE`,
:setting:`DATAFILE_EXTSIZE`, and :setting:`DATAFILE_TMP_EXTSIZE`.

Validators
~~~~~~~~~~

* The new :class:`.ProhibitNullCharactersValidator` disallows the null
character in the input of the :class:`~django.forms.CharField` form field
and its subclasses. Null character input was observed from vulnerability
scanning tools. Most databases silently discard null characters, but
psycopg2 2.7+ raises an exception when trying to save a null character to
a char/text field with PostgreSQL.

.. _backwards-incompatible-2.0:

Backwards incompatible changes in 2.0
=====================================

Removed support for bytestrings in some places
----------------------------------------------

To support native Python 2 strings, older Django versions had to accept both
bytestrings and Unicode strings. Now that Python 2 support is dropped,
bytestrings should only be encountered around input/output boundaries (handling
of binary fields or HTTP streams, for example). You might have to update your
code to limit bytestring usage to a minimum, as Django no longer accepts
bytestrings in certain code paths. Python's :option:`-b` option may help detect
that mistake in your code.

For example, ``reverse()`` now uses ``str()`` instead of ``force_text()`` to
coerce the ``args`` and ``kwargs`` it receives, prior to their placement in
the URL. For bytestrings, this creates a string with an undesired ``b`` prefix
as well as additional quotes (``str(b'foo')`` is ``"b'foo'"``). To adapt, call
``decode()`` on the bytestring before passing it to ``reverse()``.

Database backend API
--------------------

This section describes changes that may be needed in third-party database
backends.

* The ``DatabaseOperations.datetime_cast_date_sql()``,
``datetime_cast_time_sql()``, ``datetime_trunc_sql()``,
``datetime_extract_sql()``, and ``date_interval_sql()`` methods now return
only the SQL to perform the operation instead of SQL and a list of
parameters.

* Third-party database backends should add a ``DatabaseWrapper.display_name``
attribute with the name of the database that your backend works with. Django
may use it in various messages, such as in system checks.

* The first argument of ``SchemaEditor._alter_column_type_sql()`` is now
``model`` rather than ``table``.

* The first argument of ``SchemaEditor._create_index_name()`` is now
``table_name`` rather than ``model``.

* To enable ``FOR UPDATE OF`` support, set
``DatabaseFeatures.has_select_for_update_of = True``. If the database
requires that the arguments to ``OF`` be columns rather than tables, set
``DatabaseFeatures.select_for_update_of_column = True``.

* To enable support for :class:`~django.db.models.expressions.Window`
expressions, set ``DatabaseFeatures.supports_over_clause`` to ``True``. You
may need to customize the ``DatabaseOperations.window_start_rows_start_end()``
and/or ``window_start_range_start_end()`` methods.

* Third-party database backends should add a
``DatabaseOperations.cast_char_field_without_max_length`` attribute with the
database data type that will be used in the
:class:`~django.db.models.functions.Cast` function for a ``CharField`` if the
``max_length`` argument isn't provided.

* The first argument of ``DatabaseCreation._clone_test_db()`` and
``get_test_db_clone_settings()`` is now ``suffix`` rather
than ``number`` (in case you want to rename the signatures in your backend
for consistency). ``django.test`` also now passes those values as strings
rather than as integers.

* Third-party database backends should add a
``DatabaseIntrospection.get_sequences()`` method based on the stub in
``BaseDatabaseIntrospection``.

Dropped support for Oracle 11.2
-------------------------------

The end of upstream support for Oracle 11.2 is Dec. 2020. Django 1.11 will be
supported until April 2020 which almost reaches this date. Django 2.0
officially supports Oracle 12.1+.

Default MySQL isolation level is read committed
-----------------------------------------------

MySQL's default isolation level, repeatable read, may cause data loss in
typical Django usage. To prevent that and for consistency with other databases,
the default isolation level is now read committed. You can use the
:setting:`DATABASES` setting to :ref:`use a different isolation level
<mysql-isolation-level>`, if needed.

:attr:`AbstractUser.last_name <django.contrib.auth.models.User.last_name>` ``max_length`` increased to 150
----------------------------------------------------------------------------------------------------------

A migration for :attr:`django.contrib.auth.models.User.last_name` is included.
If you have a custom user model inheriting from ``AbstractUser``, you'll need
to generate and apply a database migration for your user model.

If you want to preserve the 30 character limit for last names, use a custom
form::

from django.contrib.auth.forms import UserChangeForm

class MyUserChangeForm(UserChangeForm):
last_name = forms.CharField(max_length=30, required=False)

If you wish to keep this restriction in the admin when editing users, set
``UserAdmin.form`` to use this form::

from django.contrib.auth.admin import UserAdmin
from django.contrib.auth.models import User

class MyUserAdmin(UserAdmin):
form = MyUserChangeForm

admin.site.unregister(User)
admin.site.register(User, MyUserAdmin)

``QuerySet.reverse()`` and ``last()`` are prohibited after slicing
------------------------------------------------------------------

Calling ``QuerySet.reverse()`` or ``last()`` on a sliced queryset leads to
unexpected results due to the slice being applied after reordering. This is
now prohibited, e.g.::

>>> Model.objects.all()[:2].reverse()
Traceback (most recent call last):
...
TypeError: Cannot reverse a query once a slice has been taken.

Form fields no longer accept optional arguments as positional arguments
-----------------------------------------------------------------------

To help prevent runtime errors due to incorrect ordering of form field
arguments, optional arguments of built-in form fields are no longer accepted
as positional arguments. For example::

forms.IntegerField(25, 10)

raises an exception and should be replaced with::

forms.IntegerField(max_value=25, min_value=10)

``call_command()`` validates the options it receives
----------------------------------------------------

``call_command()`` now validates that the argument parser of the command being
called defines all of the options passed to ``call_command()``.

For custom management commands that use options not created using
``parser.add_argument()``, add a ``stealth_options`` attribute on the command::

class MyCommand(BaseCommand):
stealth_options = ('option_name', ...)

Indexes no longer accept positional arguments
---------------------------------------------

For example::

models.Index(['headline', '-pub_date'], 'index_name')

raises an exception and should be replaced with::

models.Index(fields=['headline', '-pub_date'], name='index_name')

Foreign key constraints are now enabled on SQLite
-------------------------------------------------

This will appear as a backwards-incompatible change (``IntegrityError:
FOREIGN KEY constraint failed``) if attempting to save an existing model
instance that's violating a foreign key constraint.

Foreign keys are now created with ``DEFERRABLE INITIALLY DEFERRED`` instead of
``DEFERRABLE IMMEDIATE``. Thus, tables may need to be rebuilt to recreate
foreign keys with the new definition, particularly if you're using a pattern
like this::

from django.db import transaction

with transaction.atomic():
Book.objects.create(author_id=1)
Author.objects.create(id=1)

If you don't recreate the foreign key as ``DEFERRED``, the first ``create()``
would fail now that foreign key constraints are enforced.

Backup your database first! After upgrading to Django 2.0, you can then
rebuild tables using a script similar to this::

from django.apps import apps
from django.db import connection

for app in apps.get_app_configs():
for model in app.get_models(include_auto_created=True):
if model._meta.managed and not (model._meta.proxy or model._meta.swapped):
for base in model.__bases__:
if hasattr(base, '_meta'):
base._meta.local_many_to_many = []
model._meta.local_many_to_many = []
with connection.schema_editor() as editor:
editor._remake_table(model)

This script hasn't received extensive testing and needs adaption for various
cases such as multiple databases. Feel free to contribute improvements.

In addition, because of a table alteration limitation of SQLite, it's prohibited
to perform :class:`~django.db.migrations.operations.RenameModel` and
:class:`~django.db.migrations.operations.RenameField` operations on models or
fields referenced by other models in a transaction. In order to allow migrations
containing these operations to be applied, you must set the
``Migration.atomic`` attribute to ``False``.

Miscellaneous
-------------

* The ``SessionAuthenticationMiddleware`` class is removed. It provided no
functionality since session authentication is unconditionally enabled in
Django 1.10.

* The default HTTP error handlers (``handler404``, etc.) are now callables
instead of dotted Python path strings. Django favors callable references
since they provide better performance and debugging experience.

* :class:`~django.views.generic.base.RedirectView` no longer silences
``NoReverseMatch`` if the ``pattern_name`` doesn't exist.

* When ``USE_L10N`` is off, :class:`~django.forms.FloatField` and
:class:`~django.forms.DecimalField` now respect :setting:`DECIMAL_SEPARATOR`
and :setting:`THOUSAND_SEPARATOR` during validation. For example, with the
settings::

USE_L10N = False
USE_THOUSAND_SEPARATOR = True
DECIMAL_SEPARATOR = ','
THOUSAND_SEPARATOR = '.'

an input of ``"1.345"`` is now converted to ``1345`` instead of ``1.345``.

* Subclasses of :class:`~django.contrib.auth.models.AbstractBaseUser` are no
longer required to implement ``get_short_name()`` and ``get_full_name()``.
(The base implementations that raise ``NotImplementedError`` are removed.)
``django.contrib.admin`` uses these methods if implemented but doesn't
require them. Third-party apps that use these methods may want to adopt a
similar approach.

* The ``FIRST_DAY_OF_WEEK`` and ``NUMBER_GROUPING`` format settings are now
kept as integers in JavaScript and JSON i18n view outputs.

* :meth:`~django.test.TransactionTestCase.assertNumQueries` now ignores
connection configuration queries. Previously, if a test opened a new database
connection, those queries could be included as part of the
``assertNumQueries()`` count.

* The default size of the Oracle test tablespace is increased from 20M to 50M
and the default autoextend size is increased from 10M to 25M.

* To improve performance when streaming large result sets from the database,
:meth:`.QuerySet.iterator` now fetches 2000 rows at a time instead of 100.
The old behavior can be restored using the ``chunk_size`` parameter. For
example::

Book.objects.iterator(chunk_size=100)

* Providing unknown package names in the ``packages`` argument of the
:class:`~django.views.i18n.JavaScriptCatalog` view now raises ``ValueError``
instead of passing silently.

* A model instance's primary key now appears in the default ``Model.__str__()``
method, e.g. ``Question object (1)``.

* ``makemigrations`` now detects changes to the model field ``limit_choices_to``
option. Add this to your existing migrations or accept an auto-generated
migration for fields that use it.

* Performing queries that require :ref:`automatic spatial transformations
<automatic-spatial-transformations>` now raises ``NotImplementedError``
on MySQL instead of silently using non-transformed geometries.

* ``django.core.exceptions.DjangoRuntimeWarning`` is removed. It was only used
in the cache backend as an intermediate class in ``CacheKeyWarning``'s
inheritance of ``RuntimeWarning``.

* Renamed ``BaseExpression._output_field`` to ``output_field``. You may need
to update custom expressions.

* In older versions, forms and formsets combine their ``Media`` with widget
``Media`` by concatenating the two. The combining now tries to :ref:`preserve
the relative order of elements in each list <form-media-asset-order>`.
``MediaOrderConflictWarning`` is issued if the order can't be preserved.

* ``django.contrib.gis.gdal.OGRException`` is removed. It's been an alias for
``GDALException`` since Django 1.8.

* Support for GEOS 3.3.x is dropped.

* The way data is selected for ``GeometryField`` is changed to improve
performance, and in raw SQL queries, those fields must now be wrapped in
``connection.ops.select``. See the :ref:`Raw queries note<gis-raw-sql>` in
the GIS tutorial for an example.

.. _deprecated-features-2.0:

Features deprecated in 2.0
==========================

``context`` argument of ``Field.from_db_value()`` and ``Expression.convert_value()``
------------------------------------------------------------------------------------

The ``context`` argument of ``Field.from_db_value()`` and
``Expression.convert_value()`` is unused as it's always an empty dictionary.
The signature of both methods is now::

(self, value, expression, connection)

instead of::

(self, value, expression, connection, context)

Support for the old signature in custom fields and expressions remains until
Django 3.0.

Miscellaneous
-------------

* The ``django.db.backends.postgresql_psycopg2`` module is deprecated in favor
of ``django.db.backends.postgresql``. It's been an alias since Django 1.9.
This only affects code that imports from the module directly. The
``DATABASES`` setting can still use
``'django.db.backends.postgresql_psycopg2'``, though you can simplify that by
using the ``'django.db.backends.postgresql'`` name added in Django 1.9.

* ``django.shortcuts.render_to_response()`` is deprecated in favor of
:func:`django.shortcuts.render`. ``render()`` takes the same arguments
except that it also requires a ``request``.

* The ``DEFAULT_CONTENT_TYPE`` setting is deprecated. It doesn't interact well
with third-party apps and is obsolete since HTML5 has mostly superseded
XHTML.

* ``HttpRequest.xreadlines()`` is deprecated in favor of iterating over the
request.

* The ``field_name`` keyword argument to :meth:`.QuerySet.earliest` and
:meth:`.QuerySet.latest` is deprecated in favor of passing the field
names as arguments. Write ``.earliest('pub_date')`` instead of
``.earliest(field_name='pub_date')``.

.. _removed-features-2.0:

Features removed in 2.0
=======================

These features have reached the end of their deprecation cycle and are removed
in Django 2.0.

See :ref:`deprecated-features-1.9` for details on these changes, including how
to remove usage of these features.

* The ``weak`` argument to ``django.dispatch.signals.Signal.disconnect()`` is
removed.

* ``django.db.backends.base.BaseDatabaseOperations.check_aggregate_support()``
is removed.

* The ``django.forms.extras`` package is removed.

* The ``assignment_tag`` helper is removed.

* The ``host`` argument to ``SimpleTestCase.assertsRedirects()`` is removed.
The compatibility layer which allows absolute URLs to be considered equal to
relative ones when the path is identical is also removed.

* ``Field.rel`` and ``Field.remote_field.to`` are removed.

* The ``on_delete`` argument for ``ForeignKey`` and ``OneToOneField`` is now
required in models and migrations. Consider squashing migrations so that you
have fewer of them to update.

* ``django.db.models.fields.add_lazy_relation()`` is removed.

* When time zone support is enabled, database backends that don't support time
zones no longer convert aware datetimes to naive values in UTC anymore when
such values are passed as parameters to SQL queries executed outside of the
ORM, e.g. with ``cursor.execute()``.

* ``django.contrib.auth.tests.utils.skipIfCustomUser()`` is removed.

* The ``GeoManager`` and ``GeoQuerySet`` classes are removed.

* The ``django.contrib.gis.geoip`` module is removed.

* The ``supports_recursion`` check for template loaders is removed from:

* ``django.template.engine.Engine.find_template()``
* ``django.template.loader_tags.ExtendsNode.find_template()``
* ``django.template.loaders.base.Loader.supports_recursion()``
* ``django.template.loaders.cached.Loader.supports_recursion()``

* The ``load_template`` and ``load_template_sources`` template loader methods
are removed.

* The ``template_dirs`` argument for template loaders is removed:

* ``django.template.loaders.base.Loader.get_template()``
* ``django.template.loaders.cached.Loader.cache_key()``
* ``django.template.loaders.cached.Loader.get_template()``
* ``django.template.loaders.cached.Loader.get_template_sources()``
* ``django.template.loaders.filesystem.Loader.get_template_sources()``

* ``django.template.loaders.base.Loader.__call__()`` is removed.

* Support for custom error views that don't accept an ``exception`` parameter
is removed.

* The ``mime_type`` attribute of ``django.utils.feedgenerator.Atom1Feed`` and
``django.utils.feedgenerator.RssFeed`` is removed.

* The ``app_name`` argument to ``include()`` is removed.

* Support for passing a 3-tuple (including ``admin.site.urls``) as the first
argument to ``include()`` is removed.

* Support for setting a URL instance namespace without an application namespace
is removed.

* ``Field._get_val_from_obj()`` is removed.

* ``django.template.loaders.eggs.Loader`` is removed.

* The ``current_app`` parameter to the ``contrib.auth`` function-based views is
removed.

* The ``callable_obj`` keyword argument to
``SimpleTestCase.assertRaisesMessage()`` is removed.

* Support for the ``allow_tags`` attribute on ``ModelAdmin`` methods is
removed.

* The ``enclosure`` keyword argument to ``SyndicationFeed.add_item()`` is
removed.

* The ``django.template.loader.LoaderOrigin`` and
``django.template.base.StringOrigin`` aliases for
``django.template.base.Origin`` are removed.

See :ref:`deprecated-features-1.10` for details on these changes.

* The ``makemigrations --exit`` option is removed.

* Support for direct assignment to a reverse foreign key or many-to-many
relation is removed.

* The ``get_srid()`` and ``set_srid()`` methods of
``django.contrib.gis.geos.GEOSGeometry`` are removed.

* The ``get_x()``, ``set_x()``, ``get_y()``, ``set_y()``, ``get_z()``, and
``set_z()`` methods of ``django.contrib.gis.geos.Point`` are removed.

* The ``get_coords()`` and ``set_coords()`` methods of
``django.contrib.gis.geos.Point`` are removed.

* The ``cascaded_union`` property of ``django.contrib.gis.geos.MultiPolygon``
is removed.

* ``django.utils.functional.allow_lazy()`` is removed.

* The ``shell --plain`` option is removed.

* The ``django.core.urlresolvers`` module is removed in favor of its new
location, ``django.urls``.

* ``CommaSeparatedIntegerField`` is removed, except for support in historical
migrations.

* The template ``Context.has_key()`` method is removed.

* Support for the ``django.core.files.storage.Storage.accessed_time()``,
``created_time()``, and ``modified_time()`` methods is removed.

* Support for query lookups using the model name when
``Meta.default_related_name`` is set is removed.

* The MySQL ``__search`` lookup is removed.

* The shim for supporting custom related manager classes without a
``_apply_rel_filters()`` method is removed.

* Using ``User.is_authenticated()`` and ``User.is_anonymous()`` as methods
rather than properties is no longer supported.

* The ``Model._meta.virtual_fields`` attribute is removed.

* The keyword arguments ``virtual_only`` in ``Field.contribute_to_class()`` and
``virtual`` in ``Model._meta.add_field()`` are removed.

* The ``javascript_catalog()`` and ``json_catalog()`` views are removed.

* ``django.contrib.gis.utils.precision_wkt()`` is removed.

* In multi-table inheritance, implicit promotion of a ``OneToOneField`` to a
``parent_link`` is removed.

* Support for ``Widget._format_value()`` is removed.

* ``FileField`` methods ``get_directory_name()`` and ``get_filename()`` are
removed.

* The ``mark_for_escaping()`` function and the classes it uses: ``EscapeData``,
``EscapeBytes``, ``EscapeText``, ``EscapeString``, and ``EscapeUnicode`` are
removed.

* The ``escape`` filter now uses ``django.utils.html.conditional_escape()``.

* ``Manager.use_for_related_fields`` is removed.

* Model ``Manager`` inheritance follows MRO inheritance rules. The requirement
to use ``Meta.manager_inheritance_from_future`` to opt-in to the behavior is
removed.

* Support for old-style middleware using ``settings.MIDDLEWARE_CLASSES`` is
removed.


============================

1.11.29

Not secure
============================

*March 4, 2020*

Django 1.11.29 fixes a security issue in 1.11.28.

CVE-2020-9402: Potential SQL injection via ``tolerance`` parameter in GIS functions and aggregates on Oracle
============================================================================================================

GIS functions and aggregates on Oracle were subject to SQL injection,
using a suitably crafted ``tolerance``.


============================

1.11.28

Not secure
============================

*February 3, 2020*

Django 1.11.28 fixes a security issue in 1.11.27.

CVE-2020-7471: Potential SQL injection via ``StringAgg(delimiter)``
===================================================================

:class:`~django.contrib.postgres.aggregates.StringAgg` aggregation function was
subject to SQL injection, using a suitably crafted ``delimiter``.


============================

1.11.27

Not secure
============================

*December 18, 2019*

Django 1.11.27 fixes a security issue and a data loss bug in 1.11.26.

CVE-2019-19844: Potential account hijack via password reset form
================================================================

By submitting a suitably crafted email address making use of Unicode
characters, that compared equal to an existing user email when lower-cased for
comparison, an attacker could be sent a password reset token for the matched
account.

In order to avoid this vulnerability, password reset requests now compare the
submitted email using the stricter, recommended algorithm for case-insensitive
comparison of two identifiers from `Unicode Technical Report 36, section
2.11.2(B)(2)`__. Upon a match, the email containing the reset token will be
sent to the email address on record rather than the submitted address.

.. __: https://www.unicode.org/reports/tr36/#Recommendations_General

Bugfixes
========

* Fixed a data loss possibility in
:class:`~django.contrib.postgres.forms.SplitArrayField`. When using with
``ArrayField(BooleanField())``, all values after the first ``True`` value
were marked as checked instead of preserving passed values (:ticket:`31073`).


============================

1.11.26

Not secure
============================

*November 4, 2019*

Django 1.11.26 fixes a regression in 1.11.25.

Bugfixes
========

* Fixed a crash when using a ``contains``, ``contained_by``, ``has_key``,
``has_keys``, or ``has_any_keys`` lookup on
``django.contrib.postgres.fields.JSONField``, if the right or left hand
side of an expression is a key transform (:ticket:`30826`).


============================

1.11.25

Not secure
============================

*October 1, 2019*

Django 1.11.25 fixes a regression in 1.11.23.

Bugfixes
========

* Fixed a crash when filtering with a ``Subquery()`` annotation of a queryset
containing ``django.contrib.postgres.fields.JSONField`` or
:class:`~django.contrib.postgres.fields.HStoreField` (:ticket:`30769`).


============================

1.11.24

Not secure
============================

*September 2, 2019*

Django 1.11.24 fixes a regression in 1.11.23.

Bugfixes
========

* Fixed crash of ``KeyTransform()`` for
``django.contrib.postgres.fields.JSONField`` and
:class:`~django.contrib.postgres.fields.HStoreField` when using on
expressions with params (:ticket:`30672`).


============================

1.11.23

Not secure
============================

*August 1, 2019*

Django 1.11.23 fixes security issues in 1.11.22.

CVE-2019-14232: Denial-of-service possibility in ``django.utils.text.Truncator``
================================================================================

If ``django.utils.text.Truncator``'s ``chars()`` and ``words()`` methods
were passed the ``html=True`` argument, they were extremely slow to evaluate
certain inputs due to a catastrophic backtracking vulnerability in a regular
expression. The ``chars()`` and ``words()`` methods are used to implement the
:tfilter:`truncatechars_html` and :tfilter:`truncatewords_html` template
filters, which were thus vulnerable.

The regular expressions used by ``Truncator`` have been simplified in order to
avoid potential backtracking issues. As a consequence, trailing punctuation may
now at times be included in the truncated output.

CVE-2019-14233: Denial-of-service possibility in ``strip_tags()``
=================================================================

Due to the behavior of the underlying ``HTMLParser``,
:func:`django.utils.html.strip_tags` would be extremely slow to evaluate
certain inputs containing large sequences of nested incomplete HTML entities.
The ``strip_tags()`` method is used to implement the corresponding
:tfilter:`striptags` template filter, which was thus also vulnerable.

``strip_tags()`` now avoids recursive calls to ``HTMLParser`` when progress
removing tags, but necessarily incomplete HTML entities, stops being made.

Remember that absolutely NO guarantee is provided about the results of
``strip_tags()`` being HTML safe. So NEVER mark safe the result of a
``strip_tags()`` call without escaping it first, for example with
:func:`django.utils.html.escape`.

CVE-2019-14234: SQL injection possibility in key and index lookups for ``JSONField``/``HStoreField``
====================================================================================================

:lookup:`Key and index lookups <jsonfield.key>` for
``django.contrib.postgres.fields.JSONField`` and :lookup:`key lookups
<hstorefield.key>` for :class:`~django.contrib.postgres.fields.HStoreField`
were subject to SQL injection, using a suitably crafted dictionary, with
dictionary expansion, as the ``**kwargs`` passed to ``QuerySet.filter()``.

CVE-2019-14235: Potential memory exhaustion in ``django.utils.encoding.uri_to_iri()``
=====================================================================================

If passed certain inputs, :func:`django.utils.encoding.uri_to_iri` could lead
to significant memory usage due to excessive recursion when re-percent-encoding
invalid UTF-8 octet sequences.

``uri_to_iri()`` now avoids recursion when re-percent-encoding invalid UTF-8
octet sequences.


============================

1.11.22

Not secure
============================

*July 1, 2019*

Django 1.11.22 fixes a security issue in 1.11.21.

CVE-2019-12781: Incorrect HTTP detection with reverse-proxy connecting via HTTPS
--------------------------------------------------------------------------------

When deployed behind a reverse-proxy connecting to Django via HTTPS,
:attr:`django.http.HttpRequest.scheme` would incorrectly detect client
requests made via HTTP as using HTTPS. This entails incorrect results for
:meth:`~django.http.HttpRequest.is_secure`, and
:meth:`~django.http.HttpRequest.build_absolute_uri`, and that HTTP
requests would not be redirected to HTTPS in accordance with
:setting:`SECURE_SSL_REDIRECT`.

``HttpRequest.scheme`` now respects :setting:`SECURE_PROXY_SSL_HEADER`, if it
is configured, and the appropriate header is set on the request, for both HTTP
and HTTPS requests.

If you deploy Django behind a reverse-proxy that forwards HTTP requests, and
that connects to Django via HTTPS, be sure to verify that your application
correctly handles code paths relying on ``scheme``, ``is_secure()``,
``build_absolute_uri()``, and ``SECURE_SSL_REDIRECT``.


============================

1.11.21

Not secure
============================

*June 3, 2019*

Django 1.11.21 fixes a security issue in 1.11.20.

CVE-2019-12308: AdminURLFieldWidget XSS
---------------------------------------

The clickable "Current URL" link generated by ``AdminURLFieldWidget`` displayed
the provided value without validating it as a safe URL. Thus, an unvalidated
value stored in the database, or a value provided as a URL query parameter
payload, could result in an clickable JavaScript link.

``AdminURLFieldWidget`` now validates the provided value using
:class:`~django.core.validators.URLValidator` before displaying the clickable
link. You may customize the validator by passing a ``validator_class`` kwarg to
``AdminURLFieldWidget.__init__()``, e.g. when using
:attr:`~django.contrib.admin.ModelAdmin.formfield_overrides`.


============================

1.11.20

Not secure
============================

*February 11, 2019*

Django 1.11.20 fixes a packaging error in 1.11.19.

Bugfixes
========

* Corrected packaging error from 1.11.19 (:ticket:`30175`).


============================

1.11.18

Not secure
============================

*January 4, 2019*

Django 1.11.18 fixes a security issue in 1.11.17.

CVE-2019-3498: Content spoofing possibility in the default 404 page
-------------------------------------------------------------------

An attacker could craft a malicious URL that could make spoofed content appear
on the default page generated by the ``django.views.defaults.page_not_found()``
view.

The URL path is no longer displayed in the default 404 template and the
``request_path`` context variable is now quoted to fix the issue for custom
templates that use the path.


============================

1.11.17

Not secure
============================

*December 3, 2018*

Django 1.11.17 fixes several bugs in 1.11.16 and adds compatibility with
Python 3.7.

Bugfixes
========

* Prevented repetitive calls to ``geos_version_tuple()`` in the ``WKBWriter``
class in an attempt to fix a random crash involving ``LooseVersion`` since
Django 1.11.14 (:ticket:`29959`).


============================

1.11.16

Not secure
============================

*October 1, 2018*

Django 1.11.16 fixes a data loss bug in 1.11.15.

Bugfixes
========

* Fixed a race condition in ``QuerySet.update_or_create()`` that could result
in data loss (:ticket:`29499`).


============================

1.11.15

Not secure
============================

*August 1, 2018*

Django 1.11.15 fixes a security issue in 1.11.14.

CVE-2018-14574: Open redirect possibility in ``CommonMiddleware``
=================================================================

If the :class:`~django.middleware.common.CommonMiddleware` and the
:setting:`APPEND_SLASH` setting are both enabled, and if the project has a
URL pattern that accepts any path ending in a slash (many content management
systems have such a pattern), then a request to a maliciously crafted URL of
that site could lead to a redirect to another site, enabling phishing and other
attacks.

``CommonMiddleware`` now escapes leading slashes to prevent redirects to other
domains.


============================

1.11.14

Not secure
============================

*July 2, 2018*

Django 1.11.14 fixes several bugs in 1.11.13.

Bugfixes
========

* Fixed ``WKBWriter.write()`` and ``write_hex()`` for empty polygons on
GEOS 3.6.1+ (:ticket:`29460`).

* Fixed a regression in Django 1.10 that could result in large memory usage
when making edits using ``ModelAdmin.list_editable`` (:ticket:`28462`).


============================

1.11.13

Not secure
============================

*May 1, 2018*

Django 1.11.13 fixes several bugs in 1.11.12.

Bugfixes
========

* Fixed a regression in Django 1.11.8 where altering a field with a unique
constraint may drop and rebuild more foreign keys than necessary
(:ticket:`29193`).

* Fixed crashes in ``django.contrib.admindocs`` when a view is a callable
object, such as ``django.contrib.syndication.views.Feed`` (:ticket:`29296`).

* Fixed a regression in Django 1.11.12 where ``QuerySet.values()`` or
``values_list()`` after combining an annotated and unannotated queryset with
``union()``, ``difference()``, or ``intersection()`` crashed due to mismatching
columns (:ticket:`29286`).


============================

1.11.12

Not secure
============================

*April 2, 2018*

Django 1.11.12 fixes two bugs in 1.11.11.

Bugfixes
========

* Fixed a regression in Django 1.11.8 where combining two annotated
``values_list()`` querysets with ``union()``, ``difference()``, or
``intersection()`` crashed due to mismatching columns (:ticket:`29229`).

* Fixed a regression in Django 1.11 where an empty choice could be initially
selected for the ``SelectMultiple`` and ``CheckboxSelectMultiple`` widgets
(:ticket:`29273`).


============================

1.11.11

Not secure
============================

*March 6, 2018*

Django 1.11.11 fixes two security issues in 1.11.10.

CVE-2018-7536: Denial-of-service possibility in ``urlize`` and ``urlizetrunc`` template filters
===============================================================================================

The ``django.utils.html.urlize()`` function was extremely slow to evaluate
certain inputs due to catastrophic backtracking vulnerabilities in two regular
expressions. The ``urlize()`` function is used to implement the ``urlize`` and
``urlizetrunc`` template filters, which were thus vulnerable.

The problematic regular expressions are replaced with parsing logic that
behaves similarly.

CVE-2018-7537: Denial-of-service possibility in ``truncatechars_html`` and ``truncatewords_html`` template filters
==================================================================================================================

If ``django.utils.text.Truncator``'s ``chars()`` and ``words()`` methods were
passed the ``html=True`` argument, they were extremely slow to evaluate certain
inputs due to a catastrophic backtracking vulnerability in a regular
expression. The ``chars()`` and ``words()`` methods are used to implement the
``truncatechars_html`` and ``truncatewords_html`` template filters, which were
thus vulnerable.

The backtracking problem in the regular expression is fixed.


============================

1.11.10

Not secure
============================

*February 1, 2018*

Django 1.11.10 fixes a security issue and several bugs in 1.11.9.

CVE-2018-6188: Information leakage in ``AuthenticationForm``
============================================================

A regression in Django 1.11.8 made
:class:`~django.contrib.auth.forms.AuthenticationForm` run its
``confirm_login_allowed()`` method even if an incorrect password is entered.
This can leak information about a user, depending on what messages
``confirm_login_allowed()`` raises. If ``confirm_login_allowed()`` isn't
overridden, an attacker enter an arbitrary username and see if that user has
been set to ``is_active=False``. If ``confirm_login_allowed()`` is overridden,
more sensitive details could be leaked.

This issue is fixed with the caveat that ``AuthenticationForm`` can no longer
raise the "This account is inactive." error if the authentication backend
rejects inactive users (the default authentication backend, ``ModelBackend``,
has done that since Django 1.10). This issue will be revisited for Django 2.1
as a fix to address the caveat will likely be too invasive for inclusion in
older versions.

Bugfixes
========

* Fixed incorrect foreign key nullification if a model has two foreign keys to
the same model and a target model is deleted (:ticket:`29016`).

* Fixed a regression where ``contrib.auth.authenticate()`` crashes if an
authentication backend doesn't accept ``request`` and a later one does
(:ticket:`29071`).

* Fixed crash when entering an invalid uuid in ``ModelAdmin.raw_id_fields``
(:ticket:`29094`).


===========================

1.11.9

Not secure
===========================

*January 1, 2018*

Django 1.11.9 fixes several bugs in 1.11.8.

Bugfixes
========

* Fixed a regression in Django 1.11 that added newlines between ``MultiWidget``'s
subwidgets (:ticket:`28890`).

* Fixed incorrect class-based model index name generation for models with
quoted ``db_table`` (:ticket:`28876`).

* Fixed incorrect foreign key constraint name for models with quoted
``db_table`` (:ticket:`28876`).

* Fixed a regression in caching of a ``GenericForeignKey`` when the referenced
model instance uses more than one level of multi-table inheritance
(:ticket:`28856`).


===========================

1.11.8

Not secure
===========================

*December 2, 2017*

Django 1.11.8 fixes several bugs in 1.11.7.

Bugfixes
========

* Reallowed, following a regression in Django 1.10, ``AuthenticationForm`` to
raise the inactive user error when using ``ModelBackend`` (:ticket:`28645`).

* Added support for ``QuerySet.values()`` and ``values_list()`` for
``union()``, ``difference()``, and ``intersection()`` queries
(:ticket:`28781`).

* Fixed incorrect index name truncation when using a namespaced ``db_table``
(:ticket:`28792`).

* Made ``QuerySet.iterator()`` use server-side cursors on PostgreSQL after
``values()`` and ``values_list()`` (:ticket:`28817`).

* Fixed crash on SQLite and MySQL when ordering by a filtered subquery that
uses ``nulls_first`` or ``nulls_last`` (:ticket:`28848`).

* Made query lookups for ``CICharField``, ``CIEmailField``, and ``CITextField``
use a ``citext`` cast (:ticket:`28702`).

* Fixed a regression in caching of a ``GenericForeignKey`` when the referenced
model instance uses multi-table inheritance (:ticket:`28856`).

* Fixed "Cannot change column 'x': used in a foreign key constraint" crash on
MySQL with a sequence of ``AlterField`` and/or ``RenameField`` operations in
a migration (:ticket:`28305`).


===========================

1.11.7

Not secure
===========================

*November 1, 2017*

Django 1.11.7 fixes several bugs in 1.11.6.

Bugfixes
========

* Prevented ``cache.get_or_set()`` from caching ``None`` if the ``default``
argument is a callable that returns ``None`` (:ticket:`28601`).

* Fixed the Basque ``DATE_FORMAT`` string (:ticket:`28710`).

* Made ``QuerySet.reverse()`` affect ``nulls_first`` and ``nulls_last``
(:ticket:`28722`).

* Fixed unquoted table names in ``Subquery`` SQL when using ``OuterRef``
(:ticket:`28689`).


===========================

1.11.6

Not secure
===========================

*October 5, 2017*

Django 1.11.6 fixes several bugs in 1.11.5.

Bugfixes
========

* Made the ``CharField`` form field convert whitespace-only values to the
``empty_value`` when ``strip`` is enabled (:ticket:`28555`).

* Fixed crash when using the name of a model's autogenerated primary key
(``id``) in an ``Index``'s ``fields`` (:ticket:`28597`).

* Fixed a regression in Django 1.9 where a custom view error handler such as
``handler404`` that accesses ``csrf_token`` could cause CSRF verification
failures on other pages (:ticket:`28488`).


===========================

1.11.5

Not secure
===========================

*September 5, 2017*

Django 1.11.5 fixes a security issue and several bugs in 1.11.4.

CVE-2017-12794: Possible XSS in traceback section of technical 500 debug page
=============================================================================

In older versions, HTML autoescaping was disabled in a portion of the template
for the technical 500 debug page. Given the right circumstances, this allowed
a cross-site scripting attack. This vulnerability shouldn't affect most
production sites since you shouldn't run with ``DEBUG = True`` (which makes
this page accessible) in your production settings.

Bugfixes
========

* Fixed GEOS version parsing if the version has a commit hash at the end (new
in GEOS 3.6.2) (:ticket:`28441`).

* Added compatibility for ``cx_Oracle`` 6 (:ticket:`28498`).

* Fixed select widget rendering when option values are tuples (:ticket:`28502`).

* Django 1.11 inadvertently changed the sequence and trigger naming scheme on
Oracle. This causes errors on INSERTs for some tables if
``'use_returning_into': False`` is in the ``OPTIONS`` part of ``DATABASES``.
The pre-1.11 naming scheme is now restored. Unfortunately, it necessarily
requires an update to Oracle tables created with Django 1.11.[1-4]. Use the
upgrade script in :ticket:`28451` comment 8 to update sequence and trigger
names to use the pre-1.11 naming scheme.

* Added POST request support to ``LogoutView``, for equivalence with the
function-based ``logout()`` view (:ticket:`28513`).

* Omitted ``pages_per_range`` from ``BrinIndex.deconstruct()`` if it's ``None``
(:ticket:`25809`).

* Fixed a regression where ``SelectDateWidget`` localized the years in the
select box (:ticket:`28530`).

* Fixed a regression in 1.11.4 where ``runserver`` crashed with non-Unicode
system encodings on Python 2 + Windows (:ticket:`28487`).

* Fixed a regression in Django 1.10 where changes to a ``ManyToManyField``
weren't logged in the admin change history (:ticket:`27998`) and prevented
``ManyToManyField`` initial data in model forms from being affected by
subsequent model changes (:ticket:`28543`).

* Fixed non-deterministic results or an ``AssertionError`` crash in some
queries with multiple joins (:ticket:`26522`).

* Fixed a regression in ``contrib.auth``'s ``login()`` and ``logout()`` views
where they ignored positional arguments (:ticket:`28550`).


===========================

1.11.4

Not secure
===========================

*August 1, 2017*

Django 1.11.4 fixes several bugs in 1.11.3.

Bugfixes
========

* Fixed a regression in 1.11.3 on Python 2 where non-ASCII ``format`` values
for date/time widgets results in an empty ``value`` in the widget's HTML
(:ticket:`28355`).

* Fixed ``QuerySet.union()`` and ``difference()`` when combining with
a queryset raising ``EmptyResultSet`` (:ticket:`28378`).

* Fixed a regression in pickling of ``LazyObject`` on Python 2 when the wrapped
object doesn't have ``__reduce__()`` (:ticket:`28389`).

* Fixed crash in ``runserver``'s ``autoreload`` with Python 2 on Windows with
non-``str`` environment variables (:ticket:`28174`).

* Corrected ``Field.has_changed()`` to return ``False`` for disabled form
fields: ``BooleanField``, ``MultipleChoiceField``, ``MultiValueField``,
``FileField``, ``ModelChoiceField``, and ``ModelMultipleChoiceField``.

* Fixed ``QuerySet.count()`` for ``union()``, ``difference()``, and
``intersection()`` queries. (:ticket:`28399`).

* Fixed ``ClearableFileInput`` rendering as a subwidget of ``MultiWidget``
(:ticket:`28414`). Custom ``clearable_file_input.html`` widget templates
will need to adapt for the fact that context values
``checkbox_name``, ``checkbox_id``, ``is_initial``, ``input_text``,
``initial_text``, and ``clear_checkbox_label`` are now attributes of
``widget`` rather than appearing in the top-level context.

* Fixed queryset crash when using a ``GenericRelation`` to a proxy model
(:ticket:`28418`).


===========================

1.11.3

Not secure
===========================

*July 1, 2017*

Django 1.11.3 fixes several bugs in 1.11.2.

Bugfixes
========

* Removed an incorrect deprecation warning about a missing ``renderer``
argument if a ``Widget.render()`` method accepts ``**kwargs``
(:ticket:`28265`).

* Fixed a regression causing ``Model.__init__()`` to crash if a field has an
instance only descriptor (:ticket:`28269`).

* Fixed an incorrect ``DisallowedModelAdminLookup`` exception when using
a nested reverse relation in ``list_filter`` (:ticket:`28262`).

* Fixed admin's ``FieldListFilter.get_queryset()`` crash on invalid input
(:ticket:`28202`).

* Fixed invalid HTML for a required ``AdminFileWidget`` (:ticket:`28278`).

* Fixed model initialization to set the name of class-based model indexes
for models that only inherit ``models.Model`` (:ticket:`28282`).

* Fixed crash in admin's inlines when a model has an inherited non-editable
primary key (:ticket:`27967`).

* Fixed ``QuerySet.union()``, ``intersection()``, and ``difference()`` when
combining with an ``EmptyQuerySet`` (:ticket:`28293`).

* Prevented ``Paginator``’s unordered object list warning from evaluating a
``QuerySet`` (:ticket:`28284`).

* Fixed the value of ``redirect_field_name`` in ``LoginView``’s template
context. It's now an empty string (as it is for the original function-based
``login()`` view) if the corresponding parameter isn't sent in a request (in
particular, when the login page is accessed directly) (:ticket:`28229`).

* Prevented attribute values in the ``django/forms/widgets/attrs.html``
template from being localized so that numeric attributes (e.g. ``max`` and
``min``) of ``NumberInput`` work correctly (:ticket:`28303`).

* Removed casting of the option value to a string in the template context of
the ``CheckboxSelectMultiple``, ``NullBooleanSelect``, ``RadioSelect``,
``SelectMultiple``, and ``Select`` widgets (:ticket:`28176`). In Django
1.11.1, casting was added in Python to avoid localization of numeric values
in Django templates, but this made some use cases more difficult. Casting is
now done in the template using the ``|stringformat:'s'`` filter.

* Prevented a primary key alteration from adding a foreign key constraint if
``db_constraint=False`` (:ticket:`28298`).

* Fixed ``UnboundLocalError`` crash in ``RenameField`` with nonexistent field
(:ticket:`28350`).

* Fixed a regression preventing a model field's ``limit_choices_to`` from being
evaluated when a ``ModelForm`` is instantiated (:ticket:`28345`).


===========================

1.11.2

Not secure
===========================

*June 1, 2017*

Django 1.11.2 adds a minor feature and fixes several bugs in 1.11.1. Also, the
latest string translations from Transifex are incorporated.

Minor feature
=============

The new ``LiveServerTestCase.port`` attribute reallows the use case of binding
to a specific port following the :ref:`bind to port zero
<liveservertestcase-port-zero-change>` change in Django 1.11.

Bugfixes
========

* Added detection for GDAL 2.1 and 2.0, and removed detection for unsupported
versions 1.7 and 1.8 (:ticket:`28181`).

* Changed ``contrib.gis`` to raise ``ImproperlyConfigured`` rather than
``GDALException`` if ``gdal`` isn't installed, to allow third-party apps to
catch that exception (:ticket:`28178`).

* Fixed ``django.utils.http.is_safe_url()`` crash on invalid IPv6 URLs
(:ticket:`28142`).

* Fixed regression causing pickling of model fields to crash (:ticket:`28188`).

* Fixed ``django.contrib.auth.authenticate()`` when multiple authentication
backends don't accept a positional ``request`` argument (:ticket:`28207`).

* Fixed introspection of index field ordering on PostgreSQL (:ticket:`28197`).

* Fixed a regression where ``Model._state.adding`` wasn't set correctly on
multi-table inheritance parent models after saving a child model
(:ticket:`28210`).

* Allowed ``DjangoJSONEncoder`` to serialize
``django.utils.deprecation.CallableBool`` (:ticket:`28230`).

* Relaxed the validation added in Django 1.11 of the fields in the ``defaults``
argument of ``QuerySet.get_or_create()`` and ``update_or_create()`` to
reallow settable model properties (:ticket:`28222`).

* Fixed ``MultipleObjectMixin.paginate_queryset()`` crash on Python 2 if the
``InvalidPage`` message contains non-ASCII (:ticket:`28204`).

* Prevented ``Subquery`` from adding an unnecessary ``CAST`` which resulted in
invalid SQL (:ticket:`28199`).

* Corrected detection of GDAL 2.1 on Windows (:ticket:`28181`).

* Made date-based generic views return a 404 rather than crash when given an
out of range date (:ticket:`28209`).

* Fixed a regression where ``file_move_safe()`` crashed when moving files to a
CIFS mount (:ticket:`28170`).

* Moved the ``ImageField`` file extension validation added in Django 1.11 from
the model field to the form field to reallow the use case of storing images
without an extension (:ticket:`28242`).


===========================

1.11.1

Not secure
===========================

*May 6, 2017*

Django 1.11.1 adds a minor feature and fixes several bugs in 1.11.

Allowed disabling server-side cursors on PostgreSQL
===================================================

The change in Django 1.11 to make :meth:`.QuerySet.iterator()` use server-side
cursors on PostgreSQL prevents running Django with PgBouncer in transaction
pooling mode. To reallow that, use the :setting:`DISABLE_SERVER_SIDE_CURSORS
<DATABASE-DISABLE_SERVER_SIDE_CURSORS>` setting in :setting:`DATABASES`.

See :ref:`transaction-pooling-server-side-cursors` for more discussion.

Bugfixes
========

* Made migrations respect ``Index``’s ``name`` argument. If you created a
named index with Django 1.11, ``makemigrations`` will create a migration to
recreate the index with the correct name (:ticket:`28051`).

* Fixed a crash when using a ``__icontains`` lookup on a ``ArrayField``
(:ticket:`28038`).

* Fixed a crash when using a two-tuple in ``EmailMessage``’s ``attachments``
argument (:ticket:`28042`).

* Fixed ``QuerySet.filter()`` crash when it references the name of a
``OneToOneField`` primary key (:ticket:`28047`).

* Fixed empty POST data table appearing instead of "No POST data" in HTML debug
page (:ticket:`28079`).

* Restored ``BoundField``\s without any ``choices`` evaluating to ``True``
(:ticket:`28058`).

* Prevented ``SessionBase.cycle_key()`` from losing session data if
``_session_cache`` isn't populated (:ticket:`28066`).

* Fixed layout of ``ReadOnlyPasswordHashWidget`` (used in the admin's user
change page) (:ticket:`28097`).

* Allowed prefetch calls on managers with custom ``ModelIterable`` subclasses
(:ticket:`28096`).

* Fixed change password link in the ``contrib.auth`` admin for ``el``,
``es_MX``, and ``pt`` translations (:ticket:`28100`).

* Restored the output of the ``class`` attribute in the ``<ul>`` of widgets
that use the ``multiple_input.html`` template. This fixes
``ModelAdmin.radio_fields`` with ``admin.HORIZONTAL`` (:ticket:`28059`).

* Fixed crash in ``BaseGeometryWidget.subwidgets()`` (:ticket:`28039`).

* Fixed exception reraising in ORM query execution when ``cursor.execute()``
fails and the subsequent ``cursor.close()`` also fails (:ticket:`28091`).

* Fixed a regression where ``CheckboxSelectMultiple``, ``NullBooleanSelect``,
``RadioSelect``, ``SelectMultiple``, and ``Select`` localized option values
(:ticket:`28075`).

* Corrected the stack level of unordered queryset pagination warnings
(:ticket:`28109`).

* Fixed a regression causing incorrect queries for ``__in`` subquery lookups
when models use ``ForeignKey.to_field`` (:ticket:`28101`).

* Fixed crash when overriding the template of
``django.views.static.directory_index()`` (:ticket:`28122`).

* Fixed a regression in formset ``min_num`` validation with unchanged forms
that have initial data (:ticket:`28130`).

* Prepared for ``cx_Oracle`` 6.0 support (:ticket:`28138`).

* Updated the ``contrib.postgres`` ``SplitArrayWidget`` to use template-based
widget rendering (:ticket:`28040`).

* Fixed crash in ``BaseGeometryWidget.get_context()`` when overriding existing
``attrs`` (:ticket:`28105`).

* Prevented ``AddIndex`` and ``RemoveIndex`` from mutating model state
(:ticket:`28043`).

* Prevented migrations from dropping database indexes from ``Meta.indexes``
when changing ``Field.db_index`` to ``False`` (:ticket:`28052`).

* Fixed a regression in choice ordering in form fields with grouped and
non-grouped options (:ticket:`28157`).

* Fixed crash in ``BaseInlineFormSet._construct_form()`` when using
``save_as_new`` (:ticket:`28159`).

* Fixed a regression where ``Model._state.db`` wasn't set correctly on
multi-table inheritance parent models after saving a child model
(:ticket:`28166`).

* Corrected the return type of ``ArrayField(CITextField())`` values retrieved
from the database (:ticket:`28161`).

* Fixed ``QuerySet.prefetch_related()`` crash when fetching relations in nested
``Prefetch`` objects (:ticket:`27554`).

* Prevented hiding GDAL errors if it's not installed when using ``contrib.gis``
(:ticket:`28160`). (It's a required dependency as of Django 1.11.)

* Fixed a regression causing ``__in`` lookups on a foreign key to fail when
using the foreign key's parent model as the lookup value (:ticket:`28175`).


=========================

1.11

Not secure
=========================

*April 4, 2017*

Welcome to Django 1.11!

These release notes cover the :ref:`new features <whats-new-1.11>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-1.11>` you'll
want to be aware of when upgrading from Django 1.10 or older versions. We've
:ref:`begun the deprecation process for some features
<deprecated-features-1.11>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Django 1.11 is designated as a :term:`long-term support release
<Long-term support release>`. It will receive security updates for at least
three years after its release. Support for the previous LTS, Django 1.8, will
end in April 2018.

Python compatibility
====================

Django 1.11 requires Python 2.7, 3.4, 3.5, 3.6, or 3.7 (as of 1.11.17). We
**highly recommend** and only officially support the latest release of each
series.

The Django 1.11.x series is the last to support Python 2. The next major
release, Django 2.0, will only support Python 3.4+.

Deprecating warnings are no longer loud by default
==================================================

Unlike older versions of Django, Django's own deprecation warnings are no
longer displayed by default. This is consistent with Python's default behavior.

This change allows third-party apps to support both Django 1.11 LTS and Django
1.8 LTS without having to add code to avoid deprecation warnings.

Following the release of Django 2.0, we suggest that third-party app authors
drop support for all versions of Django prior to 1.11. At that time, you should
be able run your package's tests using ``python -Wd`` so that deprecation
warnings do appear. After making the deprecation warning fixes, your app should
be compatible with Django 2.0.

.. _whats-new-1.11:

What's new in Django 1.11
=========================

Class-based model indexes
-------------------------

The new :mod:`django.db.models.indexes` module contains classes which ease
creating database indexes. Indexes are added to models using the
:attr:`Meta.indexes <django.db.models.Options.indexes>` option.

The :class:`~django.db.models.Index` class creates a b-tree index, as if you
used :attr:`~django.db.models.Field.db_index` on the model field or
:attr:`~django.db.models.Options.index_together` on the model ``Meta`` class.
It can be subclassed to support different index types, such as
:class:`~django.contrib.postgres.indexes.GinIndex`. It also allows defining the
order (ASC/DESC) for the columns of the index.

Template-based widget rendering
-------------------------------

To ease customizing widgets, form widget rendering is now done using the
template system rather than in Python. See :doc:`/ref/forms/renderers`.

You may need to adjust any custom widgets that you've written for a few
:ref:`backwards incompatible changes <template-widget-incompatibilities-1-11>`.

``Subquery`` expressions
------------------------

The new :class:`~django.db.models.Subquery` and
:class:`~django.db.models.Exists` database expressions allow creating
explicit subqueries. Subqueries may refer to fields from the outer queryset
using the :class:`~django.db.models.OuterRef` class.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* :attr:`.ModelAdmin.date_hierarchy` can now reference fields across relations.

* The new :meth:`ModelAdmin.get_exclude()
<django.contrib.admin.ModelAdmin.get_exclude>` hook allows specifying the
exclude fields based on the request or model instance.

* The ``popup_response.html`` template can now be overridden per app, per
model, or by setting the :attr:`.ModelAdmin.popup_response_template`
attribute.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* The default iteration count for the PBKDF2 password hasher is increased by
20%.

* The :class:`~django.contrib.auth.views.LoginView` and
:class:`~django.contrib.auth.views.LogoutView` class-based views supersede the
deprecated ``login()`` and ``logout()`` function-based views.

* The :class:`~django.contrib.auth.views.PasswordChangeView`,
:class:`~django.contrib.auth.views.PasswordChangeDoneView`,
:class:`~django.contrib.auth.views.PasswordResetView`,
:class:`~django.contrib.auth.views.PasswordResetDoneView`,
:class:`~django.contrib.auth.views.PasswordResetConfirmView`, and
:class:`~django.contrib.auth.views.PasswordResetCompleteView` class-based
views supersede the deprecated ``password_change()``,
``password_change_done()``, ``password_reset()``, ``password_reset_done()``,
``password_reset_confirm()``, and ``password_reset_complete()`` function-based
views.

* The new ``post_reset_login`` attribute for
:class:`~django.contrib.auth.views.PasswordResetConfirmView` allows
automatically logging in a user after a successful password reset.
If you have multiple ``AUTHENTICATION_BACKENDS`` configured, use the
``post_reset_login_backend`` attribute to choose which one to use.

* To avoid the possibility of leaking a password reset token via the HTTP
Referer header (for example, if the reset page includes a reference to CSS or
JavaScript hosted on another domain), the
:class:`~django.contrib.auth.views.PasswordResetConfirmView` (but not the
deprecated ``password_reset_confirm()`` function-based view) stores the token
in a session and redirects to itself to present the password change form to
the user without the token in the URL.

* :func:`~django.contrib.auth.update_session_auth_hash` now rotates the session
key to allow a password change to invalidate stolen session cookies.

* The new ``success_url_allowed_hosts`` attribute for
:class:`~django.contrib.auth.views.LoginView` and
:class:`~django.contrib.auth.views.LogoutView` allows specifying a set of
hosts that are safe for redirecting after login and logout.

* Added password validators ``help_text`` to
:class:`~django.contrib.auth.forms.UserCreationForm`.

* The ``HttpRequest`` is now passed to :func:`~django.contrib.auth.authenticate`
which in turn passes it to the authentication backend if it accepts a
``request`` argument.

* The :func:`~django.contrib.auth.signals.user_login_failed` signal now
receives a ``request`` argument.

* :class:`~django.contrib.auth.forms.PasswordResetForm` supports custom user
models that use an email field named something other than ``'email'``.
Set :attr:`CustomUser.EMAIL_FIELD
<django.contrib.auth.models.CustomUser.EMAIL_FIELD>` to the name of the field.

* :func:`~django.contrib.auth.get_user_model` can now be called at import time,
even in modules that define models.

:mod:`django.contrib.contenttypes`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* When stale content types are detected in the
:djadmin:`remove_stale_contenttypes` command, there's now a list of related
objects such as ``auth.Permission``\s that will also be deleted. Previously,
only the content types were listed (and this prompt was after ``migrate``
rather than in a separate command).

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :meth:`.GEOSGeometry.from_gml` and :meth:`.OGRGeometry.from_gml`
methods allow creating geometries from GML.

* Added support for the :lookup:`dwithin` lookup on SpatiaLite.

* The :class:`~django.contrib.gis.db.models.functions.Area` function,
:class:`~django.contrib.gis.db.models.functions.Distance` function, and
distance lookups now work with geodetic coordinates on SpatiaLite.

* The OpenLayers-based form widgets now use ``OpenLayers.js`` from
``https://cdnjs.cloudflare.com`` which is more suitable for production use
than the old ``https://openlayers.org/`` source. They are also updated to use
OpenLayers 3.

* PostGIS migrations can now change field dimensions.

* Added the ability to pass the ``size``, ``shape``, and ``offset`` parameters
when creating :class:`~django.contrib.gis.gdal.GDALRaster` objects.

* Added SpatiaLite support for the
:class:`~django.contrib.gis.db.models.functions.IsValid` function,
:class:`~django.contrib.gis.db.models.functions.MakeValid` function, and
:lookup:`isvalid` lookup.

* Added Oracle support for the
:class:`~django.contrib.gis.db.models.functions.AsGML` function,
:class:`~django.contrib.gis.db.models.functions.BoundingCircle` function,
:class:`~django.contrib.gis.db.models.functions.IsValid` function, and
:lookup:`isvalid` lookup.

:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new ``distinct`` argument for
:class:`~django.contrib.postgres.aggregates.StringAgg` determines if
concatenated values will be distinct.

* The new :class:`~django.contrib.postgres.indexes.GinIndex` and
:class:`~django.contrib.postgres.indexes.BrinIndex` classes allow
creating ``GIN`` and ``BRIN`` indexes in the database.

* ``django.contrib.postgres.fields.JSONField`` accepts a new ``encoder``
parameter to specify a custom class to encode data types not supported by the
standard encoder.

* The new :class:`~django.contrib.postgres.fields.CIText` mixin and
:class:`~django.contrib.postgres.operations.CITextExtension` migration
operation allow using PostgreSQL's ``citext`` extension for case-insensitive
lookups. Three fields are provided: :class:`.CICharField`,
:class:`.CIEmailField`, and :class:`.CITextField`.

* The new :class:`~django.contrib.postgres.aggregates.JSONBAgg` allows
aggregating values as a JSON array.

* The :class:`~django.contrib.postgres.fields.HStoreField` (model field) and
:class:`~django.contrib.postgres.forms.HStoreField` (form field) allow
storing null values.

Cache
~~~~~

* Memcached backends now pass the contents of :setting:`OPTIONS <CACHES-OPTIONS>`
as keyword arguments to the client constructors, allowing for more advanced
control of client behavior. See the :ref:`cache arguments <cache_arguments>`
documentation for examples.

* Memcached backends now allow defining multiple servers as a comma-delimited
string in :setting:`LOCATION <CACHES-LOCATION>`, for convenience with
third-party services that use such strings in environment variables.

CSRF
~~~~

* Added the :setting:`CSRF_USE_SESSIONS` setting to allow storing the CSRF
token in the user's session rather than in a cookie.

Database backends
~~~~~~~~~~~~~~~~~

* Added the ``skip_locked`` argument to :meth:`.QuerySet.select_for_update()`
on PostgreSQL 9.5+ and Oracle to execute queries with
``FOR UPDATE SKIP LOCKED``.

* Added the :setting:`TEST['TEMPLATE'] <TEST_TEMPLATE>` setting to let
PostgreSQL users specify a template for creating the test database.

* :meth:`.QuerySet.iterator()` now uses :ref:`server-side cursors
<psycopg2:server-side-cursors>` on PostgreSQL. This feature transfers some of
the worker memory load (used to hold query results) to the database and might
increase database memory usage.

* Added MySQL support for the ``'isolation_level'`` option in
:setting:`OPTIONS` to allow specifying the :ref:`transaction isolation level
<mysql-isolation-level>`. To avoid possible data loss, it's recommended to
switch from MySQL's default level, repeatable read, to read committed.

* Added support for ``cx_Oracle`` 5.3.

Email
~~~~~

* Added the :setting:`EMAIL_USE_LOCALTIME` setting to allow sending SMTP date
headers in the local time zone rather than in UTC.

* ``EmailMessage.attach()`` and ``attach_file()`` now fall back to MIME type
:mimetype:`application/octet-stream` when binary content that can't be
decoded as UTF-8 is specified for a :mimetype:`text/*` attachment.

File Storage
~~~~~~~~~~~~

* To make it wrappable by :class:`io.TextIOWrapper`,
:class:`~django.core.files.File` now has the ``readable()``, ``writable()``,
and ``seekable()`` methods.

Forms
~~~~~

* The new ``empty_value`` attribute on :class:`~django.forms.CharField`,
:class:`~django.forms.EmailField`, :class:`~django.forms.RegexField`,
:class:`~django.forms.SlugField`, and :class:`~django.forms.URLField` allows
specifying the Python value to use to represent "empty".

* The new :meth:`Form.get_initial_for_field()
<django.forms.Form.get_initial_for_field>` method returns initial data for a
form field.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* Number formatting and the :setting:`NUMBER_GROUPING` setting support
non-uniform digit grouping.

Management Commands
~~~~~~~~~~~~~~~~~~~

* The new :option:`loaddata --exclude` option allows excluding models and apps
while loading data from fixtures.

* The new :option:`diffsettings --default` option allows specifying a settings
module other than Django's default settings to compare against.

* ``app_label``\s arguments now limit the :option:`showmigrations --plan`
output.

Migrations
~~~~~~~~~~

* Added support for serialization of ``uuid.UUID`` objects.

Models
~~~~~~

* Added support for callable values in the ``defaults`` argument of
:meth:`QuerySet.update_or_create()
<django.db.models.query.QuerySet.update_or_create>` and
:meth:`~django.db.models.query.QuerySet.get_or_create`.

* :class:`~django.db.models.ImageField` now has a default
:data:`~django.core.validators.validate_image_file_extension` validator.
(This validator moved to the form field in :doc:`Django 1.11.2 <1.11.2>`.)

* Added support for time truncation to
:class:`~django.db.models.functions.Trunc` functions.

* Added the :class:`~django.db.models.functions.ExtractWeek` function to
extract the week from :class:`~django.db.models.DateField` and
:class:`~django.db.models.DateTimeField` and exposed it through the
:lookup:`week` lookup.

* Added the :class:`~django.db.models.functions.TruncTime` function to truncate
:class:`~django.db.models.DateTimeField` to its time component and exposed it
through the :lookup:`time` lookup.

* Added support for expressions in :meth:`.QuerySet.values` and
:meth:`~.QuerySet.values_list`.

* Added support for query expressions on lookups that take multiple arguments,
such as ``range``.

* You can now use the ``unique=True`` option with
:class:`~django.db.models.FileField`.

* Added the ``nulls_first`` and ``nulls_last`` parameters to
:class:`Expression.asc() <django.db.models.Expression.asc>` and
:meth:`~django.db.models.Expression.desc` to control
the ordering of null values.

* The new ``F`` expression ``bitleftshift()`` and ``bitrightshift()`` methods
allow :ref:`bitwise shift operations <using-f-expressions-in-filters>`.

* Added :meth:`.QuerySet.union`, :meth:`~.QuerySet.intersection`, and
:meth:`~.QuerySet.difference`.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* Added :meth:`QueryDict.fromkeys() <django.http.QueryDict.fromkeys>`.

* :class:`~django.middleware.common.CommonMiddleware` now sets the
``Content-Length`` response header for non-streaming responses.

* Added the :setting:`SECURE_HSTS_PRELOAD` setting to allow appending the
``preload`` directive to the ``Strict-Transport-Security`` header.

* :class:`~django.middleware.http.ConditionalGetMiddleware` now adds the
``ETag`` header to responses.

Serialization
~~~~~~~~~~~~~

* The new ``django.core.serializers.base.Serializer.stream_class`` attribute
allows subclasses to customize the default stream.

* The encoder used by the :ref:`JSON serializer <serialization-formats-json>`
can now be customized by passing a ``cls`` keyword argument to the
``serializers.serialize()`` function.

* :class:`~django.core.serializers.json.DjangoJSONEncoder` now serializes
:class:`~datetime.timedelta` objects (used by
:class:`~django.db.models.DurationField`).

Templates
~~~~~~~~~

* :meth:`~django.utils.safestring.mark_safe` can now be used as a decorator.

* The :class:`~django.template.backends.jinja2.Jinja2` template backend now
supports context processors by setting the ``'context_processors'`` option in
:setting:`OPTIONS <TEMPLATES-OPTIONS>`.

* The :ttag:`regroup` tag now returns ``namedtuple``\s instead of dictionaries
so you can unpack the group object directly in a loop, e.g.
``{% for grouper, list in regrouped %}``.

* Added a :ttag:`resetcycle` template tag to allow resetting the sequence of
the :ttag:`cycle` template tag.

* You can now specify specific directories for a particular
:class:`filesystem.Loader <django.template.loaders.filesystem.Loader>`.

Tests
~~~~~

* Added :meth:`.DiscoverRunner.get_test_runner_kwargs` to allow customizing the
keyword arguments passed to the test runner.

* Added the :option:`test --debug-mode` option to help troubleshoot test
failures by setting the :setting:`DEBUG` setting to ``True``.

* The new :func:`django.test.utils.setup_databases` (moved from
``django.test.runner``) and :func:`~django.test.utils.teardown_databases`
functions make it easier to build custom test runners.

* Added support for :meth:`python:unittest.TestCase.subTest`’s when using the
:option:`test --parallel` option.

* ``DiscoverRunner`` now runs the system checks at the start of a test run.
Override the :meth:`.DiscoverRunner.run_checks` method if you want to disable
that.

Validators
~~~~~~~~~~

* Added :class:`~django.core.validators.FileExtensionValidator` to validate
file extensions and
:data:`~django.core.validators.validate_image_file_extension` to validate
image files.

.. _backwards-incompatible-1.11:

Backwards incompatible changes in 1.11
======================================

:mod:`django.contrib.gis`
-------------------------

* To simplify the codebase and because it's easier to install than when
``contrib.gis`` was first released, :ref:`gdalbuild` is now a required
dependency for GeoDjango. In older versions, it's only required for SQLite.

* ``contrib.gis.maps`` is removed as it interfaces with a retired version of
the Google Maps API and seems to be unmaintained. If you're using it,
:ticket:`let us know <14284>`.

* The ``GEOSGeometry`` equality operator now also compares SRID.

* The OpenLayers-based form widgets now use OpenLayers 3, and the
``gis/openlayers.html`` and ``gis/openlayers-osm.html`` templates have been
updated. Check your project if you subclass these widgets or extend the
templates. Also, the new widgets work a bit differently than the old ones.
Instead of using a toolbar in the widget, you click to draw, click and drag
to move the map, and click and drag a point/vertex/corner to move it.

* Support for SpatiaLite < 4.0 is dropped.

* Support for GDAL 1.7 and 1.8 is dropped.

* The widgets in ``contrib.gis.forms.widgets`` and the admin's
``OpenLayersWidget`` use the :doc:`form rendering API </ref/forms/renderers>`
rather than ``loader.render_to_string()``. If you're using a custom widget
template, you'll need to be sure your form renderer can locate it. For
example, you could use the :class:`~django.forms.renderers.TemplatesSetting`
renderer.

:mod:`django.contrib.staticfiles`
---------------------------------

* ``collectstatic`` may now fail during post-processing when using a hashed
static files storage if a reference loop exists (e.g. ``'foo.css'``
references ``'bar.css'`` which itself references ``'foo.css'``) or if the
chain of files referencing other files is too deep to resolve in several
passes. In the latter case, increase the number of passes using
:attr:`.ManifestStaticFilesStorage.max_post_process_passes`.

* When using ``ManifestStaticFilesStorage``, static files not found in the
manifest at runtime now raise a ``ValueError`` instead of returning an
unchanged path. You can revert to the old behavior by setting
:attr:`.ManifestStaticFilesStorage.manifest_strict` to ``False``.

Database backend API
--------------------

This section describes changes that may be needed in third-party database
backends.

* The ``DatabaseOperations.time_trunc_sql()`` method is added to support
``TimeField`` truncation. It accepts a ``lookup_type`` and ``field_name``
arguments and returns the appropriate SQL to truncate the given time field
``field_name`` to a time object with only the given specificity. The
``lookup_type`` argument can be either ``'hour'``, ``'minute'``, or
``'second'``.

* The ``DatabaseOperations.datetime_cast_time_sql()`` method is added to
support the :lookup:`time` lookup. It accepts a ``field_name`` and ``tzname``
arguments and returns the SQL necessary to cast a datetime value to time value.

* To enable ``FOR UPDATE SKIP LOCKED`` support, set
``DatabaseFeatures.has_select_for_update_skip_locked = True``.

* The new ``DatabaseFeatures.supports_index_column_ordering`` attribute
specifies if a database allows defining ordering for columns in indexes. The
default value is ``True`` and the ``DatabaseIntrospection.get_constraints()``
method should include an ``'orders'`` key in each of the returned
dictionaries with a list of ``'ASC'`` and/or ``'DESC'`` values corresponding
to the ordering of each column in the index.

* :djadmin:`inspectdb` no longer calls ``DatabaseIntrospection.get_indexes()``
which is deprecated. Custom database backends should ensure all types of
indexes are returned by ``DatabaseIntrospection.get_constraints()``.

* Renamed the ``ignores_quoted_identifier_case`` feature to
``ignores_table_name_case`` to more accurately reflect how it is used.

* The ``name`` keyword argument is added to the
``DatabaseWrapper.create_cursor(self, name=None)`` method to allow usage of
server-side cursors on backends that support it.

Dropped support for PostgreSQL 9.2 and PostGIS 2.0
--------------------------------------------------

Upstream support for PostgreSQL 9.2 ends in September 2017. As a consequence,
Django 1.11 sets PostgreSQL 9.3 as the minimum version it officially supports.

Support for PostGIS 2.0 is also removed as PostgreSQL 9.2 is the last version
to support it.

Also, the minimum supported version of psycopg2 is increased from 2.4.5 to
2.5.4.

.. _liveservertestcase-port-zero-change:

``LiveServerTestCase`` binds to port zero
-----------------------------------------

Rather than taking a port range and iterating to find a free port,
``LiveServerTestCase`` binds to port zero and relies on the operating system
to assign a free port. The ``DJANGO_LIVE_TEST_SERVER_ADDRESS`` environment
variable is no longer used, and as it's also no longer used, the
``manage.py test --liveserver`` option is removed.

If you need to bind ``LiveServerTestCase`` to a specific port, use the ``port``
attribute added in Django 1.11.2.

Protection against insecure redirects in :mod:`django.contrib.auth` and ``i18n`` views
--------------------------------------------------------------------------------------

``LoginView``, ``LogoutView`` (and the deprecated function-based equivalents),
and :func:`~django.views.i18n.set_language` protect users from being redirected
to non-HTTPS ``next`` URLs when the app is running over HTTPS.

``QuerySet.get_or_create()`` and ``update_or_create()`` validate arguments
--------------------------------------------------------------------------

To prevent typos from passing silently,
:meth:`~django.db.models.query.QuerySet.get_or_create` and
:meth:`~django.db.models.query.QuerySet.update_or_create` check that their
arguments are model fields. This should be backwards-incompatible only in the
fact that it might expose a bug in your project.

``pytz`` is a required dependency and support for ``settings.TIME_ZONE = None`` is removed
------------------------------------------------------------------------------------------

To simplify Django's timezone handling, ``pytz`` is now a required dependency.
It's automatically installed along with Django.

Support for ``settings.TIME_ZONE = None`` is removed as the behavior isn't
commonly used and is questionably useful. If you want to automatically detect
the timezone based on the system timezone, you can use `tzlocal
<https://pypi.org/project/tzlocal/>`_::

from tzlocal import get_localzone

TIME_ZONE = get_localzone().zone

This works similar to ``settings.TIME_ZONE = None`` except that it also sets
``os.environ['TZ']``. `Let us know
<https://groups.google.com/g/django-developers/c/OAV3FChfuPM/discussion>`__
if there's a use case where you find you can't adapt your code to set a
``TIME_ZONE``.

HTML changes in admin templates
-------------------------------

``<p class="help">`` is replaced with a ``<div>`` tag to allow including lists
inside help text.

Read-only fields are wrapped in ``<div class="readonly">...</div>`` instead of
``<p>...</p>`` to allow any kind of HTML as the field's content.

.. _template-widget-incompatibilities-1-11:

Changes due to the introduction of template-based widget rendering
------------------------------------------------------------------

Some undocumented classes in ``django.forms.widgets`` are removed:

* ``SubWidget``
* ``RendererMixin``, ``ChoiceFieldRenderer``, ``RadioFieldRenderer``,
``CheckboxFieldRenderer``
* ``ChoiceInput``, ``RadioChoiceInput``, ``CheckboxChoiceInput``

The undocumented ``Select.render_option()`` method is removed.

The ``Widget.format_output()`` method is removed. Use a custom widget template
instead.

Some widget values, such as ``<select>`` options, are now localized if
``settings.USE_L10N=True``. You could revert to the old behavior with custom
widget templates that uses the :ttag:`localize` template tag to turn off
localization.

``django.template.backends.django.Template.render()`` prohibits non-dict context
--------------------------------------------------------------------------------

For compatibility with multiple template engines,
``django.template.backends.django.Template.render()`` (returned from high-level
template loader APIs such as ``loader.get_template()``) must receive a
dictionary of context rather than ``Context`` or ``RequestContext``. If you
were passing either of the two classes, pass a dictionary instead -- doing so
is backwards-compatible with older versions of Django.

Model state changes in migration operations
-------------------------------------------

To improve the speed of applying migrations, rendering of related models is
delayed until an operation that needs them (e.g. ``RunPython``). If you have a
custom operation that works with model classes or model instances from the
``from_state`` argument in ``database_forwards()`` or ``database_backwards()``,
you must render model states using the ``clear_delayed_apps_cache()`` method as
described in :ref:`writing your own migration operation
<writing-your-own-migration-operation>`.

Server-side cursors on PostgreSQL
---------------------------------

The change to make :meth:`.QuerySet.iterator()` use server-side cursors on
PostgreSQL prevents running Django with PgBouncer in transaction pooling mode.
To reallow that, use the :setting:`DISABLE_SERVER_SIDE_CURSORS
<DATABASE-DISABLE_SERVER_SIDE_CURSORS>` setting (added in Django 1.11.1) in
:setting:`DATABASES`.

See :ref:`transaction-pooling-server-side-cursors` for more discussion.

Miscellaneous
-------------

* If no items in the feed have a ``pubdate`` or ``updateddate`` attribute,
:meth:`SyndicationFeed.latest_post_date()
<django.utils.feedgenerator.SyndicationFeed.latest_post_date>` now returns
the current UTC date/time, instead of a datetime without any timezone
information.

* CSRF failures are logged to the ``django.security.csrf`` logger instead of
``django.request``.

* :setting:`ALLOWED_HOSTS` validation is no longer disabled when running tests.
If your application includes tests with custom host names, you must include
those host names in :setting:`ALLOWED_HOSTS`. See
:ref:`topics-testing-advanced-multiple-hosts`.

* Using a foreign key's id (e.g. ``'field_id'``) in ``ModelAdmin.list_display``
displays the related object's ID. Remove the ``_id`` suffix if you want the
old behavior of the string representation of the object.

* In model forms, :class:`~django.db.models.CharField` with ``null=True`` now
saves ``NULL`` for blank values instead of empty strings.

* On Oracle, :meth:`Model.validate_unique()
<django.db.models.Model.validate_unique>` no longer checks empty strings for
uniqueness as the database interprets the value as ``NULL``.

* If you subclass :class:`.AbstractUser` and override ``clean()``, be sure it
calls ``super()``. :meth:`.BaseUserManager.normalize_email` is called in a
new :meth:`.AbstractUser.clean` method so that normalization is applied in
cases like model form validation.

* ``EmailField`` and ``URLField`` no longer accept the ``strip`` keyword
argument. Remove it because it doesn't have an effect in older versions of
Django as these fields always strip whitespace.

* The ``checked`` and ``selected`` attribute rendered by form widgets now uses
HTML5 boolean syntax rather than XHTML's ``checked='checked'`` and
``selected='selected'``.

* :meth:`RelatedManager.add()
<django.db.models.fields.related.RelatedManager.add>`,
:meth:`~django.db.models.fields.related.RelatedManager.remove`,
:meth:`~django.db.models.fields.related.RelatedManager.clear`, and
:meth:`~django.db.models.fields.related.RelatedManager.set` now
clear the ``prefetch_related()`` cache.

* To prevent possible loss of saved settings,
:func:`~django.test.utils.setup_test_environment` now raises an exception if
called a second time before calling
:func:`~django.test.utils.teardown_test_environment`.

* The undocumented ``DateTimeAwareJSONEncoder`` alias for
:class:`~django.core.serializers.json.DjangoJSONEncoder` (renamed in Django
1.0) is removed.

* The :class:`cached template loader <django.template.loaders.cached.Loader>`
is now enabled if :setting:`OPTIONS['loaders'] <TEMPLATES-OPTIONS>` isn't
specified and :setting:`OPTIONS['debug'] <TEMPLATES-OPTIONS>` is ``False``
(the latter option defaults to the value of :setting:`DEBUG`). This could
be backwards-incompatible if you have some :ref:`template tags that aren't
thread safe <template_tag_thread_safety>`.

* The prompt for stale content type deletion no longer occurs after running the
``migrate`` command. Use the new :djadmin:`remove_stale_contenttypes` command
instead.

* The admin's widget for ``IntegerField`` uses ``type="number"`` rather than
``type="text"``.

* Conditional HTTP headers are now parsed and compared according to the
:rfc:`7232` Conditional Requests specification rather than the older
:rfc:`2616`.

* :func:`~django.utils.cache.patch_response_headers` no longer adds a
``Last-Modified`` header. According to the :rfc:`7234section-4.2.2`, this
header is useless alongside other caching headers that provide an explicit
expiration time, e.g. ``Expires`` or ``Cache-Control``.
:class:`~django.middleware.cache.UpdateCacheMiddleware` and
:func:`~django.utils.cache.add_never_cache_headers` call
``patch_response_headers()`` and therefore are also affected by this change.

* In the admin templates, ``<p class="help">`` is replaced with a ``<div>`` tag
to allow including lists inside help text.

* :class:`~django.middleware.http.ConditionalGetMiddleware` no longer sets the
``Date`` header as web servers set that header. It also no longer sets the
``Content-Length`` header as this is now done by
:class:`~django.middleware.common.CommonMiddleware`.

If you have a middleware that modifies a response's content and appears
before ``CommonMiddleware`` in the ``MIDDLEWARE`` or ``MIDDLEWARE_CLASSES``
settings, you must reorder your middleware so that responses aren't modified
after ``Content-Length`` is set, or have the response modifying middleware
reset the ``Content-Length`` header.

* :meth:`~django.apps.AppConfig.get_model` and
:meth:`~django.apps.AppConfig.get_models` now raise
:exc:`~django.core.exceptions.AppRegistryNotReady` if they're called before
models of all applications have been loaded. Previously they only required
the target application's models to be loaded and thus could return models
without all their relations set up. If you need the old behavior of
``get_model()``, set the ``require_ready`` argument to ``False``.

* The unused ``BaseCommand.can_import_settings`` attribute is removed.

* The undocumented ``django.utils.functional.lazy_property`` is removed.

* For consistency with non-multipart requests, ``MultiPartParser.parse()`` now
leaves ``request.POST`` immutable. If you're modifying that ``QueryDict``,
you must now first copy it, e.g. ``request.POST.copy()``.

* Support for ``cx_Oracle`` < 5.2 is removed.

* Support for IPython < 1.0 is removed from the ``shell`` command.

* The signature of private API ``Widget.build_attrs()`` changed from
``extra_attrs=None, **kwargs`` to ``base_attrs, extra_attrs=None``.

* File-like objects (e.g., :class:`~io.StringIO` and :class:`~io.BytesIO`)
uploaded to an :class:`~django.db.models.ImageField` using the test client
now require a ``name`` attribute with a value that passes the
:data:`~django.core.validators.validate_image_file_extension` validator.
See the note in :meth:`.Client.post`.

* :class:`~django.db.models.FileField` now moves rather than copies the file
it receives. With the default file upload settings, files larger than
:setting:`FILE_UPLOAD_MAX_MEMORY_SIZE` now have the same permissions as
temporary files (often ``0o600``) rather than the system's standard umask
(often ``0o6644``). Set the :setting:`FILE_UPLOAD_PERMISSIONS` if you need
the same permission regardless of file size.

.. _deprecated-features-1.11:

Features deprecated in 1.11
===========================

``models.permalink()`` decorator
--------------------------------

Use :func:`django.urls.reverse` instead. For example::

from django.db import models

class MyModel(models.Model):
...

models.permalink
def url(self):
return ('guitarist_detail', [self.slug])

becomes::

from django.db import models
from django.urls import reverse

class MyModel(models.Model):
...

def url(self):
return reverse('guitarist_detail', args=[self.slug])

Miscellaneous
-------------

* ``contrib.auth``’s ``login()`` and ``logout()`` function-based views are
deprecated in favor of new class-based views
:class:`~django.contrib.auth.views.LoginView` and
:class:`~django.contrib.auth.views.LogoutView`.

* The unused ``extra_context`` parameter of
``contrib.auth.views.logout_then_login()`` is deprecated.

* ``contrib.auth``’s ``password_change()``, ``password_change_done()``,
``password_reset()``, ``password_reset_done()``, ``password_reset_confirm()``,
and ``password_reset_complete()`` function-based views are deprecated in favor
of new class-based views
:class:`~django.contrib.auth.views.PasswordChangeView`,
:class:`~django.contrib.auth.views.PasswordChangeDoneView`,
:class:`~django.contrib.auth.views.PasswordResetView`,
:class:`~django.contrib.auth.views.PasswordResetDoneView`,
:class:`~django.contrib.auth.views.PasswordResetConfirmView`, and
:class:`~django.contrib.auth.views.PasswordResetCompleteView`.

* ``django.test.runner.setup_databases()`` is moved to
:func:`django.test.utils.setup_databases`. The old location is deprecated.

* ``django.utils.translation.string_concat()`` is deprecated in
favor of :func:`django.utils.text.format_lazy`. ``string_concat(*strings)``
can be replaced by ``format_lazy('{}' * len(strings), *strings)``.

* For the ``PyLibMCCache`` cache backend, passing ``pylibmc`` behavior settings
as top-level attributes of ``OPTIONS`` is deprecated. Set them under a
``behaviors`` key within ``OPTIONS`` instead.

* The ``host`` parameter of ``django.utils.http.is_safe_url()`` is deprecated
in favor of the new ``allowed_hosts`` parameter.

* Silencing exceptions raised while rendering the
:ttag:`{% include %} <include>` template tag is deprecated as the behavior is
often more confusing than helpful. In Django 2.1, the exception will be
raised.

* ``DatabaseIntrospection.get_indexes()`` is deprecated in favor of
``DatabaseIntrospection.get_constraints()``.

* :func:`~django.contrib.auth.authenticate` now passes a ``request`` argument
to the ``authenticate()`` method of authentication backends. Support for
methods that don't accept ``request`` as the first positional argument will
be removed in Django 2.1.

* The ``USE_ETAGS`` setting is deprecated in favor of
:class:`~django.middleware.http.ConditionalGetMiddleware` which now adds the
``ETag`` header to responses regardless of the setting. ``CommonMiddleware``
and ``django.utils.cache.patch_response_headers()`` will no longer set ETags
when the deprecation ends.

* ``Model._meta.has_auto_field`` is deprecated in favor of checking if
``Model._meta.auto_field is not None``.

* Using regular expression groups with ``iLmsu`` in ``url()`` is deprecated.
The only group that's useful is ``(?i)`` for case-insensitive URLs, however,
case-insensitive URLs aren't a good practice because they create multiple
entries for search engines, for example. An alternative solution could be to
create a :data:`~django.conf.urls.handler404` that looks for uppercase
characters in the URL and redirects to a lowercase equivalent.

* The ``renderer`` argument is added to the :meth:`Widget.render()
<django.forms.Widget.render>` method. Methods that don't accept that argument
will work through a deprecation period.


===========================

1.10.8

Not secure
===========================

*September 5, 2017*

Django 1.10.8 fixes a security issue in 1.10.7.

CVE-2017-12794: Possible XSS in traceback section of technical 500 debug page
=============================================================================

In older versions, HTML autoescaping was disabled in a portion of the template
for the technical 500 debug page. Given the right circumstances, this allowed
a cross-site scripting attack. This vulnerability shouldn't affect most
production sites since you shouldn't run with ``DEBUG = True`` (which makes
this page accessible) in your production settings.


===========================

1.10.7

Not secure
===========================

*April 4, 2017*

Django 1.10.7 fixes two security issues and a bug in 1.10.6.

CVE-2017-7233: Open redirect and possible XSS attack via user-supplied numeric redirect URLs
============================================================================================

Django relies on user input in some cases (e.g.
``django.contrib.auth.views.login()`` and :doc:`i18n </topics/i18n/index>`)
to redirect the user to an "on success" URL. The security check for these
redirects (namely ``django.utils.http.is_safe_url()``) considered some numeric
URLs (e.g. ``http:999999999``) "safe" when they shouldn't be.

Also, if a developer relies on ``is_safe_url()`` to provide safe redirect
targets and puts such a URL into a link, they could suffer from an XSS attack.

CVE-2017-7234: Open redirect vulnerability in ``django.views.static.serve()``
=============================================================================

A maliciously crafted URL to a Django site using the
:func:`~django.views.static.serve` view could redirect to any other domain. The
view no longer does any redirects as they don't provide any known, useful
functionality.

Note, however, that this view has always carried a warning that it is not
hardened for production use and should be used only as a development aid.

Bugfixes
========

* Made admin's ``RelatedFieldWidgetWrapper`` use the wrapped widget's
``value_omitted_from_data()`` method (:ticket:`27905`).

* Fixed model form ``default`` fallback for ``SelectMultiple``
(:ticket:`27993`).


===========================

1.10.6

Not secure
===========================

*March 1, 2017*

Django 1.10.6 fixes several bugs in 1.10.5.

Bugfixes
========

* Fixed ``ClearableFileInput``’s "Clear" checkbox on model form fields where
the model field has a ``default`` (:ticket:`27805`).

* Fixed ``RequestDataTooBig`` and ``TooManyFieldsSent`` exceptions crashing
rather than generating a bad request response (:ticket:`27820`).

* Fixed a crash on Oracle and PostgreSQL when subtracting ``DurationField``
or ``IntegerField`` from ``DateField`` (:ticket:`27828`).

* Fixed query expression date subtraction accuracy on PostgreSQL for
differences larger than a month (:ticket:`27856`).

* Fixed a ``GDALException`` raised by ``GDALClose`` on GDAL ≥ 2.0
(:ticket:`27479`).


===========================

1.10.5

Not secure
===========================

*January 4, 2017*

Django 1.10.5 fixes several bugs in 1.10.4.

Bugfixes
========

* Fixed a crash in the debug view if ``request.user`` can't be retrieved, such
as if the database is unavailable (:ticket:`27567`).

* Fixed occasional missing plural forms in ``JavaScriptCatalog``
(:ticket:`27418`).

* Fixed a regression in the ``timesince`` and ``timeuntil`` filters that caused
incorrect results for dates in a leap year (:ticket:`27637`).

* Fixed a regression where ``collectstatic`` overwrote newer files in remote
storages (:ticket:`27658`).


===========================

1.10.4

Not secure
===========================

*December 1, 2016*

Django 1.10.4 fixes several bugs in 1.10.3.

Bugfixes
========

* Quoted the Oracle test user's password in queries to fix the "ORA-00922:
missing or invalid option" error when the password starts with a number or
special character (:ticket:`27420`).

* Fixed incorrect ``app_label`` / ``model_name`` arguments for
``allow_migrate()`` in ``makemigrations`` migration consistency checks
(:ticket:`27461`).

* Made ``Model.delete(keep_parents=True)`` preserve parent reverse
relationships in multi-table inheritance (:ticket:`27407`).

* Fixed a ``QuerySet.update()`` crash on SQLite when updating a
``DateTimeField`` with an ``F()`` expression and a ``timedelta``
(:ticket:`27544`).

* Prevented ``LocaleMiddleware`` from redirecting on URLs that should return
404 when using ``prefix_default_language=False`` (:ticket:`27402`).

* Prevented an unnecessary index from being created on an InnoDB ``ForeignKey``
when the field was added after the model was created (:ticket:`27558`).


===========================

1.10.3

Not secure
===========================

*November 1, 2016*

Django 1.10.3 fixes two security issues and several bugs in 1.10.2.

User with hardcoded password created when running tests on Oracle
=================================================================

When running tests with an Oracle database, Django creates a temporary database
user. In older versions, if a password isn't manually specified in the database
settings ``TEST`` dictionary, a hardcoded password is used. This could allow
an attacker with network access to the database server to connect.

This user is usually dropped after the test suite completes, but not when using
the ``manage.py test --keepdb`` option or if the user has an active session
(such as an attacker's connection).

A randomly generated password is now used for each test run.

DNS rebinding vulnerability when ``DEBUG=True``
===============================================

Older versions of Django don't validate the ``Host`` header against
``settings.ALLOWED_HOSTS`` when ``settings.DEBUG=True``. This makes them
vulnerable to a `DNS rebinding attack
<https://benmmurphy.github.io/blog/2016/07/11/rails-webconsole-dns-rebinding/>`_.

While Django doesn't ship a module that allows remote code execution, this is
at least a cross-site scripting vector, which could be quite serious if
developers load a copy of the production database in development or connect to
some production services for which there's no development instance, for
example. If a project uses a package like the ``django-debug-toolbar``, then
the attacker could execute arbitrary SQL, which could be especially bad if the
developers connect to the database with a superuser account.

``settings.ALLOWED_HOSTS`` is now validated regardless of ``DEBUG``. For
convenience, if ``ALLOWED_HOSTS`` is empty and ``DEBUG=True``, the following
variations of localhost are allowed ``['localhost', '127.0.0.1', '::1']``. If
your local settings file has your production ``ALLOWED_HOSTS`` value, you must
now omit it to get those fallback values.

Bugfixes
========

* Allowed ``User.is_authenticated`` and ``User.is_anonymous`` properties to be
tested for ``set`` membership (:ticket:`27309`).

* Fixed a performance regression when running ``migrate`` in projects
with ``RenameModel`` operations (:ticket:`27279`).

* Added ``model_name`` to the ``allow_migrate()`` calls in ``makemigrations``
(:ticket:`27200`).

* Made the ``JavaScriptCatalog`` view respect the ``packages`` argument;
previously it was ignored (:ticket:`27374`).

* Fixed ``QuerySet.bulk_create()`` on PostgreSQL when the number of objects is
a multiple plus one of ``batch_size`` (:ticket:`27385`).

* Prevented ``i18n_patterns()`` from using too much of the URL as the language
to fix a use case for ``prefix_default_language=False`` (:ticket:`27063`).

* Replaced a possibly incorrect redirect from ``SessionMiddleware`` when a
session is destroyed in a concurrent request with a ``SuspiciousOperation``
to indicate that the request can't be completed (:ticket:`27363`).


===========================

1.10.2

Not secure
===========================

*October 1, 2016*

Django 1.10.2 fixes several bugs in 1.10.1.

Bugfixes
========

* Fixed a crash in MySQL database validation where ``SELECT sql_mode``
doesn't return a result (:ticket:`27180`).

* Allowed combining ``contrib.postgres.search.SearchQuery`` with more than one
``&`` or ``|`` operators (:ticket:`27143`).

* Disabled system check for URL patterns beginning with a '/' when
``APPEND_SLASH=False`` (:ticket:`27238`).

* Fixed model form ``default`` fallback for ``CheckboxSelectMultiple``,
``MultiWidget``, ``FileInput``, ``SplitDateTimeWidget``, ``SelectDateWidget``,
and ``SplitArrayWidget`` (:ticket:`27186`). Custom widgets affected by this
issue should implement :meth:`~django.forms.Widget.value_omitted_from_data`.

* Fixed a crash in ``runserver`` logging during a "Broken pipe" error
(:ticket:`27271`).

* Fixed a regression where unchanged localized date/time fields were listed as
changed in the admin's model history messages (:ticket:`27302`).


===========================

1.10.1

Not secure
===========================

*September 1, 2016*

Django 1.10.1 fixes several bugs in 1.10.

Bugfixes
========

* Fixed a crash in MySQL connections where ``SELECT SQL_AUTO_IS_NULL``
doesn't return a result (:ticket:`26991`).

* Allowed ``User.is_authenticated`` and ``User.is_anonymous`` properties to be
compared using ``==``, ``!=``, and ``|`` (:ticket:`26988`, :ticket:`27154`).

* Removed the broken ``BaseCommand.usage()`` method which was for
``optparse`` support (:ticket:`27000`).

* Fixed a checks framework crash with an empty ``Meta.default_permissions``
(:ticket:`26997`).

* Fixed a regression in the number of queries when using ``RadioSelect`` with a
``ModelChoiceField`` form field (:ticket:`27001`).

* Fixed a crash if ``request.META['CONTENT_LENGTH']`` is an empty string
(:ticket:`27005`).

* Fixed the ``isnull`` lookup on a ``ForeignKey`` with its ``to_field``
pointing to a ``CharField`` or pointing to a ``CharField`` defined with
``primary_key=True`` (:ticket:`26983`).

* Prevented the ``migrate`` command from raising
``InconsistentMigrationHistory`` in the presence of unapplied squashed
migrations (:ticket:`27004`).

* Fixed a regression in ``Client.force_login()`` which required specifying a
``backend`` rather than automatically using the first one if multiple
backends are configured (:ticket:`27027`).

* Made ``QuerySet.bulk_create()`` properly initialize model instances on
backends, such as PostgreSQL, that support returning the IDs of the created
records so that many-to-many relationships can be used on the new objects
(:ticket:`27026`).

* Fixed crash of ``django.views.static.serve()`` with ``show_indexes`` enabled
(:ticket:`26973`).

* Fixed ``ClearableFileInput`` to avoid the ``required`` HTML attribute when
initial data exists (:ticket:`27037`).

* Fixed annotations with database functions when combined with lookups on
PostGIS (:ticket:`27014`).

* Reallowed the ``{% for %}`` tag to unpack any iterable (:ticket:`27058`).

* Made ``makemigrations`` skip inconsistent history checks on non-default
databases if database routers aren't in use or if no apps can be migrated
to the database (:ticket:`27054`, :ticket:`27110`, :ticket:`27142`).

* Removed duplicated managers in ``Model._meta.managers`` (:ticket:`27073`).

* Fixed ``contrib.admindocs`` crash when a view is in a class, such as some of
the admin views (:ticket:`27018`).

* Reverted a few admin checks that checked ``field.many_to_many`` back to
``isinstance(field, models.ManyToManyField)`` since it turned out the checks
weren't suitable to be generalized like that (:ticket:`26998`).

* Added the database alias to the ``InconsistentMigrationHistory`` message
raised by ``makemigrations`` and ``migrate`` (:ticket:`27089`).

* Fixed the creation of ``ContentType`` and ``Permission`` objects for models
of applications without migrations when calling the ``migrate`` command with
no migrations to apply (:ticket:`27044`).

* Included the already applied migration state changes in the ``Apps`` instance
provided to the ``pre_migrate`` signal receivers to allow ``ContentType``
renaming to be performed on model rename (:ticket:`27100`).

* Reallowed subclassing ``UserCreationForm`` without ``USERNAME_FIELD`` in
``Meta.fields`` (:ticket:`27111`).

* Fixed a regression in model forms where model fields with a ``default`` that
didn't appear in POST data no longer used the ``default`` (:ticket:`27039`).


=========================

1.10

Not secure
=========================

*August 1, 2016*

Welcome to Django 1.10!

These release notes cover the :ref:`new features <whats-new-1.10>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-1.10>` you'll
want to be aware of when upgrading from Django 1.9 or older versions. We've
:ref:`dropped some features <removed-features-1.10>` that have reached the end
of their deprecation cycle, and we've :ref:`begun the deprecation process for
some features <deprecated-features-1.10>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Python compatibility
====================

Like Django 1.9, Django 1.10 requires Python 2.7, 3.4, or 3.5. We **highly
recommend** and only officially support the latest release of each series.

.. _whats-new-1.10:

What's new in Django 1.10
=========================

Full text search for PostgreSQL
-------------------------------

``django.contrib.postgres`` now includes a :doc:`collection of database
functions </ref/contrib/postgres/search>` to allow the use of the full text
search engine. You can search across multiple fields in your relational
database, combine the searches with other lookups, use different language
configurations and weightings, and rank the results by relevance.

It also now includes trigram support, using the :lookup:`trigram_similar`
lookup, and the :class:`~django.contrib.postgres.search.TrigramSimilarity` and
:class:`~django.contrib.postgres.search.TrigramDistance` expressions.

New-style middleware
--------------------

:doc:`A new style of middleware is introduced </topics/http/middleware>` to
solve the lack of strict request/response layering of the old-style of
middleware described in `DEP 0005
<https://github.com/django/deps/blob/main/final/0005-improved-middleware.rst>`_.
You'll need to :ref:`adapt old, custom middleware <upgrading-middleware>` and
switch from the ``MIDDLEWARE_CLASSES`` setting to the new :setting:`MIDDLEWARE`
setting to take advantage of the improvements.

Official support for Unicode usernames
--------------------------------------

The :class:`~django.contrib.auth.models.User` model in ``django.contrib.auth``
originally only accepted ASCII letters and numbers in usernames. Although it
wasn't a deliberate choice, Unicode characters have always been accepted when
using Python 3.

The username validator now explicitly accepts Unicode characters by
default on Python 3 only.

Custom user models may use the new
:class:`~django.contrib.auth.validators.ASCIIUsernameValidator` or
:class:`~django.contrib.auth.validators.UnicodeUsernameValidator`.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* For sites running on a subpath, the default :attr:`URL for the "View site"
link <django.contrib.admin.AdminSite.site_url>` at the top of each admin page
will now point to ``request.META['SCRIPT_NAME']`` if set, instead of ``/``.

* The success message that appears after adding or editing an object now
contains a link to the object's change form.

* All inline JavaScript is removed so you can enable the
``Content-Security-Policy`` HTTP header if you wish.

* The new :attr:`InlineModelAdmin.classes
<django.contrib.admin.InlineModelAdmin.classes>` attribute allows specifying
classes on inline fieldsets. Inlines with a ``collapse`` class will be
initially collapsed and their header will have a small "show" link.

* If a user doesn't have the add permission, the ``object-tools`` block on a
model's changelist will now be rendered (without the add button). This makes
it easier to add custom tools in this case.

* The :class:`~django.contrib.admin.models.LogEntry` model now stores change
messages in a JSON structure so that the message can be dynamically translated
using the current active language. A new ``LogEntry.get_change_message()``
method is now the preferred way of retrieving the change message.

* Selected objects for fields in ``ModelAdmin.raw_id_fields`` now have a link
to object's change form.

* Added "No date" and "Has date" choices for ``DateFieldListFilter`` if the
field is nullable.

* The jQuery library embedded in the admin is upgraded from version 2.1.4 to
2.2.3.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* Added support for the :ref:`Argon2 password hash <argon2_usage>`. It's
recommended over PBKDF2, however, it's not the default as it requires a
third-party library.

* The default iteration count for the PBKDF2 password hasher has been increased
by 25%. This backwards compatible change will not affect users who have
subclassed ``django.contrib.auth.hashers.PBKDF2PasswordHasher`` to change the
default value.

* The ``django.contrib.auth.views.logout()`` view sends "no-cache" headers
to prevent an issue where Safari caches redirects and prevents a user from
being able to log out.

* Added the optional ``backend`` argument to :func:`django.contrib.auth.login`
to allow using it without credentials.

* The new :setting:`LOGOUT_REDIRECT_URL` setting controls the redirect of the
``django.contrib.auth.views.logout()`` view, if the view doesn't get a
``next_page`` argument.

* The new ``redirect_authenticated_user`` parameter for the
``django.contrib.auth.views.login()`` view allows redirecting
authenticated users visiting the login page.

* The new :class:`~django.contrib.auth.backends.AllowAllUsersModelBackend` and
:class:`~django.contrib.auth.backends.AllowAllUsersRemoteUserBackend` ignore
the value of ``User.is_active``, while
:class:`~django.contrib.auth.backends.ModelBackend` and
:class:`~django.contrib.auth.backends.RemoteUserBackend` now reject inactive
users.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* :ref:`Distance lookups <distance-lookups>` now accept expressions as the
distance value parameter.

* The new :attr:`GEOSGeometry.unary_union
<django.contrib.gis.geos.GEOSGeometry.unary_union>` property computes the
union of all the elements of this geometry.

* Added the :meth:`GEOSGeometry.covers()
<django.contrib.gis.geos.GEOSGeometry.covers>` binary predicate.

* Added the :meth:`GDALBand.statistics()
<django.contrib.gis.gdal.GDALBand.statistics>` method and
:attr:`~django.contrib.gis.gdal.GDALBand.mean`
and :attr:`~django.contrib.gis.gdal.GDALBand.std` attributes.

* Added support for the :class:`~django.contrib.gis.db.models.MakeLine`
aggregate and :class:`~django.contrib.gis.db.models.functions.GeoHash`
function on SpatiaLite.

* Added support for the
:class:`~django.contrib.gis.db.models.functions.Difference`,
:class:`~django.contrib.gis.db.models.functions.Intersection`, and
:class:`~django.contrib.gis.db.models.functions.SymDifference`
functions on MySQL.

* Added support for instantiating empty GEOS geometries.

* The new :attr:`~django.contrib.gis.geos.WKTWriter.trim` and
:attr:`~django.contrib.gis.geos.WKTWriter.precision` properties
of :class:`~django.contrib.gis.geos.WKTWriter` allow controlling
output of the fractional part of the coordinates in WKT.

* Added the :attr:`LineString.closed
<django.contrib.gis.geos.LineString.closed>` and
:attr:`MultiLineString.closed
<django.contrib.gis.geos.MultiLineString.closed>` properties.

* The :doc:`GeoJSON serializer </ref/contrib/gis/serializers>` now outputs the
primary key of objects in the ``properties`` dictionary if specific fields
aren't specified.

* The ability to replicate input data on the :meth:`GDALBand.data()
<django.contrib.gis.gdal.GDALBand.data>` method was added. Band data can
now be updated with repeated values efficiently.

* Added database functions
:class:`~django.contrib.gis.db.models.functions.IsValid` and
:class:`~django.contrib.gis.db.models.functions.MakeValid`, as well as the
:lookup:`isvalid` lookup, all for PostGIS. This allows filtering and
repairing invalid geometries on the database side.

* Added raster support for all :doc:`spatial lookups
</ref/contrib/gis/geoquerysets>`.

:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* For convenience, :class:`~django.contrib.postgres.fields.HStoreField` now
casts its keys and values to strings.

:mod:`django.contrib.sessions`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The :djadmin:`clearsessions` management command now removes file-based
sessions.

:mod:`django.contrib.sites`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The :class:`~django.contrib.sites.models.Site` model now supports
:ref:`natural keys <topics-serialization-natural-keys>`.

:mod:`django.contrib.staticfiles`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The :ttag:`static` template tag now uses ``django.contrib.staticfiles``
if it's in ``INSTALLED_APPS``. This is especially useful for third-party apps
which can now always use ``{% load static %}`` (instead of
``{% load staticfiles %}`` or ``{% load static from staticfiles %}``) and
not worry about whether or not the ``staticfiles`` app is installed.

* You can :ref:`more easily customize <customize-staticfiles-ignore-patterns>`
the ``collectstatic --ignore`` option with a custom ``AppConfig``.

Cache
~~~~~

* The file-based cache backend now uses the highest pickling protocol.

CSRF
~~~~

* The default :setting:`CSRF_FAILURE_VIEW`, ``views.csrf.csrf_failure()`` now
accepts an optional ``template_name`` parameter, defaulting to
``'403_csrf.html'``, to control the template used to render the page.

* To protect against `BREACH`_ attacks, the CSRF protection mechanism now
changes the form token value on every request (while keeping an invariant
secret which can be used to validate the different tokens).

.. _BREACH: http://breachattack.com/

Database backends
~~~~~~~~~~~~~~~~~

* Temporal data subtraction was unified on all backends.

* If the database supports it, backends can set
``DatabaseFeatures.can_return_ids_from_bulk_insert=True`` and implement
``DatabaseOperations.fetch_returned_insert_ids()`` to set primary keys
on objects created using ``QuerySet.bulk_create()``.

* Added keyword arguments to the ``as_sql()`` methods of various expressions
(``Func``, ``When``, ``Case``, and ``OrderBy``) to allow database backends to
customize them without mutating ``self``, which isn't safe when using
different database backends. See the ``arg_joiner`` and ``**extra_context``
parameters of :meth:`Func.as_sql() <django.db.models.Func.as_sql>` for an
example.

File Storage
~~~~~~~~~~~~

* Storage backends now present a timezone-aware API with new methods
:meth:`~django.core.files.storage.Storage.get_accessed_time`,
:meth:`~django.core.files.storage.Storage.get_created_time`, and
:meth:`~django.core.files.storage.Storage.get_modified_time`. They return a
timezone-aware ``datetime`` if :setting:`USE_TZ` is ``True`` and a naive
``datetime`` in the local timezone otherwise.

* The new :meth:`Storage.generate_filename()
<django.core.files.storage.Storage.generate_filename>` method makes it easier
to implement custom storages that don't use the ``os.path`` calls previously
in :class:`~django.db.models.FileField`.

Forms
~~~~~

* Form and widget ``Media`` is now served using
:mod:`django.contrib.staticfiles` if installed.

* The ``<input>`` tag rendered by :class:`~django.forms.CharField` now includes
a ``minlength`` attribute if the field has a ``min_length``.

* Required form fields now have the ``required`` HTML attribute. Set the new
:attr:`Form.use_required_attribute <django.forms.Form.use_required_attribute>`
attribute to ``False`` to disable it. The ``required`` attribute isn't
included on forms of formsets because the browser validation may not be
correct when adding and deleting formsets.

Generic Views
~~~~~~~~~~~~~

* The :class:`~django.views.generic.base.View` class can now be imported from
``django.views``.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* The :func:`~django.conf.urls.i18n.i18n_patterns` helper function can now be
used in a root URLConf specified using :attr:`request.urlconf
<django.http.HttpRequest.urlconf>`.

* By setting the new ``prefix_default_language`` parameter for
:func:`~django.conf.urls.i18n.i18n_patterns` to ``False``, you can allow
accessing the default language without a URL prefix.

* :func:`~django.views.i18n.set_language` now returns a 204 status code (No
Content) for AJAX requests when there is no ``next`` parameter in ``POST`` or
``GET``.

* The :class:`~django.views.i18n.JavaScriptCatalog` and
:class:`~django.views.i18n.JSONCatalog` class-based views supersede the
deprecated ``javascript_catalog()`` and ``json_catalog()`` function-based
views. The new views are almost equivalent to the old ones except that by
default the new views collect all JavaScript strings in the ``djangojs``
translation domain from all installed apps rather than only the JavaScript
strings from :setting:`LOCALE_PATHS`.

Management Commands
~~~~~~~~~~~~~~~~~~~

* :func:`~django.core.management.call_command` now returns the value returned
from the ``command.handle()`` method.

* The new :option:`check --fail-level` option allows specifying the message
level that will cause the command to exit with a non-zero status.

* The new :option:`makemigrations --check` option makes the command exit
with a non-zero status when model changes without migrations are detected.

* :djadmin:`makemigrations` now displays the path to the migration files that
it generates.

* The :option:`shell --interface` option now accepts ``python`` to force use of
the "plain" Python interpreter.

* The new :option:`shell --command` option lets you run a command as Django and
exit, instead of opening the interactive shell.

* Added a warning to :djadmin:`dumpdata` if a proxy model is specified (which
results in no output) without its concrete parent.

* The new :attr:`BaseCommand.requires_migrations_checks
<django.core.management.BaseCommand.requires_migrations_checks>` attribute
may be set to ``True`` if you want your command to print a warning, like
:djadmin:`runserver` does, if the set of migrations on disk don't match the
migrations in the database.

* To assist with testing, :func:`~django.core.management.call_command` now
accepts a command object as the first argument.

* The :djadmin:`shell` command supports tab completion on systems using
``libedit``, e.g. macOS.

* The :djadmin:`inspectdb` command lets you choose what tables should be
inspected by specifying their names as arguments.

Migrations
~~~~~~~~~~

* Added support for serialization of ``enum.Enum`` objects.

* Added the ``elidable`` argument to the
:class:`~django.db.migrations.operations.RunSQL` and
:class:`~django.db.migrations.operations.RunPython` operations to allow them
to be removed when squashing migrations.

* Added support for :ref:`non-atomic migrations <non-atomic-migrations>` by
setting the ``atomic`` attribute on a ``Migration``.

* The ``migrate`` and ``makemigrations`` commands now :ref:`check for a
consistent migration history <migration-history-consistency>`. If they find
some unapplied dependencies of an applied migration,
``InconsistentMigrationHistory`` is raised.

* The :func:`~django.db.models.signals.pre_migrate` and
:func:`~django.db.models.signals.post_migrate` signals now dispatch their
migration ``plan`` and ``apps``.

Models
~~~~~~

* Reverse foreign keys from proxy models are now propagated to their
concrete class. The reverse relation attached by a
:class:`~django.db.models.ForeignKey` pointing to a proxy model is now
accessible as a descriptor on the proxied model class and may be referenced in
queryset filtering.

* The new :meth:`Field.rel_db_type() <django.db.models.Field.rel_db_type>`
method returns the database column data type for fields such as ``ForeignKey``
and ``OneToOneField`` that point to another field.

* The :attr:`~django.db.models.Func.arity` class attribute is added to
:class:`~django.db.models.Func`. This attribute can be used to set the number
of arguments the function accepts.

* Added :class:`~django.db.models.BigAutoField` which acts much like an
:class:`~django.db.models.AutoField` except that it is guaranteed
to fit numbers from ``1`` to ``9223372036854775807``.

* :meth:`QuerySet.in_bulk() <django.db.models.query.QuerySet.in_bulk>`
may be called without any arguments to return all objects in the queryset.

* :attr:`~django.db.models.ForeignKey.related_query_name` now supports
app label and class interpolation using the ``'%(app_label)s'`` and
``'%(class)s'`` strings.

* Allowed overriding model fields inherited from abstract base classes.

* The :func:`~django.db.models.prefetch_related_objects` function is now a
public API.

* :meth:`QuerySet.bulk_create() <django.db.models.query.QuerySet.bulk_create>`
sets the primary key on objects when using PostgreSQL.

* Added the :class:`~django.db.models.functions.Cast` database function.

* A proxy model may now inherit multiple proxy models that share a common
non-abstract parent class.

* Added :class:`~django.db.models.functions.Extract` functions to extract
datetime components as integers, such as year and hour.

* Added :class:`~django.db.models.functions.Trunc` functions to truncate a date
or datetime to a significant component. They enable queries like
sales-per-day or sales-per-hour.

* ``Model.__init__()`` now sets values of virtual fields from its keyword
arguments.

* The new :attr:`Meta.base_manager_name
<django.db.models.Options.base_manager_name>` and
:attr:`Meta.default_manager_name
<django.db.models.Options.default_manager_name>` options allow controlling
the :attr:`~django.db.models.Model._base_manager` and
:attr:`~django.db.models.Model._default_manager`, respectively.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* Added ``request.user`` to the debug view.

* Added :class:`~django.http.HttpResponse` methods
:meth:`~django.http.HttpResponse.readable()` and
:meth:`~django.http.HttpResponse.seekable()` to make an instance a
stream-like object and allow wrapping it with :py:class:`io.TextIOWrapper`.

* Added the :attr:`HttpRequest.content_type
<django.http.HttpRequest.content_type>` and
:attr:`~django.http.HttpRequest.content_params` attributes which are
parsed from the ``CONTENT_TYPE`` header.

* The parser for ``request.COOKIES`` is simplified to better match the behavior
of browsers. ``request.COOKIES`` may now contain cookies that are invalid
according to :rfc:`6265` but are possible to set via ``document.cookie``.

Serialization
~~~~~~~~~~~~~

* The ``django.core.serializers.json.DjangoJSONEncoder`` now knows how to
serialize lazy strings, typically used for translatable content.

Templates
~~~~~~~~~

* Added the ``autoescape`` option to the
:class:`~django.template.backends.django.DjangoTemplates` backend and the
:class:`~django.template.Engine` class.

* Added the ``is`` and ``is not`` comparison operators to the :ttag:`if` tag.

* Allowed :tfilter:`dictsort` to order a list of lists by an element at a
specified index.

* The :func:`~django.template.context_processors.debug` context processor
contains queries for all database aliases instead of only the default alias.

* Added relative path support for string arguments of the :ttag:`extends` and
:ttag:`include` template tags.

Tests
~~~~~

* To better catch bugs, :class:`~django.test.TestCase` now checks deferrable
database constraints at the end of each test.

* Tests and test cases can be :ref:`marked with tags <topics-tagging-tests>`
and run selectively with the new :option:`test --tag` and :option:`test
--exclude-tag` options.

* You can now login and use sessions with the test client even if
:mod:`django.contrib.sessions` is not in :setting:`INSTALLED_APPS`.

URLs
~~~~

* An addition in :func:`django.setup()` allows URL resolving that happens
outside of the request/response cycle (e.g. in management commands and
standalone scripts) to take :setting:`FORCE_SCRIPT_NAME` into account when it
is set.

Validators
~~~~~~~~~~

* :class:`~django.core.validators.URLValidator` now limits the length of
domain name labels to 63 characters and the total length of domain
names to 253 characters per :rfc:`1034`.

* :func:`~django.core.validators.int_list_validator` now accepts an optional
``allow_negative`` boolean parameter, defaulting to ``False``, to allow
negative integers.

.. _backwards-incompatible-1.10:

Backwards incompatible changes in 1.10
======================================

.. warning::

In addition to the changes outlined in this section, be sure to review the
:ref:`removed-features-1.10` for the features that have reached the end of
their deprecation cycle and therefore been removed. If you haven't updated
your code within the deprecation timeline for a given feature, its removal
may appear as a backwards incompatible change.

Database backend API
--------------------

* GIS's ``AreaField`` uses an unspecified underlying numeric type that could in
practice be any numeric Python type. ``decimal.Decimal`` values retrieved
from the database are now converted to ``float`` to make it easier to combine
them with values used by the GIS libraries.

* In order to enable temporal subtraction you must set the
``supports_temporal_subtraction`` database feature flag to ``True`` and
implement the ``DatabaseOperations.subtract_temporals()`` method. This
method should return the SQL and parameters required to compute the
difference in microseconds between the ``lhs`` and ``rhs`` arguments in the
datatype used to store :class:`~django.db.models.DurationField`.

``select_related()`` prohibits non-relational fields for nested relations
-------------------------------------------------------------------------

Django 1.8 added validation for non-relational fields in ``select_related()``::

>>> Book.objects.select_related('title')
Traceback (most recent call last):
...
FieldError: Non-relational field given in select_related: 'title'

But it didn't prohibit nested non-relation fields as it does now::

>>> Book.objects.select_related('author__name')
Traceback (most recent call last):
...
FieldError: Non-relational field given in select_related: 'name'

``_meta.get_fields()`` returns consistent reverse fields for proxy models
-------------------------------------------------------------------------

Before Django 1.10, the :meth:`~django.db.models.options.Options.get_fields`
method returned different reverse fields when called on a proxy model compared
to its proxied concrete class. This inconsistency was fixed by returning the
full set of fields pointing to a concrete class or one of its proxies in both
cases.

:attr:`AbstractUser.username <django.contrib.auth.models.User.username>` ``max_length`` increased to 150
--------------------------------------------------------------------------------------------------------

A migration for :attr:`django.contrib.auth.models.User.username` is included.
If you have a custom user model inheriting from ``AbstractUser``, you'll need
to generate and apply a database migration for your user model.

We considered an increase to 254 characters to more easily allow the use of
email addresses (which are limited to 254 characters) as usernames but rejected
it due to a MySQL limitation. When using the ``utf8mb4`` encoding (recommended
for proper Unicode support), MySQL can only create unique indexes with 191
characters by default. Therefore, if you need a longer length, please use a
custom user model.

If you want to preserve the 30 character limit for usernames, use a custom form
when creating a user or changing usernames::

from django.contrib.auth.forms import UserCreationForm

class MyUserCreationForm(UserCreationForm):
username = forms.CharField(
max_length=30,
help_text='Required. 30 characters or fewer. Letters, digits and /./+/-/_ only.',
)

If you wish to keep this restriction in the admin, set ``UserAdmin.add_form``
to use this form::

from django.contrib.auth.admin import UserAdmin as BaseUserAdmin
from django.contrib.auth.models import User

class UserAdmin(BaseUserAdmin):
add_form = MyUserCreationForm

admin.site.unregister(User)
admin.site.register(User, UserAdmin)

Dropped support for PostgreSQL 9.1
----------------------------------

Upstream support for PostgreSQL 9.1 ends in September 2016. As a consequence,
Django 1.10 sets PostgreSQL 9.2 as the minimum version it officially supports.

``runserver`` output goes through logging
-----------------------------------------

Request and response handling of the ``runserver`` command is sent to the
:ref:`django-server-logger` logger instead of to ``sys.stderr``. If you
disable Django's logging configuration or override it with your own, you'll
need to add the appropriate logging configuration if you want to see that
output::

'formatters': {
'django.server': {
'()': 'django.utils.log.ServerFormatter',
'format': '[%(server_time)s] %(message)s',
}
},
'handlers': {
'django.server': {
'level': 'INFO',
'class': 'logging.StreamHandler',
'formatter': 'django.server',
},
},
'loggers': {
'django.server': {
'handlers': ['django.server'],
'level': 'INFO',
'propagate': False,
}
}

``auth.CustomUser`` and ``auth.ExtensionUser`` test models were removed
-----------------------------------------------------------------------

Since the introduction of migrations for the contrib apps in Django 1.8, the
tables of these custom user test models were not created anymore making them
unusable in a testing context.

Apps registry is no longer auto-populated when unpickling models outside of Django
----------------------------------------------------------------------------------

The apps registry is no longer auto-populated when unpickling models. This was
added in Django 1.7.2 as an attempt to allow unpickling models outside of
Django, such as in an RQ worker, without calling ``django.setup()``, but it
creates the possibility of a deadlock. To adapt your code in the case of RQ,
you can `provide your own worker script <https://python-rq.org/docs/workers/>`_
that calls ``django.setup()``.

Removed null assignment check for non-null foreign key fields
-------------------------------------------------------------

In older versions, assigning ``None`` to a non-nullable ``ForeignKey`` or
``OneToOneField`` raised ``ValueError('Cannot assign None: "model.field" does
not allow null values.')``. For consistency with other model fields which don't
have a similar check, this check is removed.

Removed weak password hashers from the default ``PASSWORD_HASHERS`` setting
---------------------------------------------------------------------------

Django 0.90 stored passwords as unsalted MD5. Django 0.91 added support for
salted SHA1 with automatic upgrade of passwords when a user logs in. Django 1.4
added PBKDF2 as the default password hasher.

If you have an old Django project with MD5 or SHA1 (even salted) encoded
passwords, be aware that these can be cracked fairly easily with today's
hardware. To make Django users acknowledge continued use of weak hashers, the
following hashers are removed from the default :setting:`PASSWORD_HASHERS`
setting::

'django.contrib.auth.hashers.SHA1PasswordHasher'
'django.contrib.auth.hashers.MD5PasswordHasher'
'django.contrib.auth.hashers.UnsaltedSHA1PasswordHasher'
'django.contrib.auth.hashers.UnsaltedMD5PasswordHasher'
'django.contrib.auth.hashers.CryptPasswordHasher'

Consider using a :ref:`wrapped password hasher <wrapping-password-hashers>` to
strengthen the hashes in your database. If that's not feasible, add the
:setting:`PASSWORD_HASHERS` setting to your project and add back any hashers
that you need.

You can check if your database has any of the removed hashers like this::

from django.contrib.auth import get_user_model
User = get_user_model()

Unsalted MD5/SHA1:
User.objects.filter(password__startswith='md5$$')
User.objects.filter(password__startswith='sha1$$')
Salted MD5/SHA1:
User.objects.filter(password__startswith='md5$').exclude(password__startswith='md5$$')
User.objects.filter(password__startswith='sha1$').exclude(password__startswith='sha1$$')
Crypt hasher:
User.objects.filter(password__startswith='crypt$$')

from django.db.models import CharField
from django.db.models.functions import Length
CharField.register_lookup(Length)
Unsalted MD5 passwords might not have an 'md5$$' prefix:
User.objects.filter(password__length=32)

``Field.get_prep_lookup()`` and ``Field.get_db_prep_lookup()`` methods are removed
----------------------------------------------------------------------------------

If you have a custom field that implements either of these methods, register a
custom lookup for it. For example::

from django.db.models import Field
from django.db.models.lookups import Exact

class MyField(Field):
...

class MyFieldExact(Exact):
def get_prep_lookup(self):
do_custom_stuff_for_myfield
....

MyField.register_lookup(MyFieldExact)

:mod:`django.contrib.gis`
-------------------------

* Support for SpatiaLite < 3.0 and GEOS < 3.3 is dropped.

* The ``add_postgis_srs()`` backwards compatibility alias for
``django.contrib.gis.utils.add_srs_entry()`` is removed.

* On Oracle/GIS, the :class:`~django.contrib.gis.db.models.functions.Area`
aggregate function now returns a ``float`` instead of ``decimal.Decimal``.
(It's still wrapped in a measure of square meters.)

* The default :class:`~django.contrib.gis.geos.GEOSGeometry` representation (WKT
output) is trimmed by default. That is, instead of
``POINT (23.0000000000000000 5.5000000000000000)``, you'll get
``POINT (23 5.5)``.

Maximum size of a request body and the number of GET/POST parameters is limited
-------------------------------------------------------------------------------

Two new settings help mitigate denial-of-service attacks via large requests:

* :setting:`DATA_UPLOAD_MAX_MEMORY_SIZE` limits the size that a request body
may be. File uploads don't count toward this limit.
* :setting:`DATA_UPLOAD_MAX_NUMBER_FIELDS` limits the number of GET/POST
parameters that are parsed.

Applications that receive unusually large form posts may need to tune these
settings.

Miscellaneous
-------------

* The ``repr()`` of a ``QuerySet`` is wrapped in ``<QuerySet >`` to
disambiguate it from a plain list when debugging.

* ``utils.version.get_version()`` returns :pep:`440` compliant release
candidate versions (e.g. '1.10rc1' instead of '1.10c1').

* CSRF token values are now required to be strings of 64 alphanumerics; values
of 32 alphanumerics, as set by older versions of Django by default, are
automatically replaced by strings of 64 characters. Other values are
considered invalid. This should only affect developers or users who replace
these tokens.

* The ``LOGOUT_URL`` setting is removed as Django hasn't made use of it
since pre-1.0. If you use it in your project, you can add it to your
project's settings. The default value was ``'/accounts/logout/'``.

* Objects with a ``close()`` method such as files and generators passed to
:class:`~django.http.HttpResponse` are now closed immediately instead of when
the WSGI server calls ``close()`` on the response.

* A redundant ``transaction.atomic()`` call in ``QuerySet.update_or_create()``
is removed. This may affect query counts tested by
``TransactionTestCase.assertNumQueries()``.

* Support for ``skip_validation`` in ``BaseCommand.execute(**options)`` is
removed. Use ``skip_checks`` (added in Django 1.7) instead.

* :djadmin:`loaddata` now raises a ``CommandError`` instead of showing a
warning when the specified fixture file is not found.

* Instead of directly accessing the ``LogEntry.change_message`` attribute, it's
now better to call the ``LogEntry.get_change_message()`` method which will
provide the message in the current language.

* The default error views now raise ``TemplateDoesNotExist`` if a nonexistent
``template_name`` is specified.

* The unused ``choices`` keyword argument of the ``Select`` and
``SelectMultiple`` widgets' ``render()`` method is removed. The ``choices``
argument of the ``render_options()`` method is also removed, making
``selected_choices`` the first argument.

* Tests that violate deferrable database constraints will now error when run on
a database that supports deferrable constraints.

* Built-in management commands now use indexing of keys in ``options``, e.g.
``options['verbosity']``, instead of ``options.get()`` and no longer perform
any type coercion. This could be a problem if you're calling commands using
``Command.execute()`` (which bypasses the argument parser that sets a default
value) instead of :func:`~django.core.management.call_command`. Instead of
calling ``Command.execute()``, pass the command object as the first argument
to ``call_command()``.

* :class:`~django.contrib.auth.backends.ModelBackend` and
:class:`~django.contrib.auth.backends.RemoteUserBackend` now reject inactive
users. This means that inactive users can't login and will be logged
out if they are switched from ``is_active=True`` to ``False``. If you need
the previous behavior, use the new
:class:`~django.contrib.auth.backends.AllowAllUsersModelBackend` or
:class:`~django.contrib.auth.backends.AllowAllUsersRemoteUserBackend`
in :setting:`AUTHENTICATION_BACKENDS` instead.

* In light of the previous change, the test client's
:meth:`~django.test.Client.login()` method no longer always rejects inactive
users but instead delegates this decision to the authentication backend.
:meth:`~django.test.Client.force_login()` also delegates the decision to the
authentication backend, so if you're using the default backends, you need to
use an active user.

* :func:`django.views.i18n.set_language` may now return a 204 status code for
AJAX requests.

* The ``base_field`` attribute of
:class:`~django.contrib.postgres.fields.RangeField` is now a type of field,
not an instance of a field. If you have created a custom subclass of
:class:`~django.contrib.postgres.fields.RangeField`, you should change the
``base_field`` attribute.

* Middleware classes are now initialized when the server starts rather than
during the first request.

* If you override ``is_authenticated()`` or ``is_anonymous()`` in a custom user
model, you must convert them to attributes or properties as described in
:ref:`the deprecation note <user-is-auth-anon-deprecation>`.

* When using :attr:`ModelAdmin.save_as=True
<django.contrib.admin.ModelAdmin.save_as>`, the "Save as new" button now
redirects to the change view for the new object instead of to the model's
changelist. If you need the previous behavior, set the new
:attr:`ModelAdmin.save_as_continue
<django.contrib.admin.ModelAdmin.save_as_continue>` attribute to ``False``.

* Required form fields now have the ``required`` HTML attribute. Set the
:attr:`Form.use_required_attribute <django.forms.Form.use_required_attribute>`
attribute to ``False`` to disable it. You could also add the ``novalidate``
attribute to ``<form>`` if you don't want browser validation. To disable
the ``required`` attribute on custom widgets, override the
:meth:`Widget.use_required_attribute() <django.forms.Widget.use_required_attribute>`
method.

* The WSGI handler no longer removes content of responses from ``HEAD``
requests or responses with a ``status_code`` of 100-199, 204, or 304. Most
web servers already implement this behavior. Responses retrieved using the
Django test client continue to have these "response fixes" applied.

* ``Model.__init__()`` now receives ``django.db.models.DEFERRED`` as the value
of deferred fields.

* The ``Model._deferred`` attribute is removed as dynamic model classes when
using ``QuerySet.defer()`` and ``only()`` is removed.

* :meth:`Storage.save() <django.core.files.storage.Storage.save>` no longer
replaces ``'\'`` with ``'/'``. This behavior is moved to
:class:`~django.core.files.storage.FileSystemStorage` since this is a storage
specific implementation detail. Any Windows user with a custom storage
implementation that relies on this behavior will need to implement it in the
custom storage's ``save()`` method.

* Private :class:`~django.db.models.FileField` methods ``get_directory_name()``
and ``get_filename()`` are no longer called (and are now deprecated) which is
a backwards incompatible change for users overriding those methods on custom
fields. To adapt such code, override ``FileField.generate_filename()`` or
:meth:`Storage.generate_filename()
<django.core.files.storage.Storage.generate_filename>` instead. It
might be possible to use :attr:`~django.db.models.FileField.upload_to` also.

* The subject of mail sent by ``AdminEmailHandler`` is no longer truncated at
989 characters. If you were counting on a limited length, truncate the subject
yourself.

* Private expressions ``django.db.models.expressions.Date`` and ``DateTime``
are removed. The new :class:`~django.db.models.functions.Trunc` expressions
provide the same functionality.

* The ``_base_manager`` and ``_default_manager`` attributes are removed from
model instances. They remain accessible on the model class.

* Accessing a deleted field on a model instance, e.g. after ``del obj.field``,
reloads the field's value instead of raising ``AttributeError``.

* If you subclass ``AbstractBaseUser`` and override ``clean()``, be sure it
calls ``super()``. :meth:`.AbstractBaseUser.normalize_username` is called in
a new :meth:`.AbstractBaseUser.clean` method.

* Private API ``django.forms.models.model_to_dict()`` returns a queryset rather
than a list of primary keys for ``ManyToManyField``\s.

* If ``django.contrib.staticfiles`` is
installed, the :ttag:`static` template tag uses the ``staticfiles`` storage
to construct the URL rather than simply joining the value with
``STATIC_ROOT``. The new approach encodes the URL, which could be
backwards-incompatible in cases such as including a fragment in a path, e.g.
``{% static 'img.svgfragment' %}``, since the ` is encoded as ``%23``.
To adapt, move the fragment outside the template tag:
``{% static 'img.svg' %}fragment``.

* When ``USE_L10N`` is ``True``, localization is now applied for the
:tfilter:`date` and :tfilter:`time` filters when no format string is
specified. The ``DATE_FORMAT`` and ``TIME_FORMAT`` specifiers from the active
locale are used instead of the settings of the same name.

.. _deprecated-features-1.10:

Features deprecated in 1.10
===========================

Direct assignment to a reverse foreign key or many-to-many relation
-------------------------------------------------------------------

Instead of assigning related objects using direct assignment::

>>> new_list = [obj1, obj2, obj3]
>>> e.related_set = new_list

Use the :meth:`~django.db.models.fields.related.RelatedManager.set` method
added in Django 1.9::

>>> e.related_set.set([obj1, obj2, obj3])

This prevents confusion about an assignment resulting in an implicit save.

Non-timezone-aware :class:`~django.core.files.storage.Storage` API
------------------------------------------------------------------

The old, non-timezone-aware methods ``accessed_time()``, ``created_time()``,
and ``modified_time()`` are deprecated in favor of the new ``get_*_time()``
methods.

Third-party storage backends should implement the new methods and mark the old
ones as deprecated. Until then, the new ``get_*_time()`` methods on the base
:class:`~django.core.files.storage.Storage` class convert ``datetime``\s from
the old methods as required and emit a deprecation warning as they do so.

Third-party storage backends may retain the old methods as long as they
wish to support earlier versions of Django.

:mod:`django.contrib.gis`
-------------------------

* The ``get_srid()`` and ``set_srid()`` methods of
:class:`~django.contrib.gis.geos.GEOSGeometry` are deprecated in favor
of the :attr:`~django.contrib.gis.geos.GEOSGeometry.srid` property.

* The ``get_x()``, ``set_x()``, ``get_y()``, ``set_y()``, ``get_z()``, and
``set_z()`` methods of :class:`~django.contrib.gis.geos.Point` are deprecated
in favor of the ``x``, ``y``, and ``z`` properties.

* The ``get_coords()`` and ``set_coords()`` methods of
:class:`~django.contrib.gis.geos.Point` are deprecated in favor of the
``tuple`` property.

* The ``cascaded_union`` property of
:class:`~django.contrib.gis.geos.MultiPolygon` is deprecated in favor of the
:attr:`~django.contrib.gis.geos.GEOSGeometry.unary_union` property.

* The ``django.contrib.gis.utils.precision_wkt()`` function is deprecated in
favor of :class:`~django.contrib.gis.geos.WKTWriter`.

``CommaSeparatedIntegerField`` model field
------------------------------------------

``CommaSeparatedIntegerField`` is deprecated in favor of
:class:`~django.db.models.CharField` with the
:func:`~django.core.validators.validate_comma_separated_integer_list`
validator::

from django.core.validators import validate_comma_separated_integer_list
from django.db import models

class MyModel(models.Model):
numbers = models.CharField(..., validators=[validate_comma_separated_integer_list])

If you're using Oracle, ``CharField`` uses a different database field type
(``NVARCHAR2``) than ``CommaSeparatedIntegerField`` (``VARCHAR2``). Depending
on your database settings, this might imply a different encoding, and thus a
different length (in bytes) for the same contents. If your stored values are
longer than the 4000 byte limit of ``NVARCHAR2``, you should use ``TextField``
(``NCLOB``) instead. In this case, if you have any queries that group by the
field (e.g. annotating the model with an aggregation or using ``distinct()``)
you'll need to change them (to defer the field).

Using a model name as a query lookup when ``default_related_name`` is set
-------------------------------------------------------------------------

Assume the following models::

from django.db import models

class Foo(models.Model):
pass

class Bar(models.Model):
foo = models.ForeignKey(Foo)

class Meta:
default_related_name = 'bars'

In older versions, :attr:`~django.db.models.Options.default_related_name`
couldn't be used as a query lookup. This is fixed and support for the old
lookup name is deprecated. For example, since ``default_related_name`` is set
in model ``Bar``, instead of using the model name ``bar`` as the lookup::

>>> bar = Bar.objects.get(pk=1)
>>> Foo.objects.get(bar=bar)

use the default_related_name ``bars``::

>>> Foo.objects.get(bars=bar)

.. _search-lookup-replacement:

``__search`` query lookup
-------------------------

The ``search`` lookup, which supports MySQL only and is extremely limited in
features, is deprecated. Replace it with a custom lookup::

from django.db import models

class Search(models.Lookup):
lookup_name = 'search'

def as_mysql(self, compiler, connection):
lhs, lhs_params = self.process_lhs(compiler, connection)
rhs, rhs_params = self.process_rhs(compiler, connection)
params = lhs_params + rhs_params
return 'MATCH (%s) AGAINST (%s IN BOOLEAN MODE)' % (lhs, rhs), params

models.CharField.register_lookup(Search)
models.TextField.register_lookup(Search)

.. _user-is-auth-anon-deprecation:

Using ``User.is_authenticated()`` and ``User.is_anonymous()`` as methods
------------------------------------------------------------------------

The ``is_authenticated()`` and ``is_anonymous()`` methods of
:class:`~django.contrib.auth.models.AbstractBaseUser` and
:class:`~django.contrib.auth.models.AnonymousUser` classes are now
properties. They will still work as methods until Django 2.0, but all usage
in Django now uses attribute access.

For example, if you use
:class:`~django.contrib.auth.middleware.AuthenticationMiddleware` and want
to know whether the user is currently logged-in you would use::

if request.user.is_authenticated:
... Do something for logged-in users.
else:
... Do something for anonymous users.

instead of ``request.user.is_authenticated()``.

This change avoids accidental information leakage if you forget to call the
method, e.g.::

if request.user.is_authenticated:
return sensitive_information

If you override these methods in a custom user model, you must change them to
properties or attributes.

Django uses a ``CallableBool`` object to allow these attributes to work as both
a property and a method. Thus, until the deprecation period ends, you cannot
compare these properties using the ``is`` operator. That is, the following
won't work::

if request.user.is_authenticated is True:
...

Custom manager classes available through ``prefetch_related`` must define a ``_apply_rel_filters()`` method
-----------------------------------------------------------------------------------------------------------

If you defined a custom manager class available through
:meth:`~django.db.models.query.QuerySet.prefetch_related` you must make sure
it defines a ``_apply_rel_filters()`` method.

This method must accept a :class:`~django.db.models.query.QuerySet` instance
as its single argument and return a filtered version of the queryset for the
model instance the manager is bound to.

The "escape" half of ``django.utils.safestring``
------------------------------------------------

The ``mark_for_escaping()`` function and the classes it uses: ``EscapeData``,
``EscapeBytes``, ``EscapeText``, ``EscapeString``, and ``EscapeUnicode`` are
deprecated.

As a result, the "lazy" behavior of the ``escape`` filter (where it would
always be applied as the last filter no matter where in the filter chain it
appeared) is deprecated. The filter will change to immediately apply
:func:`~django.utils.html.conditional_escape` in Django 2.0.

``Manager.use_for_related_fields`` and inheritance changes
----------------------------------------------------------

``Manager.use_for_related_fields`` is deprecated in favor of setting
:attr:`Meta.base_manager_name <django.db.models.Options.base_manager_name>` on
the model.

Model ``Manager`` inheritance will follow MRO inheritance rules in Django 2.0,
changing the current behavior where managers defined on non-abstract base
classes aren't inherited by child classes. A deprecating warning with
instructions on how to adapt your code is raised if you have any affected
managers. You'll either redeclare a manager from an abstract model on the child
class to override the manager from the concrete model, or you'll set the
model's ``Meta.manager_inheritance_from_future=True`` option to opt-in to the
new inheritance behavior.

During the deprecation period, ``use_for_related_fields`` will be honored and
raise a warning, even if a ``base_manager_name`` is set. This allows
third-party code to preserve legacy behavior while transitioning to the new
API. The warning can be silenced by setting
``silence_use_for_related_fields_deprecation=True`` on the manager.

Miscellaneous
-------------

* The ``makemigrations --exit`` option is deprecated in favor of the
:option:`makemigrations --check` option.

* ``django.utils.functional.allow_lazy()`` is deprecated in favor of the new
:func:`~django.utils.functional.keep_lazy` function which can be used with a
more natural decorator syntax.

* The ``shell --plain`` option is deprecated in favor of ``-i python`` or
``--interface python``.

* Importing from the ``django.core.urlresolvers`` module is deprecated in
favor of its new location, :mod:`django.urls`.

* The template ``Context.has_key()`` method is deprecated in favor of ``in``.

* The private attribute ``virtual_fields`` of ``Model._meta`` is
deprecated in favor of ``private_fields``.

* The private keyword arguments ``virtual_only`` in
``Field.contribute_to_class()`` and ``virtual`` in
``Model._meta.add_field()`` are deprecated in favor of ``private_only``
and ``private``, respectively.

* The ``javascript_catalog()`` and ``json_catalog()`` views are deprecated in
favor of class-based views :class:`~django.views.i18n.JavaScriptCatalog`
and :class:`~django.views.i18n.JSONCatalog`.

* In multi-table inheritance, implicit promotion of a ``OneToOneField`` to a
``parent_link`` is deprecated. Add ``parent_link=True`` to such fields.

* The private API ``Widget._format_value()`` is made public and renamed to
:meth:`~django.forms.Widget.format_value`. The old name will work
through a deprecation period.

* Private ``FileField`` methods ``get_directory_name()`` and ``get_filename()``
are deprecated in favor of performing this work in
:meth:`Storage.generate_filename()
<django.core.files.storage.Storage.generate_filename>`).

* Old-style middleware that uses ``settings.MIDDLEWARE_CLASSES`` are
deprecated. :ref:`Adapt old, custom middleware <upgrading-middleware>` and
use the new :setting:`MIDDLEWARE` setting.

.. _removed-features-1.10:

Features removed in 1.10
========================

These features have reached the end of their deprecation cycle and are removed
in Django 1.10. See :ref:`deprecated-features-1.8` for details, including how
to remove usage of these features.

* Support for calling a ``SQLCompiler`` directly as an alias for calling its
``quote_name_unless_alias`` method is removed.

* The ``cycle`` and ``firstof`` template tags are removed from the ``future``
template tag library.

* ``django.conf.urls.patterns()`` is removed.

* Support for the ``prefix`` argument to
``django.conf.urls.i18n.i18n_patterns()`` is removed.

* ``SimpleTestCase.urls`` is removed.

* Using an incorrect count of unpacked values in the ``for`` template tag
raises an exception rather than failing silently.

* The ability to :func:`~django.urls.reverse` URLs using a dotted Python path
is removed.

* The ability to use a dotted Python path for the ``LOGIN_URL`` and
``LOGIN_REDIRECT_URL`` settings is removed.

* Support for ``optparse`` is dropped for custom management commands.

* The class ``django.core.management.NoArgsCommand`` is removed.

* ``django.core.context_processors`` module is removed.

* ``django.db.models.sql.aggregates`` module is removed.

* ``django.contrib.gis.db.models.sql.aggregates`` module is removed.

* The following methods and properties of ``django.db.sql.query.Query`` are
removed:

* Properties: ``aggregates`` and ``aggregate_select``
* Methods: ``add_aggregate``, ``set_aggregate_mask``, and
``append_aggregate_mask``.

* ``django.template.resolve_variable`` is removed.

* The following private APIs are removed from
:class:`django.db.models.options.Options` (``Model._meta``):

* ``get_field_by_name()``
* ``get_all_field_names()``
* ``get_fields_with_model()``
* ``get_concrete_fields_with_model()``
* ``get_m2m_with_model()``
* ``get_all_related_objects()``
* ``get_all_related_objects_with_model()``
* ``get_all_related_many_to_many_objects()``
* ``get_all_related_m2m_objects_with_model()``

* The ``error_message`` argument of ``django.forms.RegexField`` is removed.

* The ``unordered_list`` filter no longer supports old style lists.

* Support for string ``view`` arguments to ``url()`` is removed.

* The backward compatible shim to rename ``django.forms.Form._has_changed()``
to ``has_changed()`` is removed.

* The ``removetags`` template filter is removed.

* The ``remove_tags()`` and ``strip_entities()`` functions in
``django.utils.html`` is removed.

* The ``is_admin_site`` argument to
``django.contrib.auth.views.password_reset()`` is removed.

* ``django.db.models.field.subclassing.SubfieldBase`` is removed.

* ``django.utils.checksums`` is removed.

* The ``original_content_type_id`` attribute on
``django.contrib.admin.helpers.InlineAdminForm`` is removed.

* The backwards compatibility shim to allow ``FormMixin.get_form()`` to be
defined with no default value for its ``form_class`` argument is removed.

* The following settings are removed, and you must upgrade to the
:setting:`TEMPLATES` setting:

* ``ALLOWED_INCLUDE_ROOTS``
* ``TEMPLATE_CONTEXT_PROCESSORS``
* ``TEMPLATE_DEBUG``
* ``TEMPLATE_DIRS``
* ``TEMPLATE_LOADERS``
* ``TEMPLATE_STRING_IF_INVALID``

* The backwards compatibility alias ``django.template.loader.BaseLoader`` is
removed.

* Django template objects returned by
:func:`~django.template.loader.get_template` and
:func:`~django.template.loader.select_template` no longer accept a
:class:`~django.template.Context` in their
:meth:`~django.template.backends.base.Template.render()` method.

* :doc:`Template response APIs </ref/template-response>` enforce the use of
:class:`dict` and backend-dependent template objects instead of
:class:`~django.template.Context` and :class:`~django.template.Template`
respectively.

* The ``current_app`` parameter for the following function and classes is
removed:

* ``django.shortcuts.render()``
* ``django.template.Context()``
* ``django.template.RequestContext()``
* ``django.template.response.TemplateResponse()``

* The ``dictionary`` and ``context_instance`` parameters for the following
functions are removed:

* ``django.shortcuts.render()``
* ``django.shortcuts.render_to_response()``
* ``django.template.loader.render_to_string()``

* The ``dirs`` parameter for the following functions is removed:

* ``django.template.loader.get_template()``
* ``django.template.loader.select_template()``
* ``django.shortcuts.render()``
* ``django.shortcuts.render_to_response()``

* Session verification is enabled regardless of whether or not
``'django.contrib.auth.middleware.SessionAuthenticationMiddleware'`` is in
``MIDDLEWARE_CLASSES``. ``SessionAuthenticationMiddleware`` no longer has
any purpose and can be removed from ``MIDDLEWARE_CLASSES``. It's kept as
a stub until Django 2.0 as a courtesy for users who don't read this note.

* Private attribute ``django.db.models.Field.related`` is removed.

* The ``--list`` option of the ``migrate`` management command is removed.

* The ``ssi`` template tag is removed.

* Support for the ``=`` comparison operator in the ``if`` template tag is
removed.

* The backwards compatibility shims to allow ``Storage.get_available_name()``
and ``Storage.save()`` to be defined without a ``max_length`` argument are
removed.

* Support for the legacy ``%(<foo>)s`` syntax in ``ModelFormMixin.success_url``
is removed.

* ``GeoQuerySet`` aggregate methods ``collect()``, ``extent()``, ``extent3d()``,
``make_line()``, and ``unionagg()`` are removed.

* The ability to specify ``ContentType.name`` when creating a content type
instance is removed.

* Support for the old signature of ``allow_migrate`` is removed.

* Support for the syntax of ``{% cycle %}`` that uses comma-separated arguments
is removed.

* The warning that :class:`~django.core.signing.Signer` issued when given an
invalid separator is now a ``ValueError``.


===========================

1.9.13

Not secure
===========================

*April 4, 2017*

Django 1.9.13 fixes two security issues and a bug in 1.9.12. This is the final
release of the 1.9.x series.

CVE-2017-7233: Open redirect and possible XSS attack via user-supplied numeric redirect URLs
============================================================================================

Django relies on user input in some cases (e.g.
``django.contrib.auth.views.login()`` and :doc:`i18n </topics/i18n/index>`)
to redirect the user to an "on success" URL. The security check for these
redirects (namely ``django.utils.http.is_safe_url()``) considered some numeric
URLs (e.g. ``http:999999999``) "safe" when they shouldn't be.

Also, if a developer relies on ``is_safe_url()`` to provide safe redirect
targets and puts such a URL into a link, they could suffer from an XSS attack.

CVE-2017-7234: Open redirect vulnerability in ``django.views.static.serve()``
=============================================================================

A maliciously crafted URL to a Django site using the
:func:`~django.views.static.serve` view could redirect to any other domain. The
view no longer does any redirects as they don't provide any known, useful
functionality.

Note, however, that this view has always carried a warning that it is not
hardened for production use and should be used only as a development aid.

Bugfixes
========

* Fixed a regression in the ``timesince`` and ``timeuntil`` filters that caused
incorrect results for dates in a leap year (:ticket:`27637`).


===========================

1.9.12

Not secure
===========================

*December 1, 2016*

Django 1.9.12 fixes a regression in 1.9.11.

Bugfixes
========

* Quoted the Oracle test user's password in queries to fix the "ORA-00922:
missing or invalid option" error when the password starts with a number or
special character (:ticket:`27420`).


===========================

1.9.11

Not secure
===========================

*November 1, 2016*

Django 1.9.11 fixes two security issues in 1.9.10.

User with hardcoded password created when running tests on Oracle
=================================================================

When running tests with an Oracle database, Django creates a temporary database
user. In older versions, if a password isn't manually specified in the database
settings ``TEST`` dictionary, a hardcoded password is used. This could allow
an attacker with network access to the database server to connect.

This user is usually dropped after the test suite completes, but not when using
the ``manage.py test --keepdb`` option or if the user has an active session
(such as an attacker's connection).

A randomly generated password is now used for each test run.

DNS rebinding vulnerability when ``DEBUG=True``
===============================================

Older versions of Django don't validate the ``Host`` header against
``settings.ALLOWED_HOSTS`` when ``settings.DEBUG=True``. This makes them
vulnerable to a `DNS rebinding attack
<https://benmmurphy.github.io/blog/2016/07/11/rails-webconsole-dns-rebinding/>`_.

While Django doesn't ship a module that allows remote code execution, this is
at least a cross-site scripting vector, which could be quite serious if
developers load a copy of the production database in development or connect to
some production services for which there's no development instance, for
example. If a project uses a package like the ``django-debug-toolbar``, then
the attacker could execute arbitrary SQL, which could be especially bad if the
developers connect to the database with a superuser account.

``settings.ALLOWED_HOSTS`` is now validated regardless of ``DEBUG``. For
convenience, if ``ALLOWED_HOSTS`` is empty and ``DEBUG=True``, the following
variations of localhost are allowed ``['localhost', '127.0.0.1', '::1']``. If
your local settings file has your production ``ALLOWED_HOSTS`` value, you must
now omit it to get those fallback values.


===========================

1.9.10

Not secure
===========================

*September 26, 2016*

Django 1.9.10 fixes a security issue in 1.9.9.

CSRF protection bypass on a site with Google Analytics
======================================================

An interaction between Google Analytics and Django's cookie parsing could allow
an attacker to set arbitrary cookies leading to a bypass of CSRF protection.

The parser for ``request.COOKIES`` is simplified to better match the behavior
of browsers and to mitigate this attack. ``request.COOKIES`` may now contain
cookies that are invalid according to :rfc:`6265` but are possible to set via
``document.cookie``.


==========================

1.9.9

Not secure
==========================

*August 1, 2016*

Django 1.9.9 fixes several bugs in 1.9.8.

Bugfixes
========

* Fixed invalid HTML in template postmortem on the debug page
(:ticket:`26938`).

* Fixed some GIS database function crashes on MySQL 5.7 (:ticket:`26657`).


==========================

1.9.8

Not secure
==========================

*July 18, 2016*

Django 1.9.8 fixes a security issue and several bugs in 1.9.7.

XSS in admin's add/change related popup
=======================================

Unsafe usage of JavaScript's ``Element.innerHTML`` could result in XSS in the
admin's add/change related popup. ``Element.textContent`` is now used to
prevent execution of the data.

The debug view also used ``innerHTML``. Although a security issue wasn't
identified there, out of an abundance of caution it's also updated to use
``textContent``.

Bugfixes
========

* Fixed missing ``varchar/text_pattern_ops`` index on ``CharField`` and
``TextField`` respectively when using ``AddField`` on PostgreSQL
(:ticket:`26889`).

* Fixed ``makemessages`` crash on Python 2 with non-ASCII file names
(:ticket:`26897`).


==========================

1.9.7

Not secure
==========================

*June 4, 2016*

Django 1.9.7 fixes several bugs in 1.9.6.

Bugfixes
========

* Removed the need for the ``request`` context processor on the admin login
page to fix a regression in 1.9 (:ticket:`26558`).

* Fixed translation of password validators' ``help_text`` in forms
(:ticket:`26544`).

* Fixed a regression causing the cached template loader to crash when using
lazy template names (:ticket:`26603`).

* Fixed ``on_commit`` callbacks execution order when callbacks make
transactions (:ticket:`26627`).

* Fixed ``HStoreField`` to raise a ``ValidationError`` instead of crashing on
non-dictionary JSON input (:ticket:`26672`).

* Fixed ``dbshell`` crash on PostgreSQL with an empty database name
(:ticket:`26698`).

* Fixed a regression in queries on a ``OneToOneField`` that has ``to_field``
and ``primary_key=True`` (:ticket:`26667`).


==========================

1.9.6

Not secure
==========================

*May 2, 2016*

Django 1.9.6 fixes several bugs in 1.9.5.

Bugfixes
========

* Added support for relative path redirects to the test client and to
``SimpleTestCase.assertRedirects()`` because Django 1.9 no longer converts
redirects to absolute URIs (:ticket:`26428`).

* Fixed ``TimeField`` microseconds round-tripping on MySQL and SQLite
(:ticket:`26498`).

* Prevented ``makemigrations`` from generating infinite migrations for a model
field that references a ``functools.partial`` (:ticket:`26475`).

* Fixed a regression where ``SessionBase.pop()`` returned ``None`` rather than
raising a ``KeyError`` for nonexistent values (:ticket:`26520`).

* Fixed a regression causing the cached template loader to crash when using
template names starting with a dash (:ticket:`26536`).

* Restored conversion of an empty string to null when saving values of
``GenericIPAddressField`` on SQLite and MySQL (:ticket:`26557`).

* Fixed a ``makemessages`` regression where temporary ``.py`` extensions were
leaked in source file paths (:ticket:`26341`).


==========================

1.9.5

Not secure
==========================

*April 1, 2016*

Django 1.9.5 fixes several bugs in 1.9.4.

Bugfixes
========

* Made ``MultiPartParser`` ignore filenames that normalize to an empty string
to fix crash in ``MemoryFileUploadHandler`` on specially crafted user input
(:ticket:`26325`).

* Fixed a race condition in ``BaseCache.get_or_set()`` (:ticket:`26332`). It
now returns the ``default`` value instead of ``False`` if there's an error
when trying to add the value to the cache.

* Fixed data loss on SQLite where ``DurationField`` values with fractional
seconds could be saved as ``None`` (:ticket:`26324`).

* The forms in ``contrib.auth`` no longer strip trailing and leading whitespace
from the password fields (:ticket:`26334`). The change requires users who set
their password to something with such whitespace after a site updated to
Django 1.9 to reset their password. It provides backwards-compatibility for
earlier versions of Django.

* Fixed a memory leak in the cached template loader (:ticket:`26306`).

* Fixed a regression that caused ``collectstatic --clear`` to fail if the
storage doesn't implement ``path()`` (:ticket:`26297`).

* Fixed a crash when using a reverse lookup with a subquery when a
``ForeignKey`` has a ``to_field`` set to something other than the primary key
(:ticket:`26373`).

* Fixed a regression in ``CommonMiddleware`` that caused spurious warnings in
logs on requests missing a trailing slash (:ticket:`26293`).

* Restored the functionality of the admin's ``raw_id_fields`` in
``list_editable`` (:ticket:`26387`).

* Fixed a regression with abstract model inheritance and explicit parent links
(:ticket:`26413`).

* Fixed a migrations crash on SQLite when renaming the primary key of a model
containing a ``ForeignKey`` to ``'self'`` (:ticket:`26384`).

* Fixed ``JSONField`` inadvertently escaping its contents when displaying values
after failed form validation (:ticket:`25532`).


==========================

1.9.4

Not secure
==========================

*March 5, 2016*

Django 1.9.4 fixes a regression on Python 2 in the 1.9.3 security release
where ``utils.http.is_safe_url()`` crashes on bytestring URLs (:ticket:`26308`).


==========================

1.9.3

Not secure
==========================

*March 1, 2016*

Django 1.9.3 fixes two security issues and several bugs in 1.9.2.

CVE-2016-2512: Malicious redirect and possible XSS attack via user-supplied redirect URLs containing basic auth
===============================================================================================================

Django relies on user input in some cases (e.g.
``django.contrib.auth.views.login()`` and :doc:`i18n </topics/i18n/index>`)
to redirect the user to an "on success" URL. The security check for these
redirects (namely ``django.utils.http.is_safe_url()``) considered some URLs
with basic authentication credentials "safe" when they shouldn't be.

For example, a URL like ``http://mysite.example.com\attacker.com`` would be
considered safe if the request's host is ``http://mysite.example.com``, but
redirecting to this URL sends the user to ``attacker.com``.

Also, if a developer relies on ``is_safe_url()`` to provide safe redirect
targets and puts such a URL into a link, they could suffer from an XSS attack.

CVE-2016-2513: User enumeration through timing difference on password hasher work factor upgrade
================================================================================================

In each major version of Django since 1.6, the default number of iterations for
the ``PBKDF2PasswordHasher`` and its subclasses has increased. This improves
the security of the password as the speed of hardware increases, however, it
also creates a timing difference between a login request for a user with a
password encoded in an older number of iterations and login request for a
nonexistent user (which runs the default hasher's default number of iterations
since Django 1.6).

This only affects users who haven't logged in since the iterations were
increased. The first time a user logs in after an iterations increase, their
password is updated with the new iterations and there is no longer a timing
difference.

The new ``BasePasswordHasher.harden_runtime()`` method allows hashers to bridge
the runtime gap between the work factor (e.g. iterations) supplied in existing
encoded passwords and the default work factor of the hasher. This method
is implemented for ``PBKDF2PasswordHasher`` and ``BCryptPasswordHasher``.
The number of rounds for the latter hasher hasn't changed since Django 1.4, but
some projects may subclass it and increase the work factor as needed.

A warning will be emitted for any :ref:`third-party password hashers that don't
implement <write-your-own-password-hasher>` a ``harden_runtime()`` method.

If you have different password hashes in your database (such as SHA1 hashes
from users who haven't logged in since the default hasher switched to PBKDF2
in Django 1.4), the timing difference on a login request for these users may be
even greater and this fix doesn't remedy that difference (or any difference
when changing hashers). You may be able to :ref:`upgrade those hashes
<wrapping-password-hashers>` to prevent a timing attack for that case.

Bugfixes
========

* Skipped URL checks (new in 1.9) if the ``ROOT_URLCONF`` setting isn't defined
(:ticket:`26155`).

* Fixed a crash on PostgreSQL that prevented using ``TIME_ZONE=None`` and
``USE_TZ=False`` (:ticket:`26177`).

* Added system checks for query name clashes of hidden relationships
(:ticket:`26162`).

* Fixed a regression for cases where
``ForeignObject.get_extra_descriptor_filter()`` returned a ``Q`` object
(:ticket:`26153`).

* Fixed regression with an ``__in=qs`` lookup for a ``ForeignKey`` with
``to_field`` set (:ticket:`26196`).

* Made ``forms.FileField`` and ``utils.translation.lazy_number()`` picklable
(:ticket:`26212`).

* Fixed :class:`~django.contrib.postgres.fields.RangeField` and
:class:`~django.contrib.postgres.fields.ArrayField` serialization with
``None`` values (:ticket:`26215`).

* Fixed a crash when filtering by a ``Decimal`` in ``RawQuery``
(:ticket:`26219`).

* Reallowed dashes in top-level domain names of URLs checked by
``URLValidator`` to fix a regression in Django 1.8 (:ticket:`26204`).

* Fixed some crashing deprecation shims in ``SimpleTemplateResponse`` that
regressed in Django 1.9 (:ticket:`26253`).

* Fixed ``BoundField`` to reallow slices of subwidgets (:ticket:`26267`).

* Changed the admin's "permission denied" message in the login template to use
``get_username`` instead of ``username`` to support custom user models
(:ticket:`26231`).

* Fixed a crash when passing a nonexistent template name to the cached template
loader's ``load_template()`` method (:ticket:`26280`).

* Prevented ``ContentTypeManager`` instances from sharing their cache
(:ticket:`26286`).

* Reverted a change in Django 1.9.2 (:ticket:`25858`) that prevented relative
lazy relationships defined on abstract models to be resolved according to
their concrete model's ``app_label`` (:ticket:`26186`).


==========================

1.9.2

Not secure
==========================

*February 1, 2016*

Django 1.9.2 fixes a security regression in 1.9 and several bugs in 1.9.1. It
also makes a small backwards incompatible change that hopefully doesn't affect
any users.

Security issue: User with "change" but not "add" permission can create objects for ``ModelAdmin``’s with ``save_as=True``
=========================================================================================================================

If a ``ModelAdmin`` uses ``save_as=True`` (not the default), the admin
provides an option when editing objects to "Save as new". A regression in
Django 1.9 prevented that form submission from raising a "Permission Denied"
error for users without the "add" permission.

Backwards incompatible change: ``.py-tpl`` files rewritten in project/app templates
===================================================================================

The addition of some Django template language syntax to the default app
template in Django 1.9 means those files now have some invalid Python syntax.
This causes difficulties for packaging systems that unconditionally
byte-compile ``*.py`` files.

To remedy this, a ``.py-tpl`` suffix is now used for the project and app
template files included in Django. The ``.py-tpl`` suffix is replaced with
``.py`` by the ``startproject`` and ``startapp`` commands. For example, a
template with the filename ``manage.py-tpl`` will be created as ``manage.py``.

Please file a ticket if you have a custom project template containing
``.py-tpl`` files and find this behavior problematic.

Bugfixes
========

* Fixed a regression in ``ConditionalGetMiddleware`` causing ``If-None-Match``
checks to always return HTTP 200 (:ticket:`26024`).

* Fixed a regression that caused the "user-tools" items to display on the
admin's logout page (:ticket:`26035`).

* Fixed a crash in the translations system when the current language has no
translations (:ticket:`26046`).

* Fixed a regression that caused the incorrect day to be selected when opening
the admin calendar widget for timezones from GMT+0100 to GMT+1200
(:ticket:`24980`).

* Fixed a regression in the admin's edit related model popup that caused an
escaped value to be displayed in the select dropdown of the parent window
(:ticket:`25997`).

* Fixed a regression in 1.8.8 causing incorrect index handling in migrations on
PostgreSQL when adding ``db_index=True`` or ``unique=True`` to a
``CharField`` or ``TextField`` that already had the other specified, or when
removing one of them from a field that had both, or when adding
``unique=True`` to a field already listed in ``unique_together``
(:ticket:`26034`).

* Fixed a regression where defining a relation on an abstract model's field
using a string model name without an app_label no longer resolved that
reference to the abstract model's app if using that model in another
application (:ticket:`25858`).

* Fixed a crash when destroying an existing test database on MySQL or
PostgreSQL (:ticket:`26096`).

* Fixed CSRF cookie check on POST requests when ``USE_X_FORWARDED_PORT=True``
(:ticket:`26094`).

* Fixed a ``QuerySet.order_by()`` crash when ordering by a relational field of
a ``ManyToManyField`` ``through`` model (:ticket:`26092`).

* Fixed a regression that caused an exception when making database queries on
SQLite with more than 2000 parameters when :setting:`DEBUG` is ``True`` on
distributions that increase the ``SQLITE_MAX_VARIABLE_NUMBER`` compile-time
limit to over 2000, such as Debian (:ticket:`26063`).

* Fixed a crash when using a reverse ``OneToOneField`` in
``ModelAdmin.readonly_fields`` (:ticket:`26060`).

* Fixed a crash when calling the ``migrate`` command in a test case with the
``available_apps`` attribute pointing to an application with migrations
disabled using the ``MIGRATION_MODULES`` setting (:ticket:`26135`).

* Restored the ability for testing and debugging tools to determine the
template from which a node came from, even during template inheritance or
inclusion. Prior to Django 1.9, debugging tools could access the template
origin from the node via ``Node.token.source[0]``. This was an undocumented,
private API. The origin is now available directly on each node using the
``Node.origin`` attribute (:ticket:`25848`).

* Fixed a regression in Django 1.8.5 that broke copying a ``SimpleLazyObject``
with ``copy.copy()`` (:ticket:`26122`).

* Always included ``geometry_field`` in the GeoJSON serializer output regardless
of the ``fields`` parameter (:ticket:`26138`).

* Fixed the ``contrib.gis`` map widgets when using
``USE_THOUSAND_SEPARATOR=True`` (:ticket:`20415`).

* Made invalid forms display the initial of values of their disabled fields
(:ticket:`26129`).


==========================

1.9.1

Not secure
==========================

*January 2, 2016*

Django 1.9.1 fixes several bugs in 1.9.

Bugfixes
========

* Fixed ``BaseCache.get_or_set()`` with the ``DummyCache`` backend
(:ticket:`25840`).

* Fixed a regression in ``FormMixin`` causing forms to be validated twice
(:ticket:`25548`, :ticket:`26018`).

* Fixed a system check crash with nested ``ArrayField``\s (:ticket:`25867`).

* Fixed a state bug when migrating a ``SeparateDatabaseAndState`` operation
backwards (:ticket:`25896`).

* Fixed a regression in ``CommonMiddleware`` causing ``If-None-Match`` checks
to always return HTTP 200 (:ticket:`25900`).

* Fixed missing ``varchar/text_pattern_ops`` index on ``CharField`` and
``TextField`` respectively when using ``AlterField`` on PostgreSQL
(:ticket:`25412`).

* Fixed admin's delete confirmation page's summary counts of related objects
(:ticket:`25883`).

* Added ``from __future__ import unicode_literals`` to the default ``apps.py``
created by ``startapp`` on Python 2 (:ticket:`25909`). Add this line to your
own ``apps.py`` files created using Django 1.9 if you want your migrations
to work on both Python 2 and Python 3.

* Prevented ``QuerySet.delete()`` from crashing on MySQL when querying across
relations (:ticket:`25882`).

* Fixed evaluation of zero-length slices of ``QuerySet.values()``
(:ticket:`25894`).

* Fixed a state bug when using an ``AlterModelManagers`` operation
(:ticket:`25852`).

* Fixed ``TypedChoiceField`` change detection with nullable fields
(:ticket:`25942`).

* Fixed incorrect timezone warnings in custom admin templates that don't have
a ``data-admin-utc-offset`` attribute in the ``body`` tag.
(:ticket:`25845`).

* Fixed a regression which prevented using a language not in Django's default
language list (:setting:`LANGUAGES`) (:ticket:`25915`).

* Avoided hiding some exceptions, like an invalid ``INSTALLED_APPS`` setting,
behind ``AppRegistryNotReady`` when starting ``runserver`` (:ticket:`25510`).
This regression appeared in 1.8.5 as a side effect of fixing :ticket:`24704`
and by mistake the fix wasn't applied to the ``stable/1.9.x`` branch.

* Fixed ``migrate --fake-initial`` detection of many-to-many tables
(:ticket:`25922`).

* Restored the functionality of the admin's ``list_editable`` add and change
buttons (:ticket:`25903`).

* Fixed ``isnull`` query lookup for ``ForeignObject``
(:ticket:`25972`).

* Fixed a regression in the admin which ignored line breaks in read-only fields
instead of converting them to ``<br>`` (:ticket:`25465`).

* Fixed incorrect object reference in
``SingleObjectMixin.get_context_object_name()`` (:ticket:`26006`).

* Made ``loaddata`` skip disabling and enabling database constraints when it
doesn't load any fixtures (:ticket:`23372`).

* Restored ``contrib.auth`` hashers compatibility with ``py-bcrypt``
(:ticket:`26016`).

* Fixed a crash in ``QuerySet.values()/values_list()`` after an ``annotate()``
and ``order_by()`` when ``values()/values_list()`` includes a field not in
the ``order_by()`` (:ticket:`25316`).


========================

1.9

Not secure
========================

*December 1, 2015*

Welcome to Django 1.9!

These release notes cover the :ref:`new features <whats-new-1.9>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-1.9>` you'll
want to be aware of when upgrading from Django 1.8 or older versions. We've
:ref:`dropped some features<removed-features-1.9>` that have reached the end of
their deprecation cycle, and we've :ref:`begun the deprecation process for some
features <deprecated-features-1.9>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Python compatibility
====================

Django 1.9 requires Python 2.7, 3.4, or 3.5. We **highly recommend** and only
officially support the latest release of each series.

The Django 1.8 series is the last to support Python 3.2 and 3.3.

.. _whats-new-1.9:

What's new in Django 1.9
========================

Performing actions after a transaction commit
---------------------------------------------

The new :func:`~django.db.transaction.on_commit` hook allows performing actions
after a database transaction is successfully committed. This is useful for
tasks such as sending notification emails, creating queued tasks, or
invalidating caches.

This functionality from the `django-transaction-hooks`_ package has been
integrated into Django.

.. _django-transaction-hooks: https://pypi.org/project/django-transaction-hooks/

Password validation
-------------------

Django now offers password validation to help prevent the usage of weak
passwords by users. The validation is integrated in the included password
change and reset forms and is simple to integrate in any other code.
Validation is performed by one or more validators, configured in the new
:setting:`AUTH_PASSWORD_VALIDATORS` setting.

Four validators are included in Django, which can enforce a minimum length,
compare the password to the user's attributes like their name, ensure
passwords aren't entirely numeric, or check against an included list of common
passwords. You can combine multiple validators, and some validators have
custom configuration options. For example, you can choose to provide a custom
list of common passwords. Each validator provides a help text to explain its
requirements to the user.

By default, no validation is performed and all passwords are accepted, so if
you don't set :setting:`AUTH_PASSWORD_VALIDATORS`, you will not see any
change. In new projects created with the default :djadmin:`startproject`
template, a simple set of validators is enabled. To enable basic validation in
the included auth forms for your project, you could set, for example::

AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]

See :ref:`password-validation` for more details.

Permission mixins for class-based views
---------------------------------------

Django now ships with the mixins
:class:`~django.contrib.auth.mixins.AccessMixin`,
:class:`~django.contrib.auth.mixins.LoginRequiredMixin`,
:class:`~django.contrib.auth.mixins.PermissionRequiredMixin`, and
:class:`~django.contrib.auth.mixins.UserPassesTestMixin` to provide the
functionality of the ``django.contrib.auth.decorators`` for class-based views.
These mixins have been taken from, or are at least inspired by, the
`django-braces`_ project.

There are a few differences between Django's and ``django-braces``\'
implementation, though:

* The :attr:`~django.contrib.auth.mixins.AccessMixin.raise_exception` attribute
can only be ``True`` or ``False``. Custom exceptions or callables are not
supported.

* The :meth:`~django.contrib.auth.mixins.AccessMixin.handle_no_permission`
method does not take a ``request`` argument. The current request is available
in ``self.request``.

* The custom ``test_func()`` of :class:`~django.contrib.auth.mixins.UserPassesTestMixin`
does not take a ``user`` argument. The current user is available in
``self.request.user``.

* The :attr:`permission_required <django.contrib.auth.mixins.PermissionRequiredMixin>`
attribute supports a string (defining one permission) or a list/tuple of
strings (defining multiple permissions) that need to be fulfilled to grant
access.

* The new :attr:`~django.contrib.auth.mixins.AccessMixin.permission_denied_message`
attribute allows passing a message to the ``PermissionDenied`` exception.

.. _django-braces: https://django-braces.readthedocs.io/en/latest/index.html

New styling for ``contrib.admin``
---------------------------------

The admin sports a modern, flat design with new SVG icons which look perfect
on HiDPI screens. It still provides a fully-functional experience to `YUI's
A-grade`_ browsers. Older browser may experience varying levels of graceful
degradation.

.. _YUI's A-grade: https://github.com/yui/yui3/wiki/Graded-Browser-Support

Running tests in parallel
-------------------------

The :djadmin:`test` command now supports a :option:`--parallel <test
--parallel>` option to run a project's tests in multiple processes in parallel.

Each process gets its own database. You must ensure that different test cases
don't access the same resources. For instance, test cases that touch the
filesystem should create a temporary directory for their own use.

This option is enabled by default for Django's own test suite provided:

- the OS supports it (all but Windows)
- the database backend supports it (all the built-in backends but Oracle)

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Admin views now have ``model_admin`` or ``admin_site`` attributes.

* The URL of the admin change view has been changed (was at
``/admin/<app>/<model>/<pk>/`` by default and is now at
``/admin/<app>/<model>/<pk>/change/``). This should not affect your
application unless you have hardcoded admin URLs. In that case, replace those
links by :ref:`reversing admin URLs <admin-reverse-urls>` instead. Note that
the old URL still redirects to the new one for backwards compatibility, but
it may be removed in a future version.

* :meth:`ModelAdmin.get_list_select_related()
<django.contrib.admin.ModelAdmin.get_list_select_related>` was added to allow
changing the ``select_related()`` values used in the admin's changelist query
based on the request.

* The ``available_apps`` context variable, which lists the available
applications for the current user, has been added to the
:meth:`AdminSite.each_context() <django.contrib.admin.AdminSite.each_context>`
method.

* :attr:`AdminSite.empty_value_display
<django.contrib.admin.AdminSite.empty_value_display>` and
:attr:`ModelAdmin.empty_value_display
<django.contrib.admin.ModelAdmin.empty_value_display>` were added to override
the display of empty values in admin change list. You can also customize the
value for each field.

* Added jQuery events :ref:`when an inline form is added or removed
<admin-javascript-inline-form-events>` on the change form page.

* The time picker widget includes a '6 p.m' option for consistency of having
predefined options every 6 hours.

* JavaScript slug generation now supports Romanian characters.

:mod:`django.contrib.admindocs`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The model section of the ``admindocs`` now also describes methods that take
arguments, rather than ignoring them.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* The default iteration count for the PBKDF2 password hasher has been increased
by 20%. This backwards compatible change will not affect users who have
subclassed ``django.contrib.auth.hashers.PBKDF2PasswordHasher`` to change the
default value.

* The ``BCryptSHA256PasswordHasher`` will now update passwords if its
``rounds`` attribute is changed.

* ``AbstractBaseUser`` and ``BaseUserManager`` were moved to a new
``django.contrib.auth.base_user`` module so that they can be imported without
including ``django.contrib.auth`` in :setting:`INSTALLED_APPS` (doing so
raised a deprecation warning in older versions and is no longer supported in
Django 1.9).

* The permission argument of
:func:`~django.contrib.auth.decorators.permission_required()` accepts all
kinds of iterables, not only list and tuples.

* The new :class:`~django.contrib.auth.middleware.PersistentRemoteUserMiddleware`
makes it possible to use ``REMOTE_USER`` for setups where the header is only
populated on login pages instead of every request in the session.

* The ``django.contrib.auth.views.password_reset()`` view accepts an
``extra_email_context`` parameter.

:mod:`django.contrib.contenttypes`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* It's now possible to use
:attr:`~django.db.models.Options.order_with_respect_to` with a
``GenericForeignKey``.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* All ``GeoQuerySet`` methods have been deprecated and replaced by
:doc:`equivalent database functions </ref/contrib/gis/functions>`. As soon
as the legacy methods have been replaced in your code, you should even be
able to remove the special ``GeoManager`` from your GIS-enabled classes.

* The GDAL interface now supports instantiating file-based and in-memory
:ref:`GDALRaster objects <raster-data-source-objects>` from raw data.
Setters for raster properties such as projection or pixel values have
been added.

* For PostGIS users, the new :class:`~django.contrib.gis.db.models.RasterField`
allows :ref:`storing GDALRaster objects <creating-and-saving-raster-models>`.
It supports automatic spatial index creation and reprojection when saving a
model. It does not yet support spatial querying.

* The new :meth:`GDALRaster.warp() <django.contrib.gis.gdal.GDALRaster.warp>`
method allows warping a raster by specifying target raster properties such as
origin, width, height, or pixel size (among others).

* The new :meth:`GDALRaster.transform()
<django.contrib.gis.gdal.GDALRaster.transform>` method allows transforming a
raster into a different spatial reference system by specifying a target
``srid``.

* The new :class:`~django.contrib.gis.geoip2.GeoIP2` class allows using
MaxMind's GeoLite2 databases which includes support for IPv6 addresses.

* The default OpenLayers library version included in widgets has been updated
from 2.13 to 2.13.1.

:mod:`django.contrib.postgres`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Added support for the :lookup:`rangefield.contained_by` lookup for some built
in fields which correspond to the range fields.

* Added ``django.contrib.postgres.fields.JSONField``.

* Added :doc:`/ref/contrib/postgres/aggregates`.

* Added the :class:`~django.contrib.postgres.functions.TransactionNow` database
function.

:mod:`django.contrib.sessions`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The session model and ``SessionStore`` classes for the ``db`` and
``cached_db`` backends are refactored to allow a custom database session
backend to build upon them. See
:ref:`extending-database-backed-session-engines` for more details.

:mod:`django.contrib.sites`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* :func:`~django.contrib.sites.shortcuts.get_current_site` now handles the case
where ``request.get_host()`` returns ``domain:port``, e.g.
``example.com:80``. If the lookup fails because the host does not match a
record in the database and the host has a port, the port is stripped and the
lookup is retried with the domain part only.

:mod:`django.contrib.syndication`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Support for multiple enclosures per feed item has been added. If multiple
enclosures are defined on a RSS feed, an exception is raised as RSS feeds,
unlike Atom feeds, do not support multiple enclosures per feed item.

Cache
~~~~~

* ``django.core.cache.backends.base.BaseCache`` now has a ``get_or_set()``
method.

* :func:`django.views.decorators.cache.never_cache` now sends more persuasive
headers (added ``no-cache, no-store, must-revalidate`` to ``Cache-Control``)
to better prevent caching. This was also added in Django 1.8.8.

CSRF
~~~~

* The request header's name used for CSRF authentication can be customized
with :setting:`CSRF_HEADER_NAME`.

* The CSRF referer header is now validated against the
:setting:`CSRF_COOKIE_DOMAIN` setting if set. See :ref:`how-csrf-works` for
details.

* The new :setting:`CSRF_TRUSTED_ORIGINS` setting provides a way to allow
cross-origin unsafe requests (e.g. ``POST``) over HTTPS.

Database backends
~~~~~~~~~~~~~~~~~

* The PostgreSQL backend (``django.db.backends.postgresql_psycopg2``) is also
available as ``django.db.backends.postgresql``. The old name will continue to
be available for backwards compatibility.

File Storage
~~~~~~~~~~~~

* :meth:`Storage.get_valid_name()
<django.core.files.storage.Storage.get_valid_name>` is now called when
the :attr:`~django.db.models.FileField.upload_to` is a callable.

* :class:`~django.core.files.File` now has the ``seekable()`` method when using
Python 3.

Forms
~~~~~

* :class:`~django.forms.ModelForm` accepts the new ``Meta`` option
``field_classes`` to customize the type of the fields. See
:ref:`modelforms-overriding-default-fields` for details.

* You can now specify the order in which form fields are rendered with the
:attr:`~django.forms.Form.field_order` attribute, the ``field_order``
constructor argument , or the :meth:`~django.forms.Form.order_fields` method.

* A form prefix can be specified inside a form class, not only when
instantiating a form. See :ref:`form-prefix` for details.

* You can now :ref:`specify keyword arguments <custom-formset-form-kwargs>`
that you want to pass to the constructor of forms in a formset.

* :class:`~django.forms.SlugField` now accepts an
:attr:`~django.forms.SlugField.allow_unicode` argument to allow Unicode
characters in slugs.

* :class:`~django.forms.CharField` now accepts a
:attr:`~django.forms.CharField.strip` argument to strip input data of leading
and trailing whitespace. As this defaults to ``True`` this is different
behavior from previous releases.

* Form fields now support the :attr:`~django.forms.Field.disabled` argument,
allowing the field widget to be displayed disabled by browsers.

* It's now possible to customize bound fields by overriding a field's
:meth:`~django.forms.Field.get_bound_field()` method.

Generic Views
~~~~~~~~~~~~~

* Class-based views generated using ``as_view()`` now have ``view_class``
and ``view_initkwargs`` attributes.

* :func:`~django.utils.decorators.method_decorator` can now be used with a list
or tuple of decorators. It can also be used to :ref:`decorate classes instead
of methods <decorating-class-based-views>`.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* The :func:`django.views.i18n.set_language` view now properly redirects to
:ref:`translated URLs <url-internationalization>`, when available.

* The ``django.views.i18n.javascript_catalog()`` view now works correctly
if used multiple times with different configurations on the same page.

* The :func:`django.utils.timezone.make_aware` function gained an ``is_dst``
argument to help resolve ambiguous times during DST transitions.

* You can now use locale variants supported by gettext. These are usually used
for languages which can be written in different scripts, for example Latin
and Cyrillic (e.g. ``belatin``).

* Added the ``django.views.i18n.json_catalog()`` view to help build a custom
client-side i18n library upon Django translations. It returns a JSON object
containing a translations catalog, formatting settings, and a plural rule.

* Added the ``name_translated`` attribute to the object returned by the
:ttag:`get_language_info` template tag. Also added a corresponding template
filter: :tfilter:`language_name_translated`.

* You can now run :djadmin:`compilemessages` from the root directory of your
project and it will find all the app message files that were created by
:djadmin:`makemessages`.

* :djadmin:`makemessages` now calls ``xgettext`` once per locale directory
rather than once per translatable file. This speeds up localization builds.

* :ttag:`blocktrans` supports assigning its output to a variable using
``asvar``.

* Two new languages are available: Colombian Spanish and Scottish Gaelic.

Management Commands
~~~~~~~~~~~~~~~~~~~

* The new :djadmin:`sendtestemail` command lets you send a test email to
easily confirm that email sending through Django is working.

* To increase the readability of the SQL code generated by
:djadmin:`sqlmigrate`, the SQL code generated for each migration operation is
preceded by the operation's description.

* The :djadmin:`dumpdata` command output is now deterministically ordered.
Moreover, when the ``--output`` option is specified, it also shows a progress
bar in the terminal.

* The :djadmin:`createcachetable` command now has a ``--dry-run`` flag to
print out the SQL rather than execute it.

* The :djadmin:`startapp` command creates an ``apps.py`` file. Since it doesn't
use ``default_app_config`` (:ref:`a discouraged API
<configuring-applications-ref>`), you must specify the app config's path,
e.g. ``'polls.apps.PollsConfig'``, in :setting:`INSTALLED_APPS` for it to be
used (instead of just ``'polls'``).

* When using the PostgreSQL backend, the :djadmin:`dbshell` command can connect
to the database using the password from your settings file (instead of
requiring it to be manually entered).

* The ``django`` package may be run as a script, i.e. ``python -m django``,
which will behave the same as ``django-admin``.

* Management commands that have the ``--noinput`` option now also take
``--no-input`` as an alias for that option.

Migrations
~~~~~~~~~~

* Initial migrations are now marked with an :attr:`initial = True
<django.db.migrations.Migration.initial>` class attribute which allows
:option:`migrate --fake-initial` to more easily detect initial migrations.

* Added support for serialization of ``functools.partial`` and ``LazyObject``
instances.

* When supplying ``None`` as a value in :setting:`MIGRATION_MODULES`, Django
will consider the app an app without migrations.

* When applying migrations, the "Rendering model states" step that's displayed
when running migrate with verbosity 2 or higher now computes only the states
for the migrations that have already been applied. The model states for
migrations being applied are generated on demand, drastically reducing the
amount of required memory.

However, this improvement is not available when unapplying migrations and
therefore still requires the precomputation and storage of the intermediate
migration states.

This improvement also requires that Django no longer supports mixed migration
plans. Mixed plans consist of a list of migrations where some are being
applied and others are being unapplied. This was never officially supported
and never had a public API that supports this behavior.

* The :djadmin:`squashmigrations` command now supports specifying the starting
migration from which migrations will be squashed.

Models
~~~~~~

* :meth:`QuerySet.bulk_create() <django.db.models.query.QuerySet.bulk_create>`
now works on proxy models.

* Database configuration gained a :setting:`TIME_ZONE <DATABASE-TIME_ZONE>`
option for interacting with databases that store datetimes in local time and
don't support time zones when :setting:`USE_TZ` is ``True``.

* Added the :meth:`RelatedManager.set()
<django.db.models.fields.related.RelatedManager.set()>` method to the related
managers created by ``ForeignKey``, ``GenericForeignKey``, and
``ManyToManyField``.

* The :meth:`~django.db.models.fields.related.RelatedManager.add` method on
a reverse foreign key now has a ``bulk`` parameter to allow executing one
query regardless of the number of objects being added rather than one query
per object.

* Added the ``keep_parents`` parameter to :meth:`Model.delete()
<django.db.models.Model.delete>` to allow deleting only a child's data in a
model that uses multi-table inheritance.

* :meth:`Model.delete() <django.db.models.Model.delete>`
and :meth:`QuerySet.delete() <django.db.models.query.QuerySet.delete>` return
the number of objects deleted.

* Added a system check to prevent defining both ``Meta.ordering`` and
``order_with_respect_to`` on the same model.

* :lookup:`Date and time <year>` lookups can be chained with other lookups
(such as :lookup:`exact`, :lookup:`gt`, :lookup:`lt`, etc.). For example:
``Entry.objects.filter(pub_date__month__gt=6)``.

* Time lookups (hour, minute, second) are now supported by
:class:`~django.db.models.TimeField` for all database backends. Support for
backends other than SQLite was added but undocumented in Django 1.7.

* You can specify the ``output_field`` parameter of the
:class:`~django.db.models.Avg` aggregate in order to aggregate over
non-numeric columns, such as ``DurationField``.

* Added the :lookup:`date` lookup to :class:`~django.db.models.DateTimeField`
to allow querying the field by only the date portion.

* Added the :class:`~django.db.models.functions.Greatest` and
:class:`~django.db.models.functions.Least` database functions.

* Added the :class:`~django.db.models.functions.Now` database function, which
returns the current date and time.

* :class:`~django.db.models.Transform` is now a subclass of
:ref:`Func() <func-expressions>` which allows ``Transform``\s to be used on
the right hand side of an expression, just like regular ``Func``\s. This
allows registering some database functions like
:class:`~django.db.models.functions.Length`,
:class:`~django.db.models.functions.Lower`, and
:class:`~django.db.models.functions.Upper` as transforms.

* :class:`~django.db.models.SlugField` now accepts an
:attr:`~django.db.models.SlugField.allow_unicode` argument to allow Unicode
characters in slugs.

* Added support for referencing annotations in ``QuerySet.distinct()``.

* ``connection.queries`` shows queries with substituted parameters on SQLite.

* :doc:`Query expressions </ref/models/expressions>` can now be used when
creating new model instances using ``save()``, ``create()``, and
``bulk_create()``.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* Unless :attr:`HttpResponse.reason_phrase
<django.http.HttpResponse.reason_phrase>` is explicitly set, it now is
determined by the current value of :attr:`HttpResponse.status_code
<django.http.HttpResponse.status_code>`. Modifying the value of
``status_code`` outside of the constructor will also modify the value of
``reason_phrase``.

* The debug view now shows details of chained exceptions on Python 3.

* The default 40x error views now accept a second positional parameter, the
exception that triggered the view.

* View error handlers now support
:class:`~django.template.response.TemplateResponse`, commonly used with
class-based views.

* Exceptions raised by the ``render()`` method are now passed to the
``process_exception()`` method of each middleware.

* Request middleware can now set :attr:`HttpRequest.urlconf
<django.http.HttpRequest.urlconf>` to ``None`` to revert any changes made
by previous middleware and return to using the :setting:`ROOT_URLCONF`.

* The :setting:`DISALLOWED_USER_AGENTS` check in
:class:`~django.middleware.common.CommonMiddleware` now raises a
:class:`~django.core.exceptions.PermissionDenied` exception as opposed to
returning an :class:`~django.http.HttpResponseForbidden` so that
:data:`~django.conf.urls.handler403` is invoked.

* Added :meth:`HttpRequest.get_port() <django.http.HttpRequest.get_port>` to
fetch the originating port of the request.

* Added the ``json_dumps_params`` parameter to
:class:`~django.http.JsonResponse` to allow passing keyword arguments to the
``json.dumps()`` call used to generate the response.

* The :class:`~django.middleware.common.BrokenLinkEmailsMiddleware` now
ignores 404s when the referer is equal to the requested URL. To circumvent
the empty referer check already implemented, some web bots set the referer to
the requested URL.

Templates
~~~~~~~~~

* Template tags created with the :meth:`~django.template.Library.simple_tag`
helper can now store results in a template variable by using the ``as``
argument.

* Added a :meth:`Context.setdefault() <django.template.Context.setdefault>`
method.

* The :ref:`django.template <django-template-logger>` logger was added and
includes the following messages:

* A ``DEBUG`` level message for missing context variables.

* A ``WARNING`` level message for uncaught exceptions raised
during the rendering of an ``{% include %}`` when debug mode is off
(helpful since ``{% include %}`` silences the exception and returns an
empty string).

* The :ttag:`firstof` template tag supports storing the output in a variable
using 'as'.

* :meth:`Context.update() <django.template.Context.update>` can now be used as
a context manager.

* Django template loaders can now extend templates recursively.

* The debug page template postmortem now include output from each engine that
is installed.

* :ref:`Debug page integration <template-debug-integration>` for custom
template engines was added.

* The :class:`~django.template.backends.django.DjangoTemplates` backend gained
the ability to register libraries and builtins explicitly through the
template :setting:`OPTIONS <TEMPLATES-OPTIONS>`.

* The ``timesince`` and ``timeuntil`` filters were improved to deal with leap
years when given large time spans.

* The ``include`` tag now caches parsed templates objects during template
rendering, speeding up reuse in places such as for loops.

Tests
~~~~~

* Added the :meth:`json() <django.test.Response.json>` method to test client
responses to give access to the response body as JSON.

* Added the :meth:`~django.test.Client.force_login()` method to the test
client. Use this method to simulate the effect of a user logging into the
site while skipping the authentication and verification steps of
:meth:`~django.test.Client.login()`.

URLs
~~~~

* Regular expression lookaround assertions are now allowed in URL patterns.

* The application namespace can now be set using an ``app_name`` attribute
on the included module or object. It can also be set by passing a 2-tuple
of (<list of patterns>, <application namespace>) as the first argument to
``include()``.

* System checks have been added for common URL pattern mistakes.

Validators
~~~~~~~~~~

* Added :func:`django.core.validators.int_list_validator` to generate
validators of strings containing integers separated with a custom character.

* :class:`~django.core.validators.EmailValidator` now limits the length of
domain name labels to 63 characters per :rfc:`1034`.

* Added :func:`~django.core.validators.validate_unicode_slug` to validate slugs
that may contain Unicode characters.

.. _backwards-incompatible-1.9:

Backwards incompatible changes in 1.9
=====================================

.. warning::

In addition to the changes outlined in this section, be sure to review the
:ref:`removed-features-1.9` for the features that have reached the end of
their deprecation cycle and therefore been removed. If you haven't updated
your code within the deprecation timeline for a given feature, its removal
may appear as a backwards incompatible change.

Database backend API
--------------------

* A couple of new tests rely on the ability of the backend to introspect column
defaults (returning the result as ``Field.default``). You can set the
``can_introspect_default`` database feature to ``False`` if your backend
doesn't implement this. You may want to review the implementation on the
backends that Django includes for reference (:ticket:`24245`).

* Registering a global adapter or converter at the level of the DB-API module
to handle time zone information of :class:`~datetime.datetime` values passed
as query parameters or returned as query results on databases that don't
support time zones is discouraged. It can conflict with other libraries.

The recommended way to add a time zone to :class:`~datetime.datetime` values
fetched from the database is to register a converter for ``DateTimeField``
in ``DatabaseOperations.get_db_converters()``.

The ``needs_datetime_string_cast`` database feature was removed. Database
backends that set it must register a converter instead, as explained above.

* The ``DatabaseOperations.value_to_db_<type>()`` methods were renamed to
``adapt_<type>field_value()`` to mirror the ``convert_<type>field_value()``
methods.

* To use the new ``date`` lookup, third-party database backends may need to
implement the ``DatabaseOperations.datetime_cast_date_sql()`` method.

* The ``DatabaseOperations.time_extract_sql()`` method was added. It calls the
existing ``date_extract_sql()`` method. This method is overridden by the
SQLite backend to add time lookups (hour, minute, second) to
:class:`~django.db.models.TimeField`, and may be needed by third-party
database backends.

* The ``DatabaseOperations.datetime_cast_sql()`` method (not to be confused
with ``DatabaseOperations.datetime_cast_date_sql()`` mentioned above)
has been removed. This method served to format dates on Oracle long
before 1.0, but hasn't been overridden by any core backend in years
and hasn't been called anywhere in Django's code or tests.

* In order to support test parallelization, you must implement the
``DatabaseCreation._clone_test_db()`` method and set
``DatabaseFeatures.can_clone_databases = True``. You may have to adjust
``DatabaseCreation.get_test_db_clone_settings()``.

Default settings that were tuples are now lists
-----------------------------------------------

The default settings in ``django.conf.global_settings`` were a combination of
lists and tuples. All settings that were formerly tuples are now lists.

``is_usable`` attribute on template loaders is removed
------------------------------------------------------

Django template loaders previously required an ``is_usable`` attribute to be
defined. If a loader was configured in the template settings and this attribute
was ``False``, the loader would be silently ignored. In practice, this was only
used by the egg loader to detect if setuptools was installed. The ``is_usable``
attribute is now removed and the egg loader instead fails at runtime if
setuptools is not installed.

Related set direct assignment
-----------------------------

Direct assignment of related objects in the ORM used to perform a ``clear()``
followed by a call to ``add()``. This caused needlessly large data changes and
prevented using the :data:`~django.db.models.signals.m2m_changed` signal to
track individual changes in many-to-many relations.

Direct assignment now relies on the new
:meth:`~django.db.models.fields.related.RelatedManager.set` method on related
managers which by default only processes changes between the existing related
set and the one that's newly assigned. The previous behavior can be restored by
replacing direct assignment by a call to ``set()`` with the keyword argument
``clear=True``.

``ModelForm``, and therefore ``ModelAdmin``, internally rely on direct
assignment for many-to-many relations and as a consequence now use the new
behavior.

Filesystem-based template loaders catch more specific exceptions
----------------------------------------------------------------

When using the :class:`filesystem.Loader <django.template.loaders.filesystem.Loader>`
or :class:`app_directories.Loader <django.template.loaders.app_directories.Loader>`
template loaders, earlier versions of Django raised a
:exc:`~django.template.TemplateDoesNotExist` error if a template source existed
but was unreadable. This could happen under many circumstances, such as if
Django didn't have permissions to open the file, or if the template source was
a directory. Now, Django only silences the exception if the template source
does not exist. All other situations result in the original ``IOError`` being
raised.

HTTP redirects no longer forced to absolute URIs
------------------------------------------------

Relative redirects are no longer converted to absolute URIs. :rfc:`2616`
required the ``Location`` header in redirect responses to be an absolute URI,
but it has been superseded by :rfc:`7231` which allows relative URIs in
``Location``, recognizing the actual practice of user agents, almost all of
which support them.

Consequently, the expected URLs passed to ``assertRedirects`` should generally
no longer include the scheme and domain part of the URLs. For example,
``self.assertRedirects(response, 'http://testserver/some-url/')`` should be
replaced by ``self.assertRedirects(response, '/some-url/')`` (unless the
redirection specifically contained an absolute URL).

In the rare case that you need the old behavior (discovered with an ancient
version of Apache with ``mod_scgi`` that interprets a relative redirect as an
"internal redirect"), you can restore it by writing a custom middleware::

class LocationHeaderFix(object):
def process_response(self, request, response):
if 'Location' in response:
response['Location'] = request.build_absolute_uri(response['Location'])
return response

Dropped support for PostgreSQL 9.0
----------------------------------

Upstream support for PostgreSQL 9.0 ended in September 2015. As a consequence,
Django 1.9 sets 9.1 as the minimum PostgreSQL version it officially supports.

Dropped support for Oracle 11.1
-------------------------------

Upstream support for Oracle 11.1 ended in August 2015. As a consequence, Django
1.9 sets 11.2 as the minimum Oracle version it officially supports.

Bulk behavior of ``add()`` method of related managers
-----------------------------------------------------

To improve performance, the ``add()`` methods of the related managers created
by ``ForeignKey`` and ``GenericForeignKey`` changed from a series of
``Model.save()`` calls to a single ``QuerySet.update()`` call. The change means
that ``pre_save`` and ``post_save`` signals aren't sent anymore. You can use
the ``bulk=False`` keyword argument to revert to the previous behavior.

Template ``LoaderOrigin`` and ``StringOrigin`` are removed
----------------------------------------------------------

In previous versions of Django, when a template engine was initialized with
debug as ``True``, an instance of ``django.template.loader.LoaderOrigin`` or
``django.template.base.StringOrigin`` was set as the origin attribute on the
template object. These classes have been combined into
:class:`~django.template.base.Origin` and is now always set regardless of the
engine debug setting. For a minimal level of backwards compatibility, the old
class names will be kept as aliases to the new ``Origin`` class until
Django 2.0.

.. _default-logging-changes-19:

Changes to the default logging configuration
--------------------------------------------

To make it easier to write custom logging configurations, Django's default
logging configuration no longer defines ``django.request`` and
``django.security`` loggers. Instead, it defines a single ``django`` logger,
filtered at the ``INFO`` level, with two handlers:

* ``console``: filtered at the ``INFO`` level and only active if ``DEBUG=True``.
* ``mail_admins``: filtered at the ``ERROR`` level and only active if
``DEBUG=False``.

If you aren't overriding Django's default logging, you should see minimal
changes in behavior, but you might see some new logging to the ``runserver``
console, for example.

If you are overriding Django's default logging, you should check to see how
your configuration merges with the new defaults.

``HttpRequest`` details in error reporting
------------------------------------------

It was redundant to display the full details of the
:class:`~django.http.HttpRequest` each time it appeared as a stack frame
variable in the HTML version of the debug page and error email. Thus, the HTTP
request will now display the same standard representation as other variables
(``repr(request)``). As a result, the
``ExceptionReporterFilter.get_request_repr()`` method and the undocumented
``django.http.build_request_repr()`` function were removed.

The contents of the text version of the email were modified to provide a
traceback of the same structure as in the case of AJAX requests. The traceback
details are rendered by the ``ExceptionReporter.get_traceback_text()`` method.

Removal of time zone aware global adapters and converters for datetimes
-----------------------------------------------------------------------

Django no longer registers global adapters and converters for managing time
zone information on :class:`~datetime.datetime` values sent to the database as
query parameters or read from the database in query results. This change
affects projects that meet all the following conditions:

* The :setting:`USE_TZ` setting is ``True``.
* The database is SQLite, MySQL, Oracle, or a third-party database that
doesn't support time zones. In doubt, you can check the value of
``connection.features.supports_timezones``.
* The code queries the database outside of the ORM, typically with
``cursor.execute(sql, params)``.

If you're passing aware :class:`~datetime.datetime` parameters to such
queries, you should turn them into naive datetimes in UTC::

from django.utils import timezone
param = timezone.make_naive(param, timezone.utc)

If you fail to do so, the conversion will be performed as in earlier versions
(with a deprecation warning) up until Django 1.11. Django 2.0 won't perform any
conversion, which may result in data corruption.

If you're reading :class:`~datetime.datetime` values from the results, they
will be naive instead of aware. You can compensate as follows::

from django.utils import timezone
value = timezone.make_aware(value, timezone.utc)

You don't need any of this if you're querying the database through the ORM,
even if you're using :meth:`raw() <django.db.models.query.QuerySet.raw>`
queries. The ORM takes care of managing time zone information.

Template tag modules are imported when templates are configured
---------------------------------------------------------------

The :class:`~django.template.backends.django.DjangoTemplates` backend now
performs discovery on installed template tag modules when instantiated. This
update enables libraries to be provided explicitly via the ``'libraries'``
key of :setting:`OPTIONS <TEMPLATES-OPTIONS>` when defining a
:class:`~django.template.backends.django.DjangoTemplates` backend. Import
or syntax errors in template tag modules now fail early at instantiation time
rather than when a template with a :ttag:`{% load %}<load>` tag is first
compiled.

``django.template.base.add_to_builtins()`` is removed
-----------------------------------------------------

Although it was a private API, projects commonly used ``add_to_builtins()`` to
make template tags and filters available without using the
:ttag:`{% load %}<load>` tag. This API has been formalized. Projects should now
define built-in libraries via the ``'builtins'`` key of :setting:`OPTIONS
<TEMPLATES-OPTIONS>` when defining a
:class:`~django.template.backends.django.DjangoTemplates` backend.

.. _simple-tag-conditional-escape-fix:

``simple_tag`` now wraps tag output in ``conditional_escape``
-------------------------------------------------------------

In general, template tags do not autoescape their contents, and this behavior is
:ref:`documented <tags-auto-escaping>`. For tags like
:class:`~django.template.Library.inclusion_tag`, this is not a problem because
the included template will perform autoescaping. For ``assignment_tag()``,
the output will be escaped when it is used as a variable in the template.

For the intended use cases of :class:`~django.template.Library.simple_tag`,
however, it is very easy to end up with incorrect HTML and possibly an XSS
exploit. For example::

register.simple_tag(takes_context=True)
def greeting(context):
return "Hello {0}!".format(context['request'].user.first_name)

In older versions of Django, this will be an XSS issue because
``user.first_name`` is not escaped.

In Django 1.9, this is fixed: if the template context has ``autoescape=True``
set (the default), then ``simple_tag`` will wrap the output of the tag function
with :func:`~django.utils.html.conditional_escape`.

To fix your ``simple_tag``\s, it is best to apply the following practices:

* Any code that generates HTML should use either the template system or
:func:`~django.utils.html.format_html`.

* If the output of a ``simple_tag`` needs escaping, use
:func:`~django.utils.html.escape` or
:func:`~django.utils.html.conditional_escape`.

* If you are absolutely certain that you are outputting HTML from a trusted
source (e.g. a CMS field that stores HTML entered by admins), you can mark it
as such using :func:`~django.utils.safestring.mark_safe`.

Tags that follow these rules will be correct and safe whether they are run on
Django 1.9+ or earlier.

``Paginator.page_range``
------------------------

:attr:`Paginator.page_range <django.core.paginator.Paginator.page_range>` is
now an iterator instead of a list.

In versions of Django previous to 1.8, ``Paginator.page_range`` returned a
``list`` in Python 2 and a ``range`` in Python 3. Django 1.8 consistently
returned a list, but an iterator is more efficient.

Existing code that depends on ``list`` specific features, such as indexing,
can be ported by converting the iterator into a ``list`` using ``list()``.

Implicit ``QuerySet`` ``__in`` lookup removed
---------------------------------------------

In earlier versions, queries such as::

Model.objects.filter(related_id=RelatedModel.objects.all())

would implicitly convert to::

Model.objects.filter(related_id__in=RelatedModel.objects.all())

resulting in SQL like ``"related_id IN (SELECT id FROM ...)"``.

This implicit ``__in`` no longer happens so the "IN" SQL is now "=", and if the
subquery returns multiple results, at least some databases will throw an error.

.. _admin-browser-support-19:

``contrib.admin`` browser support
---------------------------------

The admin no longer supports Internet Explorer 8 and below, as these browsers
have reached end-of-life.

CSS and images to support Internet Explorer 6 and 7 have been removed. PNG and
GIF icons have been replaced with SVG icons, which are not supported by
Internet Explorer 8 and earlier.

The jQuery library embedded in the admin has been upgraded from version 1.11.2
to 2.1.4. jQuery 2.x has the same API as jQuery 1.x, but does not support
Internet Explorer 6, 7, or 8, allowing for better performance and a smaller
file size. If you need to support IE8 and must also use the latest version of
Django, you can override the admin's copy of jQuery with your own by creating
a Django application with this structure::

app/static/admin/js/vendor/
jquery.js
jquery.min.js

.. _syntax-error-old-setuptools-django-19:

``SyntaxError`` when installing Django setuptools 5.5.x
-------------------------------------------------------

When installing Django 1.9 or 1.9.1 with setuptools 5.5.x, you'll see::

Compiling django/conf/app_template/apps.py ...
File "django/conf/app_template/apps.py", line 4
class {{ camel_case_app_name }}Config(AppConfig):
^
SyntaxError: invalid syntax

Compiling django/conf/app_template/models.py ...
File "django/conf/app_template/models.py", line 1
{{ unicode_literals }}from django.db import models
^
SyntaxError: invalid syntax

It's safe to ignore these errors (Django will still install just fine), but you
can avoid them by upgrading setuptools to a more recent version. If you're
using pip, you can upgrade pip using ``python -m pip install -U pip`` which
will also upgrade setuptools. This is resolved in later versions of Django as
described in the :doc:`/releases/1.9.2`.

Miscellaneous
-------------

* The jQuery static files in ``contrib.admin`` have been moved into a
``vendor/jquery`` subdirectory.

* The text displayed for null columns in the admin changelist ``list_display``
cells has changed from ``(None)`` (or its translated equivalent) to ``-`` (a
dash).

* ``django.http.responses.REASON_PHRASES`` and
``django.core.handlers.wsgi.STATUS_CODE_TEXT`` have been removed. Use
Python's Standard Library instead: :data:`http.client.responses` for Python
3 and `httplib.responses`_ for Python 2.

.. _`httplib.responses`: https://docs.python.org/2/library/httplib.html#httplib.responses

* ``ValuesQuerySet`` and ``ValuesListQuerySet`` have been removed.

* The ``admin/base.html`` template no longer sets
``window.__admin_media_prefix__`` or ``window.__admin_utc_offset__``. Image
references in JavaScript that used that value to construct absolute URLs have
been moved to CSS for easier customization. The UTC offset is stored on a
data attribute of the ``<body>`` tag.

* ``CommaSeparatedIntegerField`` validation has been refined to forbid values
like ``','``, ``',1'``, and ``'1,,2'``.

* Form initialization was moved from the :meth:`ProcessFormView.get()
<django.views.generic.edit.ProcessFormView.get>` method to the new
:meth:`FormMixin.get_context_data()
<django.views.generic.edit.FormMixin.get_context_data>` method. This may be
backwards incompatible if you have overridden the ``get_context_data()``
method without calling ``super()``.

* Support for PostGIS 1.5 has been dropped.

* The ``django.contrib.sites.models.Site.domain`` field was changed to be
:attr:`~django.db.models.Field.unique`.

* In order to enforce test isolation, database queries are not allowed
by default in :class:`~django.test.SimpleTestCase` tests anymore. You
can disable this behavior by setting the ``allow_database_queries`` class
attribute to ``True`` on your test class.

* ``ResolverMatch.app_name`` was changed to contain the full namespace path in
the case of nested namespaces. For consistency with
``ResolverMatch.namespace``, the empty value is now an empty string instead
of ``None``.

* For security hardening, session keys must be at least 8 characters.

* Private function ``django.utils.functional.total_ordering()`` has been
removed. It contained a workaround for a ``functools.total_ordering()`` bug
in Python versions older than 2.7.3.

* XML serialization (either through :djadmin:`dumpdata` or the syndication
framework) used to output any characters it received. Now if the content to
be serialized contains any control characters not allowed in the XML 1.0
standard, the serialization will fail with a :exc:`ValueError`.

* :class:`~django.forms.CharField` now strips input of leading and trailing
whitespace by default. This can be disabled by setting the new
:attr:`~django.forms.CharField.strip` argument to ``False``.

* Template text that is translated and uses two or more consecutive percent
signs, e.g. ``"%%"``, may have a new ``msgid`` after ``makemessages`` is run
(most likely the translation will be marked fuzzy). The new ``msgid`` will be
marked ``", python-format"``.

* If neither :attr:`request.current_app <django.http.HttpRequest.current_app>`
nor :class:`Context.current_app <django.template.Context>` are set, the
:ttag:`url` template tag will now use the namespace of the current request.
Set ``request.current_app`` to ``None`` if you don't want to use a namespace
hint.

* The :setting:`SILENCED_SYSTEM_CHECKS` setting now silences messages of all
levels. Previously, messages of ``ERROR`` level or higher were printed to the
console.

* The ``FlatPage.enable_comments`` field is removed from the ``FlatPageAdmin``
as it's unused by the application. If your project or a third-party app makes
use of it, :ref:`create a custom ModelAdmin <flatpages-admin>` to add it back.

* The return value of
:meth:`~django.test.runner.DiscoverRunner.setup_databases` and the first
argument of :meth:`~django.test.runner.DiscoverRunner.teardown_databases`
changed. They used to be ``(old_names, mirrors)`` tuples. Now they're just
the first item, ``old_names``.

* By default :class:`~django.test.LiveServerTestCase` attempts to find an
available port in the 8081-8179 range instead of just trying port 8081.

* The system checks for :class:`~django.contrib.admin.ModelAdmin` now check
instances rather than classes.

* The private API to apply mixed migration plans has been dropped for
performance reasons. Mixed plans consist of a list of migrations where some
are being applied and others are being unapplied.

* The related model object descriptor classes in
``django.db.models.fields.related`` (private API) are moved from the
``related`` module to ``related_descriptors`` and renamed as follows:

* ``ReverseSingleRelatedObjectDescriptor`` is ``ForwardManyToOneDescriptor``
* ``SingleRelatedObjectDescriptor`` is ``ReverseOneToOneDescriptor``
* ``ForeignRelatedObjectsDescriptor`` is ``ReverseManyToOneDescriptor``
* ``ManyRelatedObjectsDescriptor`` is ``ManyToManyDescriptor``

* If you implement a custom :data:`~django.conf.urls.handler404` view, it must
return a response with an HTTP 404 status code. Use
:class:`~django.http.HttpResponseNotFound` or pass ``status=404`` to the
:class:`~django.http.HttpResponse`. Otherwise, :setting:`APPEND_SLASH` won't
work correctly with ``DEBUG=False``.

.. _deprecated-features-1.9:

Features deprecated in 1.9
==========================

``assignment_tag()``
--------------------

Django 1.4 added the ``assignment_tag`` helper to ease the creation of
template tags that store results in a template variable. The
:meth:`~django.template.Library.simple_tag` helper has gained this same
ability, making the ``assignment_tag`` obsolete. Tags that use
``assignment_tag`` should be updated to use ``simple_tag``.

``{% cycle %}`` syntax with comma-separated arguments
-----------------------------------------------------

The :ttag:`cycle` tag supports an inferior old syntax from previous Django
versions:

.. code-block:: html+django

{% cycle row1,row2,row3 %}

Its parsing caused bugs with the current syntax, so support for the old syntax
will be removed in Django 1.10 following an accelerated deprecation.

``ForeignKey`` and ``OneToOneField`` ``on_delete`` argument
-----------------------------------------------------------

In order to increase awareness about cascading model deletion, the
``on_delete`` argument of ``ForeignKey`` and ``OneToOneField`` will be required
in Django 2.0.

Update models and existing migrations to explicitly set the argument. Since the
default is ``models.CASCADE``, add ``on_delete=models.CASCADE`` to all
``ForeignKey`` and ``OneToOneField``\s that don't use a different option. You
can also pass it as the second positional argument if you don't care about
compatibility with older versions of Django.

``Field.rel`` changes
---------------------

``Field.rel`` and its methods and attributes have changed to match the related
fields API. The ``Field.rel`` attribute is renamed to ``remote_field`` and many
of its methods and attributes are either changed or renamed.

The aim of these changes is to provide a documented API for relation fields.

``GeoManager`` and ``GeoQuerySet`` custom methods
-------------------------------------------------

All custom ``GeoQuerySet`` methods (``area()``, ``distance()``, ``gml()``, ...)
have been replaced by equivalent geographic expressions in annotations (see in
new features). Hence the need to set a custom ``GeoManager`` to GIS-enabled
models is now obsolete. As soon as your code doesn't call any of the deprecated
methods, you can simply remove the ``objects = GeoManager()`` lines from your
models.

Template loader APIs have changed
---------------------------------

Django template loaders have been updated to allow recursive template
extending. This change necessitated a new template loader API. The old
``load_template()`` and ``load_template_sources()`` methods are now deprecated.
Details about the new API can be found :ref:`in the template loader
documentation <custom-template-loaders>`.

Passing a 3-tuple or an ``app_name`` to ``include()``
-----------------------------------------------------

The instance namespace part of passing a tuple as an argument to ``include()``
has been replaced by passing the ``namespace`` argument to ``include()``. For
example::

polls_patterns = [
url(...),
]

urlpatterns = [
url(r'^polls/', include((polls_patterns, 'polls', 'author-polls'))),
]

becomes::

polls_patterns = ([
url(...),
], 'polls') 'polls' is the app_name

urlpatterns = [
url(r'^polls/', include(polls_patterns, namespace='author-polls')),
]

The ``app_name`` argument to ``include()`` has been replaced by passing a
2-tuple (as above), or passing an object or module with an ``app_name``
attribute (as below). If the ``app_name`` is set in this new way, the
``namespace`` argument is no longer required. It will default to the value of
``app_name``. For example, the URL patterns in the tutorial are changed from:

.. code-block:: python
:caption: mysite/urls.py

urlpatterns = [
url(r'^polls/', include('polls.urls', namespace="polls")),
...
]

to:

.. code-block:: python
:caption: mysite/urls.py

urlpatterns = [
url(r'^polls/', include('polls.urls')), 'namespace="polls"' removed
...
]

.. code-block:: python
:caption: polls/urls.py

app_name = 'polls' added
urlpatterns = [...]

This change also means that the old way of including an ``AdminSite`` instance
is deprecated. Instead, pass ``admin.site.urls`` directly to
``django.conf.urls.url()``:

.. code-block:: python
:caption: urls.py

from django.conf.urls import url
from django.contrib import admin

urlpatterns = [
url(r'^admin/', admin.site.urls),
]

URL application namespace required if setting an instance namespace
-------------------------------------------------------------------

In the past, an instance namespace without an application namespace
would serve the same purpose as the application namespace, but it was
impossible to reverse the patterns if there was an application namespace
with the same name. Includes that specify an instance namespace require that
the included URLconf sets an application namespace.

``current_app`` parameter to ``contrib.auth`` views
---------------------------------------------------

All views in ``django.contrib.auth.views`` have the following structure::

def view(request, ..., current_app=None, ...):

...

if current_app is not None:
request.current_app = current_app

return TemplateResponse(request, template_name, context)

As of Django 1.8, ``current_app`` is set on the ``request`` object. For
consistency, these views will require the caller to set ``current_app`` on the
``request`` instead of passing it in a separate argument.

``django.contrib.gis.geoip``
----------------------------

The :mod:`django.contrib.gis.geoip2` module supersedes
``django.contrib.gis.geoip``. The new module provides a similar API except that
it doesn't provide the legacy GeoIP-Python API compatibility methods.

Miscellaneous
-------------

* The ``weak`` argument to ``django.dispatch.signals.Signal.disconnect()`` has
been deprecated as it has no effect.

* The ``check_aggregate_support()`` method of
``django.db.backends.base.BaseDatabaseOperations`` has been deprecated and
will be removed in Django 2.0. The more general ``check_expression_support()``
should be used instead.

* ``django.forms.extras`` is deprecated. You can find
:class:`~django.forms.SelectDateWidget` in ``django.forms.widgets``
(or simply ``django.forms``) instead.

* Private API ``django.db.models.fields.add_lazy_relation()`` is deprecated.

* The ``django.contrib.auth.tests.utils.skipIfCustomUser()`` decorator is
deprecated. With the test discovery changes in Django 1.6, the tests for
``django.contrib`` apps are no longer run as part of the user's project.
Therefore, the ``skipIfCustomUser`` decorator is no longer needed to
decorate tests in ``django.contrib.auth``.

* If you customized some :ref:`error handlers <error-views>`, the view
signatures with only one request parameter are deprecated. The views should
now also accept a second ``exception`` positional parameter.

* The ``django.utils.feedgenerator.Atom1Feed.mime_type`` and
``django.utils.feedgenerator.RssFeed.mime_type`` attributes are deprecated in
favor of ``content_type``.

* :class:`~django.core.signing.Signer` now issues a warning if an invalid
separator is used. This will become an exception in Django 1.10.

* ``django.db.models.Field._get_val_from_obj()`` is deprecated in favor of
``Field.value_from_object()``.

* ``django.template.loaders.eggs.Loader`` is deprecated as distributing
applications as eggs is not recommended.

* The ``callable_obj`` keyword argument to
``SimpleTestCase.assertRaisesMessage()`` is deprecated. Pass the callable as
a positional argument instead.

* The ``allow_tags`` attribute on methods of ``ModelAdmin`` has been
deprecated. Use :func:`~django.utils.html.format_html`,
:func:`~django.utils.html.format_html_join`, or
:func:`~django.utils.safestring.mark_safe` when constructing the method's
return value instead.

* The ``enclosure`` keyword argument to ``SyndicationFeed.add_item()`` is
deprecated. Use the new ``enclosures`` argument which accepts a list of
``Enclosure`` objects instead of a single one.

* The ``django.template.loader.LoaderOrigin`` and
``django.template.base.StringOrigin`` aliases for
``django.template.base.Origin`` are deprecated.

.. _removed-features-1.9:

Features removed in 1.9
=======================

These features have reached the end of their deprecation cycle and are removed
in Django 1.9. See :ref:`deprecated-features-1.7` for details, including how to
remove usage of these features.

* ``django.utils.dictconfig`` is removed.

* ``django.utils.importlib`` is removed.

* ``django.utils.tzinfo`` is removed.

* ``django.utils.unittest`` is removed.

* The ``syncdb`` command is removed.

* ``django.db.models.signals.pre_syncdb`` and
``django.db.models.signals.post_syncdb`` is removed.

* Support for ``allow_syncdb`` on database routers is removed.

* Automatic syncing of apps without migrations is removed. Migrations are
compulsory for all apps unless you pass the :option:`migrate --run-syncdb`
option.

* The SQL management commands for apps without migrations, ``sql``, ``sqlall``,
``sqlclear``, ``sqldropindexes``, and ``sqlindexes``, are removed.

* Support for automatic loading of ``initial_data`` fixtures and initial SQL
data is removed.

* All models need to be defined inside an installed application or declare an
explicit :attr:`~django.db.models.Options.app_label`. Furthermore, it isn't
possible to import them before their application is loaded. In particular, it
isn't possible to import models inside the root package of an application.

* The model and form ``IPAddressField`` is removed. A stub field remains for
compatibility with historical migrations.

* ``AppCommand.handle_app()`` is no longer supported.

* ``RequestSite`` and ``get_current_site()`` are no longer importable from
``django.contrib.sites.models``.

* FastCGI support via the ``runfcgi`` management command is removed.

* ``django.utils.datastructures.SortedDict`` is removed.

* ``ModelAdmin.declared_fieldsets`` is removed.

* The ``util`` modules that provided backwards compatibility are removed:

* ``django.contrib.admin.util``
* ``django.contrib.gis.db.backends.util``
* ``django.db.backends.util``
* ``django.forms.util``

* ``ModelAdmin.get_formsets`` is removed.

* The backward compatible shims introduced to rename the
``BaseMemcachedCache._get_memcache_timeout()`` method to
``get_backend_timeout()`` is removed.

* The ``--natural`` and ``-n`` options for :djadmin:`dumpdata` are removed.

* The ``use_natural_keys`` argument for ``serializers.serialize()`` is removed.

* Private API ``django.forms.forms.get_declared_fields()`` is removed.

* The ability to use a ``SplitDateTimeWidget`` with ``DateTimeField`` is
removed.

* The ``WSGIRequest.REQUEST`` property is removed.

* The class ``django.utils.datastructures.MergeDict`` is removed.

* The ``zh-cn`` and ``zh-tw`` language codes are removed.

* The internal ``django.utils.functional.memoize()`` is removed.

* ``django.core.cache.get_cache`` is removed.

* ``django.db.models.loading`` is removed.

* Passing callable arguments to querysets is no longer possible.

* ``BaseCommand.requires_model_validation`` is removed in favor of
``requires_system_checks``. Admin validators is replaced by admin checks.

* The ``ModelAdmin.validator_class`` and ``default_validator_class`` attributes
are removed.

* ``ModelAdmin.validate()`` is removed.

* ``django.db.backends.DatabaseValidation.validate_field`` is removed in
favor of the ``check_field`` method.

* The ``validate`` management command is removed.

* ``django.utils.module_loading.import_by_path`` is removed in favor of
``django.utils.module_loading.import_string``.

* ``ssi`` and ``url`` template tags are removed from the ``future`` template
tag library.

* ``django.utils.text.javascript_quote()`` is removed.

* Database test settings as independent entries in the database settings,
prefixed by ``TEST_``, are no longer supported.

* The ``cache_choices`` option to :class:`~django.forms.ModelChoiceField` and
:class:`~django.forms.ModelMultipleChoiceField` is removed.

* The default value of the
:attr:`RedirectView.permanent <django.views.generic.base.RedirectView.permanent>`
attribute has changed from ``True`` to ``False``.

* ``django.contrib.sitemaps.FlatPageSitemap`` is removed in favor of
``django.contrib.flatpages.sitemaps.FlatPageSitemap``.

* Private API ``django.test.utils.TestTemplateLoader`` is removed.

* The ``django.contrib.contenttypes.generic`` module is removed.


===========================

1.8.19

Not secure
===========================

*March 6, 2018*

Django 1.8.19 fixes two security issues in 1.18.18.

CVE-2018-7536: Denial-of-service possibility in ``urlize`` and ``urlizetrunc`` template filters
===============================================================================================

The ``django.utils.html.urlize()`` function was extremely slow to evaluate
certain inputs due to a catastrophic backtracking vulnerability in a regular
expression. The ``urlize()`` function is used to implement the ``urlize`` and
``urlizetrunc`` template filters, which were thus vulnerable.

The problematic regular expression is replaced with parsing logic that behaves
similarly.

CVE-2018-7537: Denial-of-service possibility in ``truncatechars_html`` and ``truncatewords_html`` template filters
==================================================================================================================

If ``django.utils.text.Truncator``'s ``chars()`` and ``words()`` methods were
passed the ``html=True`` argument, they were extremely slow to evaluate certain
inputs due to a catastrophic backtracking vulnerability in a regular
expression. The ``chars()`` and ``words()`` methods are used to implement the
``truncatechars_html`` and ``truncatewords_html`` template filters, which were
thus vulnerable.

The backtracking problem in the regular expression is fixed.


===========================

1.8.18

Not secure
===========================

*April 4, 2017*

Django 1.8.18 fixes two security issues in 1.8.17.

CVE-2017-7233: Open redirect and possible XSS attack via user-supplied numeric redirect URLs
============================================================================================

Django relies on user input in some cases (e.g.
``django.contrib.auth.views.login()`` and :doc:`i18n </topics/i18n/index>`)
to redirect the user to an "on success" URL. The security check for these
redirects (namely ``django.utils.http.is_safe_url()``) considered some numeric
URLs (e.g. ``http:999999999``) "safe" when they shouldn't be.

Also, if a developer relies on ``is_safe_url()`` to provide safe redirect
targets and puts such a URL into a link, they could suffer from an XSS attack.

CVE-2017-7234: Open redirect vulnerability in ``django.views.static.serve()``
=============================================================================

A maliciously crafted URL to a Django site using the
:func:`~django.views.static.serve` view could redirect to any other domain. The
view no longer does any redirects as they don't provide any known, useful
functionality.

Note, however, that this view has always carried a warning that it is not
hardened for production use and should be used only as a development aid.


===========================

1.8.17

Not secure
===========================

*December 1, 2016*

Django 1.8.17 fixes a regression in 1.8.16.

Bugfixes
========

* Quoted the Oracle test user's password in queries to fix the "ORA-00922:
missing or invalid option" error when the password starts with a number or
special character (:ticket:`27420`).


===========================

1.8.16

Not secure
===========================

*November 1, 2016*

Django 1.8.16 fixes two security issues in 1.8.15.

User with hardcoded password created when running tests on Oracle
=================================================================

When running tests with an Oracle database, Django creates a temporary database
user. In older versions, if a password isn't manually specified in the database
settings ``TEST`` dictionary, a hardcoded password is used. This could allow
an attacker with network access to the database server to connect.

This user is usually dropped after the test suite completes, but not when using
the ``manage.py test --keepdb`` option or if the user has an active session
(such as an attacker's connection).

A randomly generated password is now used for each test run.

DNS rebinding vulnerability when ``DEBUG=True``
===============================================

Older versions of Django don't validate the ``Host`` header against
``settings.ALLOWED_HOSTS`` when ``settings.DEBUG=True``. This makes them
vulnerable to a `DNS rebinding attack
<https://benmmurphy.github.io/blog/2016/07/11/rails-webconsole-dns-rebinding/>`_.

While Django doesn't ship a module that allows remote code execution, this is
at least a cross-site scripting vector, which could be quite serious if
developers load a copy of the production database in development or connect to
some production services for which there's no development instance, for
example. If a project uses a package like the ``django-debug-toolbar``, then
the attacker could execute arbitrary SQL, which could be especially bad if the
developers connect to the database with a superuser account.

``settings.ALLOWED_HOSTS`` is now validated regardless of ``DEBUG``. For
convenience, if ``ALLOWED_HOSTS`` is empty and ``DEBUG=True``, the following
variations of localhost are allowed ``['localhost', '127.0.0.1', '::1']``. If
your local settings file has your production ``ALLOWED_HOSTS`` value, you must
now omit it to get those fallback values.


===========================

1.8.15

Not secure
===========================

*September 26, 2016*

Django 1.8.15 fixes a security issue in 1.8.14.

CSRF protection bypass on a site with Google Analytics
======================================================

An interaction between Google Analytics and Django's cookie parsing could allow
an attacker to set arbitrary cookies leading to a bypass of CSRF protection.

The parser for ``request.COOKIES`` is simplified to better match the behavior
of browsers and to mitigate this attack. ``request.COOKIES`` may now contain
cookies that are invalid according to :rfc:`6265` but are possible to set via
``document.cookie``.


===========================

1.8.14

Not secure
===========================

*July 18, 2016*

Django 1.8.14 fixes a security issue and a bug in 1.8.13.

XSS in admin's add/change related popup
=======================================

Unsafe usage of JavaScript's ``Element.innerHTML`` could result in XSS in the
admin's add/change related popup. ``Element.textContent`` is now used to
prevent execution of the data.

The debug view also used ``innerHTML``. Although a security issue wasn't
identified there, out of an abundance of caution it's also updated to use
``textContent``.

Bugfixes
========

* Fixed missing ``varchar/text_pattern_ops`` index on ``CharField`` and
``TextField`` respectively when using ``AddField`` on PostgreSQL
(:ticket:`26889`).


===========================

1.8.13

Not secure
===========================

*May 2, 2016*

Django 1.8.13 fixes several bugs in 1.8.12.

Bugfixes
========

* Fixed ``TimeField`` microseconds round-tripping on MySQL and SQLite
(:ticket:`26498`).

* Restored conversion of an empty string to null when saving values of
``GenericIPAddressField`` on SQLite and MySQL (:ticket:`26557`).


===========================

1.8.12

Not secure
===========================

*April 1, 2016*

Django 1.8.12 fixes several bugs in 1.8.11.

Bugfixes
========

* Made ``MultiPartParser`` ignore filenames that normalize to an empty string
to fix crash in ``MemoryFileUploadHandler`` on specially crafted user input
(:ticket:`26325`).

* Fixed data loss on SQLite where ``DurationField`` values with fractional
seconds could be saved as ``None`` (:ticket:`26324`).

* Restored the functionality of the admin's ``raw_id_fields`` in
``list_editable`` (:ticket:`26387`).


===========================

1.8.11

Not secure
===========================

*March 5, 2016*

Django 1.8.11 fixes a regression on Python 2 in the 1.8.10 security release
where ``utils.http.is_safe_url()`` crashes on bytestring URLs (:ticket:`26308`).


===========================

1.8.10

Not secure
===========================

*March 1, 2016*

Django 1.8.10 fixes two security issues and several bugs in 1.8.9.

CVE-2016-2512: Malicious redirect and possible XSS attack via user-supplied redirect URLs containing basic auth
===============================================================================================================

Django relies on user input in some cases (e.g.
``django.contrib.auth.views.login()`` and :doc:`i18n </topics/i18n/index>`)
to redirect the user to an "on success" URL. The security check for these
redirects (namely ``django.utils.http.is_safe_url()``) considered some URLs
with basic authentication credentials "safe" when they shouldn't be.

For example, a URL like ``http://mysite.example.com\attacker.com`` would be
considered safe if the request's host is ``http://mysite.example.com``, but
redirecting to this URL sends the user to ``attacker.com``.

Also, if a developer relies on ``is_safe_url()`` to provide safe redirect
targets and puts such a URL into a link, they could suffer from an XSS attack.

CVE-2016-2513: User enumeration through timing difference on password hasher work factor upgrade
================================================================================================

In each major version of Django since 1.6, the default number of iterations for
the ``PBKDF2PasswordHasher`` and its subclasses has increased. This improves
the security of the password as the speed of hardware increases, however, it
also creates a timing difference between a login request for a user with a
password encoded in an older number of iterations and login request for a
nonexistent user (which runs the default hasher's default number of iterations
since Django 1.6).

This only affects users who haven't logged in since the iterations were
increased. The first time a user logs in after an iterations increase, their
password is updated with the new iterations and there is no longer a timing
difference.

The new ``BasePasswordHasher.harden_runtime()`` method allows hashers to bridge
the runtime gap between the work factor (e.g. iterations) supplied in existing
encoded passwords and the default work factor of the hasher. This method
is implemented for ``PBKDF2PasswordHasher`` and ``BCryptPasswordHasher``.
The number of rounds for the latter hasher hasn't changed since Django 1.4, but
some projects may subclass it and increase the work factor as needed.

A warning will be emitted for any :ref:`third-party password hashers that don't
implement <write-your-own-password-hasher>` a ``harden_runtime()`` method.

If you have different password hashes in your database (such as SHA1 hashes
from users who haven't logged in since the default hasher switched to PBKDF2
in Django 1.4), the timing difference on a login request for these users may be
even greater and this fix doesn't remedy that difference (or any difference
when changing hashers). You may be able to :ref:`upgrade those hashes
<wrapping-password-hashers>` to prevent a timing attack for that case.

Bugfixes
========

* Fixed a crash on PostgreSQL that prevented using ``TIME_ZONE=None`` and
``USE_TZ=False`` (:ticket:`26177`).

* Added system checks for query name clashes of hidden relationships
(:ticket:`26162`).

* Made ``forms.FileField`` and ``utils.translation.lazy_number()`` picklable
(:ticket:`26212`).

* Fixed :class:`~django.contrib.postgres.fields.RangeField` and
:class:`~django.contrib.postgres.fields.ArrayField` serialization with
``None`` values (:ticket:`26215`).

* Reallowed dashes in top-level domain names of URLs checked by
``URLValidator`` to fix a regression in Django 1.8 (:ticket:`26204`).

* Fixed ``BoundField`` to reallow slices of subwidgets (:ticket:`26267`).

* Prevented ``ContentTypeManager`` instances from sharing their cache
(:ticket:`26286`).


==========================

1.8.9

Not secure
==========================

*February 1, 2016*

Django 1.8.9 fixes several bugs in 1.8.8.

Bugfixes
========

* Fixed a regression that caused the "user-tools" items to display on the
admin's logout page (:ticket:`26035`).

* Fixed a crash in the translations system when the current language has no
translations (:ticket:`26046`).

* Fixed a regression that caused the incorrect day to be selected when opening
the admin calendar widget for timezones from GMT+0100 to GMT+1200
(:ticket:`24980`).

* Fixed a regression in 1.8.8 causing incorrect index handling in migrations on
PostgreSQL when adding ``db_index=True`` or ``unique=True`` to a
``CharField`` or ``TextField`` that already had the other specified, or when
removing one of them from a field that had both, or when adding
``unique=True`` to a field already listed in ``unique_together``
(:ticket:`26034`).

* Fixed a crash when using an ``__in`` lookup inside a ``Case`` expression
(:ticket:`26071`).

* Fixed a crash when using a reverse ``OneToOneField`` in
``ModelAdmin.readonly_fields`` (:ticket:`26060`).

* Fixed a regression in Django 1.8.5 that broke copying a ``SimpleLazyObject``
with ``copy.copy()`` (:ticket:`26122`).

* Fixed the ``contrib.gis`` map widgets when using
``USE_THOUSAND_SEPARATOR=True`` (:ticket:`20415`).


==========================

1.8.8

Not secure
==========================

*January 2, 2016*

Django 1.8.8 fixes several bugs in 1.8.7.

Python 3.2 users, please be advised that we've decided to drop support for
Python 3.2 in Django 1.8.x at the end of 2016. We won't break things
intentionally after that, but we won't test subsequent releases against Python
3.2 either. Upstream support for Python 3.2 ends February 2016 so we don't find
much value in providing security updates for a version of Python that could be
insecure. To read more about the decision and to let us know if this will be
problematic for you, please read the `django-developers thread
<https://groups.google.com/g/django-developers/c/eMu5UQpUdWs/discussion>`_.

Bugfixes
========

* Fixed incorrect ``unique_together`` field name generation by ``inspectdb``
(:ticket:`25274`).

* Corrected ``__len`` query lookup on ``ArrayField`` for empty arrays
(:ticket:`25772`).

* Restored the ability to use custom formats from ``formats.py`` with
``django.utils.formats.get_format()`` and the ``date`` template filter
(:ticket:`25812`).

* Fixed a state bug when migrating a ``SeparateDatabaseAndState`` operation
backwards (:ticket:`25896`).

* Fixed missing ``varchar/text_pattern_ops`` index on ``CharField`` and
``TextField`` respectively when using ``AlterField`` on PostgreSQL
(:ticket:`25412`).

* Fixed a state bug when using an ``AlterModelManagers`` operation
(:ticket:`25852`).

* Fixed a regression which prevented using a language not in Django's default
language list (:setting:`LANGUAGES`) (:ticket:`25915`).

* ``django.views.decorators.cache.never_cache()`` now sends more persuasive
headers (added ``no-cache, no-store, must-revalidate`` to ``Cache-Control``)
to better prevent caching (:ticket:`13008`). This fixes a problem where a
page refresh in Firefox cleared the selected entries in the admin's
``filter_horizontal`` and ``filter_vertical`` widgets, which could result
in inadvertent data loss if a user didn't notice that and then submitted the
form (:ticket:`22955`).

* Fixed a regression in the admin which ignored line breaks in read-only fields
instead of converting them to ``<br>`` (:ticket:`25465`).

* Made ``loaddata`` skip disabling and enabling database constraints when it
doesn't load any fixtures (:ticket:`23372`).

* Fixed a crash in ``QuerySet.values()/values_list()`` after an ``annotate()``
and ``order_by()`` when ``values()/values_list()`` includes a field not in
the ``order_by()`` (:ticket:`25316`).


==========================

1.8.7

Not secure
==========================

*November 24, 2015*

Django 1.8.7 fixes a security issue and several bugs in 1.8.6.

Additionally, Django's vendored version of six, ``django.utils.six``, has
been upgraded to the latest release (1.10.0).

Fixed settings leak possibility in ``date`` template filter
===========================================================

If an application allows users to specify an unvalidated format for dates and
passes this format to the :tfilter:`date` filter, e.g.
``{{ last_updated|date:user_date_format }}``, then a malicious user could
obtain any secret in the application's settings by specifying a settings key
instead of a date format. e.g. ``"SECRET_KEY"`` instead of ``"j/m/Y"``.

To remedy this, the underlying function used by the ``date`` template filter,
``django.utils.formats.get_format()``, now only allows accessing the date/time
formatting settings.

Bugfixes
========

* Fixed a crash of the debug view during the autumn DST change when
:setting:`USE_TZ` is ``False`` and ``pytz`` is installed.

* Fixed a regression in 1.8.6 that caused database routers without an
``allow_migrate()`` method to crash (:ticket:`25686`).

* Fixed a regression in 1.8.6 by restoring the ability to use ``Manager``
objects for the ``queryset`` argument of ``ModelChoiceField``
(:ticket:`25683`).

* Fixed a regression in 1.8.6 that caused an application with South migrations
in the ``migrations`` directory to fail (:ticket:`25618`).

* Fixed a data loss possibility with :class:`~django.db.models.Prefetch` if
``to_attr`` is set to a ``ManyToManyField`` (:ticket:`25693`).

* Fixed a regression in 1.8 by making ``gettext()`` once again return UTF-8
bytestrings on Python 2 if the input is a bytestring (:ticket:`25720`).

* Fixed serialization of
:class:`~django.contrib.postgres.fields.DateRangeField` and
:class:`~django.contrib.postgres.fields.DateTimeRangeField` (:ticket:`24937`).

* Fixed the exact lookup of ``ArrayField`` (:ticket:`25666`).

* Fixed ``Model.refresh_from_db()`` updating of ``ForeignKey`` fields with
``on_delete=models.SET_NULL`` (:ticket:`25715`).

* Fixed a duplicate query regression in 1.8 on proxied model deletion
(:ticket:`25685`).

* Fixed ``set_FOO_order()`` crash when the ``ForeignKey`` of a model with
``order_with_respect_to`` references a model with a ``OneToOneField``
primary key (:ticket:`25786`).

* Fixed incorrect validation for ``PositiveIntegerField`` and
``PositiveSmallIntegerField`` on MySQL resulting in values greater than
4294967295 or 65535, respectively, passing validation and being silently
truncated by the database (:ticket:`25767`).


==========================

1.8.6

Not secure
==========================

*November 4, 2015*

Django 1.8.6 adds official support for Python 3.5 and fixes several bugs in
1.8.5.

Bugfixes
========

* Fixed a regression causing ``ModelChoiceField`` to ignore
``prefetch_related()`` on its queryset (:ticket:`25496`).

* Allowed "mode=memory" in SQLite test database name if supported
(:ticket:`12118`).

* Fixed system check crash on ``ForeignKey`` to abstract model
(:ticket:`25503`).

* Fixed incorrect queries when you have multiple ``ManyToManyField``\s on
different models that have the same field name, point to the same model, and
have their reverse relations disabled (:ticket:`25545`).

* Allowed filtering over a ``RawSQL`` annotation (:ticket:`25506`).

* Made the ``Concat`` database function idempotent on SQLite (:ticket:`25517`).

* Avoided a confusing stack trace when starting :djadmin:`runserver` with an
invalid :setting:`INSTALLED_APPS` setting (:ticket:`25510`). This regression
appeared in 1.8.5 as a side effect of fixing :ticket:`24704`.

* Made deferred models use their proxied model's ``_meta.apps`` for caching
and retrieval (:ticket:`25563`). This prevents any models generated in data
migrations using ``QuerySet.defer()`` from leaking to test and application
code.

* Fixed a typo in the name of the ``strictly_above`` PostGIS lookup
(:ticket:`25592`).

* Fixed crash with ``contrib.postgres.forms.SplitArrayField`` and
``IntegerField`` on invalid value (:ticket:`25597`).

* Added a helpful error message when Django and South migrations exist in the
same directory (:ticket:`25618`).

* Fixed a regression in ``URLValidator`` that allowed URLs with consecutive
dots in the domain section (like ``http://example..com/``) to pass
(:ticket:`25620`).

* Fixed a crash with ``GenericRelation`` and
``BaseModelAdmin.to_field_allowed`` (:ticket:`25622`).


==========================

1.8.5

Not secure
==========================

*October 3, 2015*

Django 1.8.5 fixes several bugs in 1.8.4.

Bugfixes
========

* Made the development server's autoreload more robust (:ticket:`24704`).

* Fixed ``AssertionError`` in some delete queries with a model containing a
field that is both a foreign and primary key (:ticket:`24951`).

* Fixed ``AssertionError`` in some complex queries (:ticket:`24525`).

* Fixed a migrations crash with ``GenericForeignKey`` (:ticket:`25040`).

* Made ``translation.override()`` clear the overridden language when a
translation isn't initially active (:ticket:`25295`).

* Fixed crash when using a value in ``ModelAdmin.list_display`` that clashed
with a reverse field on the model (:ticket:`25299`).

* Fixed autocompletion for options of non-``argparse`` management commands
(:ticket:`25372`).

* Alphabetized ordering of imports in ``from django.db import migrations,
models`` statement in newly created migrations (:ticket:`25384`).

* Fixed migrations crash on MySQL when adding a text or a blob field with an
unhashable default (:ticket:`25393`).

* Changed ``Count`` queries to execute ``COUNT(*)`` instead of ``COUNT('*')``
as versions of Django before 1.8 did (:ticket:`25377`). This may fix a
performance regression on some databases.

* Fixed custom queryset chaining with ``values()`` and ``values_list()``
(:ticket:`20625`).

* Moved the :ref:`unsaved model instance assignment data loss check
<unsaved-model-instance-check-18>` on reverse relations to ``Model.save()``
(:ticket:`25160`).

* Readded inline foreign keys to form instances when validating model formsets
(:ticket:`25431`).

* Allowed using ORM write methods after disabling autocommit with
:func:`set_autocommit(False) <django.db.transaction.set_autocommit>`
(:ticket:`24921`).

* Fixed the ``manage.py test --keepdb`` option on Oracle (:ticket:`25421`).

* Fixed incorrect queries with multiple many-to-many fields on a model with the
same 'to' model and with ``related_name`` set to '+' (:ticket:`24505`,
:ticket:`25486`).

* Fixed pickling a ``SimpleLazyObject`` wrapping a model (:ticket:`25389`).


==========================

1.8.4

Not secure
==========================

*August 18, 2015*

Django 1.8.4 fixes a security issue and several bugs in 1.8.3.

Denial-of-service possibility in ``logout()`` view by filling session store
===========================================================================

Previously, a session could be created when anonymously accessing the
``django.contrib.auth.views.logout()`` view (provided it wasn't decorated
with :func:`~django.contrib.auth.decorators.login_required` as done in the
admin). This could allow an attacker to easily create many new session records
by sending repeated requests, potentially filling up the session store or
causing other users' session records to be evicted.

The :class:`~django.contrib.sessions.middleware.SessionMiddleware` has been
modified to no longer create empty session records, including when
:setting:`SESSION_SAVE_EVERY_REQUEST` is active.

Bugfixes
========

* Added the ability to serialize values from the newly added
:class:`~django.db.models.UUIDField` (:ticket:`25019`).

* Added a system check warning if the old ``TEMPLATE_*`` settings are defined
in addition to the new ``TEMPLATES`` setting.

* Fixed ``QuerySet.raw()`` so ``InvalidQuery`` is not raised when using the
``db_column`` name of a ``ForeignKey`` field with ``primary_key=True``
(:ticket:`12768`).

* Prevented an exception in ``TestCase.setUpTestData()`` from leaking the
transaction (:ticket:`25176`).

* Fixed ``has_changed()`` method in ``contrib.postgres.forms.HStoreField``
(:ticket:`25215`, :ticket:`25233`).

* Fixed the recording of squashed migrations when running the ``migrate``
command (:ticket:`25231`).

* Moved the :ref:`unsaved model instance assignment data loss check
<unsaved-model-instance-check-18>` to ``Model.save()`` to allow easier usage
of in-memory models (:ticket:`25160`).

* Prevented ``varchar_patterns_ops`` and ``text_patterns_ops`` indexes for
``ArrayField`` (:ticket:`25180`).


==========================

1.8.3

Not secure
==========================

*July 8, 2015*

Django 1.8.3 fixes several security issues and bugs in 1.8.2.

Also, ``django.utils.deprecation.RemovedInDjango20Warning`` was renamed to
``RemovedInDjango110Warning`` as the version roadmap was revised to 1.9, 1.10,
1.11 (LTS), 2.0 (drops Python 2 support). For backwards compatibility,
``RemovedInDjango20Warning`` remains as an importable alias.

Denial-of-service possibility by filling session store
======================================================

In previous versions of Django, the session backends created a new empty record
in the session storage anytime ``request.session`` was accessed and there was a
session key provided in the request cookies that didn't already have a session
record. This could allow an attacker to easily create many new session records
simply by sending repeated requests with unknown session keys, potentially
filling up the session store or causing other users' session records to be
evicted.

The built-in session backends now create a session record only if the session
is actually modified; empty session records are not created. Thus this
potential DoS is now only possible if the site chooses to expose a
session-modifying view to anonymous users.

As each built-in session backend was fixed separately (rather than a fix in the
core sessions framework), maintainers of third-party session backends should
check whether the same vulnerability is present in their backend and correct
it if so.

Header injection possibility since validators accept newlines in input
======================================================================

Some of Django's built-in validators
(:class:`~django.core.validators.EmailValidator`, most seriously) didn't
prohibit newline characters (due to the usage of ``$`` instead of ``\Z`` in the
regular expressions). If you use values with newlines in HTTP response or email
headers, you can suffer from header injection attacks. Django itself isn't
vulnerable because :class:`~django.http.HttpResponse` and the mail sending
utilities in :mod:`django.core.mail` prohibit newlines in HTTP and SMTP
headers, respectively. While the validators have been fixed in Django, if
you're creating HTTP responses or email messages in other ways, it's a good
idea to ensure that those methods prohibit newlines as well. You might also
want to validate that any existing data in your application doesn't contain
unexpected newlines.

:func:`~django.core.validators.validate_ipv4_address`,
:func:`~django.core.validators.validate_slug`, and
:class:`~django.core.validators.URLValidator` are also affected, however, as
of Django 1.6 the ``GenericIPAddresseField``, ``IPAddressField``, ``SlugField``,
and ``URLField`` form fields which use these validators all strip the input, so
the possibility of newlines entering your data only exists if you are using
these validators outside of the form fields.

The undocumented, internally unused ``validate_integer()`` function is now
stricter as it validates using a regular expression instead of simply casting
the value using ``int()`` and checking if an exception was raised.

Denial-of-service possibility in URL validation
===============================================

:class:`~django.core.validators.URLValidator` included a regular expression
that was extremely slow to evaluate against certain invalid inputs. This regular
expression has been simplified and optimized.

Bugfixes
========

* Fixed ``BaseRangeField.prepare_value()`` to use each ``base_field``’s
``prepare_value()`` method (:ticket:`24841`).

* Fixed crash during :djadmin:`makemigrations` if a migrations module either
is missing ``__init__.py`` or is a file (:ticket:`24848`).

* Fixed ``QuerySet.exists()`` returning incorrect results after annotation with
``Count()`` (:ticket:`24835`).

* Corrected ``HStoreField.has_changed()`` (:ticket:`24844`).

* Reverted an optimization to the CSRF template context processor which caused
a regression (:ticket:`24836`).

* Fixed a regression which caused template context processors to overwrite
variables set on a ``RequestContext`` after it's created (:ticket:`24847`).

* Prevented the loss of ``null``/``not null`` column properties during field
renaming of MySQL databases (:ticket:`24817`).

* Fixed a crash when using a reverse one-to-one relation in
``ModelAdmin.list_display`` (:ticket:`24851`).

* Fixed quoting of SQL when renaming a field to ``AutoField`` in PostgreSQL
(:ticket:`24892`).

* Fixed lack of unique constraint when changing a field from
``primary_key=True`` to ``unique=True`` (:ticket:`24893`).

* Fixed queryset pickling when using ``prefetch_related()`` after deleting
objects (:ticket:`24831`).

* Allowed using ``choices`` longer than 1 day with ``DurationField``
(:ticket:`24897`).

* Fixed a crash when loading squashed migrations from two apps with a
dependency between them, where the dependent app's replaced migrations are
partially applied (:ticket:`24895`).

* Fixed recording of applied status for squashed (replacement) migrations
(:ticket:`24628`).

* Fixed queryset annotations when using ``Case`` expressions with ``exclude()``
(:ticket:`24833`).

* Corrected join promotion for multiple ``Case`` expressions. Annotating a
query with multiple ``Case`` expressions could unexpectedly filter out
results (:ticket:`24924`).

* Fixed usage of transforms in subqueries (:ticket:`24744`).

* Fixed ``SimpleTestCase.assertRaisesMessage()`` on Python 2.7.10
(:ticket:`24903`).

* Provided better backwards compatibility for the ``verbosity`` argument in
``optparse`` management commands by casting it to an integer
(:ticket:`24769`).

* Fixed ``prefetch_related()`` on databases other than PostgreSQL for models
using UUID primary keys (:ticket:`24912`).

* Fixed removing ``unique_together`` constraints on MySQL (:ticket:`24972`).

* Fixed crash when uploading images with MIME types that Pillow doesn't detect,
such as bitmap, in ``forms.ImageField`` (:ticket:`24948`).

* Fixed a regression when deleting a model through the admin that has a
``GenericRelation`` with a ``related_query_name`` (:ticket:`24940`).

* Reallowed non-ASCII values for ``ForeignKey.related_name`` on Python 3 by
fixing the false positive system check (:ticket:`25016`).

* Fixed inline forms that use a parent object that has a ``UUIDField`` primary
key and a child object that has an ``AutoField`` primary key
(:ticket:`24958`).

* Fixed a regression in the ``unordered_list`` template filter on certain
inputs (:ticket:`25031`).

* Fixed a regression in ``URLValidator`` that invalidated Punycode TLDs
(:ticket:`25059`).

* Improved ``pyinotify`` ``runserver`` polling (:ticket:`23882`).


==========================

1.8.2

Not secure
==========================

*May 20, 2015*

Django 1.8.2 fixes a security issue and several bugs in 1.8.1.

Fixed session flushing in the ``cached_db`` backend
===================================================

A change to ``session.flush()`` in the ``cached_db`` session backend in Django
1.8 mistakenly sets the session key to an empty string rather than ``None``. An
empty string is treated as a valid session key and the session cookie is set
accordingly. Any users with an empty string in their session cookie will use
the same session store. ``session.flush()`` is called by
``django.contrib.auth.logout()`` and, more seriously, by
``django.contrib.auth.login()`` when a user switches accounts. If a user is
logged in and logs in again to a different account (without logging out) the
session is flushed to avoid reuse. After the session is flushed (and its
session key becomes ``''``) the account details are set on the session and the
session is saved. Any users with an empty string in their session cookie will
now be logged into that account.

Bugfixes
========

* Fixed check for template engine alias uniqueness (:ticket:`24685`).

* Fixed crash when reusing the same ``Case`` instance in a query
(:ticket:`24752`).

* Corrected join promotion for ``Case`` expressions. For example, annotating a
query with a ``Case`` expression could unexpectedly filter out results
(:ticket:`24766`).

* Fixed negated ``Q`` objects in expressions. Cases like
``Case(When(~Q(friends__age__lte=30)))`` tried to generate a subquery which
resulted in a crash (:ticket:`24705`).

* Fixed incorrect GROUP BY clause generation on MySQL when the query's model
has a self-referential foreign key (:ticket:`24748`).

* Implemented ``ForeignKey.get_db_prep_value()`` so that ``ForeignKey``\s
pointing to :class:`~django.db.models.UUIDField` and inheritance on models
with ``UUIDField`` primary keys work correctly (:ticket:`24698`,
:ticket:`24712`).

* Fixed ``isnull`` lookup for ``HStoreField`` (:ticket:`24751`).

* Fixed a MySQL crash when a migration removes a combined index (unique_together
or index_together) containing a foreign key (:ticket:`24757`).

* Fixed session cookie deletion when using :setting:`SESSION_COOKIE_DOMAIN`
(:ticket:`24799`).

* On PostgreSQL, when no access is granted for the ``postgres`` database,
Django now falls back to the default database when it normally requires a
"no database" connection (:ticket:`24791`).

* Fixed display of ``contrib.admin``’s ``ForeignKey`` widget when it's used
in a row with other fields (:ticket:`24784`).


==========================

1.8.1

Not secure
==========================

*May 1, 2015*

Django 1.8.1 fixes several bugs in 1.8 and includes some optimizations in the
migrations framework.

Bugfixes
========

* Added support for serializing :class:`~datetime.timedelta` objects in
migrations (:ticket:`24566`).

* Restored proper parsing of the :djadmin:`testserver` command's positional
arguments (fixture names) (:ticket:`24571`).

* Prevented ``TypeError`` in translation functions ``check_for_language()`` and
``get_language_bidi()`` when translations are deactivated (:ticket:`24569`).

* Fixed :djadmin:`squashmigrations` command when using
:class:`~django.db.migrations.operations.SeparateDatabaseAndState`
(:ticket:`24278`).

* Stripped microseconds from ``datetime`` values when using an older version of
the MySQLdb DB API driver as it does not support fractional seconds
(:ticket:`24584`).

* Fixed a migration crash when altering
:class:`~django.db.models.ManyToManyField`\s (:ticket:`24513`).

* Fixed a crash with ``QuerySet.update()`` on foreign keys to one-to-one fields
(:ticket:`24578`).

* Fixed a regression in the model detail view of
:mod:`~django.contrib.admindocs` when a model has a reverse foreign key
relation (:ticket:`24624`).

* Prevented arbitrary file inclusions in :mod:`~django.contrib.admindocs`
(:ticket:`24625`).

* Fixed a crash with ``QuerySet.update()`` on foreign keys to instances with
``uuid`` primary keys (:ticket:`24611`).

* Fixed database introspection with SQLite 3.8.9 (released April 8, 2015)
(:ticket:`24637`).

* Updated ``urlpatterns`` examples generated by :djadmin:`startproject` to
remove usage of referencing views by dotted path in
``django.conf.urls.url()`` which is deprecated in Django 1.8
(:ticket:`24635`).

* Fixed queries where an expression was referenced in ``order_by()``, but wasn't
part of the select clause. An example query is
``qs.annotate(foo=F('field')).values('pk').order_by('foo'))`` (:ticket:`24615`).

* Fixed a database table name quoting regression (:ticket:`24605`).

* Prevented the loss of ``null``/``not null`` column properties during field
alteration of MySQL databases (:ticket:`24595`).

* Fixed JavaScript path of ``contrib.admin``’s related field widget when using
alternate static file storages (:ticket:`24655`).

* Fixed a migration crash when adding new relations to models
(:ticket:`24573`).

* Fixed a migration crash when applying migrations with model managers on
Python 3 that were generated on Python 2 (:ticket:`24701`).

* Restored the ability to use iterators as queryset filter arguments
(:ticket:`24719`).

* Fixed a migration crash when renaming the target model of a many-to-many
relation (:ticket:`24725`).

* Removed flushing of the test database with :option:`test --keepdb`, which
prevented apps with data migrations from using the option (:ticket:`24729`).

* Fixed ``makemessages`` crash in some locales (:ticket:`23271`).

* Fixed help text positioning of ``contrib.admin`` fields that use the
``ModelAdmin.filter_horizontal`` and ``filter_vertical`` options
(:ticket:`24676`).

* Fixed ``AttributeError: function 'GDALAllRegister' not found`` error when
initializing ``contrib.gis`` on Windows.

Optimizations
=============

* Changed ``ModelState`` to deepcopy fields instead of deconstructing and
reconstructing (:ticket:`24591`). This speeds up the rendering of model
states and reduces memory usage when running :djadmin:`manage.py migrate
<migrate>` (although other changes in this release may negate any performance
benefits).


========================

1.8

Not secure
========================

*April 1, 2015*

Welcome to Django 1.8!

These release notes cover the :ref:`new features <whats-new-1.8>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-1.8>` you'll
want to be aware of when upgrading from Django 1.7 or older versions. We've
also :ref:`begun the deprecation process for some features
<deprecated-features-1.8>`, and some features have reached the end of their
deprecation process and :ref:`have been removed <removed-features-1.8>`.

See the :doc:`/howto/upgrade-version` guide if you're updating an existing
project.

Django 1.8 has been designated as Django's second :term:`long-term support
release <Long-term support release>`. It will receive security updates for at
least three years after its release. Support for the previous LTS, Django 1.4,
will end 6 months from the release date of Django 1.8.

Python compatibility
====================

Django 1.8 requires Python 2.7, 3.2, 3.3, 3.4, or 3.5. We **highly recommend**
and only officially support the latest release of each series.

Django 1.8 is the first release to support Python 3.5.

Due to the end of upstream support for Python 3.2 in February 2016, we won't
test Django 1.8.x on Python 3.2 after the end of 2016.

.. _whats-new-1.8:

What's new in Django 1.8
========================

``Model._meta`` API
-------------------

Django now has a formalized API for :doc:`Model._meta </ref/models/meta>`,
providing an officially supported way to :ref:`retrieve fields
<model-meta-field-api>` and filter fields based on their :ref:`attributes
<model-field-attributes>`.

The ``Model._meta`` object has been part of Django since the days of pre-0.96
"Magic Removal" -- it just wasn't an official, stable API. In recognition of
this, we've endeavored to maintain backwards-compatibility with the old
API endpoint where possible. However, API endpoints that aren't part of the
new official API have been deprecated and will eventually be removed.

Multiple template engines
-------------------------

Django 1.8 defines a stable API for integrating template backends. It includes
built-in support for the Django template language and for
:class:`~django.template.backends.jinja2.Jinja2`. It supports rendering
templates with multiple engines within the same project. Learn more about the
new features in the :doc:`topic guide </topics/templates>` and check the
upgrade instructions in older versions of the documentation.

Security enhancements
---------------------

Several features of the django-secure_ third-party library have been
integrated into Django. :class:`django.middleware.security.SecurityMiddleware`
provides several security enhancements to the request/response cycle. The new
:option:`check --deploy` option allows you to check your production settings
file for ways to increase the security of your site.

.. _django-secure: https://pypi.org/project/django-secure/

New PostgreSQL specific functionality
-------------------------------------

Django now has a module with extensions for PostgreSQL specific features, such
as :class:`~django.contrib.postgres.fields.ArrayField`,
:class:`~django.contrib.postgres.fields.HStoreField`, :ref:`range-fields`, and
:lookup:`unaccent` lookup. A full breakdown of the features is available
:doc:`in the documentation </ref/contrib/postgres/index>`.

New data types
--------------

* Django now has a :class:`~django.db.models.UUIDField` for storing
universally unique identifiers. It is stored as the native ``uuid`` data type
on PostgreSQL and as a fixed length character field on other backends. There
is a corresponding :class:`form field <django.forms.UUIDField>`.

* Django now has a :class:`~django.db.models.DurationField` for storing periods
of time - modeled in Python by :class:`~python:datetime.timedelta`. It is
stored in the native ``interval`` data type on PostgreSQL, as a ``INTERVAL
DAY(9) TO SECOND(6)`` on Oracle, and as a ``bigint`` of microseconds on other
backends. Date and time related arithmetic has also been improved on all
backends. There is a corresponding :class:`form field
<django.forms.DurationField>`.

Query Expressions, Conditional Expressions, and Database Functions
------------------------------------------------------------------

:doc:`Query Expressions </ref/models/expressions>` allow you to create,
customize, and compose complex SQL expressions. This has enabled annotate
to accept expressions other than aggregates. Aggregates are now able to
reference multiple fields, as well as perform arithmetic, similar to ``F()``
objects. :meth:`~django.db.models.query.QuerySet.order_by` has also gained the
ability to accept expressions.

:doc:`Conditional Expressions </ref/models/conditional-expressions>` allow
you to use :keyword:`if` ... :keyword:`elif` ... :keyword:`else` logic within
queries.

A collection of :doc:`database functions </ref/models/database-functions>` is
also included with functionality such as
:class:`~django.db.models.functions.Coalesce`,
:class:`~django.db.models.functions.Concat`, and
:class:`~django.db.models.functions.Substr`.

``TestCase`` data setup
-----------------------

:class:`~django.test.TestCase` has been refactored to allow for data
initialization at the class level using transactions and savepoints. Database
backends which do not support transactions, like MySQL with the MyISAM storage
engine, will still be able to run these tests but won't benefit from the
improvements. Tests are now run within two nested
:func:`~django.db.transaction.atomic()` blocks: one for the whole class and one
for each test.

* The class method
:meth:`TestCase.setUpTestData() <django.test.TestCase.setUpTestData>` adds
the ability to set up test data at the class level. Using this technique can
speed up the tests as compared to using ``setUp()``.

* Fixture loading within ``TestCase`` is now performed once for the whole
``TestCase``.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* :class:`~django.contrib.admin.ModelAdmin` now has a
:meth:`~django.contrib.admin.ModelAdmin.has_module_permission`
method to allow limiting access to the module on the admin index page.

* :class:`~django.contrib.admin.InlineModelAdmin` now has an attribute
:attr:`~django.contrib.admin.InlineModelAdmin.show_change_link` that
supports showing a link to an inline object's change form.

* Use the new ``django.contrib.admin.RelatedOnlyFieldListFilter`` in
:attr:`ModelAdmin.list_filter <django.contrib.admin.ModelAdmin.list_filter>`
to limit the ``list_filter`` choices to foreign objects which are attached to
those from the ``ModelAdmin``.

* The :meth:`ModelAdmin.delete_view()
<django.contrib.admin.ModelAdmin.delete_view>` displays a summary of objects
to be deleted on the deletion confirmation page.

* The jQuery library embedded in the admin has been upgraded to version 1.11.2.

* You can now specify :attr:`AdminSite.site_url
<django.contrib.admin.AdminSite.site_url>` in order to display a link to the
front-end site.

* You can now specify :attr:`ModelAdmin.show_full_result_count
<django.contrib.admin.ModelAdmin.show_full_result_count>` to control whether
or not the full count of objects should be displayed on a filtered admin page.

* The ``AdminSite.password_change()`` method now has an ``extra_context``
parameter.

* You can now control who may login to the admin site by overriding only
:meth:`AdminSite.has_permission()
<django.contrib.admin.AdminSite.has_permission>` and
:attr:`AdminSite.login_form <django.contrib.admin.AdminSite.login_form>`.
The ``base.html`` template has a new block ``usertools`` which contains the
user-specific header. A new context variable ``has_permission``, which gets
its value from :meth:`~django.contrib.admin.AdminSite.has_permission`,
indicates whether the user may access the site.

* Foreign key dropdowns now have buttons for changing or deleting related
objects using a popup.

:mod:`django.contrib.admindocs`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* reStructuredText is now parsed in model docstrings.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* Authorization backends can now raise
:class:`~django.core.exceptions.PermissionDenied` in
:meth:`~django.contrib.auth.models.User.has_perm`
and :meth:`~django.contrib.auth.models.User.has_module_perms`
to short-circuit permission checking.
* :class:`~django.contrib.auth.forms.PasswordResetForm` now
has a method :meth:`~django.contrib.auth.forms.PasswordResetForm.send_mail`
that can be overridden to customize the mail to be sent.

* The ``max_length`` of :attr:`Permission.name
<django.contrib.auth.models.Permission.name>` has been increased from 50 to
255 characters. Please run the database migration.

* :attr:`~django.contrib.auth.models.CustomUser.USERNAME_FIELD` and
:attr:`~django.contrib.auth.models.CustomUser.REQUIRED_FIELDS` now supports
:class:`~django.db.models.ForeignKey`\s.

* The default iteration count for the PBKDF2 password hasher has been
increased by 33%. This backwards compatible change will not affect users who
have subclassed ``django.contrib.auth.hashers.PBKDF2PasswordHasher`` to
change the default value.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* A new :doc:`GeoJSON serializer </ref/contrib/gis/serializers>` is now
available.

* It is now allowed to include a subquery as a geographic lookup argument, for
example ``City.objects.filter(point__within=Country.objects.filter(continent='Africa').values('mpoly'))``.

* The SpatiaLite backend now supports ``Collect`` and ``Extent`` aggregates
when the database version is 3.0 or later.

* The PostGIS 2 ``CREATE EXTENSION postgis`` and the SpatiaLite
``SELECT InitSpatialMetaData`` initialization commands are now automatically
run by :djadmin:`migrate`.

* The GDAL interface now supports retrieving properties of
:ref:`raster (image) data file <raster-data-source-objects>`.

* Compatibility shims for ``SpatialRefSys`` and ``GeometryColumns`` changed in
Django 1.2 have been removed.

* All GDAL-related exceptions are now raised with ``GDALException``. The former
``OGRException`` has been kept for backwards compatibility but should not be
used any longer.

:mod:`django.contrib.sessions`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Session cookie is now deleted after
:meth:`~django.contrib.sessions.backends.base.SessionBase.flush()` is called.

:mod:`django.contrib.sitemaps`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :attr:`Sitemap.i18n <django.contrib.sitemaps.Sitemap.i18n>` attribute
allows you to generate a sitemap based on the :setting:`LANGUAGES` setting.

:mod:`django.contrib.sites`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* :func:`~django.contrib.sites.shortcuts.get_current_site` will now lookup
the current site based on :meth:`request.get_host()
<django.http.HttpRequest.get_host>` if the :setting:`SITE_ID` setting is not
defined.

* The default :class:`~django.contrib.sites.models.Site` created when running
``migrate`` now respects the :setting:`SITE_ID` setting (instead of always
using ``pk=1``).

Cache
~~~~~

* The ``incr()`` method of the
``django.core.cache.backends.locmem.LocMemCache`` backend is now thread-safe.

Cryptography
~~~~~~~~~~~~

* The ``max_age`` parameter of the
:meth:`django.core.signing.TimestampSigner.unsign` method now also accepts a
:py:class:`datetime.timedelta` object.

Database backends
~~~~~~~~~~~~~~~~~

* The MySQL backend no longer strips microseconds from ``datetime`` values as
MySQL 5.6.4 and up supports fractional seconds depending on the declaration
of the datetime field (when ``DATETIME`` includes fractional precision greater
than 0). New datetime database columns created with Django 1.8 and MySQL 5.6.4
and up will support microseconds. See the :ref:`MySQL database notes
<mysql-fractional-seconds>` for more details.

* The MySQL backend no longer creates explicit indexes for foreign keys when
using the InnoDB storage engine, as MySQL already creates them automatically.

* The Oracle backend no longer defines the ``connection_persists_old_columns``
feature as ``True``. Instead, Oracle will now include a cache busting clause
when getting the description of a table.

Email
~~~~~

* :ref:`Email backends <topic-email-backends>` now support the context manager
protocol for opening and closing connections.

* The SMTP email backend now supports ``keyfile`` and ``certfile``
authentication with the :setting:`EMAIL_SSL_CERTFILE` and
:setting:`EMAIL_SSL_KEYFILE` settings.

* The SMTP :class:`~django.core.mail.backends.smtp.EmailBackend` now supports
setting the ``timeout`` parameter with the :setting:`EMAIL_TIMEOUT` setting.

* :class:`~django.core.mail.EmailMessage` and ``EmailMultiAlternatives`` now
support the ``reply_to`` parameter.

File Storage
~~~~~~~~~~~~

* :meth:`Storage.get_available_name()
<django.core.files.storage.Storage.get_available_name>` and
:meth:`Storage.save() <django.core.files.storage.Storage.save>`
now take a ``max_length`` argument to implement storage-level maximum
filename length constraints. Filenames exceeding this argument will get
truncated. This prevents a database error when appending a unique suffix to a
long filename that already exists on the storage. See the :ref:`deprecation
note <storage-max-length-update>` about adding this argument to your custom
storage classes.

Forms
~~~~~

* Form widgets now render attributes with a value of ``True`` or ``False``
as HTML5 boolean attributes.

* The new :meth:`~django.forms.Form.has_error()` method allows checking
if a specific error has happened.

* If :attr:`~django.forms.Form.required_css_class` is defined on a form, then
the ``<label>`` tags for required fields will have this class present in its
attributes.

* The rendering of non-field errors in unordered lists (``<ul>``) now includes
``nonfield`` in its list of classes to distinguish them from field-specific
errors.

* :class:`~django.forms.Field` now accepts a
:attr:`~django.forms.Field.label_suffix` argument, which will override the
form's :attr:`~django.forms.Form.label_suffix`. This enables customizing the
suffix on a per-field basis — previously it wasn't possible to override
a form's :attr:`~django.forms.Form.label_suffix` while using shortcuts such
as ``{{ form.as_p }}`` in templates.

* :class:`~django.forms.SelectDateWidget` now accepts an
:attr:`~django.forms.SelectDateWidget.empty_label` argument, which will
override the top list choice label when :class:`~django.forms.DateField`
is not required.

* After an :class:`~django.forms.ImageField` has been cleaned and validated, the
``UploadedFile`` object will have an additional ``image`` attribute containing
the Pillow ``Image`` instance used to check if the file was a valid image. It
will also update ``UploadedFile.content_type`` with the image's content type
as determined by Pillow.

* You can now pass a callable that returns an iterable of choices when
instantiating a :class:`~django.forms.ChoiceField`.

Generic Views
~~~~~~~~~~~~~

* Generic views that use :class:`~django.views.generic.list.MultipleObjectMixin`
may now specify the ordering applied to the
:attr:`~django.views.generic.list.MultipleObjectMixin.queryset` by setting
:attr:`~django.views.generic.list.MultipleObjectMixin.ordering` or overriding
:meth:`~django.views.generic.list.MultipleObjectMixin.get_ordering()`.

* The new :attr:`SingleObjectMixin.query_pk_and_slug
<django.views.generic.detail.SingleObjectMixin.query_pk_and_slug>`
attribute allows changing the behavior of
:meth:`~django.views.generic.detail.SingleObjectMixin.get_object()`
so that it'll perform its lookup using both the primary key and the slug.

* The :meth:`~django.views.generic.edit.FormMixin.get_form()` method doesn't
require a ``form_class`` to be provided anymore. If not provided ``form_class``
defaults to :meth:`~django.views.generic.edit.FormMixin.get_form_class()`.

* Placeholders in :attr:`ModelFormMixin.success_url
<django.views.generic.edit.ModelFormMixin.success_url>` now support the Python
:py:meth:`str.format()` syntax. The legacy ``%(<foo>)s`` syntax is still
supported but will be removed in Django 1.10.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* :setting:`FORMAT_MODULE_PATH` can now be a list of strings representing
module paths. This allows importing several format modules from different
reusable apps. It also allows overriding those custom formats in your main
Django project.

Logging
~~~~~~~

* The :class:`django.utils.log.AdminEmailHandler` class now has a
:meth:`~django.utils.log.AdminEmailHandler.send_mail` method to make it more
subclass friendly.

Management Commands
~~~~~~~~~~~~~~~~~~~

* Database connections are now always closed after a management command called
from the command line has finished doing its job.

* Commands from alternate package formats like eggs are now also discovered.

* The new :option:`dumpdata --output` option allows specifying a file to which
the serialized data is written.

* The new :option:`makemessages --exclude` and :option:`compilemessages
--exclude` options allow excluding specific locales from processing.

* :djadmin:`compilemessages` now has a ``--use-fuzzy`` or ``-f`` option which
includes fuzzy translations into compiled files.

* The :option:`loaddata --ignorenonexistent` option now ignores data for models
that no longer exist.

* :djadmin:`runserver` now uses daemon threads for faster reloading.

* :djadmin:`inspectdb` now outputs ``Meta.unique_together``. It is also able to
introspect :class:`~django.db.models.AutoField` for MySQL and PostgreSQL
databases.

* When calling management commands with options using
:func:`~django.core.management.call_command`, the option name can match the
command line option name (without the initial dashes) or the final option
destination variable name, but in either case, the resulting option received
by the command is now always the ``dest`` name specified in the command
option definition (as long as the command uses the :mod:`argparse` module).

* The :djadmin:`dbshell` command now supports MySQL's optional SSL certificate
authority setting (``--ssl-ca``).

* The new :option:`makemigrations --name` allows giving the migration(s) a
custom name instead of a generated one.

* The :djadmin:`loaddata` command now prevents repeated fixture loading. If
:setting:`FIXTURE_DIRS` contains duplicates or a default fixture directory
path (``app_name/fixtures``), an exception is raised.

* The new ``makemigrations --exit`` option allows exiting with an error
code if no migrations are created.

* The new :djadmin:`showmigrations` command allows listing all migrations and
their dependencies in a project.

Middleware
~~~~~~~~~~

* The :attr:`CommonMiddleware.response_redirect_class
<django.middleware.common.CommonMiddleware.response_redirect_class>`
attribute allows you to customize the redirects issued by the middleware.

* A debug message will be logged to the ``django.request`` logger when a
middleware raises a :exc:`~django.core.exceptions.MiddlewareNotUsed` exception
in :setting:`DEBUG` mode.

Migrations
~~~~~~~~~~

* The :class:`~django.db.migrations.operations.RunSQL` operation can now handle
parameters passed to the SQL statements.

* It is now possible to have migrations (most probably :ref:`data migrations
<data-migrations>`) for applications without models.

* Migrations can now :ref:`serialize model managers
<using-managers-in-migrations>` as part of the model state.

* A :ref:`generic mechanism to handle the deprecation of model fields
<migrations-removing-model-fields>` was added.

* The :meth:`RunPython.noop() <django.db.migrations.operations.RunPython.noop>`
and :attr:`RunSQL.noop <django.db.migrations.operations.RunSQL.noop>` class
method/attribute were added to ease in making ``RunPython`` and ``RunSQL``
operations reversible.

* The migration operations :class:`~django.db.migrations.operations.RunPython`
and :class:`~django.db.migrations.operations.RunSQL` now call the
:meth:`allow_migrate` method of database routers. The router can use the
newly introduced ``app_label`` and ``hints`` arguments to make a routing
decision. To take advantage of this feature you need to update the router to
the new ``allow_migrate`` signature, see the :ref:`deprecation section
<deprecated-signature-of-allow-migrate>` for more details.

Models
~~~~~~

* Django now logs at most 9000 queries in ``connections.queries``, in order
to prevent excessive memory usage in long-running processes in debug mode.

* There is now a model ``Meta`` option to define a
:attr:`default related name <django.db.models.Options.default_related_name>`
for all relational fields of a model.

* Pickling models and querysets across different versions of Django isn't
officially supported (it may work, but there's no guarantee). An extra
variable that specifies the current Django version is now added to the
pickled state of models and querysets, and Django raises a ``RuntimeWarning``
when these objects are unpickled in a different version than the one in
which they were pickled.

* Added :meth:`Model.from_db() <django.db.models.Model.from_db()>` which
Django uses whenever objects are loaded using the ORM. The method allows
customizing model loading behavior.

* ``extra(select={...})`` now allows you to escape a literal ``%s`` sequence
using ``%%s``.

* :doc:`Custom Lookups</howto/custom-lookups>` can now be registered using
a decorator pattern.

* The new :attr:`Transform.bilateral <django.db.models.Transform.bilateral>`
attribute allows creating bilateral transformations. These transformations
are applied to both ``lhs`` and ``rhs`` when used in a lookup expression,
providing opportunities for more sophisticated lookups.

* SQL special characters (\, %, _) are now escaped properly when a pattern
lookup (e.g. ``contains``, ``startswith``, etc.) is used with an ``F()``
expression as the right-hand side. In those cases, the escaping is performed
by the database, which can lead to somewhat complex queries involving nested
``REPLACE`` function calls.

* You can now refresh model instances by using :meth:`Model.refresh_from_db()
<django.db.models.Model.refresh_from_db>`.

* You can now get the set of deferred fields for a model using
:meth:`Model.get_deferred_fields() <django.db.models.Model.get_deferred_fields>`.

* Model field ``default``’s are now used when primary key field's are set to
``None``.

Signals
~~~~~~~

* Exceptions from the ``(receiver, exception)`` tuples returned by
:meth:`Signal.send_robust() <django.dispatch.Signal.send_robust>` now have
their traceback attached as a ``__traceback__`` attribute.

* The ``environ`` argument, which contains the WSGI environment structure from
the request, was added to the :data:`~django.core.signals.request_started`
signal.

* You can now import the :func:`~django.test.signals.setting_changed` signal
from ``django.core.signals`` to avoid loading ``django.test`` in non-test
situations. Django no longer does so itself.

System Check Framework
~~~~~~~~~~~~~~~~~~~~~~

* :attr:`~django.core.checks.register` can now be used as a function.

Templates
~~~~~~~~~

* :tfilter:`urlize` now supports domain-only links that include characters after
the top-level domain (e.g. ``djangoproject.com/`` and
``djangoproject.com/download/``).

* :tfilter:`urlize` doesn't treat exclamation marks at the end of a domain or
its query string as part of the URL (the URL in e.g. ``'djangoproject.com!``
is ``djangoproject.com``)

* Added a :class:`locmem.Loader <django.template.loaders.locmem.Loader>`
class that loads Django templates from a Python dictionary.

* The :ttag:`now` tag can now store its output in a context variable with the
usual syntax: ``{% now 'j n Y' as varname %}``.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* ``WSGIRequest`` now respects paths starting with ``//``.

* The :meth:`HttpRequest.build_absolute_uri()
<django.http.HttpRequest.build_absolute_uri>` method now handles paths
starting with ``//`` correctly.

* If :setting:`DEBUG` is ``True`` and a request raises a
:exc:`~django.core.exceptions.SuspiciousOperation`, the response will be
rendered with a detailed error page.

* The ``query_string`` argument of :class:`~django.http.QueryDict` is now
optional, defaulting to ``None``, so a blank ``QueryDict`` can now be
instantiated with ``QueryDict()`` instead of ``QueryDict(None)`` or
``QueryDict('')``.

* The ``GET`` and ``POST`` attributes of an :class:`~django.http.HttpRequest`
object are now :class:`~django.http.QueryDict`\s rather than dictionaries,
and the ``FILES`` attribute is now a ``MultiValueDict``.
This brings this class into line with the documentation and with
``WSGIRequest``.

* The :attr:`HttpResponse.charset <django.http.HttpResponse.charset>` attribute
was added.

* ``WSGIRequestHandler`` now follows RFC in converting URI to IRI, using
``uri_to_iri()``.

* The :meth:`HttpRequest.get_full_path()
<django.http.HttpRequest.get_full_path>` method now escapes unsafe characters
from the path portion of a Uniform Resource Identifier (URI) properly.

* :class:`~django.http.HttpResponse` now implements a few additional methods
like :meth:`~django.http.HttpResponse.getvalue` so that instances can be used
as stream objects.

* The new :meth:`HttpResponse.setdefault()
<django.http.HttpResponse.setdefault>` method allows setting a header unless
it has already been set.

* You can use the new :class:`~django.http.FileResponse` to stream files.

* The :func:`~django.views.decorators.http.condition` decorator for
conditional view processing now supports the ``If-unmodified-since`` header.

Tests
~~~~~

* The :class:`RequestFactory.trace() <django.test.RequestFactory>`
and :class:`Client.trace() <django.test.Client.trace>` methods were
implemented, allowing you to create ``TRACE`` requests in your tests.

* The ``count`` argument was added to
:meth:`~django.test.SimpleTestCase.assertTemplateUsed`. This allows you to
assert that a template was rendered a specific number of times.

* The new :meth:`~django.test.SimpleTestCase.assertJSONNotEqual` assertion
allows you to test that two JSON fragments are not equal.

* Added options to the :djadmin:`test` command to preserve the test database
(:option:`--keepdb <test --keepdb>`), to run the test cases in reverse order
(:option:`--reverse <test --reverse>`), and to enable SQL logging for failing
tests (:option:`--debug-sql <test --debug-sql>`).

* Added the :attr:`~django.test.Response.resolver_match` attribute to test
client responses.

* Added several settings that allow customization of test tablespace parameters
for Oracle: :setting:`DATAFILE`, :setting:`DATAFILE_TMP`,
:setting:`DATAFILE_MAXSIZE` and :setting:`DATAFILE_TMP_MAXSIZE`.

* The :func:`~django.test.override_settings` decorator can now affect the
master router in :setting:`DATABASE_ROUTERS`.

* Added test client support for file uploads with file-like objects.

* A shared cache is now used when testing with an SQLite in-memory database when
using Python 3.4+ and SQLite 3.7.13+. This allows sharing the database
between threads.

Validators
~~~~~~~~~~

* :class:`~django.core.validators.URLValidator` now supports IPv6 addresses,
Unicode domains, and URLs containing authentication data.

.. _backwards-incompatible-1.8:

Backwards incompatible changes in 1.8
=====================================

.. warning::

In addition to the changes outlined in this section, be sure to review the
:ref:`deprecation plan <deprecation-removed-in-1.8>` for any features that
have been removed. If you haven't updated your code within the
deprecation timeline for a given feature, its removal may appear as a
backwards incompatible change.

Related object operations are run in a transaction
--------------------------------------------------

Some operations on related objects such as
:meth:`~django.db.models.fields.related.RelatedManager.add()` or direct
assignment ran multiple data modifying queries without wrapping them in
transactions. To reduce the risk of data corruption, all data modifying methods
that affect multiple related objects (i.e. ``add()``, ``remove()``,
``clear()``, and direct assignment) now perform their data modifying queries
from within a transaction, provided your database supports transactions.

This has one backwards incompatible side effect, signal handlers triggered from
these methods are now executed within the method's transaction and any
exception in a signal handler will prevent the whole operation.

.. _unsaved-model-instance-check-18:

Assigning unsaved objects to relations raises an error
------------------------------------------------------

.. note::

To more easily allow in-memory usage of models, this change was reverted in
Django 1.8.4 and replaced with a check during ``model.save()``. For example::

>>> book = Book.objects.create(name="Django")
>>> book.author = Author(name="John")
>>> book.save()
Traceback (most recent call last):
...
ValueError: save() prohibited to prevent data loss due to unsaved related object 'author'.

A similar check on assignment to reverse one-to-one relations was removed
in Django 1.8.5.

Assigning unsaved objects to a :class:`~django.db.models.ForeignKey`,
:class:`~django.contrib.contenttypes.fields.GenericForeignKey`, and
:class:`~django.db.models.OneToOneField` now raises a :exc:`ValueError`.

Previously, the assignment of an unsaved object would be silently ignored.
For example::

>>> book = Book.objects.create(name="Django")
>>> book.author = Author(name="John")
>>> book.author.save()
>>> book.save()

>>> Book.objects.get(name="Django")
>>> book.author
>>>

Now, an error will be raised to prevent data loss::

>>> book.author = Author(name="john")
Traceback (most recent call last):
...
ValueError: Cannot assign "<Author: John>": "Author" instance isn't saved in the database.

If you require allowing the assignment of unsaved instances (the old behavior)
and aren't concerned about the data loss possibility (e.g. you never save the
objects to the database), you can disable this check by using the
``ForeignKey.allow_unsaved_instance_assignment`` attribute. (This attribute was
removed in 1.8.4 as it's no longer relevant.)

Management commands that only accept positional arguments
---------------------------------------------------------

If you have written a custom management command that only accepts positional
arguments and you didn't specify the ``args`` command variable, you might get
an error like ``Error: unrecognized arguments: ...``, as variable parsing is
now based on :py:mod:`argparse` which doesn't implicitly accept positional
arguments. You can make your command backwards compatible by simply setting the
``args`` class variable. However, if you don't have to keep compatibility with
older Django versions, it's better to implement the new
:meth:`~django.core.management.BaseCommand.add_arguments` method as described
in :doc:`/howto/custom-management-commands`.

Custom test management command arguments through test runner
------------------------------------------------------------

The method to add custom arguments to the ``test`` management command through
the test runner has changed. Previously, you could provide an ``option_list``
class variable on the test runner to add more arguments (à la
:py:mod:`optparse`). Now to implement the same behavior, you have to create an
``add_arguments(cls, parser)`` class method on the test runner and call
``parser.add_argument`` to add any custom arguments, as parser is now an
:py:class:`argparse.ArgumentParser` instance.

Model check ensures auto-generated column names are within limits specified by database
---------------------------------------------------------------------------------------

A field name that's longer than the column name length supported by a database
can create problems. For example, with MySQL you'll get an exception trying to
create the column, and with PostgreSQL the column name is truncated by the
database (you may see a warning in the PostgreSQL logs).

A model check has been introduced to better alert users to this scenario before
the actual creation of database tables.

If you have an existing model where this check seems to be a false positive,
for example on PostgreSQL where the name was already being truncated, simply
use :attr:`~django.db.models.Field.db_column` to specify the name that's being
used.

The check also applies to the columns generated in an implicit
``ManyToManyField.through`` model. If you run into an issue there, use
:attr:`~django.db.models.ManyToManyField.through` to create an explicit model
and then specify :attr:`~django.db.models.Field.db_column` on its column(s)
as needed.

Query relation lookups now check object types
---------------------------------------------

Querying for model lookups now checks if the object passed is of correct type
and raises a :exc:`ValueError` if not. Previously, Django didn't care if the
object was of correct type; it just used the object's related field attribute
(e.g. ``id``) for the lookup. Now, an error is raised to prevent incorrect
lookups::

>>> book = Book.objects.create(name="Django")
>>> book = Book.objects.filter(author=book)
Traceback (most recent call last):
...
ValueError: Cannot query "<Book: Django>": Must be "Author" instance.

``select_related()`` now checks given fields
--------------------------------------------

``select_related()`` now validates that the given fields actually exist.
Previously, nonexistent fields were silently ignored. Now, an error is raised::

>>> book = Book.objects.select_related('nonexistent_field')
Traceback (most recent call last):
...
FieldError: Invalid field name(s) given in select_related: 'nonexistent_field'

The validation also makes sure that the given field is relational::

>>> book = Book.objects.select_related('name')
Traceback (most recent call last):
...
FieldError: Non-relational field given in select_related: 'name'

Default ``EmailField.max_length`` increased to 254
--------------------------------------------------

The old default 75 character ``max_length`` was not capable of storing all
possible RFC3696/5321-compliant email addresses. In order to store all
possible valid email addresses, the ``max_length`` has been increased to 254
characters. You will need to generate and apply database migrations for your
affected models (or add ``max_length=75`` if you wish to keep the length on
your current fields). A migration for
:attr:`django.contrib.auth.models.User.email` is included.

Support for PostgreSQL versions older than 9.0
----------------------------------------------

The end of upstream support periods was reached in July 2014 for PostgreSQL 8.4.
As a consequence, Django 1.8 sets 9.0 as the minimum PostgreSQL version it
officially supports.

This also includes dropping support for PostGIS 1.3 and 1.4 as these versions
are not supported on versions of PostgreSQL later than 8.4.

Django also now requires the use of Psycopg2 version 2.4.5 or higher (or 2.5+
if you want to use :mod:`django.contrib.postgres`).

Support for MySQL versions older than 5.5
-----------------------------------------

The end of upstream support periods was reached in January 2012 for MySQL 5.0
and December 2013 for MySQL 5.1. As a consequence, Django 1.8 sets 5.5 as the
minimum MySQL version it officially supports.

Support for Oracle versions older than 11.1
-------------------------------------------

The end of upstream support periods was reached in July 2010 for Oracle 9.2,
January 2012 for Oracle 10.1, and July 2013 for Oracle 10.2. As a consequence,
Django 1.8 sets 11.1 as the minimum Oracle version it officially supports.

Specific privileges used instead of roles for tests on Oracle
-------------------------------------------------------------

Earlier versions of Django granted the CONNECT and RESOURCE roles to the test
user on Oracle. These roles have been deprecated, so Django 1.8 uses the
specific underlying privileges instead. This changes the privileges required
of the main user for running tests (unless the project is configured to avoid
creating a test user). The exact privileges required now are detailed in
:ref:`Oracle notes <oracle-notes>`.

``AbstractUser.last_login`` allows null values
----------------------------------------------

The :attr:`AbstractUser.last_login <django.contrib.auth.models.User.last_login>`
field now allows null values. Previously, it defaulted to the time when the user
was created which was misleading if the user never logged in. If you are using
the default user (:class:`django.contrib.auth.models.User`), run the database
migration included in ``contrib.auth``.

If you are using a custom user model that inherits from ``AbstractUser``,
you'll need to run :djadmin:`makemigrations` and generate a migration for your
app that contains that model. Also, if wish to set ``last_login`` to ``NULL``
for users who haven't logged in, you can run this query::

from django.db import models
from django.contrib.auth import get_user_model
from django.contrib.auth.models import AbstractBaseUser

UserModel = get_user_model()
if issubclass(UserModel, AbstractBaseUser):
UserModel._default_manager.filter(
last_login=models.F('date_joined')
).update(last_login=None)

:mod:`django.contrib.gis`
-------------------------

* Support for GEOS 3.1 and GDAL 1.6 has been dropped.

* Support for SpatiaLite < 2.4 has been dropped.

* GIS-specific lookups have been refactored to use the
:class:`django.db.models.Lookup` API.

* The default ``str`` representation of
:class:`~django.contrib.gis.geos.GEOSGeometry` objects has been changed from
WKT to EWKT format (including the SRID). As this representation is used in
the serialization framework, that means that ``dumpdata`` output will now
contain the SRID value of geometry objects.

Priority of context processors for ``TemplateResponse`` brought in line with ``render``
---------------------------------------------------------------------------------------

The :class:`~django.template.response.TemplateResponse` constructor is designed to be a
drop-in replacement for the :func:`~django.shortcuts.render` function. However,
it had a slight incompatibility, in that for ``TemplateResponse``, context data
from the passed in context dictionary could be shadowed by context data returned
from context processors, whereas for ``render`` it was the other way
around. This was a bug, and the behavior of ``render`` is more appropriate,
since it allows the globally defined context processors to be overridden locally
in the view. If you were relying on the fact context data in a
``TemplateResponse`` could be overridden using a context processor, you will
need to change your code.

Overriding ``setUpClass`` / ``tearDownClass`` in test cases
-----------------------------------------------------------

The decorators :func:`~django.test.override_settings` and
:func:`~django.test.modify_settings` now act at the class level when used as
class decorators. As a consequence, when overriding ``setUpClass()`` or
``tearDownClass()``, the ``super`` implementation should always be called.

Removal of ``django.contrib.formtools``
---------------------------------------

The formtools contrib app has been moved to a separate package and the
relevant documentation pages have been updated or removed.

The new package is available `on GitHub`_ and on PyPI.

.. _on GitHub: https://github.com/jazzband/django-formtools/

Database connection reloading between tests
-------------------------------------------

Django previously closed database connections between each test within a
``TestCase``. This is no longer the case as Django now wraps the whole
``TestCase`` within a transaction. If some of your tests relied on the old
behavior, you should have them inherit from ``TransactionTestCase`` instead.

Cleanup of the ``django.template`` namespace
--------------------------------------------

If you've been relying on private APIs exposed in the ``django.template``
module, you may have to import them from ``django.template.base`` instead.

Also private APIs ``django.template.base.compile_string()``,
``django.template.loader.find_template()``, and
``django.template.loader.get_template_from_string()`` were removed.

``model`` attribute on private model relations
----------------------------------------------

In earlier versions of Django, on a model with a reverse foreign key
relationship (for example), ``model._meta.get_all_related_objects()`` returned
the relationship as a ``django.db.models.related.RelatedObject`` with the
``model`` attribute set to the source of the relationship. Now, this method
returns the relationship as ``django.db.models.fields.related.ManyToOneRel``
(private API ``RelatedObject`` has been removed), and the ``model`` attribute
is set to the target of the relationship instead of the source. The source
model is accessible on the ``related_model`` attribute instead.

Consider this example from the tutorial in Django 1.8::

>>> p = Poll.objects.get(pk=1)
>>> p._meta.get_all_related_objects()
[<ManyToOneRel: polls.choice>]
>>> p._meta.get_all_related_objects()[0].model
<class 'polls.models.Poll'>
>>> p._meta.get_all_related_objects()[0].related_model
<class 'polls.models.Choice'>

and compare it to the behavior on older versions::

>>> p._meta.get_all_related_objects()
[<RelatedObject: polls:choice related to poll>]
>>> p._meta.get_all_related_objects()[0].model
<class 'polls.models.Choice'>

To access the source model, you can use a pattern like this to write code that
will work with both Django 1.8 and older versions::

for relation in opts.get_all_related_objects():
to_model = getattr(relation, 'related_model', relation.model)

Also note that ``get_all_related_objects()`` is deprecated in 1.8.

Database backend API
--------------------

The following changes to the database backend API are documented to assist
those writing third-party backends in updating their code:

* ``BaseDatabaseXXX`` classes have been moved to ``django.db.backends.base``.
Please import them from the new locations::

from django.db.backends.base.base import BaseDatabaseWrapper
from django.db.backends.base.client import BaseDatabaseClient
from django.db.backends.base.creation import BaseDatabaseCreation
from django.db.backends.base.features import BaseDatabaseFeatures
from django.db.backends.base.introspection import BaseDatabaseIntrospection
from django.db.backends.base.introspection import FieldInfo, TableInfo
from django.db.backends.base.operations import BaseDatabaseOperations
from django.db.backends.base.schema import BaseDatabaseSchemaEditor
from django.db.backends.base.validation import BaseDatabaseValidation

* The ``data_types``, ``data_types_suffix``, and
``data_type_check_constraints`` attributes have moved from the
``DatabaseCreation`` class to ``DatabaseWrapper``.

* The ``SQLCompiler.as_sql()`` method now takes a ``subquery`` parameter
(:ticket:`24164`).

* The ``BaseDatabaseOperations.date_interval_sql()`` method now only takes a
``timedelta`` parameter.

:mod:`django.contrib.admin`
---------------------------

* ``AdminSite`` no longer takes an ``app_name`` argument and its ``app_name``
attribute has been removed. The application name is always ``admin`` (as
opposed to the instance name which you can still customize using
``AdminSite(name="...")``.

* The ``ModelAdmin.get_object()`` method (private API) now takes a third
argument named ``from_field`` in order to specify which field should match
the provided ``object_id``.

* The :meth:`ModelAdmin.response_delete()
<django.contrib.admin.ModelAdmin.response_delete>` method
now takes a second argument named ``obj_id`` which is the serialized
identifier used to retrieve the object before deletion.

Default autoescaping of functions in ``django.template.defaultfilters``
-----------------------------------------------------------------------

In order to make built-in template filters that output HTML "safe by default"
when calling them in Python code, the following functions in
``django.template.defaultfilters`` have been changed to automatically escape
their input value:

* ``join``
* ``linebreaksbr``
* ``linebreaks_filter``
* ``linenumbers``
* ``unordered_list``
* ``urlize``
* ``urlizetrunc``

You can revert to the old behavior by specifying ``autoescape=False`` if you
are passing trusted content. This change doesn't have any effect when using
the corresponding filters in templates.

Miscellaneous
-------------

* ``connections.queries`` is now a read-only attribute.

* Database connections are considered equal only if they're the same object.
They aren't hashable any more.

* :class:`~django.middleware.gzip.GZipMiddleware` used to disable compression
for some content types when the request is from Internet Explorer, in order
to work around a bug in IE6 and earlier. This behavior could affect
performance on IE7 and later. It was removed.

* ``URLField.to_python`` no longer adds a trailing slash to pathless URLs.

* The :tfilter:`length` template filter now returns ``0`` for an undefined
variable, rather than an empty string.

* ``ForeignKey.default_error_message['invalid']`` has been changed from
``'%(model)s instance with pk %(pk)r does not exist.'`` to
``'%(model)s instance with %(field)s %(value)r does not exist.'`` If you are
using this message in your own code, please update the list of interpolated
parameters. Internally, Django will continue to provide the
``pk`` parameter in ``params`` for backwards compatibility.

* ``UserCreationForm.error_messages['duplicate_username']`` is no longer used.
If you wish to customize that error message, :ref:`override it on the form
<modelforms-overriding-default-fields>` using the ``'unique'`` key in
``Meta.error_messages['username']`` or, if you have a custom form field for
``'username'``, using the ``'unique'`` key in its
:attr:`~django.forms.Field.error_messages` argument.

* The block ``usertools`` in the ``base.html`` template of
:mod:`django.contrib.admin` now requires the ``has_permission`` context
variable to be set. If you have any custom admin views that use this
template, update them to pass :meth:`AdminSite.has_permission()
<django.contrib.admin.AdminSite.has_permission>` as this new variable's
value or simply include :meth:`AdminSite.each_context(request)
<django.contrib.admin.AdminSite.each_context>` in the context.

* Internal changes were made to the :class:`~django.forms.ClearableFileInput`
widget to allow more customization. The undocumented ``url_markup_template``
attribute was removed in favor of ``template_with_initial``.

* For consistency with other major vendors, the ``en_GB`` locale now has Monday
as the first day of the week.

* Seconds have been removed from any locales that had them in ``TIME_FORMAT``,
``DATETIME_FORMAT``, or ``SHORT_DATETIME_FORMAT``.

* The default max size of the Oracle test tablespace has increased from 300M
(or 200M, before 1.7.2) to 500M.

* ``reverse()`` and ``reverse_lazy()`` now return Unicode strings instead of
bytestrings.

* The ``CacheClass`` shim has been removed from all cache backends.
These aliases were provided for backwards compatibility with Django 1.3.
If you are still using them, please update your project to use the real
class name found in the :setting:`BACKEND <CACHES-BACKEND>` key of the
:setting:`CACHES` setting.

* By default, :func:`~django.core.management.call_command` now always skips the
check framework (unless you pass it ``skip_checks=False``).

* When iterating over lines, :class:`~django.core.files.File` now uses
:pep:`universal newlines <278>`. The following are recognized as ending a
line: the Unix end-of-line convention ``'\n'``, the Windows convention
``'\r\n'``, and the old Macintosh convention ``'\r'``.

* The Memcached cache backends ``MemcachedCache`` and ``PyLibMCCache`` will
delete a key if ``set()`` fails. This is necessary to ensure the ``cache_db``
session store always fetches the most current session data.

* Private APIs ``override_template_loaders`` and ``override_with_test_loader``
in ``django.test.utils`` were removed. Override ``TEMPLATES`` with
``override_settings`` instead.

* Warnings from the MySQL database backend are no longer converted to
exceptions when :setting:`DEBUG` is ``True``.

* :class:`~django.http.HttpRequest` now has a simplified ``repr`` (e.g.
``<WSGIRequest: GET '/somepath/'>``). This won't change the behavior of
the :class:`~django.views.debug.SafeExceptionReporterFilter` class.

* Class-based views that use :class:`~django.views.generic.edit.ModelFormMixin`
will raise an :exc:`~django.core.exceptions.ImproperlyConfigured` exception
when both the ``fields`` and ``form_class`` attributes are specified.
Previously, ``fields`` was silently ignored.

* When following redirects, the test client now raises
:exc:`~django.test.client.RedirectCycleError` if it detects a loop or hits a
maximum redirect limit (rather than passing silently).

* Translatable strings set as the ``default`` parameter of the field are cast
to concrete strings later, so the return type of ``Field.get_default()`` is
different in some cases. There is no change to default values which are the
result of a callable.

* ``GenericIPAddressField.empty_strings_allowed`` is now ``False``. Database
backends that interpret empty strings as null (only Oracle among the backends
that Django includes) will no longer convert null values back to an empty
string. This is consistent with other backends.

* When the ``BaseCommand.leave_locale_alone``
attribute is ``False``, translations are now deactivated instead of forcing
the "en-us" locale. In the case your models contained non-English strings and
you counted on English translations to be activated in management commands,
this will not happen any longer. It might be that new database migrations are
generated (once) after migrating to 1.8.

* :func:`django.utils.translation.get_language()` now returns ``None`` instead
of :setting:`LANGUAGE_CODE` when translations are temporarily deactivated.

* When a translation doesn't exist for a specific literal, the fallback is now
taken from the :setting:`LANGUAGE_CODE` language (instead of from the
untranslated ``msgid`` message).

* The ``name`` field of :class:`django.contrib.contenttypes.models.ContentType`
has been removed by a migration and replaced by a property. That means it's
not possible to query or filter a ``ContentType`` by this field any longer.

Be careful if you upgrade to Django 1.8 and skip Django 1.7. If you run
``manage.py migrate --fake``, this migration will be skipped and you'll see
a ``RuntimeError: Error creating new content types.`` exception because the
``name`` column won't be dropped from the database. Use ``manage.py migrate
--fake-initial`` to fake only the initial migration instead.

* The new :option:`migrate --fake-initial` option allows faking initial
migrations. In 1.7, initial migrations were always automatically faked if all
tables created in an initial migration already existed.

* An app *without* migrations with a ``ForeignKey`` to an app *with* migrations
may now result in a foreign key constraint error when migrating the database
or running tests. In Django 1.7, this could fail silently and result in a
missing constraint. To resolve the error, add migrations to the app without
them.

.. _deprecated-features-1.8:

Features deprecated in 1.8
==========================

Selected methods in ``django.db.models.options.Options``
--------------------------------------------------------

As part of the formalization of the ``Model._meta`` API (from the
:class:`django.db.models.options.Options` class), a number of methods have been
deprecated and will be removed in Django 1.10:

* ``get_all_field_names()``
* ``get_all_related_objects()``
* ``get_all_related_objects_with_model()``
* ``get_all_related_many_to_many_objects()``
* ``get_all_related_m2m_objects_with_model()``
* ``get_concrete_fields_with_model()``
* ``get_field_by_name()``
* ``get_fields_with_model()``
* ``get_m2m_with_model()``

Loading ``cycle`` and ``firstof`` template tags from ``future`` library
-----------------------------------------------------------------------

Django 1.6 introduced ``{% load cycle from future %}`` and
``{% load firstof from future %}`` syntax for forward compatibility of the
:ttag:`cycle` and :ttag:`firstof` template tags. This syntax is now deprecated
and will be removed in Django 1.10. You can simply remove the
``{% load ... from future %}`` tags.

``django.conf.urls.patterns()``
-------------------------------

In the olden days of Django, it was encouraged to reference views as strings
in ``urlpatterns``::

urlpatterns = patterns('',
url('^$', 'myapp.views.myview'),
)

and Django would magically import ``myapp.views.myview`` internally and turn
the string into a real function reference. In order to reduce repetition when
referencing many views from the same module, the ``patterns()`` function takes
a required initial ``prefix`` argument which is prepended to all
views-as-strings in that set of ``urlpatterns``::

urlpatterns = patterns('myapp.views',
url('^$', 'myview'),
url('^other/$', 'otherview'),
)

In the modern era, we have updated the tutorial to instead recommend importing
your views module and referencing your view functions (or classes) directly.
This has a number of advantages, all deriving from the fact that we are using
normal Python in place of "Django String Magic": the errors when you mistype a
view name are less obscure, IDEs can help with autocompletion of view names,
etc.

So these days, the above use of the ``prefix`` arg is much more likely to be
written (and is better written) as::

from myapp import views

urlpatterns = patterns('',
url('^$', views.myview),
url('^other/$', views.otherview),
)

Thus ``patterns()`` serves little purpose and is a burden when teaching new users
(answering the newbie's question "why do I need this empty string as the first
argument to ``patterns()``?"). For these reasons, we are deprecating it.
Updating your code is as simple as ensuring that ``urlpatterns`` is a list of
``django.conf.urls.url()`` instances. For example::

from django.conf.urls import url
from myapp import views

urlpatterns = [
url('^$', views.myview),
url('^other/$', views.otherview),
]

Passing a string as ``view`` to ``django.conf.urls.url()``
----------------------------------------------------------

Related to the previous item, referencing views as strings in the ``url()``
function is deprecated. Pass the callable view as described in the previous
section instead.

Template-related settings
-------------------------

As a consequence of the multiple template engines refactor, several settings
are deprecated in favor of :setting:`TEMPLATES`:

* ``ALLOWED_INCLUDE_ROOTS``
* ``TEMPLATE_CONTEXT_PROCESSORS``
* ``TEMPLATE_DEBUG``
* ``TEMPLATE_DIRS``
* ``TEMPLATE_LOADERS``
* ``TEMPLATE_STRING_IF_INVALID``

``django.core.context_processors``
----------------------------------

Built-in template context processors have been moved to
``django.template.context_processors``.

``django.test.SimpleTestCase.urls``
-----------------------------------

The attribute ``SimpleTestCase.urls`` for specifying URLconf configuration in
tests has been deprecated and will be removed in Django 1.10. Use
:func:`override_settings(ROOT_URLCONF=...) <django.test.override_settings>`
instead.

``prefix`` argument to :func:`~django.conf.urls.i18n.i18n_patterns`
-------------------------------------------------------------------

Related to the previous item, the ``prefix`` argument to
:func:`django.conf.urls.i18n.i18n_patterns` has been deprecated. Simply pass a
list of ``django.conf.urls.url()`` instances instead.

Using an incorrect count of unpacked values in the :ttag:`for` template tag
---------------------------------------------------------------------------

Using an incorrect count of unpacked values in :ttag:`for` tag will raise an
exception rather than fail silently in Django 1.10.

Passing a dotted path to ``reverse()`` and :ttag:`url`
------------------------------------------------------

Reversing URLs by Python path is an expensive operation as it causes the
path being reversed to be imported. This behavior has also resulted in a
`security issue`_. Use :ref:`named URL patterns <naming-url-patterns>`
for reversing instead.

If you are using :mod:`django.contrib.sitemaps`, add the ``name`` argument to
the ``url`` that references :func:`django.contrib.sitemaps.views.sitemap`::

from django.contrib.sitemaps.views import sitemap

url(r'^sitemap\.xml$', sitemap, {'sitemaps': sitemaps},
name='django.contrib.sitemaps.views.sitemap')

to ensure compatibility when reversing by Python path is removed in Django 1.10.

Similarly for GIS sitemaps, add ``name='django.contrib.gis.sitemaps.views.kml'``
or ``name='django.contrib.gis.sitemaps.views.kmz'``.

If you are using a Python path for the :setting:`LOGIN_URL` or
:setting:`LOGIN_REDIRECT_URL` setting, use the name of the ``url()`` instead.

.. _security issue: https://www.djangoproject.com/weblog/2014/apr/21/security/#s-issue-unexpected-code-execution-using-reverse

Aggregate methods and modules
-----------------------------

The ``django.db.models.sql.aggregates`` and
``django.contrib.gis.db.models.sql.aggregates`` modules (both private API), have
been deprecated as ``django.db.models.aggregates`` and
``django.contrib.gis.db.models.aggregates`` are now also responsible
for SQL generation. The old modules will be removed in Django 1.10.

If you were using the old modules, see :doc:`Query Expressions
</ref/models/expressions>` for instructions on rewriting custom aggregates
using the new stable API.

The following methods and properties of ``django.db.models.sql.query.Query``
have also been deprecated and the backwards compatibility shims will be removed
in Django 1.10:

* ``Query.aggregates``, replaced by ``annotations``.
* ``Query.aggregate_select``, replaced by ``annotation_select``.
* ``Query.add_aggregate()``, replaced by ``add_annotation()``.
* ``Query.set_aggregate_mask()``, replaced by ``set_annotation_mask()``.
* ``Query.append_aggregate_mask()``, replaced by ``append_annotation_mask()``.

Extending management command arguments through ``Command.option_list``
----------------------------------------------------------------------

Management commands now use :py:mod:`argparse` instead of :py:mod:`optparse` to
parse command-line arguments passed to commands. This also means that the way
to add custom arguments to commands has changed: instead of extending the
``option_list`` class list, you should now override the
:meth:`~django.core.management.BaseCommand.add_arguments` method and add
arguments through ``argparse.add_argument()``. See
:ref:`this example <custom-commands-options>` for more details.

``django.core.management.NoArgsCommand``
----------------------------------------

The class ``NoArgsCommand`` is now deprecated and will be removed in Django
1.10. Use :class:`~django.core.management.BaseCommand` instead, which takes no
arguments by default.

Listing all migrations in a project
-----------------------------------

The ``--list`` option of the :djadmin:`migrate` management command is
deprecated and will be removed in Django 1.10. Use :djadmin:`showmigrations`
instead.

``cache_choices`` option of ``ModelChoiceField`` and ``ModelMultipleChoiceField``
---------------------------------------------------------------------------------

:class:`~django.forms.ModelChoiceField` and
:class:`~django.forms.ModelMultipleChoiceField` took an undocumented, untested
option ``cache_choices``. This cached querysets between multiple renderings of
the same ``Form`` object. This option is subject to an accelerated deprecation
and will be removed in Django 1.9.

``django.template.resolve_variable()``
--------------------------------------

The function has been informally marked as "Deprecated" for some time. Replace
``resolve_variable(path, context)`` with
``django.template.Variable(path).resolve(context)``.

``django.contrib.webdesign``
----------------------------

It provided the :ttag:`lorem` template tag which is now included in the
built-in tags. Simply remove ``'django.contrib.webdesign'`` from
:setting:`INSTALLED_APPS` and ``{% load webdesign %}`` from your templates.

``error_message`` argument to ``django.forms.RegexField``
---------------------------------------------------------

It provided backwards compatibility for pre-1.0 code, but its functionality is
redundant. Use ``Field.error_messages['invalid']`` instead.

Old :tfilter:`unordered_list` syntax
------------------------------------

An older (pre-1.0), more restrictive and verbose input format for the
:tfilter:`unordered_list` template filter has been deprecated::

['States', [['Kansas', [['Lawrence', []], ['Topeka', []]]], ['Illinois', []]]]

Using the new syntax, this becomes::

['States', ['Kansas', ['Lawrence', 'Topeka'], 'Illinois']]

``django.forms.Field._has_changed()``
-------------------------------------

Rename this method to :meth:`~django.forms.Field.has_changed` by removing the
leading underscore. The old name will still work until Django 1.10.

``django.utils.html.remove_tags()`` and ``removetags`` template filter
----------------------------------------------------------------------

``django.utils.html.remove_tags()`` as well as the template filter
``removetags`` have been deprecated as they cannot guarantee safe output. Their
existence is likely to lead to their use in security-sensitive contexts where
they are not actually safe.

The unused and undocumented ``django.utils.html.strip_entities()`` function has
also been deprecated.

``is_admin_site`` argument to ``django.contrib.auth.views.password_reset()``
----------------------------------------------------------------------------

It's a legacy option that should no longer be necessary.

``SubfieldBase``
----------------

``django.db.models.fields.subclassing.SubfieldBase`` has been deprecated and
will be removed in Django 1.10. Historically, it was used to handle fields where
type conversion was needed when loading from the database, but it was not used
in ``.values()`` calls or in aggregates. It has been replaced with
:meth:`~django.db.models.Field.from_db_value`.

The new approach doesn't call the :meth:`~django.db.models.Field.to_python`
method on assignment as was the case with ``SubfieldBase``. If you need that
behavior, reimplement the ``Creator`` class `from Django's source code
<https://github.com/django/django/blob/stable/1.8.x/django/db/models/fields/subclassing.py#L31-L44>`_
in your project.

``django.utils.checksums``
--------------------------

The ``django.utils.checksums`` module has been deprecated and will be removed
in Django 1.10. The functionality it provided (validating checksum using the
Luhn algorithm) was undocumented and not used in Django. The module has been
moved to the `django-localflavor`_ package (version 1.1+).

.. _django-localflavor: https://pypi.org/project/django-localflavor/

``InlineAdminForm.original_content_type_id``
--------------------------------------------

The ``original_content_type_id`` attribute on ``InlineAdminForm`` has been
deprecated and will be removed in Django 1.10. Historically, it was used
to construct the "view on site" URL. This URL is now accessible using the
``absolute_url`` attribute of the form.

``django.views.generic.edit.FormMixin.get_form()``’s ``form_class`` argument
----------------------------------------------------------------------------

``FormMixin`` subclasses that override the ``get_form()`` method should make
sure to provide a default value for the ``form_class`` argument since it's
now optional.

Rendering templates loaded by :func:`~django.template.loader.get_template()` with a :class:`~django.template.Context`
---------------------------------------------------------------------------------------------------------------------

The return type of :func:`~django.template.loader.get_template()` has changed
in Django 1.8: instead of a :class:`django.template.Template`, it returns a
``Template`` instance whose exact type depends on which backend loaded it.

Both classes provide a ``render()`` method, however, the former takes a
:class:`django.template.Context` as an argument while the latter expects a
:class:`dict`. This change is enforced through a deprecation path for Django
templates.

All this also applies to :func:`~django.template.loader.select_template()`.

:class:`~django.template.Template` and :class:`~django.template.Context` classes in template responses
------------------------------------------------------------------------------------------------------

Some methods of :class:`~django.template.response.SimpleTemplateResponse` and
:class:`~django.template.response.TemplateResponse` accepted
:class:`django.template.Context` and :class:`django.template.Template` objects
as arguments. They should now receive :class:`dict` and backend-dependent
template objects respectively.

This also applies to the return types if you have subclassed either template
response class.

Check the :doc:`template response API documentation </ref/template-response>`
for details.

``current_app`` argument of template-related APIs
-------------------------------------------------

The following functions and classes will no longer accept a ``current_app``
parameter to set an URL namespace in Django 1.10:

* ``django.shortcuts.render()``
* ``django.template.Context()``
* ``django.template.RequestContext()``
* ``django.template.response.TemplateResponse()``

Set ``request.current_app`` instead, where ``request`` is the first argument
to these functions or classes. If you're using a plain ``Context``, use a
``RequestContext`` instead.

``dictionary`` and ``context_instance`` arguments of rendering functions
------------------------------------------------------------------------

The following functions will no longer accept the ``dictionary`` and
``context_instance`` parameters in Django 1.10:

* ``django.shortcuts.render()``
* ``django.shortcuts.render_to_response()``
* ``django.template.loader.render_to_string()``

Use the ``context`` parameter instead. When ``dictionary`` is passed as a
positional argument, which is the most common idiom, no changes are needed.

If you're passing a :class:`~django.template.Context` in ``context_instance``,
pass a :class:`dict` in the ``context`` parameter instead. If you're passing a
:class:`~django.template.RequestContext`, pass the request separately in the
``request`` parameter.

``dirs`` argument of template-finding functions
-----------------------------------------------

The following functions will no longer accept a ``dirs`` parameter to override
``TEMPLATE_DIRS`` in Django 1.10:

* :func:`django.template.loader.get_template()`
* :func:`django.template.loader.select_template()`
* :func:`django.shortcuts.render()`
* ``django.shortcuts.render_to_response()``

The parameter didn't work consistently across different template loaders and
didn't work for included templates.

``django.template.loader.BaseLoader``
-------------------------------------

``django.template.loader.BaseLoader`` was renamed to
``django.template.loaders.base.Loader``. If you've written a custom template
loader that inherits ``BaseLoader``, you must inherit ``Loader`` instead.

``django.test.utils.TestTemplateLoader``
----------------------------------------

Private API ``django.test.utils.TestTemplateLoader`` is deprecated in favor of
``django.template.loaders.locmem.Loader`` and will be removed in Django 1.9.

.. _storage-max-length-update:

Support for the ``max_length`` argument on custom ``Storage`` classes
---------------------------------------------------------------------

``Storage`` subclasses should add ``max_length=None`` as a parameter to
:meth:`~django.core.files.storage.Storage.get_available_name` and/or
:meth:`~django.core.files.storage.Storage.save` if they override either method.
Support for storages that do not accept this argument will be removed in
Django 1.10.

``qn`` replaced by ``compiler``
-------------------------------

In previous Django versions, various internal ORM methods (mostly ``as_sql``
methods) accepted a ``qn`` (for "quote name") argument, which was a reference
to a function that quoted identifiers for sending to the database. In Django
1.8, that argument has been renamed to ``compiler`` and is now a full
``SQLCompiler`` instance. For backwards-compatibility, calling a
``SQLCompiler`` instance performs the same name-quoting that the ``qn``
function used to. However, this backwards-compatibility shim is immediately
deprecated: you should rename your ``qn`` arguments to ``compiler``, and call
``compiler.quote_name_unless_alias(...)`` where you previously called
``qn(...)``.

Default value of ``RedirectView.permanent``
-------------------------------------------

The default value of the
:attr:`RedirectView.permanent <django.views.generic.base.RedirectView.permanent>`
attribute will change from ``True`` to ``False`` in Django 1.9.

Using ``AuthenticationMiddleware`` without ``SessionAuthenticationMiddleware``
------------------------------------------------------------------------------

``django.contrib.auth.middleware.SessionAuthenticationMiddleware`` was
added in Django 1.7. In Django 1.7.2, its functionality was moved to
``auth.get_user()`` and, for backwards compatibility, enabled only if
``'django.contrib.auth.middleware.SessionAuthenticationMiddleware'`` appears in
``MIDDLEWARE_CLASSES``.

In Django 1.10, session verification will be enabled regardless of whether or not
``SessionAuthenticationMiddleware`` is enabled (at which point
``SessionAuthenticationMiddleware`` will have no significance). You can add it
to your ``MIDDLEWARE_CLASSES`` sometime before then to opt-in. Please read the
:ref:`upgrade considerations <session-invalidation-on-password-change>` first.

``django.contrib.sitemaps.FlatPageSitemap``
-------------------------------------------

``django.contrib.sitemaps.FlatPageSitemap`` has moved to
``django.contrib.flatpages.sitemaps.FlatPageSitemap``. The old import location
is deprecated and will be removed in Django 1.9.

Model ``Field.related``
-----------------------

Private attribute ``django.db.models.Field.related`` is deprecated in favor
of ``Field.rel``. The latter is an instance of
``django.db.models.fields.related.ForeignObjectRel`` which replaces
``django.db.models.related.RelatedObject``. The ``django.db.models.related``
module has been removed and the ``Field.related`` attribute will be removed in
Django 1.10.

``ssi`` template tag
--------------------

The ``ssi`` template tag allows files to be included in a template by
absolute path. This is of limited use in most deployment situations, and
the :ttag:`include` tag often makes more sense. This tag is now deprecated and
will be removed in Django 1.10.

``=`` as comparison operator in ``if`` template tag
---------------------------------------------------

Using a single equals sign with the ``{% if %}`` template tag for equality
testing was undocumented and untested. It's now deprecated in favor of ``==``.

``%(<foo>)s`` syntax in ``ModelFormMixin.success_url``
------------------------------------------------------

The legacy ``%(<foo>)s`` syntax in :attr:`ModelFormMixin.success_url
<django.views.generic.edit.ModelFormMixin.success_url>` is deprecated and
will be removed in Django 1.10.

``GeoQuerySet`` aggregate methods
---------------------------------

The ``collect()``, ``extent()``, ``extent3d()``, ``make_line()``, and
``unionagg()`` aggregate methods are deprecated and should be replaced by their
function-based aggregate equivalents (``Collect``, ``Extent``, ``Extent3D``,
``MakeLine``, and ``Union``).

.. _deprecated-signature-of-allow-migrate:

Signature of the ``allow_migrate`` router method
------------------------------------------------

The signature of the :meth:`allow_migrate` method of database routers has
changed from ``allow_migrate(db, model)`` to
``allow_migrate(db, app_label, model_name=None, **hints)``.

When ``model_name`` is set, the value that was previously given through the
``model`` positional argument may now be found inside the ``hints`` dictionary
under the key ``'model'``.

After switching to the new signature the router will also be called by the
:class:`~django.db.migrations.operations.RunPython` and
:class:`~django.db.migrations.operations.RunSQL` operations.

.. _removed-features-1.8:

Features removed in 1.8
=======================

These features have reached the end of their deprecation cycle and are removed
in Django 1.8. See :ref:`deprecated-features-1.6` for details, including how to
remove usage of these features.

* ``django.contrib.comments`` is removed.

* The following transaction management APIs are removed:

- ``TransactionMiddleware``
- the decorators and context managers ``autocommit``, ``commit_on_success``,
and ``commit_manually``, defined in ``django.db.transaction``
- the functions ``commit_unless_managed`` and ``rollback_unless_managed``,
also defined in ``django.db.transaction``
- the ``TRANSACTIONS_MANAGED`` setting

* The :ttag:`cycle` and :ttag:`firstof` template tags auto-escape their
arguments.

* The ``SEND_BROKEN_LINK_EMAILS`` setting is removed.

* ``django.middleware.doc.XViewMiddleware`` is removed.

* The ``Model._meta.module_name`` alias is removed.

* The backward compatible shims introduced to rename ``get_query_set``
and similar queryset methods are removed. This affects the following classes:
``BaseModelAdmin``, ``ChangeList``, ``BaseCommentNode``,
``GenericForeignKey``, ``Manager``, ``SingleRelatedObjectDescriptor`` and
``ReverseSingleRelatedObjectDescriptor``.

* The backward compatible shims introduced to rename the attributes
``ChangeList.root_query_set`` and ``ChangeList.query_set`` are removed.

* ``django.views.defaults.shortcut`` and ``django.conf.urls.shortcut`` are
removed.

* Support for the Python Imaging Library (PIL) module is removed.

* The following private APIs are removed:

- ``django.db.backend``
- ``django.db.close_connection()``
- ``django.db.backends.creation.BaseDatabaseCreation.set_autocommit()``
- ``django.db.transaction.is_managed()``
- ``django.db.transaction.managed()``

* ``django.forms.widgets.RadioInput`` is removed.

* The module ``django.test.simple`` and the class
``django.test.simple.DjangoTestSuiteRunner`` are removed.

* The module ``django.test._doctest`` is removed.

* The ``CACHE_MIDDLEWARE_ANONYMOUS_ONLY`` setting is removed. This change
affects both ``django.middleware.cache.CacheMiddleware`` and
``django.middleware.cache.UpdateCacheMiddleware`` despite the lack of a
deprecation warning in the latter class.

* Usage of the hard-coded *Hold down "Control", or "Command" on a Mac, to select
more than one.* string to override or append to user-provided ``help_text`` in
forms for ``ManyToMany`` model fields is not performed by Django anymore
either at the model or forms layer.

* The ``Model._meta.get_(add|change|delete)_permission`` methods are removed.

* The session key ``django_language`` is no longer read for backwards
compatibility.

* Geographic Sitemaps are removed
(``django.contrib.gis.sitemaps.views.index`` and
``django.contrib.gis.sitemaps.views.sitemap``).

* ``django.utils.html.fix_ampersands``, the ``fix_ampersands`` template filter,
and ``django.utils.html.clean_html`` are removed.


===========================

1.7.11

Not secure
===========================

*November 24, 2015*

Django 1.7.11 fixes a security issue and a data loss bug in 1.7.10.

Fixed settings leak possibility in ``date`` template filter
===========================================================

If an application allows users to specify an unvalidated format for dates and
passes this format to the :tfilter:`date` filter, e.g.
``{{ last_updated|date:user_date_format }}``, then a malicious user could
obtain any secret in the application's settings by specifying a settings key
instead of a date format. e.g. ``"SECRET_KEY"`` instead of ``"j/m/Y"``.

To remedy this, the underlying function used by the ``date`` template filter,
``django.utils.formats.get_format()``, now only allows accessing the date/time
formatting settings.

Bugfixes
========

* Fixed a data loss possibility with :class:`~django.db.models.Prefetch` if
``to_attr`` is set to a ``ManyToManyField`` (:ticket:`25693`).


===========================

1.7.10

Not secure
===========================

*August 18, 2015*

Django 1.7.10 fixes a security issue in 1.7.9.

Denial-of-service possibility in ``logout()`` view by filling session store
===========================================================================

Previously, a session could be created when anonymously accessing the
``django.contrib.auth.views.logout()`` view (provided it wasn't decorated
with :func:`~django.contrib.auth.decorators.login_required` as done in the
admin). This could allow an attacker to easily create many new session records
by sending repeated requests, potentially filling up the session store or
causing other users' session records to be evicted.

The :class:`~django.contrib.sessions.middleware.SessionMiddleware` has been
modified to no longer create empty session records, including when
:setting:`SESSION_SAVE_EVERY_REQUEST` is active.

Additionally, the ``contrib.sessions.backends.base.SessionBase.flush()`` and
``cache_db.SessionStore.flush()`` methods have been modified to avoid creating
a new empty session. Maintainers of third-party session backends should check
if the same vulnerability is present in their backend and correct it if so.


==========================

1.7.9

Not secure
==========================

*July 8, 2015*

Django 1.7.9 fixes several security issues and bugs in 1.7.8.

Denial-of-service possibility by filling session store
======================================================

In previous versions of Django, the session backends created a new empty record
in the session storage anytime ``request.session`` was accessed and there was a
session key provided in the request cookies that didn't already have a session
record. This could allow an attacker to easily create many new session records
simply by sending repeated requests with unknown session keys, potentially
filling up the session store or causing other users' session records to be
evicted.

The built-in session backends now create a session record only if the session
is actually modified; empty session records are not created. Thus this
potential DoS is now only possible if the site chooses to expose a
session-modifying view to anonymous users.

As each built-in session backend was fixed separately (rather than a fix in the
core sessions framework), maintainers of third-party session backends should
check whether the same vulnerability is present in their backend and correct
it if so.

Header injection possibility since validators accept newlines in input
======================================================================

Some of Django's built-in validators
(:class:`~django.core.validators.EmailValidator`, most seriously) didn't
prohibit newline characters (due to the usage of ``$`` instead of ``\Z`` in the
regular expressions). If you use values with newlines in HTTP response or email
headers, you can suffer from header injection attacks. Django itself isn't
vulnerable because :class:`~django.http.HttpResponse` and the mail sending
utilities in :mod:`django.core.mail` prohibit newlines in HTTP and SMTP
headers, respectively. While the validators have been fixed in Django, if
you're creating HTTP responses or email messages in other ways, it's a good
idea to ensure that those methods prohibit newlines as well. You might also
want to validate that any existing data in your application doesn't contain
unexpected newlines.

:func:`~django.core.validators.validate_ipv4_address`,
:func:`~django.core.validators.validate_slug`, and
:class:`~django.core.validators.URLValidator` are also affected, however, as
of Django 1.6 the ``GenericIPAddresseField``, ``IPAddressField``, ``SlugField``,
and ``URLField`` form fields which use these validators all strip the input, so
the possibility of newlines entering your data only exists if you are using
these validators outside of the form fields.

The undocumented, internally unused ``validate_integer()`` function is now
stricter as it validates using a regular expression instead of simply casting
the value using ``int()`` and checking if an exception was raised.

Bugfixes
========

* Prevented the loss of ``null``/``not null`` column properties during field
renaming of MySQL databases (:ticket:`24817`).

* Fixed ``SimpleTestCase.assertRaisesMessage()`` on Python 2.7.10
(:ticket:`24903`).


==========================

1.7.8

Not secure
==========================

*May 1, 2015*

Django 1.7.8 fixes:

* Database introspection with SQLite 3.8.9 (released April 8, 2015)
(:ticket:`24637`).

* A database table name quoting regression in 1.7.2 (:ticket:`24605`).

* The loss of ``null``/``not null`` column properties during field alteration
of MySQL databases (:ticket:`24595`).


==========================

1.7.7

Not secure
==========================

*March 18, 2015*

Django 1.7.7 fixes several bugs and security issues in 1.7.6.

Denial-of-service possibility with ``strip_tags()``
===================================================

Last year :func:`~django.utils.html.strip_tags` was changed to work
iteratively. The problem is that the size of the input it's processing can
increase on each iteration which results in an infinite loop in
``strip_tags()``. This issue only affects versions of Python that haven't
received :bpo:`a bugfix in HTMLParser <20288>`; namely Python < 2.7.7 and
3.3.5. Some operating system vendors have also backported the fix for the
Python bug into their packages of earlier versions.

To remedy this issue, ``strip_tags()`` will now return the original input if
it detects the length of the string it's processing increases. Remember that
absolutely NO guarantee is provided about the results of ``strip_tags()`` being
HTML safe. So NEVER mark safe the result of a ``strip_tags()`` call without
escaping it first, for example with :func:`~django.utils.html.escape`.

Mitigated possible XSS attack via user-supplied redirect URLs
=============================================================

Django relies on user input in some cases (e.g.
``django.contrib.auth.views.login()`` and :doc:`i18n </topics/i18n/index>`)
to redirect the user to an "on success" URL. The security checks for these
redirects (namely ``django.utils.http.is_safe_url()``) accepted URLs with
leading control characters and so considered URLs like ``\x08javascript:...``
safe. This issue doesn't affect Django currently, since we only put this URL
into the ``Location`` response header and browsers seem to ignore JavaScript
there. Browsers we tested also treat URLs prefixed with control characters such
as ``%08//example.com`` as relative paths so redirection to an unsafe target
isn't a problem either.

However, if a developer relies on ``is_safe_url()`` to
provide safe redirect targets and puts such a URL into a link, they could
suffer from an XSS attack as some browsers such as Google Chrome ignore control
characters at the start of a URL in an anchor ``href``.

Bugfixes
========

* Fixed renaming of classes in migrations where renaming a subclass would
cause incorrect state to be recorded for objects that referenced the
superclass (:ticket:`24354`).

* Stopped writing migration files in dry run mode when merging migration
conflicts. When ``makemigrations --merge`` is called with ``verbosity=3`` the
migration file is written to ``stdout`` (:ticket:`24427`).


==========================

1.7.6

Not secure
==========================

*March 9, 2015*

Django 1.7.6 fixes a security issue and several bugs in 1.7.5.

Mitigated an XSS attack via properties in ``ModelAdmin.readonly_fields``
========================================================================

The :attr:`ModelAdmin.readonly_fields
<django.contrib.admin.ModelAdmin.readonly_fields>` attribute in the Django
admin allows displaying model fields and model attributes. While the former
were correctly escaped, the latter were not. Thus untrusted content could be
injected into the admin, presenting an exploitation vector for XSS attacks.

In this vulnerability, every model attribute used in ``readonly_fields`` that
is not an actual model field (e.g. a :class:`property`) will **fail to be
escaped** even if that attribute is not marked as safe. In this release,
autoescaping is now correctly applied.

Bugfixes
========

* Fixed crash when coercing ``ManyRelatedManager`` to a string
(:ticket:`24352`).

* Fixed a bug that prevented migrations from adding a foreign key constraint
when converting an existing field to a foreign key (:ticket:`24447`).


==========================

1.7.5

Not secure
==========================

*February 25, 2015*

Django 1.7.5 fixes several bugs in 1.7.4.

Bugfixes
========

* Reverted a fix that prevented a migration crash when unapplying
``contrib.contenttypes``’s or ``contrib.auth``’s first migration
(:ticket:`24075`) due to severe impact on the test performance
(:ticket:`24251`) and problems in multi-database setups (:ticket:`24298`).

* Fixed a regression that prevented custom fields inheriting from
``ManyToManyField`` from being recognized in migrations (:ticket:`24236`).

* Fixed crash in ``contrib.sites`` migrations when a default database isn't
used (:ticket:`24332`).

* Added the ability to set the isolation level on PostgreSQL with psycopg2 ≥
2.4.2 (:ticket:`24318`). It was advertised as a new feature in Django 1.6
but it didn't work in practice.

* Formats for the Azerbaijani locale (``az``) have been added.


==========================

1.7.4

Not secure
==========================

*January 27, 2015*

Django 1.7.4 fixes several bugs in 1.7.3.

Bugfixes
========

* Fixed a migration crash when unapplying ``contrib.contenttypes``’s or
``contrib.auth``’s first migration (:ticket:`24075`).

* Made the migration's ``RenameModel`` operation rename ``ManyToManyField``
tables (:ticket:`24135`).

* Fixed a migration crash on MySQL when migrating from a ``OneToOneField`` to a
``ForeignKey`` (:ticket:`24163`).

* Prevented the ``static.serve`` view from producing ``ResourceWarning``\s in
certain circumstances (security fix regression, :ticket:`24193`).

* Fixed schema check for ManyToManyField to look for internal type instead
of checking class instance, so you can write custom m2m-like fields with the
same behavior. (:ticket:`24104`).


==========================

1.7.3

Not secure
==========================

*January 13, 2015*

Django 1.7.3 fixes several security issues and bugs in 1.7.2.

WSGI header spoofing via underscore/dash conflation
===================================================

When HTTP headers are placed into the WSGI environ, they are normalized by
converting to uppercase, converting all dashes to underscores, and prepending
``HTTP_``. For instance, a header ``X-Auth-User`` would become
``HTTP_X_AUTH_USER`` in the WSGI environ (and thus also in Django's
``request.META`` dictionary).

Unfortunately, this means that the WSGI environ cannot distinguish between
headers containing dashes and headers containing underscores: ``X-Auth-User``
and ``X-Auth_User`` both become ``HTTP_X_AUTH_USER``. This means that if a
header is used in a security-sensitive way (for instance, passing
authentication information along from a front-end proxy), even if the proxy
carefully strips any incoming value for ``X-Auth-User``, an attacker may be
able to provide an ``X-Auth_User`` header (with underscore) and bypass this
protection.

In order to prevent such attacks, both Nginx and Apache 2.4+ strip all headers
containing underscores from incoming requests by default. Django's built-in
development server now does the same. Django's development server is not
recommended for production use, but matching the behavior of common production
servers reduces the surface area for behavior changes during deployment.

Mitigated possible XSS attack via user-supplied redirect URLs
=============================================================

Django relies on user input in some cases (e.g.
``django.contrib.auth.views.login()`` and :doc:`i18n </topics/i18n/index>`)
to redirect the user to an "on success" URL. The security checks for these
redirects (namely ``django.utils.http.is_safe_url()``) didn't strip leading
whitespace on the tested URL and as such considered URLs like
``\njavascript:...`` safe. If a developer relied on ``is_safe_url()`` to
provide safe redirect targets and put such a URL into a link, they could suffer
from a XSS attack. This bug doesn't affect Django currently, since we only put
this URL into the ``Location`` response header and browsers seem to ignore
JavaScript there.

Denial-of-service attack against ``django.views.static.serve``
==============================================================

In older versions of Django, the :func:`django.views.static.serve` view read
the files it served one line at a time. Therefore, a big file with no newlines
would result in memory usage equal to the size of that file. An attacker could
exploit this and launch a denial-of-service attack by simultaneously requesting
many large files. This view now reads the file in chunks to prevent large
memory usage.

Note, however, that this view has always carried a warning that it is not
hardened for production use and should be used only as a development aid. Now
may be a good time to audit your project and serve your files in production
using a real front-end web server if you are not doing so.

Database denial-of-service with ``ModelMultipleChoiceField``
============================================================

Given a form that uses ``ModelMultipleChoiceField`` and
``show_hidden_initial=True`` (not a documented API), it was possible for a user
to cause an unreasonable number of SQL queries by submitting duplicate values
for the field's data. The validation logic in ``ModelMultipleChoiceField`` now
deduplicates submitted values to address this issue.

Bugfixes
========

* The default iteration count for the PBKDF2 password hasher has been
increased by 25%. This part of the normal major release process was
inadvertently omitted in 1.7. This backwards compatible change will not
affect users who have subclassed
``django.contrib.auth.hashers.PBKDF2PasswordHasher`` to change the
default value.

* Fixed a crash in the CSRF middleware when handling non-ASCII referer header
(:ticket:`23815`).

* Fixed a crash in the ``django.contrib.auth.redirect_to_login`` view when
passing a ``reverse_lazy()`` result on Python 3 (:ticket:`24097`).

* Added correct formats for Greek (``el``) (:ticket:`23967`).

* Fixed a migration crash when unapplying a migration where multiple operations
interact with the same model (:ticket:`24110`).


==========================

1.7.2

Not secure
==========================

*January 2, 2015*

Django 1.7.2 fixes several bugs in 1.7.1.

Additionally, Django's vendored version of six, ``django.utils.six``, has
been upgraded to the latest release (1.9.0).

Bugfixes
========

* Fixed migration's renaming of auto-created many-to-many tables when changing
:attr:`Meta.db_table <django.db.models.Options.db_table>` (:ticket:`23630`).

* Fixed a migration crash when adding an explicit ``id`` field to a model on
SQLite (:ticket:`23702`).

* Added a warning for duplicate models when a module is reloaded. Previously a
``RuntimeError`` was raised every time two models clashed in the app registry.
(:ticket:`23621`).

* Prevented :djadmin:`flush` from loading initial data for migrated apps
(:ticket:`23699`).

* Fixed a :djadmin:`makemessages` regression in 1.7.1 when
:setting:`STATIC_ROOT` has the default ``None`` value (:ticket:`23717`).

* Added GeoDjango compatibility with mysqlclient database driver.

* Fixed MySQL 5.6+ crash with ``GeometryField``\s in migrations
(:ticket:`23719`).

* Fixed a migration crash when removing a field that is referenced in
``AlterIndexTogether`` or ``AlterUniqueTogether`` (:ticket:`23614`).

* Updated the first day of the week in the Ukrainian locale to Monday.

* Added support for transactional spatial metadata initialization on
SpatiaLite 4.1+ (:ticket:`23152`).

* Fixed a migration crash that prevented changing a nullable field with a
default to non-nullable with the same default (:ticket:`23738`).

* Fixed a migration crash when adding ``GeometryField``\s with ``blank=True``
on PostGIS (:ticket:`23731`).

* Allowed usage of ``DateTimeField()`` as ``Transform.output_field``
(:ticket:`23420`).

* Fixed a migration serializing bug involving ``float("nan")`` and
``float("inf")`` (:ticket:`23770`).

* Fixed a regression where custom form fields having a ``queryset`` attribute
but no ``limit_choices_to`` could not be used in a
:class:`~django.forms.ModelForm` (:ticket:`23795`).

* Fixed a custom field type validation error with MySQL backend when
``db_type`` returned ``None`` (:ticket:`23761`).

* Fixed a migration crash when a field is renamed that is part of an
``index_together`` (:ticket:`23859`).

* Fixed :djadmin:`squashmigrations` to respect the ``--no-optimize`` parameter
(:ticket:`23799`).

* Made :class:`~django.db.migrations.operations.RenameModel` reversible
(:ticket:`22248`)

* Avoided unnecessary rollbacks of migrations from other apps when migrating
backwards (:ticket:`23410`).

* Fixed a rare query error when using deeply nested subqueries
(:ticket:`23605`).

* Fixed a crash in migrations when deleting a field that is part of a
``index/unique_together`` constraint (:ticket:`23794`).

* Fixed ``django.core.files.File.__repr__()`` when the file's ``name`` contains
Unicode characters (:ticket:`23888`).

* Added missing context to the admin's ``delete_selected`` view that prevented
custom site header, etc. from appearing (:ticket:`23898`).

* Fixed a regression with dynamically generated inlines and allowed field
references in the admin (:ticket:`23754`).

* Fixed an infinite loop bug for certain cyclic migration dependencies, and made
the error message for cyclic dependencies much more helpful.

* Added missing ``index_together`` handling for SQLite (:ticket:`23880`).

* Fixed a crash when ``RunSQL`` SQL content was collected by the schema editor,
typically when using ``sqlmigrate`` (:ticket:`23909`).

* Fixed a regression in ``contrib.admin`` add/change views which caused some
``ModelAdmin`` methods to receive the incorrect ``obj`` value
(:ticket:`23934`).

* Fixed ``runserver`` crash when socket error message contained Unicode
characters (:ticket:`23946`).

* Fixed serialization of ``type`` when adding a ``deconstruct()`` method
(:ticket:`23950`).

* Prevented the
``django.contrib.auth.middleware.SessionAuthenticationMiddleware`` from
setting a ``"Vary: Cookie"`` header on all responses (:ticket:`23939`).

* Fixed a crash when adding ``blank=True`` to ``TextField()`` on MySQL
(:ticket:`23920`).

* Fixed index creation by the migration infrastructure, particularly when
dealing with PostgreSQL specific ``{text|varchar}_pattern_ops`` indexes
(:ticket:`23954`).

* Fixed bug in ``makemigrations`` that created broken migration files when
dealing with multiple table inheritance and inheriting from more than one
model (:ticket:`23956`).

* Fixed a crash when a ``MultiValueField`` has invalid data (:ticket:`23674`).

* Fixed a crash in the admin when using "Save as new" and also deleting a
related inline (:ticket:`23857`).

* Always converted ``related_name`` to text (Unicode), since that is required
on Python 3 for interpolation. Removed conversion of ``related_name`` to text
in migration deconstruction (:ticket:`23455` and :ticket:`23982`).

* Enlarged the sizes of tablespaces which are created by default for testing
on Oracle (the main tablespace was increased from 200M to 300M and the
temporary tablespace from 100M to 150M). This was required to accommodate
growth in Django's own test suite (:ticket:`23969`).

* Fixed ``timesince`` filter translations in Korean (:ticket:`23989`).

* Fixed the SQLite ``SchemaEditor`` to properly add defaults in the absence of
a user specified ``default``. For example, a ``CharField`` with ``blank=True``
didn't set existing rows to an empty string which resulted in a crash when
adding the ``NOT NULL`` constraint (:ticket:`23987`).

* ``makemigrations`` no longer prompts for a default value when adding
``TextField()`` or ``CharField()`` without a ``default`` (:ticket:`23405`).

* Fixed a migration crash when adding ``order_with_respect_to`` to a table
with existing rows (:ticket:`23983`).

* Restored the ``pre_migrate`` signal if all apps have migrations
(:ticket:`23975`).

* Made admin system checks run for custom ``AdminSite``\s (:ticket:`23497`).

* Ensured the app registry is fully populated when unpickling models. When an
external script (like a queueing infrastructure) reloads pickled models, it
could crash with an ``AppRegistryNotReady`` exception (:ticket:`24007`).

* Added quoting to field indexes in the SQL generated by migrations to prevent
a crash when the index name requires it (:ticket:`24015`).

* Added ``datetime.time`` support to migrations questioner (:ticket:`23998`).

* Fixed admindocs crash on apps installed as eggs (:ticket:`23525`).

* Changed migrations autodetector to generate an ``AlterModelOptions`` operation
instead of ``DeleteModel`` and ``CreateModel`` operations when changing
``Meta.managed``. This prevents data loss when changing ``managed`` from
``False`` to ``True`` and vice versa (:ticket:`24037`).

* Enabled the ``sqlsequencereset`` command on apps with migrations
(:ticket:`24054`).

* Added tablespace SQL to apps with migrations (:ticket:`24051`).

* Corrected ``contrib.sites`` default site creation in a multiple database
setup (:ticket:`24000`).

* Restored support for objects that aren't :class:`str` or :class:`bytes` in
``django.utils.safestring.mark_for_escaping()`` on Python 3.

* Supported strings escaped by third-party libraries with the ``__html__``
convention in the template engine (:ticket:`23831`).

* Prevented extraneous ``DROP DEFAULT`` SQL in migrations (:ticket:`23581`).

* Restored the ability to use more than five levels of subqueries
(:ticket:`23758`).

* Fixed crash when ``ValidationError`` is initialized with a ``ValidationError``
that is initialized with a dictionary (:ticket:`24008`).

* Prevented a crash on apps without migrations when running ``migrate --list``
(:ticket:`23366`).


==========================

1.7.1

Not secure
==========================

*October 22, 2014*

Django 1.7.1 fixes several bugs in 1.7.

Bugfixes
========

* Allowed related many-to-many fields to be referenced in the admin
(:ticket:`23604`).

* Added a more helpful error message if you try to migrate an app without first
creating the ``contenttypes`` table (:ticket:`22411`).

* Modified migrations dependency algorithm to avoid possible infinite recursion.

* Fixed a ``UnicodeDecodeError`` when the ``flush`` error message contained
Unicode characters (:ticket:`22882`).

* Reinstated missing ``CHECK`` SQL clauses which were omitted on some backends
when not using migrations (:ticket:`23416`).

* Fixed serialization of ``type`` objects in migrations (:ticket:`22951`).

* Allowed inline and hidden references to admin fields (:ticket:`23431`).

* The ``deconstructible`` decorator now fails with a ``ValueError`` if the
decorated object cannot automatically be imported (:ticket:`23418`).

* Fixed a typo in an ``inlineformset_factory()`` error message that caused a
crash (:ticket:`23451`).

* Restored the ability to use :setting:`ABSOLUTE_URL_OVERRIDES` with the
``'auth.User'`` model (:ticket:`11775`). As a side effect, the setting now
adds a ``get_absolute_url()`` method to any model that appears in
``ABSOLUTE_URL_OVERRIDES`` but doesn't define ``get_absolute_url()``.

* Avoided masking some ``ImportError`` exceptions during application loading
(:ticket:`22920`).

* Empty ``index_together`` or ``unique_together`` model options no longer
results in infinite migrations (:ticket:`23452`).

* Fixed crash in ``contrib.sitemaps`` if ``lastmod`` returned a ``date`` rather
than a ``datetime`` (:ticket:`23403`).

* Allowed migrations to work with ``app_label``\s that have the same last
part (e.g. ``django.contrib.auth`` and ``vendor.auth``) (:ticket:`23483`).

* Restored the ability to deepcopy ``F`` objects (:ticket:`23492`).

* Formats for Welsh (``cy``) and several Chinese locales (``zh_CN``,
``zh_Hans``, ``zh_Hant`` and ``zh_TW``) have been added. Formats for
Macedonian have been fixed (trailing dot removed, :ticket:`23532`).

* Added quoting of constraint names in the SQL generated by migrations to
prevent crash with uppercase characters in the name (:ticket:`23065`).

* Fixed renaming of models with a self-referential many-to-many field
(``ManyToManyField('self')``) (:ticket:`23503`).

* Added the :meth:`~django.contrib.admin.InlineModelAdmin.get_extra()`,
:meth:`~django.contrib.admin.InlineModelAdmin.get_max_num()`, and
:meth:`~django.contrib.admin.InlineModelAdmin.get_min_num()` hooks to
:class:`~django.contrib.contenttypes.admin.GenericInlineModelAdmin`
(:ticket:`23539`).

* Made ``migrations.RunSQL`` no longer require percent sign escaping. This is
now consistent with ``cursor.execute()`` (:ticket:`23426`).

* Made the :setting:`SERIALIZE <TEST_SERIALIZE>` entry in the
:setting:`TEST <DATABASE-TEST>` dictionary usable (:ticket:`23421`).

* Fixed bug in migrations that prevented foreign key constraints to unmanaged
models with a custom primary key (:ticket:`23415`).

* Added ``SchemaEditor`` for MySQL GIS backend so that spatial indexes will be
created for apps with migrations (:ticket:`23538`).

* Added ``SchemaEditor`` for Oracle GIS backend so that spatial metadata and
indexes will be created for apps with migrations (:ticket:`23537`).

* Coerced the ``related_name`` model field option to Unicode during migration
generation to generate migrations that work with both Python 2 and 3
(:ticket:`23455`).

* Fixed ``MigrationWriter`` to handle builtin types without imports
(:ticket:`23560`).

* Fixed ``deepcopy`` on ``ErrorList`` (:ticket:`23594`).

* Made the :mod:`~django.contrib.admindocs` view to browse view details check
if the view specified in the URL exists in the URLconf. Previously it was
possible to import arbitrary packages from the Python path. This was not
considered a security issue because ``admindocs`` is only accessible to staff
users (:ticket:`23601`).

* Fixed ``UnicodeDecodeError`` crash in ``AdminEmailHandler`` with non-ASCII
characters in the request (:ticket:`23593`).

* Fixed missing ``get_or_create`` and ``update_or_create`` on related managers
causing ``IntegrityError`` (:ticket:`23611`).

* Made :func:`~django.utils.http.urlsafe_base64_decode` return the proper
type (bytestring) on Python 3 (:ticket:`23333`).

* :djadmin:`makemigrations` can now serialize timezone-aware values
(:ticket:`23365`).

* Added a prompt to the migrations questioner when removing the null constraint
from a field to prevent an IntegrityError on existing NULL rows
(:ticket:`23609`).

* Fixed generic relations in ``ModelAdmin.list_filter`` (:ticket:`23616`).

* Restored RFC compliance for the SMTP backend on Python 3 (:ticket:`23063`).

* Fixed a crash while parsing cookies containing invalid content
(:ticket:`23638`).

* The system check framework now raises error **models.E020** when the
class method ``Model.check()`` is unreachable (:ticket:`23615`).

* Made the Oracle test database creation drop the test user in the event of an
unclean exit of a previous test run (:ticket:`23649`).

* Fixed :djadmin:`makemigrations` to detect changes to
:attr:`Meta.db_table <django.db.models.Options.db_table>` (:ticket:`23629`).

* Fixed a regression when feeding the Django test client with an empty data
string (:ticket:`21740`).

* Fixed a regression in :djadmin:`makemessages` where static files were
unexpectedly ignored (:ticket:`23583`).


========================

1.7

Not secure
========================

*September 2, 2014*

Welcome to Django 1.7!

These release notes cover the :ref:`new features <whats-new-1.7>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-1.7>` you'll
want to be aware of when upgrading from Django 1.6 or older versions. We've
:ref:`begun the deprecation process for some features
<deprecated-features-1.7>`, and some features have reached the end of their
deprecation process and :ref:`have been removed <removed-features-1.7>`.

Python compatibility
====================

Django 1.7 requires Python 2.7, 3.2, 3.3, or 3.4. We **highly recommend** and
only officially support the latest release of each series.

The Django 1.6 series is the last to support Python 2.6. Django 1.7 is the
first release to support Python 3.4.

This change should affect only a small number of Django users, as most
operating-system vendors today are shipping Python 2.7 or newer as their default
version. If you're still using Python 2.6, however, you'll need to stick to
Django 1.6 until you can upgrade your Python version. Per :doc:`our support
policy </internals/release-process>`, Django 1.6 will continue to receive
security support until the release of Django 1.8.

.. _whats-new-1.7:

What's new in Django 1.7
========================

Schema migrations
-----------------

Django now has built-in support for schema migrations. It allows models
to be updated, changed, and deleted by creating migration files that represent
the model changes and which can be run on any development, staging or production
database.

Migrations are covered in :doc:`their own documentation</topics/migrations>`,
but a few of the key features are:

* ``syncdb`` has been deprecated and replaced by ``migrate``. Don't worry -
calls to ``syncdb`` will still work as before.

* A new ``makemigrations`` command provides an easy way to autodetect changes
to your models and make migrations for them.

``django.db.models.signals.pre_syncdb`` and
``django.db.models.signals.post_syncdb`` have been deprecated,
to be replaced by :data:`~django.db.models.signals.pre_migrate` and
:data:`~django.db.models.signals.post_migrate` respectively. These
new signals have slightly different arguments. Check the
documentation for details.

* The ``allow_syncdb`` method on database routers is now called ``allow_migrate``,
but still performs the same function. Routers with ``allow_syncdb`` methods
will still work, but that method name is deprecated and you should change
it as soon as possible (nothing more than renaming is required).

* ``initial_data`` fixtures are no longer loaded for apps with migrations; if
you want to load initial data for an app, we suggest you create a migration for
your application and define a :class:`~django.db.migrations.operations.RunPython`
or :class:`~django.db.migrations.operations.RunSQL` operation in the ``operations`` section of the migration.

* Test rollback behavior is different for apps with migrations; in particular,
Django will no longer emulate rollbacks on non-transactional databases or
inside ``TransactionTestCase`` :ref:`unless specifically requested
<test-case-serialized-rollback>`.

* It is not advised to have apps without migrations depend on (have a
:class:`~django.db.models.ForeignKey` or
:class:`~django.db.models.ManyToManyField` to) apps with migrations.

.. _app-loading-refactor-17-release-note:

App-loading refactor
--------------------

Historically, Django applications were tightly linked to models. A singleton
known as the "app cache" dealt with both installed applications and models.
The models module was used as an identifier for applications in many APIs.

As the concept of :doc:`Django applications </ref/applications>` matured, this
code showed some shortcomings. It has been refactored into an "app registry"
where models modules no longer have a central role and where it's possible to
attach configuration data to applications.

Improvements thus far include:

* Applications can run code at startup, before Django does anything else, with
the :meth:`~django.apps.AppConfig.ready` method of their configuration.

* Application labels are assigned correctly to models even when they're
defined outside of ``models.py``. You don't have to set
:attr:`~django.db.models.Options.app_label` explicitly any more.

* It is possible to omit ``models.py`` entirely if an application doesn't
have any models.

* Applications can be relabeled with the :attr:`~django.apps.AppConfig.label`
attribute of application configurations, to work around label conflicts.

* The name of applications can be customized in the admin with the
:attr:`~django.apps.AppConfig.verbose_name` of application configurations.

* The admin automatically calls :func:`~django.contrib.admin.autodiscover()`
when Django starts. You can consequently remove this line from your
URLconf.

* Django imports all application configurations and models as soon as it
starts, through a deterministic and straightforward process. This should
make it easier to diagnose import issues such as import loops.

New method on Field subclasses
------------------------------

To help power both schema migrations and to enable easier addition of
composite keys in future releases of Django, the
:class:`~django.db.models.Field` API now has a new required method:
``deconstruct()``.

This method takes no arguments, and returns a tuple of four items:

* ``name``: The field's attribute name on its parent model, or ``None`` if it
is not part of a model
* ``path``: A dotted, Python path to the class of this field, including the class name.
* ``args``: Positional arguments, as a list
* ``kwargs``: Keyword arguments, as a dict

These four values allow any field to be serialized into a file, as well as
allowing the field to be copied safely, both essential parts of these new features.

This change should not affect you unless you write custom Field subclasses;
if you do, you may need to reimplement the ``deconstruct()`` method if your
subclass changes the method signature of ``__init__`` in any way. If your
field just inherits from a built-in Django field and doesn't override ``__init__``,
no changes are necessary.

If you do need to override ``deconstruct()``, a good place to start is the
built-in Django fields (``django/db/models/fields/__init__.py``) as several
fields, including ``DecimalField`` and ``DateField``, override it and show how
to call the method on the superclass and simply add or remove extra arguments.

This also means that all arguments to fields must themselves be serializable;
to see what we consider serializable, and to find out how to make your own
classes serializable, read the
:ref:`migration serialization documentation <migration-serializing>`.

Calling custom ``QuerySet`` methods from the ``Manager``
--------------------------------------------------------

Historically, the recommended way to make reusable model queries was to create
methods on a custom ``Manager`` class. The problem with this approach was that
after the first method call, you'd get back a ``QuerySet`` instance and
couldn't call additional custom manager methods.

Though not documented, it was common to work around this issue by creating a
custom ``QuerySet`` so that custom methods could be chained; but the solution
had a number of drawbacks:

* The custom ``QuerySet`` and its custom methods were lost after the first
call to ``values()`` or ``values_list()``.

* Writing a custom ``Manager`` was still necessary to return the custom
``QuerySet`` class and all methods that were desired on the ``Manager``
had to be proxied to the ``QuerySet``. The whole process went against
the DRY principle.

The :meth:`QuerySet.as_manager() <django.db.models.query.QuerySet.as_manager>`
class method can now directly :ref:`create Manager with QuerySet methods
<create-manager-with-queryset-methods>`::

class FoodQuerySet(models.QuerySet):
def pizzas(self):
return self.filter(kind='pizza')

def vegetarian(self):
return self.filter(vegetarian=True)

class Food(models.Model):
kind = models.CharField(max_length=50)
vegetarian = models.BooleanField(default=False)
objects = FoodQuerySet.as_manager()

Food.objects.pizzas().vegetarian()

Using a custom manager when traversing reverse relations
--------------------------------------------------------

It is now possible to :ref:`specify a custom manager
<using-custom-reverse-manager>` when traversing a reverse relationship::

class Blog(models.Model):
pass

class Entry(models.Model):
blog = models.ForeignKey(Blog)

objects = models.Manager() Default Manager
entries = EntryManager() Custom Manager

b = Blog.objects.get(id=1)
b.entry_set(manager='entries').all()

New system check framework
--------------------------

We've added a new :doc:`System check framework </ref/checks>` for
detecting common problems (like invalid models) and providing hints for
resolving those problems. The framework is extensible so you can add your
own checks for your own apps and libraries.

To perform system checks, you use the :djadmin:`check` management command.
This command replaces the older ``validate`` management command.

New ``Prefetch`` object for advanced ``prefetch_related`` operations.
---------------------------------------------------------------------

The new :class:`~django.db.models.Prefetch` object allows customizing
prefetch operations.

You can specify the ``QuerySet`` used to traverse a given relation
or customize the storage location of prefetch results.

This enables things like filtering prefetched relations, calling
:meth:`~django.db.models.query.QuerySet.select_related()` from a prefetched
relation, or prefetching the same relation multiple times with different
querysets. See :meth:`~django.db.models.query.QuerySet.prefetch_related()`
for more details.

Admin shortcuts support time zones
----------------------------------

The "today" and "now" shortcuts next to date and time input widgets in the
admin are now operating in the :ref:`current time zone
<default-current-time-zone>`. Previously, they used the browser time zone,
which could result in saving the wrong value when it didn't match the current
time zone on the server.

In addition, the widgets now display a help message when the browser and
server time zone are different, to clarify how the value inserted in the field
will be interpreted.

Using database cursors as context managers
------------------------------------------

Prior to Python 2.7, database cursors could be used as a context manager. The
specific backend's cursor defined the behavior of the context manager. The
behavior of magic method lookups was changed with Python 2.7 and cursors were
no longer usable as context managers.

Django 1.7 allows a cursor to be used as a context manager. That is,
the following can be used::

with connection.cursor() as c:
c.execute(...)

instead of::

c = connection.cursor()
try:
c.execute(...)
finally:
c.close()

Custom lookups
--------------

It is now possible to write custom lookups and transforms for the ORM.
Custom lookups work just like Django's built-in lookups (e.g. ``lte``,
``icontains``) while transforms are a new concept.

The :class:`django.db.models.Lookup` class provides a way to add lookup
operators for model fields. As an example it is possible to add ``day_lte``
operator for ``DateFields``.

The :class:`django.db.models.Transform` class allows transformations of
database values prior to the final lookup. For example it is possible to
write a ``year`` transform that extracts year from the field's value.
Transforms allow for chaining. After the ``year`` transform has been added
to ``DateField`` it is possible to filter on the transformed value, for
example ``qs.filter(author__birthdate__year__lte=1981)``.

For more information about both custom lookups and transforms refer to
the :doc:`custom lookups </howto/custom-lookups>` documentation.

Improvements to ``Form`` error handling
---------------------------------------

``Form.add_error()``
~~~~~~~~~~~~~~~~~~~~

Previously there were two main patterns for handling errors in forms:

* Raising a :exc:`~django.core.exceptions.ValidationError` from within certain
functions (e.g. ``Field.clean()``, ``Form.clean_<fieldname>()``, or
``Form.clean()`` for non-field errors.)

* Fiddling with ``Form._errors`` when targeting a specific field in
``Form.clean()`` or adding errors from outside of a "clean" method
(e.g. directly from a view).

Using the former pattern was straightforward since the form can guess from the
context (i.e. which method raised the exception) where the errors belong and
automatically process them. This remains the canonical way of adding errors
when possible. However the latter was fiddly and error-prone, since the burden
of handling edge cases fell on the user.

The new :meth:`~django.forms.Form.add_error()` method allows adding errors
to specific form fields from anywhere without having to worry about the details
such as creating instances of ``django.forms.utils.ErrorList`` or dealing with
``Form.cleaned_data``. This new API replaces manipulating ``Form._errors``
which now becomes a private API.

See :ref:`validating-fields-with-clean` for an example using
``Form.add_error()``.

Error metadata
~~~~~~~~~~~~~~

The :exc:`~django.core.exceptions.ValidationError` constructor accepts metadata
such as error ``code`` or ``params`` which are then available for interpolating
into the error message (see :ref:`raising-validation-error` for more details);
however, before Django 1.7 those metadata were discarded as soon as the errors
were added to :attr:`Form.errors <django.forms.Form.errors>`.

:attr:`Form.errors <django.forms.Form.errors>` and
``django.forms.utils.ErrorList`` now store the ``ValidationError`` instances
so these metadata can be retrieved at any time through the new
:meth:`Form.errors.as_data <django.forms.Form.errors.as_data()>` method.

The retrieved ``ValidationError`` instances can then be identified thanks to
their error ``code`` which enables things like rewriting the error's message
or writing custom logic in a view when a given error is present. It can also
be used to serialize the errors in a custom format such as XML.

The new :meth:`Form.errors.as_json() <django.forms.Form.errors.as_json()>`
method is a convenience method which returns error messages along with error
codes serialized as JSON. ``as_json()`` uses ``as_data()`` and gives an idea
of how the new system could be extended.

Error containers and backward compatibility
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Heavy changes to the various error containers were necessary in order
to support the features above, specifically
:attr:`Form.errors <django.forms.Form.errors>`,
``django.forms.utils.ErrorList``, and the internal storages of
:exc:`~django.core.exceptions.ValidationError`. These containers which used
to store error strings now store ``ValidationError`` instances and public APIs
have been adapted to make this as transparent as possible, but if you've been
using private APIs, some of the changes are backwards incompatible; see
:ref:`validation-error-constructor-and-internal-storage` for more details.

Minor features
--------------

:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* You can now implement :attr:`~django.contrib.admin.AdminSite.site_header`,
:attr:`~django.contrib.admin.AdminSite.site_title`, and
:attr:`~django.contrib.admin.AdminSite.index_title` attributes on a custom
:class:`~django.contrib.admin.AdminSite` in order to easily change the admin
site's page title and header text. No more needing to override templates!

* Buttons in :mod:`django.contrib.admin` now use the ``border-radius`` CSS
property for rounded corners rather than GIF background images.

* Some admin templates now have ``app-<app_name>`` and ``model-<model_name>``
classes in their ``<body>`` tag to allow customizing the CSS per app or per
model.

* The admin changelist cells now have a ``field-<field_name>`` class in the
HTML to enable style customizations.

* The admin's search fields can now be customized per-request thanks to the new
:meth:`django.contrib.admin.ModelAdmin.get_search_fields` method.

* The :meth:`ModelAdmin.get_fields()
<django.contrib.admin.ModelAdmin.get_fields>` method may be overridden to
customize the value of :attr:`ModelAdmin.fields
<django.contrib.admin.ModelAdmin.fields>`.

* In addition to the existing ``admin.site.register`` syntax, you can use the
new :func:`~django.contrib.admin.register` decorator to register a
:class:`~django.contrib.admin.ModelAdmin`.

* You may specify :meth:`ModelAdmin.list_display_links
<django.contrib.admin.ModelAdmin.list_display_links>` ``= None`` to disable
links on the change list page grid.

* You may now specify :attr:`ModelAdmin.view_on_site
<django.contrib.admin.ModelAdmin.view_on_site>` to control whether or not to
display the "View on site" link.

* You can specify a descending ordering for a :attr:`ModelAdmin.list_display
<django.contrib.admin.ModelAdmin.list_display>` value by prefixing the
``admin_order_field`` value with a hyphen.

* The :meth:`ModelAdmin.get_changeform_initial_data()
<django.contrib.admin.ModelAdmin.get_changeform_initial_data>` method may be
overridden to define custom behavior for setting initial change form data.

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~

* Any ``**kwargs`` passed to
:meth:`~django.contrib.auth.models.User.email_user()` are passed to the
underlying :meth:`~django.core.mail.send_mail()` call.

* The :func:`~django.contrib.auth.decorators.permission_required` decorator can
take a list of permissions as well as a single permission.

* You can override the new :meth:`AuthenticationForm.confirm_login_allowed()
<django.contrib.auth.forms.AuthenticationForm.confirm_login_allowed>` method
to more easily customize the login policy.

* ``django.contrib.auth.views.password_reset()`` takes an optional
``html_email_template_name`` parameter used to send a multipart HTML email
for password resets.

* The :meth:`AbstractBaseUser.get_session_auth_hash()
<django.contrib.auth.models.AbstractBaseUser.get_session_auth_hash>`
method was added and if your :setting:`AUTH_USER_MODEL` inherits from
:class:`~django.contrib.auth.models.AbstractBaseUser`, changing a user's
password now invalidates old sessions if the
``django.contrib.auth.middleware.SessionAuthenticationMiddleware`` is
enabled. See :ref:`session-invalidation-on-password-change` for more details.

``django.contrib.formtools``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Calls to ``WizardView.done()`` now include a ``form_dict`` to allow easier
access to forms by their step name.

:mod:`django.contrib.gis`
~~~~~~~~~~~~~~~~~~~~~~~~~

* The default OpenLayers library version included in widgets has been updated
from 2.11 to 2.13.

* Prepared geometries now also support the ``crosses``, ``disjoint``,
``overlaps``, ``touches`` and ``within`` predicates, if GEOS 3.3 or later is
installed.

:mod:`django.contrib.messages`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The backends for :mod:`django.contrib.messages` that use cookies, will now
follow the :setting:`SESSION_COOKIE_SECURE` and
:setting:`SESSION_COOKIE_HTTPONLY` settings.

* The :ref:`messages context processor <message-displaying>` now adds a
dictionary of default levels under the name ``DEFAULT_MESSAGE_LEVELS``.

* :class:`~django.contrib.messages.storage.base.Message` objects now have a
``level_tag`` attribute that contains the string representation of the
message level.

:mod:`django.contrib.redirects`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* :class:`~django.contrib.redirects.middleware.RedirectFallbackMiddleware`
has two new attributes
(:attr:`~django.contrib.redirects.middleware.RedirectFallbackMiddleware.response_gone_class`
and
:attr:`~django.contrib.redirects.middleware.RedirectFallbackMiddleware.response_redirect_class`)
that specify the types of :class:`~django.http.HttpResponse` instances the
middleware returns.

:mod:`django.contrib.sessions`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The ``"django.contrib.sessions.backends.cached_db"`` session backend now
respects :setting:`SESSION_CACHE_ALIAS`. In previous versions, it always used
the ``default`` cache.

:mod:`django.contrib.sitemaps`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The :mod:`sitemap framework<django.contrib.sitemaps>` now makes use of
:attr:`~django.contrib.sitemaps.Sitemap.lastmod` to set a ``Last-Modified``
header in the response. This makes it possible for the
:class:`~django.middleware.http.ConditionalGetMiddleware` to handle
conditional ``GET`` requests for sitemaps which set ``lastmod``.

:mod:`django.contrib.sites`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The new :class:`django.contrib.sites.middleware.CurrentSiteMiddleware` allows
setting the current site on each request.

:mod:`django.contrib.staticfiles`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The :ref:`static files storage classes <staticfiles-storages>` may be
subclassed to override the permissions that collected static files and
directories receive by setting the
:attr:`~django.core.files.storage.FileSystemStorage.file_permissions_mode`
and :attr:`~django.core.files.storage.FileSystemStorage.directory_permissions_mode`
parameters. See :djadmin:`collectstatic` for example usage.

* The ``CachedStaticFilesStorage`` backend gets a sibling class called
:class:`~django.contrib.staticfiles.storage.ManifestStaticFilesStorage`
that doesn't use the cache system at all but instead a JSON file called
``staticfiles.json`` for storing the mapping between the original file name
(e.g. ``css/styles.css``) and the hashed file name (e.g.
``css/styles.55e7cbb9ba48.css``). The ``staticfiles.json`` file is created
when running the :djadmin:`collectstatic` management command and should
be a less expensive alternative for remote storages such as Amazon S3.

See the :class:`~django.contrib.staticfiles.storage.ManifestStaticFilesStorage`
docs for more information.

* :djadmin:`findstatic` now accepts verbosity flag level 2, meaning it will
show the relative paths of the directories it searched. See
:djadmin:`findstatic` for example output.

:mod:`django.contrib.syndication`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* The :class:`~django.utils.feedgenerator.Atom1Feed` syndication feed's
``updated`` element now utilizes ``updateddate`` instead of ``pubdate``,
allowing the ``published`` element to be included in the feed (which
relies on ``pubdate``).

Cache
~~~~~

* Access to caches configured in :setting:`CACHES` is now available via
:data:`django.core.cache.caches`. This dict-like object provides a different
instance per thread. It supersedes ``django.core.cache.get_cache()`` which
is now deprecated.

* If you instantiate cache backends directly, be aware that they aren't
thread-safe any more, as :data:`django.core.cache.caches` now yields
different instances per thread.

* Defining the :setting:`TIMEOUT <CACHES-TIMEOUT>` argument of the
:setting:`CACHES` setting as ``None`` will set the cache keys as
"non-expiring" by default. Previously, it was only possible to pass
``timeout=None`` to the cache backend's ``set()`` method.

Cross Site Request Forgery
~~~~~~~~~~~~~~~~~~~~~~~~~~

* The :setting:`CSRF_COOKIE_AGE` setting facilitates the use of session-based
CSRF cookies.

Email
~~~~~

* :func:`~django.core.mail.send_mail` now accepts an ``html_message``
parameter for sending a multipart :mimetype:`text/plain` and
:mimetype:`text/html` email.

* The SMTP :class:`~django.core.mail.backends.smtp.EmailBackend` now accepts a
``timeout`` parameter.

File Storage
~~~~~~~~~~~~

* File locking on Windows previously depended on the PyWin32 package; if it
wasn't installed, file locking failed silently. That dependency has been
removed, and file locking is now implemented natively on both Windows
and Unix.

File Uploads
~~~~~~~~~~~~

* The new :attr:`UploadedFile.content_type_extra
<django.core.files.uploadedfile.UploadedFile.content_type_extra>` attribute
contains extra parameters passed to the ``content-type`` header on a file
upload.

* The new :setting:`FILE_UPLOAD_DIRECTORY_PERMISSIONS` setting controls
the file system permissions of directories created during file upload, like
:setting:`FILE_UPLOAD_PERMISSIONS` does for the files themselves.

* The :attr:`FileField.upload_to <django.db.models.FileField.upload_to>`
attribute is now optional. If it is omitted or given ``None`` or an empty
string, a subdirectory won't be used for storing the uploaded files.

* Uploaded files are now explicitly closed before the response is delivered to
the client. Partially uploaded files are also closed as long as they are
named ``file`` in the upload handler.

* :meth:`Storage.get_available_name()
<django.core.files.storage.Storage.get_available_name>` now appends an
underscore plus a random 7 character alphanumeric string (e.g.
``"_x3a1gho"``), rather than iterating through an underscore followed by a
number (e.g. ``"_1"``, ``"_2"``, etc.) to prevent a denial-of-service attack.
This change was also made in the 1.6.6, 1.5.9, and 1.4.14 security releases.

Forms
~~~~~

* The ``<label>`` and ``<input>`` tags rendered by
:class:`~django.forms.RadioSelect` and
:class:`~django.forms.CheckboxSelectMultiple` when looping over the radio
buttons or checkboxes now include ``for`` and ``id`` attributes, respectively.
Each radio button or checkbox includes an ``id_for_label`` attribute to
output the element's ID.

* The ``<textarea>`` tags rendered by :class:`~django.forms.Textarea` now
include a ``maxlength`` attribute if the :class:`~django.db.models.TextField`
model field has a ``max_length``.

* :attr:`Field.choices<django.db.models.Field.choices>` now allows you to
customize the "empty choice" label by including a tuple with an empty string
or ``None`` for the key and the custom label as the value. The default blank
option ``"----------"`` will be omitted in this case.

* :class:`~django.forms.MultiValueField` allows optional subfields by setting
the ``require_all_fields`` argument to ``False``. The ``required`` attribute
for each individual field will be respected, and a new ``incomplete``
validation error will be raised when any required fields are empty.

* The :meth:`~django.forms.Form.clean` method on a form no longer needs to
return ``self.cleaned_data``. If it does return a changed dictionary then
that will still be used.

* After a temporary regression in Django 1.6, it's now possible again to make
:class:`~django.forms.TypedChoiceField` ``coerce`` method return an arbitrary
value.

* :attr:`SelectDateWidget.months
<django.forms.SelectDateWidget.months>` can be used to
customize the wording of the months displayed in the select widget.

* The ``min_num`` and ``validate_min`` parameters were added to
:func:`~django.forms.formsets.formset_factory` to allow validating
a minimum number of submitted forms.

* The metaclasses used by ``Form`` and ``ModelForm`` have been reworked to
support more inheritance scenarios. The previous limitation that prevented
inheriting from both ``Form`` and ``ModelForm`` simultaneously have been
removed as long as ``ModelForm`` appears first in the MRO.

* It's now possible to remove a field from a ``Form`` when subclassing by
setting the name to ``None``.

* It's now possible to customize the error messages for ``ModelForm``’s
``unique``, ``unique_for_date``, and ``unique_together`` constraints.
In order to support ``unique_together`` or any other ``NON_FIELD_ERROR``,
``ModelForm`` now looks for the ``NON_FIELD_ERROR`` key in the
``error_messages`` dictionary of the ``ModelForm``’s inner ``Meta`` class.
See :ref:`considerations regarding model's error_messages
<considerations-regarding-model-errormessages>` for more details.

Internationalization
~~~~~~~~~~~~~~~~~~~~

* The :attr:`django.middleware.locale.LocaleMiddleware.response_redirect_class`
attribute allows you to customize the redirects issued by the middleware.

* The :class:`~django.middleware.locale.LocaleMiddleware` now stores the user's
selected language with the session key ``_language``. This should only be
accessed using the ``LANGUAGE_SESSION_KEY`` constant. Previously it was
stored with the key ``django_language`` and the ``LANGUAGE_SESSION_KEY``
constant did not exist, but keys reserved for Django should start with an
underscore. For backwards compatibility ``django_language`` is still read
from in 1.7. Sessions will be migrated to the new key as they are written.

* The :ttag:`blocktrans` tag now supports a ``trimmed`` option. This
option will remove newline characters from the beginning and the end of the
content of the ``{% blocktrans %}`` tag, replace any whitespace at the
beginning and end of a line and merge all lines into one using a space
character to separate them. This is quite useful for indenting the content of
a ``{% blocktrans %}`` tag without having the indentation characters end up
in the corresponding entry in the ``.po`` file, which makes the translation
process easier.

* When you run :djadmin:`makemessages` from the root directory of your project,
any extracted strings will now be automatically distributed to the proper
app or project message file. See :ref:`how-to-create-language-files` for
details.

* The :djadmin:`makemessages` command now always adds the ``--previous``
command line flag to the ``msgmerge`` command, keeping previously translated
strings in ``.po`` files for fuzzy strings.

* The following settings to adjust the language cookie options were introduced:
:setting:`LANGUAGE_COOKIE_AGE`, :setting:`LANGUAGE_COOKIE_DOMAIN`
and :setting:`LANGUAGE_COOKIE_PATH`.

* Added :doc:`/topics/i18n/formatting` for Esperanto.

Management Commands
~~~~~~~~~~~~~~~~~~~

* The new :option:`--no-color` option for ``django-admin`` disables the
colorization of management command output.

* The new :option:`dumpdata --natural-foreign` and :option:`dumpdata
--natural-primary` options, and the new ``use_natural_foreign_keys`` and
``use_natural_primary_keys`` arguments for ``serializers.serialize()``, allow
the use of natural primary keys when serializing.

* It is no longer necessary to provide the cache table name or the
``--database`` option for the :djadmin:`createcachetable` command.
Django takes this information from your settings file. If you have configured
multiple caches or multiple databases, all cache tables are created.

* The :djadmin:`runserver` command received several improvements:

* On Linux systems, if pyinotify_ is installed, the development server will
reload immediately when a file is changed. Previously, it polled the
filesystem for changes every second. That caused a small delay before
reloads and reduced battery life on laptops.

.. _pyinotify: https://pypi.org/project/pyinotify/

* In addition, the development server automatically reloads when a
translation file is updated, i.e. after running
:djadmin:`compilemessages`.

* All HTTP requests are logged to the console, including requests for static
files or ``favicon.ico`` that used to be filtered out.

* Management commands can now produce syntax colored output under Windows if
the ANSICON third-party tool is installed and active.

* :djadmin:`collectstatic` command with symlink option is now supported on
Windows NT 6 (Windows Vista and newer).

* Initial SQL data now works better if the sqlparse_ Python library is
installed.

Note that it's deprecated in favor of the
:class:`~django.db.migrations.operations.RunSQL` operation of migrations,
which benefits from the improved behavior.

.. _sqlparse: https://pypi.org/project/sqlparse/

Models
~~~~~~

* The :meth:`QuerySet.update_or_create()
<django.db.models.query.QuerySet.update_or_create>` method was added.

* The new :attr:`~django.db.models.Options.default_permissions` model
``Meta`` option allows you to customize (or disable) creation of the default
add, change, and delete permissions.

* Explicit :class:`~django.db.models.OneToOneField` for
:ref:`multi-table-inheritance` are now discovered in abstract classes.

* It is now possible to avoid creating a backward relation for
:class:`~django.db.models.OneToOneField` by setting its
:attr:`~django.db.models.ForeignKey.related_name` to
``'+'`` or ending it with ``'+'``.

* :class:`F expressions <django.db.models.F>` support the power operator
(``**``).

* The ``remove()`` and ``clear()`` methods of the related managers created by
``ForeignKey`` and ``GenericForeignKey`` now accept the ``bulk`` keyword
argument to control whether or not to perform operations in bulk
(i.e. using ``QuerySet.update()``). Defaults to ``True``.

* It is now possible to use ``None`` as a query value for the :lookup:`iexact`
lookup.

* It is now possible to pass a callable as value for the attribute
:attr:`~django.db.models.ForeignKey.limit_choices_to` when defining a
``ForeignKey`` or ``ManyToManyField``.

* Calling :meth:`only() <django.db.models.query.QuerySet.only>` and
:meth:`defer() <django.db.models.query.QuerySet.defer>` on the result of
:meth:`QuerySet.values() <django.db.models.query.QuerySet.values>` now raises
an error (before that, it would either result in a database error or
incorrect data).

* You can use a single list for :attr:`~django.db.models.Options.index_together`
(rather than a list of lists) when specifying a single set of fields.

* Custom intermediate models having more than one foreign key to any of the
models participating in a many-to-many relationship are now permitted,
provided you explicitly specify which foreign keys should be used by setting
the new :attr:`ManyToManyField.through_fields <django.db.models.ManyToManyField.through_fields>`
argument.

* Assigning a model instance to a non-relation field will now throw an error.
Previously this used to work if the field accepted integers as input as it
took the primary key.

* Integer fields are now validated against database backend specific min and
max values based on their :meth:`internal_type <django.db.models.Field.get_internal_type>`.
Previously model field validation didn't prevent values out of their associated
column data type range from being saved resulting in an integrity error.

* It is now possible to explicitly :meth:`~django.db.models.query.QuerySet.order_by`
a relation ``_id`` field by using its attribute name.

Signals
~~~~~~~

* The ``enter`` argument was added to the
:data:`~django.test.signals.setting_changed` signal.

* The model signals can be now be connected to using a ``str`` of the
``'app_label.ModelName'`` form – just like related fields – to lazily
reference their senders.

Templates
~~~~~~~~~

* The :meth:`Context.push() <django.template.Context.push>` method now returns
a context manager which automatically calls :meth:`pop()
<django.template.Context.pop>` upon exiting the ``with`` statement.
Additionally, :meth:`push() <django.template.Context.push>` now accepts
parameters that are passed to the ``dict`` constructor used to build the new
context level.

* The new :meth:`Context.flatten() <django.template.Context.flatten>` method
returns a ``Context``'s stack as one flat dictionary.

* ``Context`` objects can now be compared for equality (internally, this
uses :meth:`Context.flatten() <django.template.Context.flatten>` so the
internal structure of each ``Context``'s stack doesn't matter as long as their
flattened version is identical).

* The :ttag:`widthratio` template tag now accepts an ``"as"`` parameter to
capture the result in a variable.

* The :ttag:`include` template tag will now also accept anything with a
``render()`` method (such as a ``Template``) as an argument. String
arguments will be looked up using
:func:`~django.template.loader.get_template` as always.

* It is now possible to :ttag:`include` templates recursively.

* Template objects now have an origin attribute set when
``TEMPLATE_DEBUG`` is ``True``. This allows template origins to be
inspected and logged outside of the ``django.template`` infrastructure.

* ``TypeError`` exceptions are no longer silenced when raised during the
rendering of a template.

* The following functions now accept a ``dirs`` parameter which is a list or
tuple to override ``TEMPLATE_DIRS``:

* :func:`django.template.loader.get_template()`
* :func:`django.template.loader.select_template()`
* :func:`django.shortcuts.render()`
* ``django.shortcuts.render_to_response()``

* The :tfilter:`time` filter now accepts timezone-related :ref:`format
specifiers <date-and-time-formatting-specifiers>` ``'e'``, ``'O'`` , ``'T'``
and ``'Z'`` and is able to digest :ref:`time-zone-aware
<naive_vs_aware_datetimes>` ``datetime`` instances performing the expected
rendering.

* The :ttag:`cache` tag will now try to use the cache called
"template_fragments" if it exists and fall back to using the default cache
otherwise. It also now accepts an optional ``using`` keyword argument to
control which cache it uses.

* The new :tfilter:`truncatechars_html` filter truncates a string to be no
longer than the specified number of characters, taking HTML into account.

Requests and Responses
~~~~~~~~~~~~~~~~~~~~~~

* The new :attr:`HttpRequest.scheme <django.http.HttpRequest.scheme>` attribute
specifies the scheme of the request (``http`` or ``https`` normally).


* The shortcut :func:`redirect() <django.shortcuts.redirect>` now supports
relative URLs.

* The new :class:`~django.http.JsonResponse` subclass of
:class:`~django.http.HttpResponse` helps easily create JSON-encoded responses.

Tests
~~~~~

* :class:`~django.test.runner.DiscoverRunner` has two new attributes,
:attr:`~django.test.runner.DiscoverRunner.test_suite` and
:attr:`~django.test.runner.DiscoverRunner.test_runner`, which facilitate
overriding the way tests are collected and run.

* The ``fetch_redirect_response`` argument was added to
:meth:`~django.test.SimpleTestCase.assertRedirects`. Since the test
client can't fetch externals URLs, this allows you to use ``assertRedirects``
with redirects that aren't part of your Django app.

* Correct handling of scheme when making comparisons in
:meth:`~django.test.SimpleTestCase.assertRedirects`.

* The ``secure`` argument was added to all the request methods of
:class:`~django.test.Client`. If ``True``, the request will be made
through HTTPS.

* :meth:`~django.test.TransactionTestCase.assertNumQueries` now prints
out the list of executed queries if the assertion fails.

* The ``WSGIRequest`` instance generated by the test handler is now attached to
the :attr:`django.test.Response.wsgi_request` attribute.

* The database settings for testing have been collected into a dictionary
named :setting:`TEST <DATABASE-TEST>`.

Utilities
~~~~~~~~~

* Improved :func:`~django.utils.html.strip_tags` accuracy (but it still cannot
guarantee an HTML-safe result, as stated in the documentation).

Validators
~~~~~~~~~~

* :class:`~django.core.validators.RegexValidator` now accepts the optional
:attr:`~django.core.validators.RegexValidator.flags` and
Boolean :attr:`~django.core.validators.RegexValidator.inverse_match` arguments.
The :attr:`~django.core.validators.RegexValidator.inverse_match` attribute
determines if the :exc:`~django.core.exceptions.ValidationError` should
be raised when the regular expression pattern matches (``True``) or does not
match (``False``, by default) the provided ``value``. The
:attr:`~django.core.validators.RegexValidator.flags` attribute sets the flags
used when compiling a regular expression string.

* :class:`~django.core.validators.URLValidator` now accepts an optional
``schemes`` argument which allows customization of the accepted URI schemes
(instead of the defaults ``http(s)`` and ``ftp(s)``).

* :func:`~django.core.validators.validate_email` now accepts addresses with
IPv6 literals, like ``example[2001:db8::1]``, as specified in RFC 5321.

.. _backwards-incompatible-1.7:

Backwards incompatible changes in 1.7
=====================================

.. warning::

In addition to the changes outlined in this section, be sure to review the
:ref:`deprecation plan <deprecation-removed-in-1.7>` for any features that
have been removed. If you haven't updated your code within the
deprecation timeline for a given feature, its removal may appear as a
backwards incompatible change.

``allow_syncdb`` / ``allow_migrate``
------------------------------------

While Django will still look at ``allow_syncdb`` methods even though they
should be renamed to ``allow_migrate``, there is a subtle difference in which
models get passed to these methods.

For apps with migrations, ``allow_migrate`` will now get passed
:ref:`historical models <historical-models>`, which are special versioned models
without custom attributes, methods or managers. Make sure your ``allow_migrate``
methods are only referring to fields or other items in ``model._meta``.

initial_data
------------

Apps with migrations will not load ``initial_data`` fixtures when they have
finished migrating. Apps without migrations will continue to load these fixtures
during the phase of ``migrate`` which emulates the old ``syncdb`` behavior,
but any new apps will not have this support.

Instead, you are encouraged to load initial data in migrations if you need it
(using the ``RunPython`` operation and your model classes);
this has the added advantage that your initial data will not need updating
every time you change the schema.

Additionally, like the rest of Django's old ``syncdb`` code, ``initial_data``
has been started down the deprecation path and will be removed in Django 1.9.

``deconstruct()`` and serializability
-------------------------------------

Django now requires all Field classes and all of their constructor arguments
to be serializable. If you modify the constructor signature in your custom
Field in any way, you'll need to implement a ``deconstruct()`` method;
we've expanded the custom field documentation with :ref:`instructions
on implementing this method <custom-field-deconstruct-method>`.

The requirement for all field arguments to be
:ref:`serializable <migration-serializing>` means that any custom class
instances being passed into Field constructors - things like custom Storage
subclasses, for instance - need to have a :ref:`deconstruct method defined on
them as well <custom-deconstruct-method>`, though Django provides a handy
class decorator that will work for most applications.

App-loading changes
-------------------

Start-up sequence
~~~~~~~~~~~~~~~~~

Django 1.7 loads application configurations and models as soon as it starts.
While this behavior is more straightforward and is believed to be more robust,
regressions cannot be ruled out. See :ref:`applications-troubleshooting` for
solutions to some problems you may encounter.

Standalone scripts
~~~~~~~~~~~~~~~~~~

If you're using Django in a plain Python script — rather than a management
command — and you rely on the :envvar:`DJANGO_SETTINGS_MODULE` environment
variable, you must now explicitly initialize Django at the beginning of your
script with::

>>> import django
>>> django.setup()

Otherwise, you will hit an ``AppRegistryNotReady`` exception.

WSGI scripts
~~~~~~~~~~~~

Until Django 1.3, the recommended way to create a WSGI application was::

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

In Django 1.4, support for WSGI was improved and the API changed to::

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

If you're still using the former style in your WSGI script, you need to
upgrade to the latter, or you will hit an ``AppRegistryNotReady`` exception.

App registry consistency
~~~~~~~~~~~~~~~~~~~~~~~~

It is no longer possible to have multiple installed applications with the same
label. In previous versions of Django, this didn't always work correctly, but
didn't crash outright either.

If you have two apps with the same label, you should create an
:class:`~django.apps.AppConfig` for one of them and override its
:class:`~django.apps.AppConfig.label` there. You should then adjust your code
wherever it references this application or its models with the old label.

It isn't possible to import the same model twice through different paths any
more. As of Django 1.6, this may happen only if you're manually putting a
directory and a subdirectory on :envvar:`PYTHONPATH`. Refer to the section on
the new project layout in the :doc:`1.4 release notes </releases/1.4>` for
migration instructions.

You should make sure that:

* All models are defined in applications that are listed in
:setting:`INSTALLED_APPS` or have an explicit
:attr:`~django.db.models.Options.app_label`.

* Models aren't imported as a side-effect of loading their application.
Specifically, you shouldn't import models in the root module of an
application nor in the module that define its configuration class.

Django will enforce these requirements as of version 1.9, after a deprecation
period.

Subclassing AppCommand
~~~~~~~~~~~~~~~~~~~~~~

Subclasses of :class:`~django.core.management.AppCommand` must now implement a
:meth:`~django.core.management.AppCommand.handle_app_config` method instead of
``handle_app()``. This method receives an :class:`~django.apps.AppConfig`
instance instead of a models module.

Introspecting applications
~~~~~~~~~~~~~~~~~~~~~~~~~~

Since :setting:`INSTALLED_APPS` now supports application configuration classes
in addition to application modules, you should review code that accesses this
setting directly and use the app registry (:attr:`django.apps.apps`) instead.

The app registry has preserved some features of the old app cache. Even though
the app cache was a private API, obsolete methods and arguments will be
removed through a standard deprecation path, with the exception of the
following changes that take effect immediately:

* ``get_model`` raises :exc:`LookupError` instead of returning ``None`` when no
model is found.

* The ``only_installed`` argument of ``get_model`` and ``get_models`` no
longer exists, nor does the ``seed_cache`` argument of ``get_model``.

Management commands and order of :setting:`INSTALLED_APPS`
----------------------------------------------------------

When several applications provide management commands with the same name,
Django loads the command from the application that comes first in
:setting:`INSTALLED_APPS`. Previous versions loaded the command from the
application that came last.

This brings discovery of management commands in line with other parts of
Django that rely on the order of :setting:`INSTALLED_APPS`, such as static
files, templates, and translations.

.. _validation-error-constructor-and-internal-storage:

``ValidationError`` constructor and internal storage
----------------------------------------------------

The behavior of the ``ValidationError`` constructor has changed when it
receives a container of errors as an argument (e.g. a ``list`` or an
``ErrorList``):

* It converts any strings it finds to instances of ``ValidationError``
before adding them to its internal storage.

* It doesn't store the given container but rather copies its content to its
own internal storage; previously the container itself was added to the
``ValidationError`` instance and used as internal storage.

This means that if you access the ``ValidationError`` internal storages, such
as ``error_list``; ``error_dict``; or the return value of
``update_error_dict()`` you may find instances of ``ValidationError`` where you
would have previously found strings.

Also if you directly assigned the return value of ``update_error_dict()``
to ``Form._errors`` you may inadvertently add ``list`` instances where
``ErrorList`` instances are expected. This is a problem because unlike a
simple ``list``, an ``ErrorList`` knows how to handle instances of
``ValidationError``.

Most use-cases that warranted using these private APIs are now covered by
the newly introduced :meth:`Form.add_error() <django.forms.Form.add_error()>`
method::

Old pattern:
try:
...
except ValidationError as e:
self._errors = e.update_error_dict(self._errors)

New pattern:
try:
...
except ValidationError as e:
self.add_error(None, e)

If you need both Django <= 1.6 and 1.7 compatibility you can't use
:meth:`Form.add_error() <django.forms.Form.add_error()>` since it
wasn't available before Django 1.7, but you can use the following
workaround to convert any ``list`` into ``ErrorList``::

try:
...
except ValidationError as e:
self._errors = e.update_error_dict(self._errors)

Additional code to ensure ``ErrorDict`` is exclusively
composed of ``ErrorList`` instances.
for field, error_list in self._errors.items():
if not isinstance(error_list, self.error_class):
self._errors[field] = self.error_class(error_list)

Behavior of ``LocMemCache`` regarding pickle errors
---------------------------------------------------

An inconsistency existed in previous versions of Django regarding how pickle
errors are handled by different cache backends.
``django.core.cache.backends.locmem.LocMemCache`` used to fail silently when
such an error occurs, which is inconsistent with other backends and leads to
cache-specific errors. This has been fixed in Django 1.7, see
:ticket:`21200` for more details.

Cache keys are now generated from the request's absolute URL
------------------------------------------------------------

Previous versions of Django generated cache keys using a request's path and
query string but not the scheme or host. If a Django application was serving
multiple subdomains or domains, cache keys could collide. In Django 1.7, cache
keys vary by the absolute URL of the request including scheme, host, path, and
query string. For example, the URL portion of a cache key is now generated from
``https://www.example.com/path/to/?key=val`` rather than ``/path/to/?key=val``.
The cache keys generated by Django 1.7 will be different from the keys
generated by older versions of Django. After upgrading to Django 1.7, the first
request to any previously cached URL will be a cache miss.

Passing ``None`` to ``Manager.db_manager()``
--------------------------------------------

In previous versions of Django, it was possible to use
``db_manager(using=None)`` on a model manager instance to obtain a manager
instance using default routing behavior, overriding any manually specified
database routing. In Django 1.7, a value of ``None`` passed to db_manager will
produce a router that *retains* any manually assigned database routing -- the
manager will *not* be reset. This was necessary to resolve an inconsistency in
the way routing information cascaded over joins. See :ticket:`13724` for more
details.

pytz may be required
--------------------

If your project handles datetimes before 1970 or after 2037 and Django raises
a :exc:`ValueError` when encountering them, you will have to install pytz_. You
may be affected by this problem if you use Django's time zone-related date
formats or :mod:`django.contrib.syndication`.

.. _pytz: https://pypi.org/project/pytz/

``remove()`` and ``clear()`` methods of related managers
--------------------------------------------------------

The ``remove()`` and ``clear()`` methods of the related managers created by
``ForeignKey``, ``GenericForeignKey``, and ``ManyToManyField`` suffered from a
number of issues. Some operations ran multiple data modifying queries without
wrapping them in a transaction, and some operations didn't respect default
filtering when it was present (i.e. when the default manager on the related
model implemented a custom ``get_queryset()``).

Fixing the issues introduced some backward incompatible changes:

- The default implementation of ``remove()`` for ``ForeignKey`` related managers
changed from a series of ``Model.save()`` calls to a single
``QuerySet.update()`` call. The change means that ``pre_save`` and
``post_save`` signals aren't sent anymore. You can use the ``bulk=False``
keyword argument to revert to the previous behavior.

- The ``remove()`` and ``clear()`` methods for ``GenericForeignKey`` related
managers now perform bulk delete. The ``Model.delete()`` method isn't called
on each instance anymore. You can use the ``bulk=False`` keyword argument to
revert to the previous behavior.

- The ``remove()`` and ``clear()`` methods for ``ManyToManyField`` related
managers perform nested queries when filtering is involved, which may or
may not be an issue depending on your database and your data itself.
See :ref:`this note <nested-queries-performance>` for more details.

Admin login redirection strategy
--------------------------------

Historically, the Django admin site passed the request from an unauthorized or
unauthenticated user directly to the login view, without HTTP redirection. In
Django 1.7, this behavior changed to conform to a more traditional workflow
where any unauthorized request to an admin page will be redirected (by HTTP
status code 302) to the login page, with the ``next`` parameter set to the
referring path. The user will be redirected there after a successful login.

Note also that the admin login form has been updated to not contain the
``this_is_the_login_form`` field (now unused) and the ``ValidationError`` code
has been set to the more regular ``invalid_login`` key.

``select_for_update()`` requires a transaction
----------------------------------------------

Historically, queries that use
:meth:`~django.db.models.query.QuerySet.select_for_update()` could be
executed in autocommit mode, outside of a transaction. Before Django
1.6, Django's automatic transactions mode allowed this to be used to
lock records until the next write operation. Django 1.6 introduced
database-level autocommit; since then, execution in such a context
voids the effect of ``select_for_update()``. It is, therefore, assumed
now to be an error and raises an exception.

This change was made because such errors can be caused by including an
app which expects global transactions (e.g. :setting:`ATOMIC_REQUESTS
<DATABASE-ATOMIC_REQUESTS>` set to ``True``), or Django's old autocommit
behavior, in a project which runs without them; and further, such
errors may manifest as data-corruption bugs. It was also made in
Django 1.6.3.

This change may cause test failures if you use ``select_for_update()``
in a test class which is a subclass of
:class:`~django.test.TransactionTestCase` rather than
:class:`~django.test.TestCase`.

Contrib middleware removed from default ``MIDDLEWARE_CLASSES``
--------------------------------------------------------------

The :ref:`app-loading refactor <app-loading-refactor-17-release-note>`
deprecated using models from apps which are not part of the
:setting:`INSTALLED_APPS` setting. This exposed an incompatibility between
the default :setting:`INSTALLED_APPS` and ``MIDDLEWARE_CLASSES`` in the
global defaults (``django.conf.global_settings``). To bring these settings in
sync and prevent deprecation warnings when doing things like testing reusable
apps with minimal settings,
:class:`~django.contrib.sessions.middleware.SessionMiddleware`,
:class:`~django.contrib.auth.middleware.AuthenticationMiddleware`, and
:class:`~django.contrib.messages.middleware.MessageMiddleware` were removed
from the defaults. These classes will still be included in the default settings
generated by :djadmin:`startproject`. Most projects will not be affected by
this change but if you were not previously declaring the
``MIDDLEWARE_CLASSES`` in your project settings and relying on the
global default you should ensure that the new defaults are in line with your
project's needs. You should also check for any code that accesses
``django.conf.global_settings.MIDDLEWARE_CLASSES`` directly.

Miscellaneous
-------------

* The :meth:`django.core.files.uploadhandler.FileUploadHandler.new_file()`
method is now passed an additional ``content_type_extra`` parameter. If you
have a custom :class:`~django.core.files.uploadhandler.FileUploadHandler`
that implements ``new_file()``, be sure it accepts this new parameter.

* :class:`ModelFormSet<django.forms.models.BaseModelFormSet>`\s no longer
delete instances when ``save(commit=False)`` is called. See
:attr:`~django.forms.formsets.BaseFormSet.can_delete` for instructions on how
to manually delete objects from deleted forms.

* Loading empty fixtures emits a ``RuntimeWarning`` rather than raising
:exc:`~django.core.management.CommandError`.

* :func:`django.contrib.staticfiles.views.serve` will now raise an
:exc:`~django.http.Http404` exception instead of
:exc:`~django.core.exceptions.ImproperlyConfigured` when :setting:`DEBUG`
is ``False``. This change removes the need to conditionally add the view to
your root URLconf, which in turn makes it safe to reverse by name. It also
removes the ability for visitors to generate spurious HTTP 500 errors by
requesting static files that don't exist or haven't been collected yet.

* The :meth:`django.db.models.Model.__eq__` method is now defined in a
way where instances of a proxy model and its base model are considered
equal when primary keys match. Previously only instances of exact same
class were considered equal on primary key match.

* The :meth:`django.db.models.Model.__eq__` method has changed such that
two ``Model`` instances without primary key values won't be considered
equal (unless they are the same instance).

* The :meth:`django.db.models.Model.__hash__` method will now raise ``TypeError``
when called on an instance without a primary key value. This is done to
avoid mutable ``__hash__`` values in containers.

* :class:`~django.db.models.AutoField` columns in SQLite databases will now be
created using the ``AUTOINCREMENT`` option, which guarantees monotonic
increments. This will cause primary key numbering behavior to change on
SQLite, becoming consistent with most other SQL databases. This will only
apply to newly created tables. If you have a database created with an older
version of Django, you will need to migrate it to take advantage of this
feature. For example, you could do the following:

) Use :djadmin:`dumpdata` to save your data.
) Rename the existing database file (keep it as a backup).
) Run :djadmin:`migrate` to create the updated schema.
) Use :djadmin:`loaddata` to import the fixtures you exported in (1).

* ``django.contrib.auth.models.AbstractUser`` no longer defines a
:meth:`~django.db.models.Model.get_absolute_url()` method. The old definition
returned ``"/users/%s/" % urlquote(self.username)`` which was arbitrary
since applications may or may not define such a url in ``urlpatterns``.
Define a ``get_absolute_url()`` method on your own custom user object or use
:setting:`ABSOLUTE_URL_OVERRIDES` if you want a URL for your user.

* The static asset-serving functionality of the
:class:`django.test.LiveServerTestCase` class has been simplified: Now it's
only able to serve content already present in :setting:`STATIC_ROOT` when
tests are run. The ability to transparently serve all the static assets
(similarly to what one gets with :setting:`DEBUG = True <DEBUG>` at
development-time) has been moved to a new class that lives in the
``staticfiles`` application (the one actually in charge of such feature):
:class:`django.contrib.staticfiles.testing.StaticLiveServerTestCase`. In other
words, ``LiveServerTestCase`` itself is less powerful but at the same time
has less magic.

Rationale behind this is removal of dependency of non-contrib code on
contrib applications.

* The old cache URI syntax (e.g. ``"locmem://"``) is no longer supported. It
still worked, even though it was not documented or officially supported. If
you're still using it, please update to the current :setting:`CACHES` syntax.

* The default ordering of ``Form`` fields in case of inheritance has changed to
follow normal Python MRO. Fields are now discovered by iterating through the
MRO in reverse with the topmost class coming last. This only affects you if
you relied on the default field ordering while having fields defined on both
the current class *and* on a parent ``Form``.

* The ``required`` argument of
:class:`~django.forms.SelectDateWidget` has been removed.
This widget now respects the form field's ``is_required`` attribute like
other widgets.

* ``Widget.is_hidden`` is now a read-only property, getting its value by
introspecting the presence of ``input_type == 'hidden'``.

* :meth:`~django.db.models.query.QuerySet.select_related` now chains in the
same way as other similar calls like ``prefetch_related``. That is,
``select_related('foo', 'bar')`` is equivalent to
``select_related('foo').select_related('bar')``. Previously the latter would
have been equivalent to ``select_related('bar')``.

* GeoDjango dropped support for GEOS < 3.1.

* The ``init_connection_state`` method of database backends now executes in
autocommit mode (unless you set :setting:`AUTOCOMMIT <DATABASE-AUTOCOMMIT>`
to ``False``). If you maintain a custom database backend, you should check
that method.

* The ``django.db.backends.BaseDatabaseFeatures.allows_primary_key_0``
attribute has been renamed to ``allows_auto_pk_0`` to better describe it.
It's ``True`` for all database backends included with Django except MySQL
which does allow primary keys with value 0. It only forbids *autoincrement*
primary keys with value 0.

* Shadowing model fields defined in a parent model has been forbidden as this
creates ambiguity in the expected model behavior. In addition, clashing
fields in the model inheritance hierarchy result in a system check error.
For example, if you use multi-inheritance, you need to define custom primary
key fields on parent models, otherwise the default ``id`` fields will clash.
See :ref:`model-multiple-inheritance-topic` for details.

* ``django.utils.translation.parse_accept_lang_header()`` now returns
lowercase locales, instead of the case as it was provided. As locales should
be treated case-insensitive this allows us to speed up locale detection.

* ``django.utils.translation.get_language_from_path()`` and
``django.utils.translation.trans_real.get_supported_language_variant()``
now no longer have a ``supported`` argument.

* The ``shortcut`` view in ``django.contrib.contenttypes.views`` now supports
protocol-relative URLs (e.g. ``//example.com``).

* :class:`~django.contrib.contenttypes.fields.GenericRelation` now supports an
optional ``related_query_name`` argument. Setting ``related_query_name`` adds
a relation from the related object back to the content type for filtering,
ordering and other query operations.

* When running tests on PostgreSQL, the :setting:`USER` will need read access
to the built-in ``postgres`` database. This is in lieu of the previous
behavior of connecting to the actual non-test database.

* As part of the :doc:`System check framework </ref/checks>`, :ref:`fields,
models, and model managers <field-checking>` all implement a ``check()``
method that is registered with the check framework. If you have an existing
method called ``check()`` on one of these objects, you will need to rename it.

* As noted above in the "Cache" section of "Minor Features", defining the
:setting:`TIMEOUT <CACHES-TIMEOUT>` argument of the
:setting:`CACHES` setting as ``None`` will set the cache keys as
"non-expiring". Previously, with the memcache backend, a
:setting:`TIMEOUT <CACHES-TIMEOUT>` of ``0`` would set non-expiring keys,
but this was inconsistent with the set-and-expire (i.e. no caching) behavior
of ``set("key", "value", timeout=0)``. If you want non-expiring keys,
please update your settings to use ``None`` instead of ``0`` as the latter
now designates set-and-expire in the settings as well.

* The ``sql*`` management commands now respect the ``allow_migrate()`` method
of :setting:`DATABASE_ROUTERS`. If you have models synced to non-default
databases, use the ``--database`` flag to get SQL for those models
(previously they would always be included in the output).

* Decoding the query string from URLs now falls back to the ISO-8859-1 encoding
when the input is not valid UTF-8.

* With the addition of the
``django.contrib.auth.middleware.SessionAuthenticationMiddleware`` to
the default project template (pre-1.7.2 only), a database must be created
before accessing a page using :djadmin:`runserver`.

* The addition of the ``schemes`` argument to ``URLValidator`` will appear
as a backwards-incompatible change if you were previously using a custom
regular expression to validate schemes. Any scheme not listed in ``schemes``
will fail validation, even if the regular expression matches the given URL.

.. _deprecated-features-1.7:

Features deprecated in 1.7
==========================

``django.core.cache.get_cache``
-------------------------------

``django.core.cache.get_cache`` has been supplanted by
:data:`django.core.cache.caches`.

``django.utils.dictconfig``/``django.utils.importlib``
------------------------------------------------------

``django.utils.dictconfig`` and ``django.utils.importlib`` were copies of
respectively :mod:`logging.config` and :mod:`importlib` provided for Python
versions prior to 2.7. They have been deprecated.

``django.utils.module_loading.import_by_path``
----------------------------------------------

The current ``django.utils.module_loading.import_by_path`` function
catches ``AttributeError``, ``ImportError``, and ``ValueError`` exceptions,
and re-raises :exc:`~django.core.exceptions.ImproperlyConfigured`. Such
exception masking makes it needlessly hard to diagnose circular import
problems, because it makes it look like the problem comes from inside Django.
It has been deprecated in favor of
:meth:`~django.utils.module_loading.import_string`.

``django.utils.tzinfo``
-----------------------

``django.utils.tzinfo`` provided two :class:`~datetime.tzinfo` subclasses,
``LocalTimezone`` and ``FixedOffset``. They've been deprecated in favor of
more correct alternatives provided by :mod:`django.utils.timezone`,
:func:`django.utils.timezone.get_default_timezone` and
:func:`django.utils.timezone.get_fixed_timezone`.

``django.utils.unittest``
-------------------------

``django.utils.unittest`` provided uniform access to the ``unittest2`` library
on all Python versions. Since ``unittest2`` became the standard library's
:mod:`unittest` module in Python 2.7, and Django 1.7 drops support for older
Python versions, this module isn't useful anymore. It has been deprecated. Use
:mod:`unittest` instead.

``django.utils.datastructures.SortedDict``
------------------------------------------

As :class:`~collections.OrderedDict` was added to the standard library in
Python 2.7, ``SortedDict`` is no longer needed and has been deprecated.

The two additional, deprecated methods provided by ``SortedDict`` (``insert()``
and ``value_for_index()``) have been removed. If you relied on these methods to
alter structures like form fields, you should now treat these ``OrderedDict``\s
as immutable objects and override them to change their content.

For example, you might want to override ``MyFormClass.base_fields`` (although
this attribute isn't considered a public API) to change the ordering of fields
for all ``MyFormClass`` instances; or similarly, you could override
``self.fields`` from inside ``MyFormClass.__init__()``, to change the fields
for a particular form instance. For example (from Django itself)::

PasswordChangeForm.base_fields = OrderedDict(
(k, PasswordChangeForm.base_fields[k])
for k in ['old_password', 'new_password1', 'new_password2']
)

Custom SQL location for models package
--------------------------------------

Previously, if models were organized in a package (``myapp/models/``) rather
than simply ``myapp/models.py``, Django would look for initial SQL data in
``myapp/models/sql/``. This bug has been fixed so that Django
will search ``myapp/sql/`` as documented. After this issue was fixed, migrations
were added which deprecates initial SQL data. Thus, while this change still
exists, the deprecation is irrelevant as the entire feature will be removed in
Django 1.9.

Reorganization of ``django.contrib.sites``
------------------------------------------

``django.contrib.sites`` provides reduced functionality when it isn't in
:setting:`INSTALLED_APPS`. The app-loading refactor adds some constraints in
that situation. As a consequence, two objects were moved, and the old
locations are deprecated:

* :class:`~django.contrib.sites.requests.RequestSite` now lives in
``django.contrib.sites.requests``.
* :func:`~django.contrib.sites.shortcuts.get_current_site` now lives in
``django.contrib.sites.shortcuts``.

``declared_fieldsets`` attribute on ``ModelAdmin``
--------------------------------------------------

``ModelAdmin.declared_fieldsets`` has been deprecated. Despite being a private
API, it will go through a regular deprecation path. This attribute was mostly
used by methods that bypassed ``ModelAdmin.get_fieldsets()`` but this was
considered a bug and has been addressed.

Reorganization of ``django.contrib.contenttypes``
-------------------------------------------------

Since ``django.contrib.contenttypes.generic`` defined both admin and model
related objects, an import of this module could trigger unexpected side effects.
As a consequence, its contents were split into :mod:`~django.contrib.contenttypes`
submodules and the ``django.contrib.contenttypes.generic`` module is deprecated:

* :class:`~django.contrib.contenttypes.fields.GenericForeignKey` and
:class:`~django.contrib.contenttypes.fields.GenericRelation` now live in
:mod:`~django.contrib.contenttypes.fields`.
* :class:`~django.contrib.contenttypes.forms.BaseGenericInlineFormSet` and
:func:`~django.contrib.contenttypes.forms.generic_inlineformset_factory` now
live in :mod:`~django.contrib.contenttypes.forms`.
* :class:`~django.contrib.contenttypes.admin.GenericInlineModelAdmin`,
:class:`~django.contrib.contenttypes.admin.GenericStackedInline` and
:class:`~django.contrib.contenttypes.admin.GenericTabularInline` now live in
:mod:`~django.contrib.contenttypes.admin`.

``syncdb``
----------

The ``syncdb`` command has been deprecated in favor of the new :djadmin:`migrate`
command. ``migrate`` takes the same arguments as ``syncdb`` used to plus a few
more, so it's safe to just change the name you're calling and nothing else.

``util`` modules renamed to ``utils``
-------------------------------------

The following instances of ``util.py`` in the Django codebase have been renamed
to ``utils.py`` in an effort to unify all util and utils references:

* ``django.contrib.admin.util``
* ``django.contrib.gis.db.backends.util``
* ``django.db.backends.util``
* ``django.forms.util``

``get_formsets`` method on ``ModelAdmin``
-----------------------------------------

``ModelAdmin.get_formsets`` has been deprecated in favor of the new
:meth:`~django.contrib.admin.ModelAdmin.get_formsets_with_inlines`, in order to
better handle the case of selectively showing inlines on a ``ModelAdmin``.

``IPAddressField``
------------------

The ``django.db.models.IPAddressField`` and ``django.forms.IPAddressField``
fields have been deprecated in favor of
:class:`django.db.models.GenericIPAddressField` and
:class:`django.forms.GenericIPAddressField`.

``BaseMemcachedCache._get_memcache_timeout`` method
---------------------------------------------------

The ``BaseMemcachedCache._get_memcache_timeout()`` method has been renamed to
``get_backend_timeout()``. Despite being a private API, it will go through the
normal deprecation.

Natural key serialization options
---------------------------------

The ``--natural`` and ``-n`` options for :djadmin:`dumpdata` have been
deprecated. Use :option:`dumpdata --natural-foreign` instead.

Similarly, the ``use_natural_keys`` argument for ``serializers.serialize()``
has been deprecated. Use ``use_natural_foreign_keys`` instead.

Merging of ``POST`` and ``GET`` arguments into ``WSGIRequest.REQUEST``
----------------------------------------------------------------------

It was already strongly suggested that you use ``GET`` and ``POST`` instead of
``REQUEST``, because the former are more explicit. The property ``REQUEST`` is
deprecated and will be removed in Django 1.9.

``django.utils.datastructures.MergeDict`` class
-----------------------------------------------

``MergeDict`` exists primarily to support merging ``POST`` and ``GET``
arguments into a ``REQUEST`` property on ``WSGIRequest``. To merge
dictionaries, use ``dict.update()`` instead. The class ``MergeDict`` is
deprecated and will be removed in Django 1.9.

Language codes ``zh-cn``, ``zh-tw`` and ``fy-nl``
-------------------------------------------------

The currently used language codes for Simplified Chinese ``zh-cn``,
Traditional Chinese ``zh-tw`` and (Western) Frysian ``fy-nl`` are deprecated
and should be replaced by the language codes ``zh-hans``, ``zh-hant`` and
``fy`` respectively. If you use these language codes, you should rename the
locale directories and update your settings to reflect these changes. The
deprecated language codes will be removed in Django 1.9.

``django.utils.functional.memoize`` function
--------------------------------------------

The function ``memoize`` is deprecated and should be replaced by the
``functools.lru_cache`` decorator (available from Python 3.2 onward).

Django ships a backport of this decorator for older Python versions and it's
available at ``django.utils.lru_cache.lru_cache``. The deprecated function will
be removed in Django 1.9.

Geo Sitemaps
------------

Google has retired support for the Geo Sitemaps format. Hence Django support
for Geo Sitemaps is deprecated and will be removed in Django 1.8.

Passing callable arguments to queryset methods
----------------------------------------------

Callable arguments for querysets were an undocumented feature that was
unreliable. It's been deprecated and will be removed in Django 1.9.

Callable arguments were evaluated when a queryset was constructed rather than
when it was evaluated, thus this feature didn't offer any benefit compared to
evaluating arguments before passing them to queryset and created confusion that
the arguments may have been evaluated at query time.

``ADMIN_FOR`` setting
---------------------

The ``ADMIN_FOR`` feature, part of the admindocs, has been removed. You can
remove the setting from your configuration at your convenience.

``SplitDateTimeWidget`` with ``DateTimeField``
----------------------------------------------

``SplitDateTimeWidget`` support in :class:`~django.forms.DateTimeField` is
deprecated, use ``SplitDateTimeWidget`` with
:class:`~django.forms.SplitDateTimeField` instead.

``validate``
------------

The ``validate`` management command is deprecated in favor of the
:djadmin:`check` command.

``django.core.management.BaseCommand``
--------------------------------------

``requires_model_validation`` is deprecated in favor of a new
``requires_system_checks`` flag. If the latter flag is missing, then the
value of the former flag is used. Defining both ``requires_system_checks`` and
``requires_model_validation`` results in an error.

The ``check()`` method has replaced the old ``validate()`` method.

``ModelAdmin`` validators
-------------------------

The ``ModelAdmin.validator_class`` and ``default_validator_class`` attributes
are deprecated in favor of the new ``checks_class`` attribute.

The ``ModelAdmin.validate()`` method is deprecated in favor of
``ModelAdmin.check()``.

The ``django.contrib.admin.validation`` module is deprecated.

``django.db.backends.DatabaseValidation.validate_field``
--------------------------------------------------------

This method is deprecated in favor of a new ``check_field`` method.
The functionality required by ``check_field()`` is the same as that provided
by ``validate_field()``, but the output format is different. Third-party database
backends needing this functionality should provide an implementation of
``check_field()``.

Loading ``ssi`` and ``url`` template tags from ``future`` library
-----------------------------------------------------------------

Django 1.3 introduced ``{% load ssi from future %}`` and
``{% load url from future %}`` syntax for forward compatibility of the
``ssi`` and :ttag:`url` template tags. This syntax is now deprecated and
will be removed in Django 1.9. You can simply remove the
``{% load ... from future %}`` tags.

``django.utils.text.javascript_quote``
--------------------------------------

``javascript_quote()`` was an undocumented function present in ``django.utils.text``.
It was used internally in the ``javascript_catalog()`` view
whose implementation was changed to make use of ``json.dumps()`` instead.
If you were relying on this function to provide safe output from untrusted
strings, you should use ``django.utils.html.escapejs`` or the
:tfilter:`escapejs` template filter.
If all you need is to generate valid JavaScript strings, you can simply use
``json.dumps()``.

``fix_ampersands`` utils method and template filter
---------------------------------------------------

The ``django.utils.html.fix_ampersands`` method and the ``fix_ampersands``
template filter are deprecated, as the escaping of ampersands is already taken care
of by Django's standard HTML escaping features. Combining this with ``fix_ampersands``
would either result in double escaping, or, if the output is assumed to be safe,
a risk of introducing XSS vulnerabilities. Along with ``fix_ampersands``,
``django.utils.html.clean_html`` is deprecated, an undocumented function that calls
``fix_ampersands``.
As this is an accelerated deprecation, ``fix_ampersands`` and ``clean_html``
will be removed in Django 1.8.

Reorganization of database test settings
----------------------------------------

All database settings with a ``TEST_`` prefix have been deprecated in favor of
entries in a :setting:`TEST <DATABASE-TEST>` dictionary in the database
settings. The old settings will be supported until Django 1.9. For backwards
compatibility with older versions of Django, you can define both versions of
the settings as long as they match.

FastCGI support
---------------

FastCGI support via the ``runfcgi`` management command will be removed in
Django 1.9. Please deploy your project using WSGI.

Moved objects in ``contrib.sites``
----------------------------------

Following the app-loading refactor, two objects in
``django.contrib.sites.models`` needed to be moved because they must be
available without importing ``django.contrib.sites.models`` when
``django.contrib.sites`` isn't installed. Import ``RequestSite`` from
``django.contrib.sites.requests`` and ``get_current_site()`` from
``django.contrib.sites.shortcuts``. The old import locations will work until
Django 1.9.

``django.forms.forms.get_declared_fields()``
--------------------------------------------

Django no longer uses this functional internally. Even though it's a private
API, it'll go through the normal deprecation cycle.

Private Query Lookup APIs
-------------------------

Private APIs ``django.db.models.sql.where.WhereNode.make_atom()`` and
``django.db.models.sql.where.Constraint`` are deprecated in favor of the new
:doc:`custom lookups API </ref/models/lookups>`.

.. _removed-features-1.7:

Features removed in 1.7
=======================

These features have reached the end of their deprecation cycle and are removed
in Django 1.7. See :ref:`deprecated-features-1.5` for details, including how to
remove usage of these features.

* ``django.utils.simplejson`` is removed.

* ``django.utils.itercompat.product`` is removed.

* INSTALLED_APPS and TEMPLATE_DIRS are no longer corrected from a plain
string into a tuple.

* :class:`~django.http.HttpResponse`,
:class:`~django.template.response.SimpleTemplateResponse`,
:class:`~django.template.response.TemplateResponse`,
``render_to_response()``, :func:`~django.contrib.sitemaps.views.index`, and
:func:`~django.contrib.sitemaps.views.sitemap` no longer take a ``mimetype``
argument

* :class:`~django.http.HttpResponse` immediately consumes its content if it's
an iterator.

* The ``AUTH_PROFILE_MODULE`` setting, and the ``get_profile()`` method on
the User model are removed.

* The ``cleanup`` management command is removed.

* The ``daily_cleanup.py`` script is removed.

* :meth:`~django.db.models.query.QuerySet.select_related` no longer has a
``depth`` keyword argument.

* The ``get_warnings_state()``/``restore_warnings_state()``
functions from :mod:`django.test.utils` and the ``save_warnings_state()``/
``restore_warnings_state()``
:ref:`django.test.*TestCase <django-testcase-subclasses>` are removed.

* The ``check_for_test_cookie`` method in
:class:`~django.contrib.auth.forms.AuthenticationForm` is removed.

* The version of ``django.contrib.auth.views.password_reset_confirm()`` that
supports base36 encoded user IDs
(``django.contrib.auth.views.password_reset_confirm_uidb36``) is removed.

* The ``django.utils.encoding.StrAndUnicode`` mix-in is removed.


===========================

1.6.11

Not secure
===========================

*March 18, 2015*

Django 1.6.11 fixes two security issues in 1.6.10.

Denial-of-service possibility with ``strip_tags()``
===================================================

Last year :func:`~django.utils.html.strip_tags` was changed to work
iteratively. The problem is that the size of the input it's processing can
increase on each iteration which results in an infinite loop in
``strip_tags()``. This issue only affects versions of Python that haven't
received :bpo:`a bugfix in HTMLParser <20288>`; namely Python < 2.7.7 and
3.3.5. Some operating system vendors have also backported the fix for the
Python bug into their packages of earlier versions.

To remedy this issue, ``strip_tags()`` will now return the original input if
it detects the length of the string it's processing increases. Remember that
absolutely NO guarantee is provided about the results of ``strip_tags()`` being
HTML safe. So NEVER mark safe the result of a ``strip_tags()`` call without
escaping it first, for example with :func:`~django.utils.html.escape`.

Mitigated possible XSS attack via user-supplied redirect URLs
=============================================================

Django relies on user input in some cases (e.g.
``django.contrib.auth.views.login()`` and :doc:`i18n </topics/i18n/index>`)
to redirect the user to an "on success" URL. The security checks for these
redirects (namely ``django.utils.http.is_safe_url()``) accepted URLs with
leading control characters and so considered URLs like ``\x08javascript:...``
safe. This issue doesn't affect Django currently, since we only put this URL
into the ``Location`` response header and browsers seem to ignore JavaScript
there. Browsers we tested also treat URLs prefixed with control characters such
as ``%08//example.com`` as relative paths so redirection to an unsafe target
isn't a problem either.

However, if a developer relies on ``is_safe_url()`` to
provide safe redirect targets and puts such a URL into a link, they could
suffer from an XSS attack as some browsers such as Google Chrome ignore control
characters at the start of a URL in an anchor ``href``.


===========================

1.6.10

Not secure
===========================

*January 13, 2015*

Django 1.6.10 fixes several security issues in 1.6.9.

WSGI header spoofing via underscore/dash conflation
===================================================

When HTTP headers are placed into the WSGI environ, they are normalized by
converting to uppercase, converting all dashes to underscores, and prepending
``HTTP_``. For instance, a header ``X-Auth-User`` would become
``HTTP_X_AUTH_USER`` in the WSGI environ (and thus also in Django's
``request.META`` dictionary).

Unfortunately, this means that the WSGI environ cannot distinguish between
headers containing dashes and headers containing underscores: ``X-Auth-User``
and ``X-Auth_User`` both become ``HTTP_X_AUTH_USER``. This means that if a
header is used in a security-sensitive way (for instance, passing
authentication information along from a front-end proxy), even if the proxy
carefully strips any incoming value for ``X-Auth-User``, an attacker may be
able to provide an ``X-Auth_User`` header (with underscore) and bypass this
protection.

In order to prevent such attacks, both Nginx and Apache 2.4+ strip all headers
containing underscores from incoming requests by default. Django's built-in
development server now does the same. Django's development server is not
recommended for production use, but matching the behavior of common production
servers reduces the surface area for behavior changes during deployment.

Mitigated possible XSS attack via user-supplied redirect URLs
=============================================================

Django relies on user input in some cases (e.g.
``django.contrib.auth.views.login()`` and :doc:`i18n </topics/i18n/index>`)
to redirect the user to an "on success" URL. The security checks for these
redirects (namely ``django.utils.http.is_safe_url()``) didn't strip leading
whitespace on the tested URL and as such considered URLs like
``\njavascript:...`` safe. If a developer relied on ``is_safe_url()`` to
provide safe redirect targets and put such a URL into a link, they could suffer
from a XSS attack. This bug doesn't affect Django currently, since we only put
this URL into the ``Location`` response header and browsers seem to ignore
JavaScript there.

Denial-of-service attack against ``django.views.static.serve``
==============================================================

In older versions of Django, the :func:`django.views.static.serve` view read
the files it served one line at a time. Therefore, a big file with no newlines
would result in memory usage equal to the size of that file. An attacker could
exploit this and launch a denial-of-service attack by simultaneously requesting
many large files. This view now reads the file in chunks to prevent large
memory usage.

Note, however, that this view has always carried a warning that it is not
hardened for production use and should be used only as a development aid. Now
may be a good time to audit your project and serve your files in production
using a real front-end web server if you are not doing so.

Database denial-of-service with ``ModelMultipleChoiceField``
============================================================

Given a form that uses ``ModelMultipleChoiceField`` and
``show_hidden_initial=True`` (not a documented API), it was possible for a user
to cause an unreasonable number of SQL queries by submitting duplicate values
for the field's data. The validation logic in ``ModelMultipleChoiceField`` now
deduplicates submitted values to address this issue.


==========================

1.6.9

Not secure
==========================

*January 2, 2015*

Django 1.6.9 fixes a regression in the 1.6.6 security release.

Additionally, Django's vendored version of six, ``django.utils.six``, has
been upgraded to the latest release (1.9.0).

Bugfixes
========

* Fixed a regression with dynamically generated inlines and allowed field
references in the admin (:ticket:`23754`).


==========================

1.6.8

Not secure
==========================

*October 22, 2014*

Django 1.6.8 fixes a couple regressions in the 1.6.6 security release.

Bugfixes
========

* Allowed related many-to-many fields to be referenced in the admin
(:ticket:`23604`).

* Allowed inline and hidden references to admin fields (:ticket:`23431`).


==========================

1.6.7

Not secure
==========================

*September 2, 2014*

Django 1.6.7 fixes several bugs in 1.6.6, including a regression related to
a security fix in that release.

Bugfixes
========

* Allowed inherited and m2m fields to be referenced in the admin
(:ticket:`23329`).
* Fixed a crash when using ``QuerySet.defer()`` with ``select_related()``
(:ticket:`23370`).


==========================

1.6.6

Not secure
==========================

*August 20, 2014*

Django 1.6.6 fixes several security issues and bugs in 1.6.5.

``reverse()`` could generate URLs pointing to other hosts
=========================================================

In certain situations, URL reversing could generate scheme-relative URLs (URLs
starting with two slashes), which could unexpectedly redirect a user to a
different host. An attacker could exploit this, for example, by redirecting
users to a phishing site designed to ask for user's passwords.

To remedy this, URL reversing now ensures that no URL starts with two slashes
(//), replacing the second slash with its URL encoded counterpart (%2F). This
approach ensures that semantics stay the same, while making the URL relative to
the domain and not to the scheme.

File upload denial-of-service
=============================

Before this release, Django's file upload handing in its default configuration
may degrade to producing a huge number of ``os.stat()`` system calls when a
duplicate filename is uploaded. Since ``stat()`` may invoke IO, this may produce
a huge data-dependent slowdown that slowly worsens over time. The net result is
that given enough time, a user with the ability to upload files can cause poor
performance in the upload handler, eventually causing it to become very slow
simply by uploading 0-byte files. At this point, even a slow network connection
and few HTTP requests would be all that is necessary to make a site unavailable.

We've remedied the issue by changing the algorithm for generating file names
if a file with the uploaded name already exists.
:meth:`Storage.get_available_name()
<django.core.files.storage.Storage.get_available_name>` now appends an
underscore plus a random 7 character alphanumeric string (e.g. ``"_x3a1gho"``),
rather than iterating through an underscore followed by a number (e.g. ``"_1"``,
``"_2"``, etc.).

``RemoteUserMiddleware`` session hijacking
==========================================

When using the :class:`~django.contrib.auth.middleware.RemoteUserMiddleware`
and the ``RemoteUserBackend``, a change to the ``REMOTE_USER`` header between
requests without an intervening logout could result in the prior user's session
being co-opted by the subsequent user. The middleware now logs the user out on
a failed login attempt.

Data leakage via query string manipulation in ``contrib.admin``
===============================================================

In older versions of Django it was possible to reveal any field's data by
modifying the "popup" and "to_field" parameters of the query string on an admin
change form page. For example, requesting a URL like
``/admin/auth/user/?_popup=1&t=password`` and viewing the page's HTML allowed
viewing the password hash of each user. While the admin requires users to have
permissions to view the change form pages in the first place, this could leak
data if you rely on users having access to view only certain fields on a model.

To address the issue, an exception will now be raised if a ``to_field`` value
that isn't a related field to a model that has been registered with the admin
is specified.

Bugfixes
========

* Corrected email and URL validation to reject a trailing dash
(:ticket:`22579`).

* Prevented indexes on PostgreSQL virtual fields (:ticket:`22514`).

* Prevented edge case where values of FK fields could be initialized with a
wrong value when an inline model formset is created for a relationship
defined to point to a field other than the PK (:ticket:`13794`).

* Restored ``pre_delete`` signals for ``GenericRelation`` cascade deletion
(:ticket:`22998`).

* Fixed transaction handling when specifying non-default database in
``createcachetable`` and ``flush`` (:ticket:`23089`).

* Fixed the "ORA-01843: not a valid month" errors when using Unicode
with older versions of Oracle server (:ticket:`20292`).

* Restored bug fix for sending Unicode email with Python 2.6.5 and below
(:ticket:`19107`).

* Prevented ``UnicodeDecodeError`` in ``runserver`` with non-UTF-8 and
non-English locale (:ticket:`23265`).

* Fixed JavaScript errors while editing multi-geometry objects in the OpenLayers
widget (:ticket:`23137`, :ticket:`23293`).

* Prevented a crash on Python 3 with query strings containing unencoded
non-ASCII characters (:ticket:`22996`).


==========================

1.6.5

Not secure
==========================

*May 14, 2014*

Django 1.6.5 fixes two security issues and several bugs in 1.6.4.

Issue: Caches may incorrectly be allowed to store and serve private data
========================================================================

In certain situations, Django may allow caches to store private data
related to a particular session and then serve that data to requests
with a different session, or no session at all. This can lead to
information disclosure and can be a vector for cache poisoning.

When using Django sessions, Django will set a ``Vary: Cookie`` header to
ensure caches do not serve cached data to requests from other sessions.
However, older versions of Internet Explorer (most likely only Internet
Explorer 6, and Internet Explorer 7 if run on Windows XP or Windows Server
2003) are unable to handle the ``Vary`` header in combination with many content
types. Therefore, Django would remove the header if the request was made by
Internet Explorer.

To remedy this, the special behavior for these older Internet Explorer versions
has been removed, and the ``Vary`` header is no longer stripped from the response.
In addition, modifications to the ``Cache-Control`` header for all Internet Explorer
requests with a ``Content-Disposition`` header have also been removed as they
were found to have similar issues.

Issue: Malformed redirect URLs from user input not correctly validated
======================================================================

The validation for redirects did not correctly validate some malformed URLs,
which are accepted by some browsers. This allows a user to be redirected to
an unsafe URL unexpectedly.

Django relies on user input in some cases (e.g.
``django.contrib.auth.views.login()``, ``django.contrib.comments``, and
:doc:`i18n </topics/i18n/index>`) to redirect the user to an "on success" URL.
The security checks for these redirects (namely
``django.utils.http.is_safe_url()``) did not correctly validate some malformed
URLs, such as ``http:\\\\\\djangoproject.com``, which are accepted by some
browsers with more liberal URL parsing.

To remedy this, the validation in ``is_safe_url()`` has been tightened to be able
to handle and correctly validate these malformed URLs.

Bugfixes
========

* Made the ``year_lookup_bounds_for_datetime_field`` Oracle backend method
Python 3 compatible (:ticket:`22551`).

* Fixed ``pgettext_lazy`` crash when receiving bytestring content on Python 2
(:ticket:`22565`).

* Fixed the SQL generated when filtering by a negated ``Q`` object that contains
a ``F`` object. (:ticket:`22429`).

* Avoided overwriting data fetched by ``select_related()`` in certain cases
which could cause minor performance regressions
(:ticket:`22508`).


==========================

1.6.4

Not secure
==========================

*April 28, 2014*

Django 1.6.4 fixes several bugs in 1.6.3.

Bugfixes
========

* Added backwards compatibility support for the :mod:`django.contrib.messages`
cookie format of Django 1.4 and earlier to facilitate upgrading to 1.6 from
1.4 (:ticket:`22426`).

* Restored the ability to ``reverse()`` views created using
:func:`functools.partial()` (:ticket:`22486`).

* Fixed the ``object_id`` of the ``LogEntry`` that's created after a user
password change in the admin (:ticket:`22515`).


==========================

1.6.3

Not secure
==========================

*April 21, 2014*

Django 1.6.3 fixes several bugs in 1.6.2, including three security issues,
and makes one backwards-incompatible change:

Unexpected code execution using ``reverse()``
=============================================

Django's URL handling is based on a mapping of regex patterns
(representing the URLs) to callable views, and Django's own processing
consists of matching a requested URL against those patterns to
determine the appropriate view to invoke.

Django also provides a convenience function -- ``reverse()`` -- which performs
this process in the opposite direction. The ``reverse()`` function takes
information about a view and returns a URL which would invoke that view. Use
of ``reverse()`` is encouraged for application developers, as the output of
``reverse()`` is always based on the current URL patterns, meaning developers
do not need to change other code when making changes to URLs.

One argument signature for ``reverse()`` is to pass a dotted Python
path to the desired view. In this situation, Django will import the
module indicated by that dotted path as part of generating the
resulting URL. If such a module has import-time side effects, those
side effects will occur.

Thus it is possible for an attacker to cause unexpected code
execution, given the following conditions:

1. One or more views are present which construct a URL based on user
input (commonly, a "next" parameter in a querystring indicating
where to redirect upon successful completion of an action).

2. One or more modules are known to an attacker to exist on the
server's Python import path, which perform code execution with side
effects on importing.

To remedy this, ``reverse()`` will now only accept and import dotted
paths based on the view-containing modules listed in the project's :doc:`URL
pattern configuration </topics/http/urls>`, so as to ensure that only modules
the developer intended to be imported in this fashion can or will be imported.

Caching of anonymous pages could reveal CSRF token
==================================================

Django includes both a :doc:`caching framework </topics/cache>` and a system
for :doc:`preventing cross-site request forgery (CSRF) attacks
</ref/csrf/>`. The CSRF-protection system is based on a random nonce
sent to the client in a cookie which must be sent by the client on future
requests and, in forms, a hidden value which must be submitted back with the
form.

The caching framework includes an option to cache responses to
anonymous (i.e., unauthenticated) clients.

When the first anonymous request to a given page is by a client which
did not have a CSRF cookie, the cache framework will also cache the
CSRF cookie and serve the same nonce to other anonymous clients who
do not have a CSRF cookie. This can allow an attacker to obtain a
valid CSRF cookie value and perform attacks which bypass the check for
the cookie.

To remedy this, the caching framework will no longer cache such
responses. The heuristic for this will be:

1. If the incoming request did not submit any cookies, and

2. If the response did send one or more cookies, and

3. If the ``Vary: Cookie`` header is set on the response, then the
response will not be cached.

MySQL typecasting
=================

The MySQL database is known to "typecast" on certain queries; for
example, when querying a table which contains string values, but using
a query which filters based on an integer value, MySQL will first
silently coerce the strings to integers and return a result based on that.

If a query is performed without first converting values to the
appropriate type, this can produce unexpected results, similar to what
would occur if the query itself had been manipulated.

Django's model field classes are aware of their own types and most
such classes perform explicit conversion of query arguments to the
correct database-level type before querying. However, three model
field classes did not correctly convert their arguments:

* :class:`~django.db.models.FilePathField`
* :class:`~django.db.models.GenericIPAddressField`
* ``IPAddressField``

These three fields have been updated to convert their arguments to the
correct types before querying.

Additionally, developers of custom model fields are now warned via
documentation to ensure their custom field classes will perform
appropriate type conversions, and users of the :meth:`raw()
<django.db.models.query.QuerySet.raw>` and :meth:`extra()
<django.db.models.query.QuerySet.extra>` query methods -- which allow the
developer to supply raw SQL or SQL fragments -- will be advised to ensure they
perform appropriate manual type conversions prior to executing queries.

``select_for_update()`` requires a transaction
==============================================

Historically, queries that use
:meth:`~django.db.models.query.QuerySet.select_for_update()` could be
executed in autocommit mode, outside of a transaction. Before Django
1.6, Django's automatic transactions mode allowed this to be used to
lock records until the next write operation. Django 1.6 introduced
database-level autocommit; since then, execution in such a context
voids the effect of ``select_for_update()``. It is, therefore, assumed
now to be an error and raises an exception.

This change was made because such errors can be caused by including an
app which expects global transactions (e.g. :setting:`ATOMIC_REQUESTS
<DATABASE-ATOMIC_REQUESTS>` set to ``True``), or Django's old autocommit
behavior, in a project which runs without them; and further, such
errors may manifest as data-corruption bugs.

This change may cause test failures if you use ``select_for_update()``
in a test class which is a subclass of
:class:`~django.test.TransactionTestCase` rather than
:class:`~django.test.TestCase`.

Other bugfixes and changes
==========================

* Content retrieved from the GeoIP library is now properly decoded from its
default ``iso-8859-1`` encoding
(:ticket:`21996`).

* Fixed ``AttributeError`` when using
:meth:`~django.db.models.query.QuerySet.bulk_create` with ``ForeignObject``
(:ticket:`21566`).

* Fixed crash of ``QuerySet``\s that use ``F() + timedelta()`` when their query
was compiled more once
(:ticket:`21643`).

* Prevented custom ``widget`` class attribute of
:class:`~django.forms.IntegerField` subclasses from being overwritten by the
code in their ``__init__`` method
(:ticket:`22245`).

* Improved :func:`~django.utils.html.strip_tags` accuracy (but it still cannot
guarantee an HTML-safe result, as stated in the documentation).

* Fixed a regression in the :mod:`django.contrib.gis` SQL compiler for
non-concrete fields (:ticket:`22250`).

* Fixed :attr:`ModelAdmin.preserve_filters
<django.contrib.admin.ModelAdmin.preserve_filters>` when running a site with
a URL prefix (:ticket:`21795`).

* Fixed a crash in the ``find_command`` management utility when the ``PATH``
environment variable wasn't set
(:ticket:`22256`).

* Fixed :djadmin:`changepassword` on Windows
(:ticket:`22364`).

* Avoided shadowing deadlock exceptions on MySQL
(:ticket:`22291`).

* Wrapped database exceptions in ``_set_autocommit``
(:ticket:`22321`).

* Fixed atomicity when closing a database connection or when the database server
disconnects (:ticket:`21239` and :ticket:`21202`)

* Fixed regression in ``prefetch_related`` that caused the related objects
query to include an unnecessary join
(:ticket:`21760`).

Additionally, Django's vendored version of six, ``django.utils.six`` has been
upgraded to the latest release (1.6.1).


==========================

1.6.2

Not secure
==========================

*February 6, 2014*

This is Django 1.6.2, a bugfix release for Django 1.6. Django 1.6.2 fixes
several bugs in 1.6.1:

* Prevented the base geometry object of a prepared geometry to be garbage
collected, which could lead to crash Django
(:ticket:`21662`).

* Fixed a crash when executing the :djadmin:`changepassword` command when the
user object representation contained non-ASCII characters
(:ticket:`21627`).

* The :djadmin:`collectstatic` command will raise an error rather than
default to using the current working directory if :setting:`STATIC_ROOT` is
not set. Combined with the ``--clear`` option, the previous behavior could
wipe anything below the current working directory
(:ticket:`21581`).

* Fixed mail encoding on Python 3.3.3+
(:ticket:`21093`).

* Fixed an issue where when
``settings.DATABASES['default']['AUTOCOMMIT'] = False``, the connection
wasn't in autocommit mode but Django pretended it was.

* Fixed a regression in multiple-table inheritance ``exclude()`` queries
(:ticket:`21787`).

* Added missing items to ``django.utils.timezone.__all__``
(:ticket:`21880`).

* Fixed a field misalignment issue with ``select_related()`` and model
inheritance
(:ticket:`21413`).

* Fixed join promotion for negated ``AND`` conditions
(:ticket:`21748`).

* Oracle database introspection now works with boolean and float fields
(:ticket:`19884`).

* Fixed an issue where lazy objects weren't actually marked as safe when passed
through :func:`~django.utils.safestring.mark_safe` and could end up being
double-escaped (:ticket:`21882`).

Additionally, Django's vendored version of six, ``django.utils.six`` has been
upgraded to the latest release (1.5.2).


==========================

1.6.1

Not secure
==========================

*December 12, 2013*

This is Django 1.6.1, a bugfix release for Django 1.6. In addition to the bug
fixes listed below, translations submitted since the 1.6 release are also
included.

Bug fixes
=========

* Fixed ``BCryptSHA256PasswordHasher`` with ``py-bcrypt`` and Python 3
(21398).
* Fixed a regression that prevented a ``ForeignKey`` with a hidden reverse
manager (``related_name`` ending with '+') from being used as a lookup for
``prefetch_related`` (21410).
* Fixed :meth:`Queryset.datetimes<django.db.models.query.QuerySet.datetimes>`
raising ``AttributeError`` in some situations (21432).
* Fixed :class:`~django.contrib.auth.backends.ModelBackend` raising
``UnboundLocalError`` if :func:`~django.contrib.auth.get_user_model`
raised an error (21439).
* Fixed a regression that prevented editable ``GenericRelation`` subclasses
from working in ``ModelForms`` (21428).
* Added missing ``to_python`` method for ``ModelMultipleChoiceField`` which
is required in Django 1.6 to properly detect changes from initial values
(21568).
* Fixed ``django.contrib.humanize`` translations where the Unicode sequence
for the non-breaking space was returned verbatim (21415).
* Fixed :djadmin:`loaddata` error when fixture file name contained any dots
not related to file extensions (21457) or when fixture path was relative
but located in a subdirectory (21551).
* Fixed display of inline instances in formsets when parent has 0 for primary
key (21472).
* Fixed a regression where custom querysets for foreign keys were overwritten
if ``ModelAdmin`` had ordering set (21405).
* Removed mention of a feature in the ``--locale``/``-l`` option of the
``makemessages`` and ``compilemessages`` commands that never worked as
promised: Support of multiple locale names separated by commas. It's still
possible to specify multiple locales in one run by using the option
multiple times (21488, 17181).
* Fixed a regression that unnecessarily triggered settings configuration when
importing ``get_wsgi_application`` (21486).
* Fixed test client ``logout()`` method when using the cookie-based session
backend (21448).
* Fixed a crash when a ``GeometryField`` uses a non-geometric widget (21496).
* Fixed password hash upgrade when changing the iteration count (21535).
* Fixed a bug in the debug view when the URLconf only contains one element
(21530).
* Re-added missing search result count and reset link in changelist admin view
(21510).
* The current language is no longer saved to the session by ``LocaleMiddleware``
on every response, but rather only after a logout (21473).
* Fixed a crash when executing ``runserver`` on non-English systems and when the
formatted date in its output contained non-ASCII characters (21358).
* Fixed a crash in the debug view after an exception occurred on Python ≥ 3.3
(21443).
* Fixed a crash in :class:`~django.db.models.ImageField` on some platforms
(Homebrew and RHEL6 reported) (21355).
* Fixed a regression when using generic relations in ``ModelAdmin.list_filter``
(21431).


=========================

1.6

Not secure
=========================

.. note::

Dedicated to Malcolm Tredinnick

On March 17, 2013, the Django project and the free software community lost
a very dear friend and developer.

Malcolm was a long-time contributor to Django, a model community member, a
brilliant mind, and a friend. His contributions to Django — and to many other
open source projects — are nearly impossible to enumerate. Many on the core
Django team had their first patches reviewed by him; his mentorship enriched
us. His consideration, patience, and dedication will always be an inspiration
to us.

This release of Django is for Malcolm.

-- The Django Developers

*November 6, 2013*

Welcome to Django 1.6!

These release notes cover the :ref:`new features <whats-new-1.6>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-1.6>` you'll
want to be aware of when upgrading from Django 1.5 or older versions. We've
also dropped some features, which are detailed in :ref:`our deprecation plan
<deprecation-removed-in-1.6>`, and we've :ref:`begun the deprecation process
for some features <deprecated-features-1.6>`.

Python compatibility
====================

Django 1.6, like Django 1.5, requires Python 2.6.5 or above. Python 3 is also
officially supported. We **highly recommend** the latest minor release for each
supported Python series (2.6.X, 2.7.X, 3.2.X, and 3.3.X).

Django 1.6 will be the final release series to support Python 2.6; beginning
with Django 1.7, the minimum supported Python version will be 2.7.

Python 3.4 is not supported, but support will be added in Django 1.7.

.. _whats-new-1.6:

What's new in Django 1.6
========================

Simplified default project and app templates
--------------------------------------------

The default templates used by :djadmin:`startproject` and :djadmin:`startapp`
have been simplified and modernized. The :doc:`admin
</ref/contrib/admin/index>` is now enabled by default in new projects; the
:doc:`sites </ref/contrib/sites>` framework no longer is. :ref:`clickjacking
prevention <clickjacking-prevention>` is now on and the database defaults to
SQLite.

If the default templates don't suit your tastes, you can use :ref:`custom
project and app templates <custom-app-and-project-templates>`.

Improved transaction management
-------------------------------

Django's transaction management was overhauled. Database-level autocommit is
now turned on by default. This makes transaction handling more explicit and
should improve performance. The existing APIs were deprecated, and new APIs
were introduced, as described in the :doc:`transaction management docs
</topics/db/transactions>`.

Persistent database connections
-------------------------------

Django now supports reusing the same database connection for several requests.
This avoids the overhead of reestablishing a connection at the beginning of
each request. For backwards compatibility, this feature is disabled by
default. See :ref:`persistent-database-connections` for details.

Discovery of tests in any test module
-------------------------------------

Django 1.6 ships with a new test runner that allows more flexibility in the
location of tests. The previous runner
(``django.test.simple.DjangoTestSuiteRunner``) found tests only in the
``models.py`` and ``tests.py`` modules of a Python package in
:setting:`INSTALLED_APPS`.

The new runner (``django.test.runner.DiscoverRunner``) uses the test discovery
features built into ``unittest2`` (the version of ``unittest`` in the
Python 2.7+ standard library, and bundled with Django). With test discovery,
tests can be located in any module whose name matches the pattern ``test*.py``.

In addition, the test labels provided to ``./manage.py test`` to nominate
specific tests to run must now be full Python dotted paths (or directory
paths), rather than ``applabel.TestCase.test_method_name`` pseudo-paths. This
allows running tests located anywhere in your codebase, rather than only in
:setting:`INSTALLED_APPS`. For more details, see :doc:`/topics/testing/index`.

This change is backwards-incompatible; see the :ref:`backwards-incompatibility
notes<new-test-runner>`.

Time zone aware aggregation
---------------------------

The support for :doc:`time zones </topics/i18n/timezones>` introduced in
Django 1.4 didn't work well with :meth:`QuerySet.dates()
<django.db.models.query.QuerySet.dates>`: aggregation was always performed in
UTC. This limitation was lifted in Django 1.6. Use :meth:`QuerySet.datetimes()
<django.db.models.query.QuerySet.datetimes>` to perform time zone aware
aggregation on a :class:`~django.db.models.DateTimeField`.

Support for savepoints in SQLite
--------------------------------

Django 1.6 adds support for savepoints in SQLite, with some :ref:`limitations
<savepoints-in-sqlite>`.

``BinaryField`` model field
---------------------------

A new :class:`django.db.models.BinaryField` model field allows storage of raw
binary data in the database.

GeoDjango form widgets
----------------------

GeoDjango now provides :doc:`form fields and widgets </ref/contrib/gis/forms-api>`
for its geo-specialized fields. They are OpenLayers-based by default, but they
can be customized to use any other JS framework.

``check`` management command added for verifying compatibility
--------------------------------------------------------------

A :djadmin:`check` management command was added, enabling you to verify if your
current configuration (currently oriented at settings) is compatible with the
current version of Django.

:meth:`Model.save() <django.db.models.Model.save()>` algorithm changed
----------------------------------------------------------------------

The :meth:`Model.save() <django.db.models.Model.save()>` method now
tries to directly ``UPDATE`` the database if the instance has a primary
key value. Previously ``SELECT`` was performed to determine if ``UPDATE``
or ``INSERT`` were needed. The new algorithm needs only one query for
updating an existing row while the old algorithm needed two. See
:meth:`Model.save() <django.db.models.Model.save()>` for more details.

In some rare cases the database doesn't report that a matching row was
found when doing an ``UPDATE``. An example is the PostgreSQL ``ON UPDATE``
trigger which returns ``NULL``. In such cases it is possible to set
:attr:`django.db.models.Options.select_on_save` flag to force saving to
use the old algorithm.

Minor features
--------------

* Authentication backends can raise ``PermissionDenied`` to immediately fail
the authentication chain.

* The ``HttpOnly`` flag can be set on the CSRF cookie with
:setting:`CSRF_COOKIE_HTTPONLY`.

* The :meth:`~django.test.TransactionTestCase.assertQuerysetEqual` now checks
for undefined order and raises :exc:`ValueError` if undefined
order is spotted. The order is seen as undefined if the given ``QuerySet``
isn't ordered and there is more than one ordered value to compare against.

* Added :meth:`~django.db.models.query.QuerySet.earliest` for symmetry with
:meth:`~django.db.models.query.QuerySet.latest`.

* In addition to :lookup:`year`, :lookup:`month` and :lookup:`day`, the ORM
now supports :lookup:`hour`, :lookup:`minute` and :lookup:`second` lookups.

* Django now wraps all :pep:`249` exceptions.

* The default widgets for :class:`~django.forms.EmailField`,
:class:`~django.forms.URLField`, :class:`~django.forms.IntegerField`,
:class:`~django.forms.FloatField` and :class:`~django.forms.DecimalField` use
the new type attributes available in HTML5 (``type='email'``, ``type='url'``,
``type='number'``). Note that due to erratic support of the ``number``
input type with localized numbers in current browsers, Django only uses it
when numeric fields are not localized.

* The ``number`` argument for :ref:`lazy plural translations
<lazy-plural-translations>` can be provided at translation time rather than
at definition time.

* For custom management commands: Verification of the presence of valid
settings in commands that ask for it by using the
``BaseCommand.can_import_settings`` internal
option is now performed independently from handling of the locale that
should be active during the execution of the command. The latter can now be
influenced by the new
``BaseCommand.leave_locale_alone`` internal
option. See :ref:`management-commands-and-locales` for more details.

* The :attr:`~django.views.generic.edit.DeletionMixin.success_url` of
:class:`~django.views.generic.edit.DeletionMixin` is now interpolated with
its ``object``’s ``__dict__``.

* :class:`~django.http.HttpResponseRedirect` and
:class:`~django.http.HttpResponsePermanentRedirect` now provide an ``url``
attribute (equivalent to the URL the response will redirect to).

* The ``MemcachedCache`` cache backend now uses the latest :mod:`pickle`
protocol available.

* Added :class:`~django.contrib.messages.views.SuccessMessageMixin` which
provides a ``success_message`` attribute for
:class:`~django.views.generic.edit.FormView` based classes.

* Added the :attr:`django.db.models.ForeignKey.db_constraint` and
:attr:`django.db.models.ManyToManyField.db_constraint` options.

* The jQuery library embedded in the admin has been upgraded to version 1.9.1.

* Syndication feeds (:mod:`django.contrib.syndication`) can now pass extra
context through to feed templates using a new
:meth:`Feed.get_context_data()
<django.contrib.syndication.Feed.get_context_data>` callback.

* The admin list columns have a ``column-<field_name>`` class in the HTML
so the columns header can be styled with CSS, e.g. to set a column width.

* The :ref:`isolation level<database-isolation-level>` can be customized under
PostgreSQL.

* The :ttag:`blocktrans` template tag now respects
``TEMPLATE_STRING_IF_INVALID`` for variables not present in the
context, just like other template constructs.

* ``SimpleLazyObject``\s will now present more helpful representations in shell
debugging situations.

* Generic :class:`~django.contrib.gis.db.models.GeometryField` is now editable
with the OpenLayers widget in the admin.

* The documentation contains a :doc:`deployment checklist
</howto/deployment/checklist>`.

* The :djadmin:`diffsettings` command gained a ``--all`` option.

* ``django.forms.fields.Field.__init__`` now calls ``super()``, allowing
field mixins to implement ``__init__()`` methods that will reliably be
called.

* The ``validate_max`` parameter was added to ``BaseFormSet`` and
:func:`~django.forms.formsets.formset_factory`, and ``ModelForm`` and inline
versions of the same. The behavior of validation for formsets with
``max_num`` was clarified. The previously undocumented behavior that
hardened formsets against memory exhaustion attacks was documented,
and the undocumented limit of the higher of 1000 or ``max_num`` forms
was changed so it is always 1000 more than ``max_num``.

* Added ``BCryptSHA256PasswordHasher`` to resolve the password truncation issue
with bcrypt.

* `Pillow`_ is now the preferred image manipulation library to use with Django.
`PIL`_ is pending deprecation (support to be removed in Django 1.8).
To upgrade, you should **first** uninstall PIL, **then** install Pillow.

.. _`Pillow`: https://pypi.org/project/Pillow/
.. _`PIL`: https://pypi.org/project/PIL/

* :class:`~django.forms.ModelForm` accepts several new ``Meta``
options.

* Fields included in the ``localized_fields`` list will be localized
(by setting ``localize`` on the form field).
* The ``labels``, ``help_texts`` and ``error_messages`` options may be used
to customize the default fields, see
:ref:`modelforms-overriding-default-fields` for details.

* The ``choices`` argument to model fields now accepts an iterable of iterables
instead of requiring an iterable of lists or tuples.

* The reason phrase can be customized in HTTP responses using
:attr:`~django.http.HttpResponse.reason_phrase`.

* When giving the URL of the next page for
``django.contrib.auth.views.logout()``,
``django.contrib.auth.views.password_reset()``,
``django.contrib.auth.views.password_reset_confirm()``,
and ``django.contrib.auth.views.password_change()``, you can now pass
URL names and they will be resolved.

* The new :option:`dumpdata --pks` option specifies the primary keys of objects
to dump. This option can only be used with one model.

* Added ``QuerySet`` methods :meth:`~django.db.models.query.QuerySet.first`
and :meth:`~django.db.models.query.QuerySet.last` which are convenience
methods returning the first or last object matching the filters. Returns
``None`` if there are no objects matching.

* :class:`~django.views.generic.base.View` and
:class:`~django.views.generic.base.RedirectView` now support HTTP ``PATCH``
method.

* ``GenericForeignKey`` now takes an optional ``for_concrete_model`` argument,
which when set to ``False`` allows the field to reference proxy models. The
default is ``True`` to retain the old behavior.

* The :class:`~django.middleware.locale.LocaleMiddleware` now stores the active
language in session if it is not present there. This prevents loss of
language settings after session flush, e.g. logout.

* :exc:`~django.core.exceptions.SuspiciousOperation` has been differentiated
into a number of subclasses, and each will log to a matching named logger
under the ``django.security`` logging hierarchy. Along with this change,
a ``handler400`` mechanism and default view are used whenever
a ``SuspiciousOperation`` reaches the WSGI handler to return an
``HttpResponseBadRequest``.

* The :exc:`~django.db.models.Model.DoesNotExist` exception now includes a
message indicating the name of the attribute used for the lookup.

* The :meth:`~django.db.models.query.QuerySet.get_or_create` method no longer
requires at least one keyword argument.

* The :class:`~django.test.SimpleTestCase` class includes a new assertion
helper for testing formset errors:
:meth:`~django.test.SimpleTestCase.assertFormsetError`.

* The list of related fields added to a
:class:`~django.db.models.query.QuerySet` by
:meth:`~django.db.models.query.QuerySet.select_related` can be cleared using
``select_related(None)``.

* The :meth:`~django.contrib.admin.InlineModelAdmin.get_extra` and
:meth:`~django.contrib.admin.InlineModelAdmin.get_max_num` methods on
:class:`~django.contrib.admin.InlineModelAdmin` may be overridden to
customize the extra and maximum number of inline forms.

* Formsets now have a
:meth:`~django.forms.formsets.BaseFormSet.total_error_count` method.

* :class:`~django.forms.ModelForm` fields can now override error messages
defined in model fields by using the
:attr:`~django.forms.Field.error_messages` argument of a ``Field``’s
constructor. To take advantage of this new feature with your custom fields,
:ref:`see the updated recommendation <raising-validation-error>` for raising
a ``ValidationError``.

* :class:`~django.contrib.admin.ModelAdmin` now preserves filters on the list view
after creating, editing or deleting an object. It's possible to restore the previous
behavior of clearing filters by setting the
:attr:`~django.contrib.admin.ModelAdmin.preserve_filters` attribute to ``False``.

* Added
:meth:`FormMixin.get_prefix<django.views.generic.edit.FormMixin.get_prefix>`
(which returns
:attr:`FormMixin.prefix<django.views.generic.edit.FormMixin.prefix>` by
default) to allow customizing the :attr:`~django.forms.Form.prefix` of the
form.

* Raw queries (``Manager.raw()`` or ``cursor.execute()``) can now use the
"pyformat" parameter style, where placeholders in the query are given as
``'%(name)s'`` and the parameters are passed as a dictionary rather than
a list (except on SQLite). This has long been possible (but not officially
supported) on MySQL and PostgreSQL, and is now also available on Oracle.

* The default iteration count for the PBKDF2 password hasher has been
increased by 20%. This backwards compatible change will not affect
existing passwords or users who have subclassed
``django.contrib.auth.hashers.PBKDF2PasswordHasher`` to change the
default value. Passwords :ref:`will be upgraded <password-upgrades>` to use
the new iteration count as necessary.

.. _backwards-incompatible-1.6:

Backwards incompatible changes in 1.6
=====================================

.. warning::

In addition to the changes outlined in this section, be sure to review the
:ref:`deprecation plan <deprecation-removed-in-1.6>` for any features that
have been removed. If you haven't updated your code within the
deprecation timeline for a given feature, its removal may appear as a
backwards incompatible change.

New transaction management model
--------------------------------

Behavior changes
~~~~~~~~~~~~~~~~

Database-level autocommit is enabled by default in Django 1.6. While this
doesn't change the general spirit of Django's transaction management, there
are a few backwards-incompatibilities.

Savepoints and ``assertNumQueries``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The changes in transaction management may result in additional statements to
create, release or rollback savepoints. This is more likely to happen with
SQLite, since it didn't support savepoints until this release.

If tests using :meth:`~django.test.TransactionTestCase.assertNumQueries` fail
because of a higher number of queries than expected, check that the extra
queries are related to savepoints, and adjust the expected number of queries
accordingly.

Autocommit option for PostgreSQL
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In previous versions, database-level autocommit was only an option for
PostgreSQL, and it was disabled by default. This option is now ignored and can
be removed.

.. _new-test-runner:

New test runner
---------------

In order to maintain greater consistency with Python's ``unittest`` module, the
new test runner (``django.test.runner.DiscoverRunner``) does not automatically
support some types of tests that were supported by the previous runner:

* Tests in ``models.py`` and ``tests/__init__.py`` files will no longer be
found and run. Move them to a file whose name begins with ``test``.

* Doctests will no longer be automatically discovered. To integrate doctests in
your test suite, follow the :ref:`recommendations in the Python documentation
<doctest-unittest-api>`.

Django bundles a modified version of the :mod:`doctest` module from the Python
standard library (in ``django.test._doctest``) and includes some additional
doctest utilities. These utilities are deprecated and will be removed in Django
1.8; doctest suites should be updated to work with the standard library's
doctest module (or converted to ``unittest``-compatible tests).

If you wish to delay updates to your test suite, you can set your
:setting:`TEST_RUNNER` setting to ``django.test.simple.DjangoTestSuiteRunner``
to fully restore the old test behavior. ``DjangoTestSuiteRunner`` is deprecated
but will not be removed from Django until version 1.8.

Removal of ``django.contrib.gis.tests.GeoDjangoTestSuiteRunner`` GeoDjango custom test runner
---------------------------------------------------------------------------------------------

This is for developers working on the GeoDjango application itself and related
to the item above about changes in the test runners:

The ``django.contrib.gis.tests.GeoDjangoTestSuiteRunner`` test runner has been
removed and the standalone GeoDjango tests execution setup it implemented isn't
supported anymore. To run the GeoDjango tests simply use the new
``DiscoverRunner`` and specify the ``django.contrib.gis`` app.

Custom user models in tests
---------------------------

The introduction of the new test runner has also slightly changed the way that
test models are imported. As a result, any test that overrides ``AUTH_USER_MODEL``
to test behavior with one of Django's test user models (
``django.contrib.auth.tests.custom_user.CustomUser`` and
``django.contrib.auth.tests.custom_user.ExtensionUser``) must now
explicitly import the User model in your test module::

from django.contrib.auth.tests.custom_user import CustomUser

override_settings(AUTH_USER_MODEL='auth.CustomUser')
class CustomUserFeatureTests(TestCase):
def test_something(self):
Test code here ...

This import forces the custom user model to be registered. Without this import,
the test will be unable to swap in the custom user model, and you will get an
error reporting::

ImproperlyConfigured: AUTH_USER_MODEL refers to model 'auth.CustomUser' that has not been installed

Time zone-aware ``day``, ``month``, and ``week_day`` lookups
------------------------------------------------------------

Django 1.6 introduces time zone support for :lookup:`day`, :lookup:`month`,
and :lookup:`week_day` lookups when :setting:`USE_TZ` is ``True``. These
lookups were previously performed in UTC regardless of the current time zone.

This requires :ref:`time zone definitions in the database
<database-time-zone-definitions>`. If you're using SQLite, you must install
pytz_. If you're using MySQL, you must install pytz_ and load the time zone
tables with `mysql_tzinfo_to_sql`_.

.. _pytz: http://pytz.sourceforge.net/
.. _mysql_tzinfo_to_sql: https://dev.mysql.com/doc/refman/en/mysql-tzinfo-to-sql.html

Addition of ``QuerySet.datetimes()``
------------------------------------

When the :doc:`time zone support </topics/i18n/timezones>` added in Django 1.4
was active, :meth:`QuerySet.dates() <django.db.models.query.QuerySet.dates>`
lookups returned unexpected results, because the aggregation was performed in
UTC. To fix this, Django 1.6 introduces a new API, :meth:`QuerySet.datetimes()
<django.db.models.query.QuerySet.datetimes>`. This requires a few changes in
your code.

``QuerySet.dates()`` returns ``date`` objects
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

:meth:`QuerySet.dates() <django.db.models.query.QuerySet.dates>` now returns a
list of :class:`~datetime.date`. It used to return a list of
:class:`~datetime.datetime`.

:meth:`QuerySet.datetimes() <django.db.models.query.QuerySet.datetimes>`
returns a list of :class:`~datetime.datetime`.

``QuerySet.dates()`` no longer usable on ``DateTimeField``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

:meth:`QuerySet.dates() <django.db.models.query.QuerySet.dates>` raises an
error if it's used on :class:`~django.db.models.DateTimeField` when time
zone support is active. Use :meth:`QuerySet.datetimes()
<django.db.models.query.QuerySet.datetimes>` instead.

``date_hierarchy`` requires time zone definitions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The :attr:`~django.contrib.admin.ModelAdmin.date_hierarchy` feature of the
admin now relies on :meth:`QuerySet.datetimes()
<django.db.models.query.QuerySet.datetimes>` when it's used on a
:class:`~django.db.models.DateTimeField`.

This requires time zone definitions in the database when :setting:`USE_TZ` is
``True``. :ref:`Learn more <database-time-zone-definitions>`.

``date_list`` in generic views requires time zone definitions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For the same reason, accessing ``date_list`` in the context of a date-based
generic view requires time zone definitions in the database when the view is
based on a :class:`~django.db.models.DateTimeField` and :setting:`USE_TZ` is
``True``. :ref:`Learn more <database-time-zone-definitions>`.

New lookups may clash with model fields
---------------------------------------

Django 1.6 introduces ``hour``, ``minute``, and ``second`` lookups on
:class:`~django.db.models.DateTimeField`. If you had model fields called
``hour``, ``minute``, or ``second``, the new lookups will clash with you field
names. Append an explicit :lookup:`exact` lookup if this is an issue.

``BooleanField`` no longer defaults to ``False``
------------------------------------------------

When a :class:`~django.db.models.BooleanField` doesn't have an explicit
:attr:`~django.db.models.Field.default`, the implicit default value is
``None``. In previous version of Django, it was ``False``, but that didn't
represent accurately the lack of a value.

Code that relies on the default value being ``False`` may raise an exception
when saving new model instances to the database, because ``None`` isn't an
acceptable value for a :class:`~django.db.models.BooleanField`. You should
either specify ``default=False`` in the field definition, or ensure the field
is set to ``True`` or ``False`` before saving the object.

Translations and comments in templates
--------------------------------------

Extraction of translations after comments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Extraction of translatable literals from templates with the
:djadmin:`makemessages` command now correctly detects i18n constructs when
they are located after a ``{`` / ``}``-type comment on the same line. E.g.:

.. code-block:: html+django

{ A comment }{% trans "This literal was incorrectly ignored. Not anymore" %}

Location of translator comments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

:ref:`translator-comments-in-templates` specified using ``{`` / ``}`` need to
be at the end of a line. If they are not, the comments are ignored and
:djadmin:`makemessages` will generate a warning. For example:

.. code-block:: html+django

{ Translators: This is ignored }{% trans "Translate me" %}
{{ title }}{ Translators: Extracted and associated with 'Welcome' below }
<h1>{% trans "Welcome" %}</h1>

Quoting in ``reverse()``
------------------------

When reversing URLs, Django didn't apply ``django.utils.http.urlquote``
to arguments before interpolating them in URL patterns. This bug is fixed in
Django 1.6. If you worked around this bug by applying URL quoting before
passing arguments to ``reverse()``, this may result in double-quoting. If this
happens, simply remove the URL quoting from your code. You will also have to
replace special characters in URLs used in
:func:`~django.test.SimpleTestCase.assertRedirects` with their encoded
versions.

Storage of IP addresses in the comments app
-------------------------------------------

The comments app now uses a
``GenericIPAddressField`` for storing commenters' IP addresses, to support
comments submitted from IPv6 addresses. Until now, it stored them in an
``IPAddressField``, which is only meant to support IPv4. When saving a comment
made from an IPv6 address, the address would be silently truncated on MySQL
databases, and raise an exception on Oracle. You will need to change the
column type in your database to benefit from this change.

For MySQL, execute this query on your project's database:

.. code-block:: sql

ALTER TABLE django_comments MODIFY ip_address VARCHAR(39);

For Oracle, execute this query:

.. code-block:: sql

ALTER TABLE DJANGO_COMMENTS MODIFY (ip_address VARCHAR2(39));

If you do not apply this change, the behavior is unchanged: on MySQL, IPv6
addresses are silently truncated; on Oracle, an exception is generated. No
database change is needed for SQLite or PostgreSQL databases.

Percent literals in ``cursor.execute`` queries
----------------------------------------------

When you are running raw SQL queries through the
:ref:`cursor.execute <executing-custom-sql>` method, the rule about doubling
percent literals (``%``) inside the query has been unified. Past behavior
depended on the database backend. Now, across all backends, you only need to
double literal percent characters if you are also providing replacement
parameters. For example::

No parameters, no percent doubling
cursor.execute("SELECT foo FROM bar WHERE baz = '30%'")

Parameters passed, non-placeholders have to be doubled
cursor.execute("SELECT foo FROM bar WHERE baz = '30%%' and id = %s", [self.id])

``SQLite`` users need to check and update such queries.

.. _m2m-help_text:

Help text of model form fields for ManyToManyField fields
---------------------------------------------------------

HTML rendering of model form fields corresponding to
:class:`~django.db.models.ManyToManyField` model fields used to get the
hard-coded sentence:

*Hold down "Control", or "Command" on a Mac, to select more than one.*

(or its translation to the active locale) imposed as the help legend shown along
them if neither :attr:`model <django.db.models.Field.help_text>` nor :attr:`form
<django.forms.Field.help_text>` ``help_text`` attributes were specified by the
user (or this string was appended to any ``help_text`` that was provided).

Since this happened at the model layer, there was no way to prevent the text
from appearing in cases where it wasn't applicable such as form fields that
implement user interactions that don't involve a keyboard and/or a mouse.

Starting with Django 1.6, as an ad-hoc temporary backward-compatibility
provision, the logic to add the "Hold down..." sentence has been moved to the
model form field layer and modified to add the text only when the associated
widget is :class:`~django.forms.SelectMultiple` or selected subclasses.

The change can affect you in a backward incompatible way if you employ custom
model form fields and/or widgets for ``ManyToManyField`` model fields whose UIs
do rely on the automatic provision of the mentioned hard-coded sentence. These
form field implementations need to adapt to the new scenario by providing their
own handling of the ``help_text`` attribute.

Applications that use Django :doc:`model form </topics/forms/modelforms>`
facilities together with Django built-in form :doc:`fields </ref/forms/fields>`
and :doc:`widgets </ref/forms/widgets>` aren't affected but need to be aware of
what's described in :ref:`m2m-help_text-deprecation` below.

QuerySet iteration
------------------

The ``QuerySet`` iteration was changed to immediately convert all fetched
rows to ``Model`` objects. In Django 1.5 and earlier the fetched rows were
converted to ``Model`` objects in chunks of 100.

Existing code will work, but the amount of rows converted to objects
might change in certain use cases. Such usages include partially looping
over a queryset or any usage which ends up doing ``__bool__`` or
``__contains__``.

Notably most database backends did fetch all the rows in one go already in
1.5.

It is still possible to convert the fetched rows to ``Model`` objects
lazily by using the :meth:`~django.db.models.query.QuerySet.iterator()`
method.

:meth:`BoundField.label_tag<django.forms.BoundField.label_tag>` now includes the form's :attr:`~django.forms.Form.label_suffix`
-------------------------------------------------------------------------------------------------------------------------------

This is consistent with how methods like
:meth:`Form.as_p<django.forms.Form.as_p>` and
:meth:`Form.as_ul<django.forms.Form.as_ul>` render labels.

If you manually render ``label_tag`` in your templates:

.. code-block:: html+django

{{ form.my_field.label_tag }}: {{ form.my_field }}

you'll want to remove the colon (or whatever other separator you may be
using) to avoid duplicating it when upgrading to Django 1.6. The following
template in Django 1.6 will render identically to the above template in Django
1.5, except that the colon will appear inside the ``<label>`` element.

.. code-block:: html+django

{{ form.my_field.label_tag }} {{ form.my_field }}

will render something like:

.. code-block:: html

<label for="id_my_field">My Field:</label> <input id="id_my_field" type="text" name="my_field" />

If you want to keep the current behavior of rendering ``label_tag`` without
the ``label_suffix``, instantiate the form ``label_suffix=''``. You can also
customize the ``label_suffix`` on a per-field basis using the new
``label_suffix`` parameter on :meth:`~django.forms.BoundField.label_tag`.

Admin views ``_changelist_filters`` GET parameter
-------------------------------------------------

To achieve preserving and restoring list view filters, admin views now
pass around the ``_changelist_filters`` GET parameter. It's important that you
account for that change if you have custom admin templates or if your tests
rely on the previous URLs. If you want to revert to the original behavior you
can set the
:attr:`~django.contrib.admin.ModelAdmin.preserve_filters` attribute to ``False``.

``django.contrib.auth`` password reset uses base 64 encoding of ``User`` PK
---------------------------------------------------------------------------

Past versions of Django used base 36 encoding of the ``User`` primary key in
the password reset views and URLs
(``django.contrib.auth.views.password_reset_confirm()``). Base 36 encoding is
sufficient if the user primary key is an integer, however, with the
introduction of custom user models in Django 1.5, that assumption may no longer
be true.

``django.contrib.auth.views.password_reset_confirm()`` has been modified to
take a ``uidb64`` parameter instead of ``uidb36``. If you are reversing this
view, for example in a custom ``password_reset_email.html`` template, be sure
to update your code.

A temporary shim for ``django.contrib.auth.views.password_reset_confirm()``
that will allow password reset links generated prior to Django 1.6 to continue
to work has been added to provide backwards compatibility; this will be removed
in Django 1.7. Thus, as long as your site has been running Django 1.6 for more
than ``PASSWORD_RESET_TIMEOUT_DAYS``, this change will have no effect.
If not (for example, if you upgrade directly from Django 1.5 to Django 1.7),
then any password reset links generated before you upgrade to Django 1.7 or
later won't work after the upgrade.

In addition, if you have any custom password reset URLs, you will need to
update them by replacing ``uidb36`` with ``uidb64`` and the dash that follows
that pattern with a slash. Also add ``_\-`` to the list of characters that may
match the ``uidb64`` pattern.

For example::

url(r'^reset/(?P<uidb36>[0-9A-Za-z]+)-(?P<token>.+)/$',
'django.contrib.auth.views.password_reset_confirm',
name='password_reset_confirm'),

becomes::

url(r'^reset/(?P<uidb64>[0-9A-Za-z_\-]+)/(?P<token>.+)/$',
'django.contrib.auth.views.password_reset_confirm',
name='password_reset_confirm'),

You may also want to add the shim to support the old style reset links. Using
the example above, you would modify the existing url by replacing
``django.contrib.auth.views.password_reset_confirm`` with
``django.contrib.auth.views.password_reset_confirm_uidb36`` and also remove
the ``name`` argument so it doesn't conflict with the new url::

url(r'^reset/(?P<uidb36>[0-9A-Za-z]+)-(?P<token>.+)/$',
'django.contrib.auth.views.password_reset_confirm_uidb36'),

You can remove this URL pattern after your app has been deployed with Django
1.6 for ``PASSWORD_RESET_TIMEOUT_DAYS``.

Default session serialization switched to JSON
----------------------------------------------

Historically, :mod:`django.contrib.sessions` used :mod:`pickle` to serialize
session data before storing it in the backend. If you're using the :ref:`signed
cookie session backend<cookie-session-backend>` and :setting:`SECRET_KEY` is
known by an attacker (there isn't an inherent vulnerability in Django that
would cause it to leak), the attacker could insert a string into their session
which, when unpickled, executes arbitrary code on the server. The technique for
doing so is simple and easily available on the internet. Although the cookie
session storage signs the cookie-stored data to prevent tampering, a
:setting:`SECRET_KEY` leak immediately escalates to a remote code execution
vulnerability.

This attack can be mitigated by serializing session data using JSON rather
than :mod:`pickle`. To facilitate this, Django 1.5.3 introduced a new setting,
:setting:`SESSION_SERIALIZER`, to customize the session serialization format.
For backwards compatibility, this setting defaulted to using :mod:`pickle`
in Django 1.5.3, but we've changed the default to JSON in 1.6. If you upgrade
and switch from pickle to JSON, sessions created before the upgrade will be
lost. While JSON serialization does not support all Python objects like
:mod:`pickle` does, we highly recommend using JSON-serialized sessions. Be
aware of the following when checking your code to determine if JSON
serialization will work for your application:

* JSON requires string keys, so you will likely run into problems if you are
using non-string keys in ``request.session``.
* Setting session expiration by passing ``datetime`` values to
:meth:`~django.contrib.sessions.backends.base.SessionBase.set_expiry` will
not work as ``datetime`` values are not serializable in JSON. You can use
integer values instead.

See the :ref:`session_serialization` documentation for more details.

Object Relational Mapper changes
--------------------------------

Django 1.6 contains many changes to the ORM. These changes fall mostly in
three categories:

1. Bug fixes (e.g. proper join clauses for generic relations, query combining,
join promotion, and join trimming fixes)
2. Preparation for new features. For example the ORM is now internally ready
for multicolumn foreign keys.
3. General cleanup.

These changes can result in some compatibility problems. For example, some
queries will now generate different table aliases. This can affect
:meth:`QuerySet.extra() <django.db.models.query.QuerySet.extra>`. In addition
some queries will now produce different results. An example is
:meth:`exclude(condition) <django.db.models.query.QuerySet.exclude>`
where the condition is a complex one (referencing multijoins inside
:class:`Q objects <django.db.models.Q>`). In many cases the affected
queries didn't produce correct results in Django 1.5 but do now.
Unfortunately there are also cases that produce different results, but
neither Django 1.5 nor 1.6 produce correct results.

Finally, there have been many changes to the ORM internal APIs.

Miscellaneous
-------------

* The ``django.db.models.query.EmptyQuerySet`` can't be instantiated any more -
it is only usable as a marker class for checking if
:meth:`~django.db.models.query.QuerySet.none` has been called:
``isinstance(qs.none(), EmptyQuerySet)``

* If your CSS/JavaScript code used to access HTML input widgets by type, you
should review it as ``type='text'`` widgets might be now output as
``type='email'``, ``type='url'`` or ``type='number'`` depending on their
corresponding field type.

* Form field's :attr:`~django.forms.Field.error_messages` that contain a
placeholder should now always use a named placeholder (``"Value '%(value)s' is
too big"`` instead of ``"Value '%s' is too big"``). See the corresponding
field documentation for details about the names of the placeholders. The
changes in 1.6 particularly affect :class:`~django.forms.DecimalField` and
:class:`~django.forms.ModelMultipleChoiceField`.

* Some :attr:`~django.forms.Field.error_messages` for
:class:`~django.forms.IntegerField`, :class:`~django.forms.EmailField`,
``IPAddressField``, :class:`~django.forms.GenericIPAddressField`, and
:class:`~django.forms.SlugField` have been suppressed because they
duplicated error messages already provided by validators tied to the fields.

* Due to a change in the form validation workflow,
:class:`~django.forms.TypedChoiceField` ``coerce`` method should always
return a value present in the ``choices`` field attribute. That limitation
should be lift again in Django 1.7.

* There have been changes in the way timeouts are handled in cache backends.
Explicitly passing in ``timeout=None`` no longer results in using the
default timeout. It will now set a non-expiring timeout. Passing 0 into the
memcache backend no longer uses the default timeout, and now will
set-and-expire-immediately the value.

* The ``django.contrib.flatpages`` app used to set custom HTTP headers for
debugging purposes. This functionality was not documented and made caching
ineffective so it has been removed, along with its generic implementation,
previously available in ``django.core.xheaders``.

* The ``XViewMiddleware`` has been moved from ``django.middleware.doc`` to
``django.contrib.admindocs.middleware`` because it is an implementation
detail of admindocs, proven not to be reusable in general.

* :class:`~django.db.models.GenericIPAddressField` will now only allow
``blank`` values if ``null`` values are also allowed. Creating a
``GenericIPAddressField`` where ``blank`` is allowed but ``null`` is not
will trigger a model validation error because ``blank`` values are always
stored as ``null``. Previously, storing a ``blank`` value in a field which
did not allow ``null`` would cause a database exception at runtime.

* If a ``NoReverseMatch`` exception is raised from a method when rendering a
template, it is not silenced. For example, ``{{ obj.view_href }}`` will
cause template rendering to fail if ``view_href()`` raises
``NoReverseMatch``. There is no change to the :ttag:`{% url %}<url>` tag, it
causes template rendering to fail like always when ``NoReverseMatch`` is
raised.

* :meth:`django.test.Client.logout` now calls
:meth:`django.contrib.auth.logout` which will send the
:func:`~django.contrib.auth.signals.user_logged_out` signal.

* :ref:`Authentication views <built-in-auth-views>` are now reversed by name,
not their locations in ``django.contrib.auth.views``. If you are using the
views without a ``name``, you should update your ``urlpatterns`` to use
``django.conf.urls.url()`` with the ``name`` parameter. For example::

(r'^reset/done/$', 'django.contrib.auth.views.password_reset_complete')

becomes::

url(r'^reset/done/$', 'django.contrib.auth.views.password_reset_complete', name='password_reset_complete')

* :class:`~django.views.generic.base.RedirectView` now has a ``pattern_name``
attribute which allows it to choose the target by reversing the URL.

* In Django 1.4 and 1.5, a blank string was unintentionally not considered to
be a valid password. This meant
:meth:`~django.contrib.auth.models.User.set_password()` would save a blank
password as an unusable password like
:meth:`~django.contrib.auth.models.User.set_unusable_password()` does, and
thus :meth:`~django.contrib.auth.models.User.check_password()` always
returned ``False`` for blank passwords. This has been corrected in this
release: blank passwords are now valid.

* The admin :attr:`~django.contrib.admin.ModelAdmin.changelist_view` previously
accepted a ``pop`` GET parameter to signify it was to be displayed in a popup.
This parameter has been renamed to ``_popup`` to be consistent with the rest
of the admin views. You should update your custom templates if they use the
previous parameter name.

* :meth:`~django.core.validators.validate_email` now accepts email addresses
with ``localhost`` as the domain.

* The new :option:`makemessages --keep-pot` option prevents deleting the
temporary ``.pot`` file generated before creating the ``.po`` file.

* The undocumented ``django.core.servers.basehttp.WSGIServerException`` has
been removed. Use ``socket.error`` provided by the standard library instead.
This change was also released in Django 1.5.5.

* The signature of :meth:`django.views.generic.base.RedirectView.get_redirect_url`
has changed and now accepts positional arguments as well (``*args, **kwargs``).
Any unnamed captured group will now be passed to ``get_redirect_url()``
which may result in a ``TypeError`` if you don't update the signature of your
custom method.

.. _deprecated-features-1.6:

Features deprecated in 1.6
==========================

Transaction management APIs
---------------------------

Transaction management was completely overhauled in Django 1.6, and the
current APIs are deprecated:

- ``django.middleware.transaction.TransactionMiddleware``
- ``django.db.transaction.autocommit``
- ``django.db.transaction.commit_on_success``
- ``django.db.transaction.commit_manually``
- the ``TRANSACTIONS_MANAGED`` setting

``django.contrib.comments``
---------------------------

Django's comment framework has been deprecated and is no longer supported. It
will be available in Django 1.6 and 1.7, and removed in Django 1.8. Most users
will be better served with a custom solution, or a hosted product like Disqus__.

The code formerly known as ``django.contrib.comments`` is `still available
in an external repository`__.

__ https://disqus.com/
__ https://github.com/django/django-contrib-comments

Support for PostgreSQL versions older than 8.4
----------------------------------------------

The end of upstream support periods was reached in December 2011 for
PostgreSQL 8.2 and in February 2013 for 8.3. As a consequence, Django 1.6 sets
8.4 as the minimum PostgreSQL version it officially supports.

You're strongly encouraged to use the most recent version of PostgreSQL
available, because of performance improvements and to take advantage of the
native streaming replication available in PostgreSQL 9.x.

Changes to :ttag:`cycle` and :ttag:`firstof`
--------------------------------------------

The template system generally escapes all variables to avoid XSS attacks.
However, due to an accident of history, the :ttag:`cycle` and :ttag:`firstof`
tags render their arguments as-is.

Django 1.6 starts a process to correct this inconsistency. The ``future``
template library provides alternate implementations of :ttag:`cycle` and
:ttag:`firstof` that autoescape their inputs. If you're using these tags,
you're encouraged to include the following line at the top of your templates to
enable the new behavior::

{% load cycle from future %}

or::

{% load firstof from future %}

The tags implementing the old behavior have been deprecated, and in Django
1.8, the old behavior will be replaced with the new behavior. To ensure
compatibility with future versions of Django, existing templates should be
modified to use the ``future`` versions.

If necessary, you can temporarily disable auto-escaping with
:func:`~django.utils.safestring.mark_safe` or :ttag:`{% autoescape off %}
<autoescape>`.

``CACHE_MIDDLEWARE_ANONYMOUS_ONLY`` setting
-------------------------------------------

``CacheMiddleware`` and ``UpdateCacheMiddleware`` used to provide a way to
cache requests only if they weren't made by a logged-in user. This mechanism
was largely ineffective because the middleware correctly takes into account the
``Vary: Cookie`` HTTP header, and this header is being set on a variety of
occasions, such as:

* accessing the session, or
* using CSRF protection, which is turned on by default, or
* using a client-side library which sets cookies, like `Google Analytics`__.

This makes the cache effectively work on a per-session basis regardless of the
``CACHE_MIDDLEWARE_ANONYMOUS_ONLY`` setting.

__ https://marketingplatform.google.com/about/analytics/

``SEND_BROKEN_LINK_EMAILS`` setting
-----------------------------------

:class:`~django.middleware.common.CommonMiddleware` used to provide basic
reporting of broken links by email when ``SEND_BROKEN_LINK_EMAILS`` is set to
``True``.

Because of intractable ordering problems between
:class:`~django.middleware.common.CommonMiddleware` and
:class:`~django.middleware.locale.LocaleMiddleware`, this feature was split
out into a new middleware:
:class:`~django.middleware.common.BrokenLinkEmailsMiddleware`.

If you're relying on this feature, you should add
``'django.middleware.common.BrokenLinkEmailsMiddleware'`` to your
``MIDDLEWARE_CLASSES`` setting and remove ``SEND_BROKEN_LINK_EMAILS``
from your settings.

``_has_changed`` method on widgets
----------------------------------

If you defined your own form widgets and defined the ``_has_changed`` method
on a widget, you should now define this method on the form field itself.

``module_name`` model _meta attribute
-------------------------------------

``Model._meta.module_name`` was renamed to ``model_name``. Despite being a
private API, it will go through a regular deprecation path.

``get_(add|change|delete)_permission`` model _meta methods
----------------------------------------------------------

``Model._meta.get_(add|change|delete)_permission`` methods were deprecated.
Even if they were not part of the public API they'll also go through
a regular deprecation path. You can replace them with
``django.contrib.auth.get_permission_codename('action', Model._meta)`` where
``'action'`` is ``'add'``, ``'change'``, or ``'delete'``.

``get_query_set`` and similar methods renamed to ``get_queryset``
-----------------------------------------------------------------

Methods that return a ``QuerySet`` such as ``Manager.get_query_set`` or
``ModelAdmin.queryset`` have been renamed to ``get_queryset``.

If you are writing a library that implements, for example, a
``Manager.get_query_set`` method, and you need to support old Django versions,
you should rename the method and conditionally add an alias with the old name::

class CustomManager(models.Manager):
def get_queryset(self):
pass ...

if django.VERSION < (1, 6):
get_query_set = get_queryset

For Django >= 1.6, models.Manager provides a get_query_set fallback
that emits a warning when used.

If you are writing a library that needs to call the ``get_queryset`` method and
must support old Django versions, you should write::

get_queryset = (some_manager.get_query_set
if hasattr(some_manager, 'get_query_set')
else some_manager.get_queryset)
return get_queryset() etc

In the general case of a custom manager that both implements its own
``get_queryset`` method and calls that method, and needs to work with older Django
versions, and libraries that have not been updated yet, it is useful to define
a ``get_queryset_compat`` method as below and use it internally to your manager::

class YourCustomManager(models.Manager):
def get_queryset(self):
return YourCustomQuerySet() for example

if django.VERSION < (1, 6):
get_query_set = get_queryset

def active(self): for example
return self.get_queryset_compat().filter(active=True)

def get_queryset_compat(self):
get_queryset = (self.get_query_set
if hasattr(self, 'get_query_set')
else self.get_queryset)
return get_queryset()

This helps to minimize the changes that are needed, but also works correctly in
the case of subclasses (such as ``RelatedManagers`` from Django 1.5) which might
override either ``get_query_set`` or ``get_queryset``.


``shortcut`` view and URLconf
-----------------------------

The ``shortcut`` view was moved from ``django.views.defaults`` to
``django.contrib.contenttypes.views`` shortly after the 1.0 release, but the
old location was never deprecated. This oversight was corrected in Django 1.6
and you should now use the new location.

The URLconf ``django.conf.urls.shortcut`` was also deprecated. If you're
including it in an URLconf, simply replace::

(r'^prefix/', include('django.conf.urls.shortcut')),

with::

(r'^prefix/(?P<content_type_id>\d+)/(?P<object_id>.*)/$', 'django.contrib.contenttypes.views.shortcut'),

``ModelForm`` without ``fields`` or ``exclude``
-----------------------------------------------

Previously, if you wanted a :class:`~django.forms.ModelForm` to use all fields on
the model, you could simply omit the ``Meta.fields`` attribute, and all fields
would be used.

This can lead to security problems where fields are added to the model and,
unintentionally, automatically become editable by end users. In some cases,
particular with boolean fields, it is possible for this problem to be completely
invisible. This is a form of `Mass assignment vulnerability
<https://en.wikipedia.org/wiki/Mass_assignment_vulnerability>`_.

For this reason, this behavior is deprecated, and using the ``Meta.exclude``
option is strongly discouraged. Instead, all fields that are intended for
inclusion in the form should be listed explicitly in the ``fields`` attribute.

If this security concern really does not apply in your case, there is a shortcut
to explicitly indicate that all fields should be used - use the special value
``"__all__"`` for the fields attribute::

class MyModelForm(ModelForm):
class Meta:
fields = "__all__"
model = MyModel

If you have custom ``ModelForms`` that only need to be used in the admin, there
is another option. The admin has its own methods for defining fields
(``fieldsets`` etc.), and so adding a list of fields to the ``ModelForm`` is
redundant. Instead, simply omit the ``Meta`` inner class of the ``ModelForm``,
or omit the ``Meta.model`` attribute. Since the ``ModelAdmin`` subclass knows
which model it is for, it can add the necessary attributes to derive a
functioning ``ModelForm``. This behavior also works for earlier Django
versions.

``UpdateView`` and ``CreateView`` without explicit fields
---------------------------------------------------------

The generic views :class:`~django.views.generic.edit.CreateView` and
:class:`~django.views.generic.edit.UpdateView`, and anything else derived from
:class:`~django.views.generic.edit.ModelFormMixin`, are vulnerable to the
security problem described in the section above, because they can automatically
create a ``ModelForm`` that uses all fields for a model.

For this reason, if you use these views for editing models, you must also supply
the ``fields`` attribute (new in Django 1.6), which is a list of model fields
and works in the same way as the :class:`~django.forms.ModelForm`
``Meta.fields`` attribute. Alternatively, you can set the ``form_class``
attribute to a ``ModelForm`` that explicitly defines the fields to be used.
Defining an ``UpdateView`` or ``CreateView`` subclass to be used with a model
but without an explicit list of fields is deprecated.

.. _m2m-help_text-deprecation:

Munging of help text of model form fields for ``ManyToManyField`` fields
------------------------------------------------------------------------

All special handling of the ``help_text`` attribute of ``ManyToManyField`` model
fields performed by standard model or model form fields as described in
:ref:`m2m-help_text` above is deprecated and will be removed in Django 1.8.

Help text of these fields will need to be handled either by applications, custom
form fields or widgets, just like happens with the rest of the model field
types.


===========================

1.5.12

Not secure
===========================

*January 2, 2015*

Django 1.5.12 fixes a regression in the 1.5.9 security release.

Bugfixes
========

* Fixed a regression with dynamically generated inlines and allowed field
references in the admin
(:ticket:`23754`).


===========================

1.5.11

Not secure
===========================

*October 22, 2014*

Django 1.5.11 fixes a couple regressions in the 1.5.9 security release.

Bugfixes
========

* Allowed related many-to-many fields to be referenced in the admin
(:ticket:`23604`).

* Allowed inline and hidden references to admin fields
(:ticket:`23431`).


===========================

1.5.10

Not secure
===========================

*September 2, 2014*

Django 1.5.10 fixes a regression in the 1.5.9 security release.

Bugfixes
========

* Allowed inherited and m2m fields to be referenced in the admin
(:ticket:`22486`)


==========================

1.5.9

Not secure
==========================

*August 20, 2014*

Django 1.5.9 fixes several security issues in 1.5.8.

``reverse()`` could generate URLs pointing to other hosts
=========================================================

In certain situations, URL reversing could generate scheme-relative URLs (URLs
starting with two slashes), which could unexpectedly redirect a user to a
different host. An attacker could exploit this, for example, by redirecting
users to a phishing site designed to ask for user's passwords.

To remedy this, URL reversing now ensures that no URL starts with two slashes
(//), replacing the second slash with its URL encoded counterpart (%2F). This
approach ensures that semantics stay the same, while making the URL relative to
the domain and not to the scheme.

File upload denial-of-service
=============================

Before this release, Django's file upload handing in its default configuration
may degrade to producing a huge number of ``os.stat()`` system calls when a
duplicate filename is uploaded. Since ``stat()`` may invoke IO, this may produce
a huge data-dependent slowdown that slowly worsens over time. The net result is
that given enough time, a user with the ability to upload files can cause poor
performance in the upload handler, eventually causing it to become very slow
simply by uploading 0-byte files. At this point, even a slow network connection
and few HTTP requests would be all that is necessary to make a site unavailable.

We've remedied the issue by changing the algorithm for generating file names
if a file with the uploaded name already exists.
:meth:`Storage.get_available_name()
<django.core.files.storage.Storage.get_available_name>` now appends an
underscore plus a random 7 character alphanumeric string (e.g. ``"_x3a1gho"``),
rather than iterating through an underscore followed by a number (e.g. ``"_1"``,
``"_2"``, etc.).

``RemoteUserMiddleware`` session hijacking
==========================================

When using the :class:`~django.contrib.auth.middleware.RemoteUserMiddleware`
and the ``RemoteUserBackend``, a change to the ``REMOTE_USER`` header between
requests without an intervening logout could result in the prior user's session
being co-opted by the subsequent user. The middleware now logs the user out on
a failed login attempt.

Data leakage via query string manipulation in ``contrib.admin``
===============================================================

In older versions of Django it was possible to reveal any field's data by
modifying the "popup" and "to_field" parameters of the query string on an admin
change form page. For example, requesting a URL like
``/admin/auth/user/?pop=1&t=password`` and viewing the page's HTML allowed
viewing the password hash of each user. While the admin requires users to have
permissions to view the change form pages in the first place, this could leak
data if you rely on users having access to view only certain fields on a model.

To address the issue, an exception will now be raised if a ``to_field`` value
that isn't a related field to a model that has been registered with the admin
is specified.


==========================

1.5.8

Not secure
==========================

*May 14, 2014*

Django 1.5.8 fixes two security issues in 1.5.8.

Caches may incorrectly be allowed to store and serve private data
=================================================================

In certain situations, Django may allow caches to store private data
related to a particular session and then serve that data to requests
with a different session, or no session at all. This can lead to
information disclosure and can be a vector for cache poisoning.

When using Django sessions, Django will set a ``Vary: Cookie`` header to
ensure caches do not serve cached data to requests from other sessions.
However, older versions of Internet Explorer (most likely only Internet
Explorer 6, and Internet Explorer 7 if run on Windows XP or Windows Server
2003) are unable to handle the ``Vary`` header in combination with many content
types. Therefore, Django would remove the header if the request was made by
Internet Explorer.

To remedy this, the special behavior for these older Internet Explorer versions
has been removed, and the ``Vary`` header is no longer stripped from the response.
In addition, modifications to the ``Cache-Control`` header for all Internet Explorer
requests with a ``Content-Disposition`` header have also been removed as they
were found to have similar issues.

Malformed redirect URLs from user input not correctly validated
===============================================================

The validation for redirects did not correctly validate some malformed URLs,
which are accepted by some browsers. This allows a user to be redirected to
an unsafe URL unexpectedly.

Django relies on user input in some cases (e.g.
``django.contrib.auth.views.login()``, ``django.contrib.comments``, and
:doc:`i18n </topics/i18n/index>`) to redirect the user to an "on success" URL.
The security checks for these redirects (namely
``django.utils.http.is_safe_url()``) did not correctly validate some malformed
URLs, such as ``http:\\\\\\djangoproject.com``, which are accepted by some
browsers with more liberal URL parsing.

To remedy this, the validation in ``is_safe_url()`` has been tightened to be able
to handle and correctly validate these malformed URLs.


==========================

1.5.7

Not secure
==========================

*April 28, 2014*

Django 1.5.7 fixes a regression in the 1.5.6 security release.

Bugfixes
========

* Restored the ability to ``reverse()`` views created using
:func:`functools.partial()` (:ticket:`22486`).


==========================

1.5.6

Not secure
==========================

*April 21, 2014*

Django 1.5.6 fixes several bugs in 1.5.5, including three security
issues.

Unexpected code execution using ``reverse()``
=============================================

Django's URL handling is based on a mapping of regex patterns
(representing the URLs) to callable views, and Django's own processing
consists of matching a requested URL against those patterns to
determine the appropriate view to invoke.

Django also provides a convenience function -- ``reverse()`` -- which performs
this process in the opposite direction. The ``reverse()`` function takes
information about a view and returns a URL which would invoke that view. Use
of ``reverse()`` is encouraged for application developers, as the output of
``reverse()`` is always based on the current URL patterns, meaning developers
do not need to change other code when making changes to URLs.

One argument signature for ``reverse()`` is to pass a dotted Python
path to the desired view. In this situation, Django will import the
module indicated by that dotted path as part of generating the
resulting URL. If such a module has import-time side effects, those
side effects will occur.

Thus it is possible for an attacker to cause unexpected code
execution, given the following conditions:

1. One or more views are present which construct a URL based on user
input (commonly, a "next" parameter in a querystring indicating
where to redirect upon successful completion of an action).

2. One or more modules are known to an attacker to exist on the
server's Python import path, which perform code execution with side
effects on importing.

To remedy this, ``reverse()`` will now only accept and import dotted
paths based on the view-containing modules listed in the project's :doc:`URL
pattern configuration </topics/http/urls>`, so as to ensure that only modules
the developer intended to be imported in this fashion can or will be imported.

Caching of anonymous pages could reveal CSRF token
==================================================

Django includes both a :doc:`caching framework </topics/cache>` and a system
for :doc:`preventing cross-site request forgery (CSRF) attacks
</ref/csrf/>`. The CSRF-protection system is based on a random nonce
sent to the client in a cookie which must be sent by the client on future
requests and, in forms, a hidden value which must be submitted back with the
form.

The caching framework includes an option to cache responses to
anonymous (i.e., unauthenticated) clients.

When the first anonymous request to a given page is by a client which
did not have a CSRF cookie, the cache framework will also cache the
CSRF cookie and serve the same nonce to other anonymous clients who
do not have a CSRF cookie. This can allow an attacker to obtain a
valid CSRF cookie value and perform attacks which bypass the check for
the cookie.

To remedy this, the caching framework will no longer cache such
responses. The heuristic for this will be:

1. If the incoming request did not submit any cookies, and

2. If the response did send one or more cookies, and

3. If the ``Vary: Cookie`` header is set on the response, then the
response will not be cached.

MySQL typecasting
=================

The MySQL database is known to "typecast" on certain queries; for
example, when querying a table which contains string values, but using
a query which filters based on an integer value, MySQL will first
silently coerce the strings to integers and return a result based on that.

If a query is performed without first converting values to the
appropriate type, this can produce unexpected results, similar to what
would occur if the query itself had been manipulated.

Django's model field classes are aware of their own types and most
such classes perform explicit conversion of query arguments to the
correct database-level type before querying. However, three model
field classes did not correctly convert their arguments:

* :class:`~django.db.models.FilePathField`
* :class:`~django.db.models.GenericIPAddressField`
* ``IPAddressField``

These three fields have been updated to convert their arguments to the
correct types before querying.

Additionally, developers of custom model fields are now warned via
documentation to ensure their custom field classes will perform
appropriate type conversions, and users of the :meth:`raw()
<django.db.models.query.QuerySet.raw>` and :meth:`extra()
<django.db.models.query.QuerySet.extra>` query methods -- which allow the
developer to supply raw SQL or SQL fragments -- will be advised to ensure they
perform appropriate manual type conversions prior to executing queries.

Bugfixes
========

* Fixed :class:`~django.contrib.auth.backends.ModelBackend` raising
``UnboundLocalError`` if :func:`~django.contrib.auth.get_user_model`
raised an error (21439).

Additionally, Django's vendored version of six, ``django.utils.six``,
has been upgraded to the latest release (1.6.1).


==========================

1.5.5

Not secure
==========================

*October 23, 2013*

Django 1.5.5 fixes a couple security-related bugs and several other bugs in the
1.5 series.

Readdressed denial-of-service via password hashers
==================================================

Django 1.5.4 imposes a 4096-byte limit on passwords in order to mitigate a
denial-of-service attack through submission of bogus but extremely large
passwords. In Django 1.5.5, we've reverted this change and instead improved
the speed of our PBKDF2 algorithm by not rehashing the key on every iteration.

Properly rotate CSRF token on login
===================================

This behavior introduced as a security hardening measure in Django 1.5.2 did
not work properly and is now fixed.

Bugfixes
========

* Fixed a data corruption bug with ``datetime_safe.datetime.combine`` (21256).
* Fixed a Python 3 incompatibility in ``django.utils.text.unescape_entities()``
(21185).
* Fixed a couple data corruption issues with ``QuerySet`` edge cases under
Oracle and MySQL (21203, 21126).
* Fixed crashes when using combinations of ``annotate()``,
``select_related()``, and ``only()`` (16436).

Backwards incompatible changes
==============================

* The undocumented ``django.core.servers.basehttp.WSGIServerException`` has
been removed. Use ``socket.error`` provided by the standard library instead.


==========================

1.5.4

Not secure
==========================

*September 14, 2013*

This is Django 1.5.4, the fourth release in the Django 1.5 series. It addresses
two security issues and one bug.

Denial-of-service via password hashers
======================================

In previous versions of Django, no limit was imposed on the plaintext
length of a password. This allowed a denial-of-service attack through
submission of bogus but extremely large passwords, tying up server
resources performing the (expensive, and increasingly expensive with
the length of the password) calculation of the corresponding hash.

As of 1.5.4, Django's authentication framework imposes a 4096-byte
limit on passwords, and will fail authentication with any submitted
password of greater length.

Corrected usage of :func:`~django.views.decorators.debug.sensitive_post_parameters` in :mod:`django.contrib.auth`’s admin
=========================================================================================================================

The decoration of the ``add_view`` and ``user_change_password`` user admin
views with :func:`~django.views.decorators.debug.sensitive_post_parameters`
did not include :func:`~django.utils.decorators.method_decorator` (required
since the views are methods) resulting in the decorator not being properly
applied. This usage has been fixed and
:func:`~django.views.decorators.debug.sensitive_post_parameters` will now
throw an exception if it's improperly used.

Bugfixes
========

* Fixed a bug that prevented a ``QuerySet`` that uses
:meth:`~django.db.models.query.QuerySet.prefetch_related` from being pickled
and unpickled more than once (the second pickling attempt raised an
exception) (21102).


==========================

1.5.3

Not secure
==========================

*September 10, 2013*

This is Django 1.5.3, the third release in the Django 1.5 series. It addresses
one security issue and also contains an opt-in feature to enhance the security
of :mod:`django.contrib.sessions`.

Directory traversal vulnerability in ``ssi`` template tag
=========================================================

In previous versions of Django it was possible to bypass the
``ALLOWED_INCLUDE_ROOTS`` setting used for security with the ``ssi``
template tag by specifying a relative path that starts with one of the allowed
roots. For example, if ``ALLOWED_INCLUDE_ROOTS = ("/var/www",)`` the following
would be possible:

.. code-block:: html+django

{% ssi "/var/www/../../etc/passwd" %}

In practice this is not a very common problem, as it would require the template
author to put the ``ssi`` file in a user-controlled variable, but it's possible
in principle.

Mitigating a remote-code execution vulnerability in :mod:`django.contrib.sessions`
==================================================================================

:mod:`django.contrib.sessions` currently uses :mod:`pickle` to serialize
session data before storing it in the backend. If you're using the :ref:`signed
cookie session backend<cookie-session-backend>` and :setting:`SECRET_KEY` is
known by an attacker (there isn't an inherent vulnerability in Django that
would cause it to leak), the attacker could insert a string into their session
which, when unpickled, executes arbitrary code on the server. The technique for
doing so is simple and easily available on the internet. Although the cookie
session storage signs the cookie-stored data to prevent tampering, a
:setting:`SECRET_KEY` leak immediately escalates to a remote code execution
vulnerability.

This attack can be mitigated by serializing session data using JSON rather
than :mod:`pickle`. To facilitate this, Django 1.5.3 introduces a new setting,
:setting:`SESSION_SERIALIZER`, to customize the session serialization format.
For backwards compatibility, this setting defaults to using :mod:`pickle`.
While JSON serialization does not support all Python objects like :mod:`pickle`
does, we highly recommend switching to JSON-serialized values. Also,
as JSON requires string keys, you will likely run into problems if you are
using non-string keys in ``request.session``. See the
:ref:`session_serialization` documentation for more details.


==========================

1.5.2

Not secure
==========================

*August 13, 2013*

This is Django 1.5.2, a bugfix and security release for Django 1.5.

Mitigated possible XSS attack via user-supplied redirect URLs
=============================================================

Django relies on user input in some cases (e.g.
``django.contrib.auth.views.login()``, ``django.contrib.comments``, and
:doc:`i18n </topics/i18n/index>`) to redirect the user to an "on success" URL.
The security checks for these redirects (namely
``django.utils.http.is_safe_url()``) didn't check if the scheme is ``http(s)``
and as such allowed ``javascript:...`` URLs to be entered. If a developer
relied on ``is_safe_url()`` to provide safe redirect targets and put such a
URL into a link, they could suffer from a XSS attack. This bug doesn't affect
Django currently, since we only put this URL into the ``Location`` response
header and browsers seem to ignore JavaScript there.

XSS vulnerability in :mod:`django.contrib.admin`
================================================

If a :class:`~django.db.models.URLField` is used in Django 1.5, it displays the
current value of the field and a link to the target on the admin change page.
The display routine of this widget was flawed and allowed for XSS.

Bugfixes
========

* Fixed a crash with :meth:`~django.db.models.query.QuerySet.prefetch_related`
(19607) as well as some ``pickle`` regressions with ``prefetch_related``
(20157 and 20257).
* Fixed a regression in :mod:`django.contrib.gis` in the Google Map output on
Python 3 (20773).
* Made ``DjangoTestSuiteRunner.setup_databases`` properly handle aliases for
the default database (19940) and prevented ``teardown_databases`` from
attempting to tear down aliases (20681).
* Fixed the ``django.core.cache.backends.memcached.MemcachedCache`` backend's
``get_many()`` method on Python 3 (20722).
* Fixed :mod:`django.contrib.humanize` translation syntax errors. Affected
languages: Mexican Spanish, Mongolian, Romanian, Turkish (20695).
* Added support for wheel packages (19252).
* The CSRF token now rotates when a user logs in.
* Some Python 3 compatibility fixes including 20212 and 20025.
* Fixed some rare cases where :meth:`~django.db.models.query.QuerySet.get`
exceptions recursed infinitely (20278).
* :djadmin:`makemessages` no longer crashes with ``UnicodeDecodeError``
(20354).
* Fixed ``geojson`` detection with SpatiaLite.
* :meth:`~django.test.SimpleTestCase.assertContains` once again works with
binary content (20237).
* Fixed :class:`~django.db.models.ManyToManyField` if it has a Unicode ``name``
parameter (20207).
* Ensured that the WSGI request's path is correctly based on the
``SCRIPT_NAME`` environment variable or the :setting:`FORCE_SCRIPT_NAME`
setting, regardless of whether or not either has a trailing slash (20169).
* Fixed an obscure bug with the :func:`~django.test.override_settings`
decorator. If you hit an ``AttributeError: 'Settings' object has no attribute
'_original_allowed_hosts'`` exception, it's probably fixed (20636).


==========================

1.5.1

Not secure
==========================

*March 28, 2013*

This is Django 1.5.1, a bugfix release for Django 1.5. It's completely backwards
compatible with Django 1.5, but includes a handful of fixes.

The biggest fix is for a memory leak introduced in Django 1.5. Under certain
circumstances, repeated iteration over querysets could leak memory - sometimes
quite a bit of it. If you'd like more information, the details are in
:ticket:`our ticket tracker <19895>` (and in :bpo:`a related issue <17468>` in
Python itself).

If you've noticed memory problems under Django 1.5, upgrading to 1.5.1 should
fix those issues.

Django 1.5.1 also includes a couple smaller fixes:

* Module-level warnings emitted during tests are no longer silently hidden
(:ticket:`18985`).
* Prevented filtering on password hashes in the user admin (:ticket:`20078`).


========================

1.5

Not secure
========================

*February 26, 2013*

Welcome to Django 1.5!

These release notes cover the :ref:`new features <whats-new-1.5>`, as well as
some :ref:`backwards incompatible changes <backwards-incompatible-1.5>` you'll
want to be aware of when upgrading from Django 1.4 or older versions. We've
also dropped some features, which are detailed in :ref:`our deprecation plan
<deprecation-removed-in-1.5>`, and we've :ref:`begun the deprecation process
for some features <deprecated-features-1.5>`.

Overview
========

The biggest new feature in Django 1.5 is the `configurable User model`_. Before
Django 1.5, applications that wanted to use Django's auth framework
(:mod:`django.contrib.auth`) were forced to use Django's definition of a "user".
In Django 1.5, you can now swap out the ``User`` model for one that you write
yourself. This could be a simple extension to the existing ``User`` model -- for
example, you could add a Twitter or Facebook ID field -- or you could completely
replace the ``User`` with one totally customized for your site.

Django 1.5 is also the first release with `Python 3 support`_! We're labeling
this support "experimental" because we don't yet consider it production-ready,
but everything's in place for you to start porting your apps to Python 3.
Our next release, Django 1.6, will support Python 3 without reservations.

Other notable new features in Django 1.5 include:

* `Support for saving a subset of model's fields`_ -
:meth:`Model.save() <django.db.models.Model.save()>` now accepts an
``update_fields`` argument, letting you specify which fields are
written back to the database when you call ``save()``. This can help
in high-concurrency operations, and can improve performance.

* Better `support for streaming responses <explicit-streaming-responses>`_ via
the new :class:`~django.http.StreamingHttpResponse` response class.

* `GeoDjango`_ now supports PostGIS 2.0.

* ... and more; `see below <what-s-new-in-django-1-5>`_.

Wherever possible we try to introduce new features in a backwards-compatible
manner per :doc:`our API stability policy </misc/api-stability>`.
However, as with previous releases, Django 1.5 ships with some minor
:ref:`backwards incompatible changes <backwards-incompatible-1.5>`; people
upgrading from previous versions of Django should read that list carefully.

One deprecated feature worth noting is the shift to "new-style" :ttag:`url` tag.
Prior to Django 1.3, syntax like ``{% url myview %}`` was interpreted
incorrectly (Django considered ``"myview"`` to be a literal name of a view, not
a template variable named ``myview``). Django 1.3 and above introduced the
``{% load url from future %}`` syntax to bring in the corrected behavior where
``myview`` was seen as a variable.

The upshot of this is that if you are not using ``{% load url from future %}`