Ormar

✨ Features

* `ForeignKey` and `ManyToMany` now support `skip_reverse: bool = False` flag [118](https://github.com/collerek/ormar/issues/118).
If you set `skip_reverse` flag internally the field is still registered on the other
side of the relationship so you can:
* `filter` by related models fields from reverse model
* `order_by` by related models fields from reverse model

But you cannot:
* access the related field from reverse model with `related_name`
* even if you `select_related` from reverse side of the model the returned models won't be populated in reversed instance (the join is not prevented so you still can `filter` and `order_by`)
* the relation won't be populated in `dict()` and `json()`
* you cannot pass the nested related objects when populating from `dict()` or `json()` (also through `fastapi`). It will be either ignored or raise error depending on `extra` setting in pydantic `Config`.
* `Model.save_related()` now can save whole data tree in once [148](https://github.com/collerek/ormar/discussions/148)
meaning:
* it knows if it should save main `Model` or related `Model` first to preserve the relation
* it saves main `Model` if
* it's not `saved`,
* has no `pk` value
* or `save_all=True` flag is set

in those cases you don't have to split save into two calls (`save()` and `save_related()`)
* it supports also `ManyToMany` relations
* it supports also optional `Through` model values for m2m relations
* Add possibility to customize `Through` model relation field names.
* By default `Through` model relation names default to related model name in lowercase.
So in example like this:
python
... course declaration ommited
class Student(ormar.Model):
class Meta:
database = database
metadata = metadata

id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)
courses = ormar.ManyToMany(Course)

will produce default Through model like follows (example simplified)
class StudentCourse(ormar.Model):
class Meta:
database = database
metadata = metadata
tablename = "students_courses"

id: int = ormar.Integer(primary_key=True)
student = ormar.ForeignKey(Student) default name
course = ormar.ForeignKey(Course) default name

* To customize the names of fields/relation in Through model now you can use new parameters to `ManyToMany`:
* `through_relation_name` - name of the field leading to the model in which `ManyToMany` is declared
* `through_reverse_relation_name` - name of the field leading to the model to which `ManyToMany` leads to

Example:
python
... course declaration ommited
class Student(ormar.Model):
class Meta:
database = database
metadata = metadata

id: int = ormar.Integer(primary_key=True)
name: str = ormar.String(max_length=100)
courses = ormar.ManyToMany(Course,
through_relation_name="student_id",
through_reverse_relation_name="course_id")

will produce default Through model like follows (example simplified)
class StudentCourse(ormar.Model):
class Meta:
database = database
metadata = metadata
tablename = "students_courses"

id: int = ormar.Integer(primary_key=True)
student_id = ormar.ForeignKey(Student) set by through_relation_name
course_id = ormar.ForeignKey(Course) set by through_reverse_relation_name

✨ Features

* `Model.save_related(follow=False)` now accept also two additional arguments: `Model.save_related(follow=False, save_all=False, exclude=None)`.
* `save_all:bool` -> By default (so with `save_all=False`) `ormar` only upserts models that are not saved (so new or updated ones),
with `save_all=True` all related models are saved, regardless of `saved` status, which might be useful if updated
models comes from api call, so are not changed in the backend.
* `exclude: Union[Set, Dict, None]` -> set/dict of relations to exclude from save, those relation won't be saved even with `follow=True` and `save_all=True`.
To exclude nested relations pass a nested dictionary like: `exclude={"child":{"sub_child": {"exclude_sub_child_realtion"}}}`. The allowed values follow
the `fields/exclude_fields` (from `QuerySet`) methods schema so when in doubt you can refer to docs in queries -> selecting subset of fields -> fields.
* `Model.update()` method now accepts `_columns: List[str] = None` parameter, that accepts list of column names to update. If passed only those columns will be updated in database.
Note that `update()` does not refresh the instance of the Model, so if you change more columns than you pass in `_columns` list your Model instance will have different values than the database!
* `Model.dict()` method previously included only directly related models or nested models if they were not nullable and not virtual,
now all related models not previously visited without loops are included in `dict()`. This should be not breaking
as just more data will be dumped to dict, but it should not be missing.
* `QuerySet.delete(each=False, **kwargs)` previously required that you either pass a `filter` (by `**kwargs` or as a separate `filter()` call) or set `each=True` now also accepts
`exclude()` calls that generates NOT filter. So either `each=True` needs to be set to delete whole table or at least one of `filter/exclude` clauses.
* Same thing applies to `QuerySet.update(each=False, **kwargs)` which also previously required that you either pass a `filter` (by `**kwargs` or as a separate `filter()` call) or set `each=True` now also accepts
`exclude()` calls that generates NOT filter. So either `each=True` needs to be set to update whole table or at least one of `filter/exclude` clauses.
* Same thing applies to `QuerysetProxy.update(each=False, **kwargs)` which also previously required that you either pass a `filter` (by `**kwargs` or as a separate `filter()` call) or set `each=True` now also accepts
`exclude()` calls that generates NOT filter. So either `each=True` needs to be set to update whole table or at least one of `filter/exclude` clauses.

🐛 Fixes

* Fix improper relation field resolution in `QuerysetProxy` if fk column has different database alias.
* Fix hitting recursion error with very complicated models structure with loops when calling `dict()`.
* Fix bug when two non-relation fields were merged (appended) in query result when they were not relation fields (i.e. JSON)
* Fix bug when during translation to dict from list the same relation name is used in chain but leads to different models
* Fix bug when bulk_create would try to save also `property_field` decorated methods and `pydantic` fields
* Fix wrong merging of deeply nested chain of reversed relations

💬 Other

* Performance optimizations
* Split tests into packages based on tested area

Features

* add `get_or_none(**kwargs)` method to `QuerySet` and `QuerysetProxy`. It is exact equivalent of `get(**kwargs)` but instead of raising `ormar.NoMatch` exception if there is no db record matching the criteria, `get_or_none` simply returns `None`.

Fixes

* Fix dialect dependent quoting of column and table names in order_by clauses not working
properly in postgres.

Breaking

* Dropped supported for long deprecated notation of field definition in which you use ormar fields as type hints i.e. `test_field: ormar.Integger() = None`
* Improved type hints -> `mypy` can properly resolve related models fields (`ForeignKey` and `ManyToMany`) as well as return types of `QuerySet` methods.
Those mentioned are now returning proper model (i.e. `Book`) instead or `ormar.Model` type. There is still problem with reverse sides of relation and `QuerysetProxy` methods,
to ease type hints now those return `Any`. Partially fixes 112.

Features

* add `select_all(follow: bool = False)` method to `QuerySet` and `QuerysetProxy`.
It is kind of equivalent of the Model's `load_all()` method but can be used directly in a query.
By default `select_all()` adds only directly related models, with `follow=True` also related models
of related models are added without loops in relations. Note that it's not and end `async` model
so you still have to issue `get()`, `all()` etc. as `select_all()` returns a QuerySet (or proxy)
like `fields()` or `order_by()`. 131

Internals

* `ormar` fields are no longer stored as classes in `Meta.model_fields` dictionary
but instead they are stored as instances.

Features
* Add possibility to change default ordering of relations and models.
* To change model sorting pass `orders_by = [columns]` where `columns: List[str]` to model `Meta` class
* To change relation order_by pass `orders_by = [columns]` where `columns: List[str]`
* To change reverse relation order_by pass `related_orders_by = [columns]` where `columns: List[str]`
* Arguments can be column names or `-{col_name}` to sort descending
* In relations you can sort only by directly related model columns
or for `ManyToMany` columns also `Through` model columns `"{through_field_name}__{column_name}"`
* Order in which order_by clauses are applied is as follows:
* Explicitly passed `order_by()` calls in query
* Relation passed `orders_by` if exists
* Model `Meta` class `orders_by`
* Model primary key column asc (fallback, used if none of above provided)
* Add 4 new aggregated functions -> `min`, `max`, `sum` and `avg` that are their
corresponding sql equivalents.
* You can pass one or many column names including related columns.
* As of now each column passed is aggregated separately (so `sum(col1+col2)` is not possible,
you can have `sum(col1, col2)` and later add 2 returned sums in python)
* You cannot `sum` and `avg` non numeric columns
* If you aggregate on one column, the single value is directly returned as a result
* If you aggregate on multiple columns a dictionary with column: result pairs is returned
* Add 4 new signals -> `pre_relation_add`, `post_relation_add`, `pre_relation_remove` and `post_relation_remove`
* The newly added signals are emitted for `ManyToMany` relations (both sides)
and reverse side of `ForeignKey` relation (same as `QuerysetProxy` is exposed).
* Signals recieve following args: `sender: Type[Model]` - sender class,
`instance: Model` - instance to which related model is added, `child: Model` - model being added,
`relation_name: str` - name of the relation to which child is added,
for add signals also `passed_kwargs: Dict` - dict of kwargs passed to `add()`

Changes
* `Through` models for ManyToMany relations are now instantiated on creation, deletion and update, so you can provide not only
autoincrement int as a primary key but any column type with default function provided.
* Since `Through` models are now instantiated you can also subscribe to `Through` model
pre/post save/update/delete signals
* `pre_update` signals receivers now get also passed_args argument which is a
dict of values passed to update function if any (else empty dict)

Fixes
* `pre_update` signal now is sent before the extraction of values so you can modify the passed
instance in place and modified fields values will be reflected in database
* `bulk_update` now works correctly also with `UUID` primary key column type

Features
* Add possibility to encrypt the selected field(s) in the database
* As minimum you need to provide `encrypt_secret` and `encrypt_backend`
* `encrypt_backend` can be one of the `ormar.EncryptBackends` enum (`NONE, FERNET, HASH, CUSTOM`) - default: `NONE`
* When custom backend is selected you need to provide your backend class that subclasses `ormar.fields.EncryptBackend`
* You cannot encrypt `primary_key` column and relation columns (FK and M2M).
* Provided are 2 backends: HASH and FERNET
* HASH is a one-way hash (like for password), never decrypted on retrieval
* FERNET is a two-way encrypt/decrypt backend
* Note that in FERNET backend you loose `filtering` possibility altogether as part of the encrypted value is a timestamp.
* Note that in HASH backend you can filter by full value but filters like `contain` will not work as comparison is make on encrypted values
* Note that adding `encrypt_backend` changes the database column type to `TEXT`, which needs to be reflected in db either by migration or manual change

Fixes
* (Advanced/ Internal) Restore custom sqlalchemy types (by `types.TypeDecorator` subclass) functionality that ceased to working so `process_result_value` was never called