Eland

------------------

Added
^^^^^

* Added support for ``eland.Series.unique()`` (`448`_, contributed by `V1NAY8`_)
* Added ``--ca-certs`` and ``--insecure`` options to ``eland_import_hub_model`` for configuring TLS (`441`_)

.. _448: https://github.com/elastic/eland/pull/448
.. _441: https://github.com/elastic/eland/pull/441

------------------

Added
^^^^^

* Added support for Natural Language Processing (NLP) models using PyTorch (`394`_)
* Added new extra ``eland[pytorch]`` for installing all dependencies needed for PyTorch (`394`_)
* Added a CLI script ``eland_import_hub_model`` for uploading HuggingFace models to Elasticsearch (`403`_)
* Added support for v8.0 of the Python Elasticsearch client (`415`_)
* Added a warning if Eland detects it's communicating with an incompatible Elasticsearch version (`419`_)
* Added support for ``number_samples`` to LightGBM and Scikit-Learn models (`397`_, contributed by `V1NAY8`_)
* Added ability to use datetime types for filtering dataframes (`284`_, contributed by `Fju`_)
* Added pandas ``datetime64`` type to use the Elasticsearch ``date`` type (`425`_, contributed by `Ashton-Sidhu`_)
* Added ``es_verify_mapping_compatibility`` parameter to disable schema enforcement with ``pandas_to_eland`` (`423`_, contributed by `Ashton-Sidhu`_)

Changed
^^^^^^^

* Changed ``to_pandas()`` to only use Point-in-Time and ``search_after`` instead of using Scroll APIs
for pagination.

.. _Fju: https://github.com/Fju
.. _Ashton-Sidhu: https://github.com/Ashton-Sidhu
.. _419: https://github.com/elastic/eland/pull/419
.. _415: https://github.com/elastic/eland/pull/415
.. _397: https://github.com/elastic/eland/pull/397
.. _394: https://github.com/elastic/eland/pull/394
.. _403: https://github.com/elastic/eland/pull/403
.. _284: https://github.com/elastic/eland/pull/284
.. _424: https://github.com/elastic/eland/pull/425
.. _423: https://github.com/elastic/eland/pull/423

---------------------

Added
^^^^^

* Added support for ``DataFrame.iterrows()`` and ``DataFrame.itertuples()`` (`380`_, contributed by `kxbin`_)

Performance
^^^^^^^^^^^

* Simplified result collectors to increase performance transforming Elasticsearch results to pandas (`378`_, contributed by `V1NAY8`_)
* Changed search pagination function to yield batches of hits (`379`_)

.. _kxbin: https://github.com/kxbin
.. _378: https://github.com/elastic/eland/pull/378
.. _379: https://github.com/elastic/eland/pull/379
.. _380: https://github.com/elastic/eland/pull/380

---------------------

Added
^^^^^

* Added support for Pandas 1.3.x (`362`_, contributed by `V1NAY8`_)
* Added support for LightGBM 3.x (`362`_, contributed by `V1NAY8`_)
* Added ``DataFrame.idxmax()`` and ``DataFrame.idxmin()`` methods (`353`_, contributed by `V1NAY8`_)
* Added type hints to ``eland.ndframe`` and ``eland.operations`` (`366`_, contributed by `V1NAY8`_)

Removed
^^^^^^^

* Removed support for Pandas <1.2 (`364`_)
* Removed support for Python 3.6 to match Pandas (`364`_)

Changed
^^^^^^^

* Changed paginated search function to use `Point-in-Time`_ and `Search After`_ features
instead of Scroll when connected to Elasticsearch 7.12+ (`370`_ and `376`_, contributed by `V1NAY8`_)
* Optimized the ``FieldMappings.aggregate_field_name()`` method (`373`_, contributed by `V1NAY8`_)

.. _Point-in-Time: https://www.elastic.co/guide/en/elasticsearch/reference/current/point-in-time-api.html
.. _Search After: https://www.elastic.co/guide/en/elasticsearch/reference/7.14/paginate-search-results.html#search-after
.. _353: https://github.com/elastic/eland/pull/353
.. _362: https://github.com/elastic/eland/pull/362
.. _364: https://github.com/elastic/eland/pull/364
.. _366: https://github.com/elastic/eland/pull/366
.. _370: https://github.com/elastic/eland/pull/370
.. _373: https://github.com/elastic/eland/pull/373
.. _376: https://github.com/elastic/eland/pull/376

---------------------

Added
^^^^^

* Added ``DataFrame.quantile()``, ``Series.quantile()``, and
``DataFrameGroupBy.quantile()`` aggregations (`318`_ and `356`_, contributed by `V1NAY8`_)

Changed
^^^^^^^

* Changed the error raised when ``es_index_pattern`` doesn't point to any indices
to be more user-friendly (`346`_)

Fixed
^^^^^

* Fixed a warning about conflicting field types when wildcards are used
in ``es_index_pattern`` (`346`_)

* Fixed sorting when using ``DataFrame.groupby()`` with ``dropna``
(`322`_, contributed by `V1NAY8`_)

* Fixed deprecated usage ``numpy.int`` in favor of ``numpy.int_`` (`354`_, contributed by `V1NAY8`_)

.. _318: https://github.com/elastic/eland/pull/318
.. _322: https://github.com/elastic/eland/pull/322
.. _346: https://github.com/elastic/eland/pull/346
.. _354: https://github.com/elastic/eland/pull/354
.. _356: https://github.com/elastic/eland/pull/356

---------------------

Added
^^^^^

* Added support for Pandas 1.2.0 (`336`_)

* Added ``DataFrame.mode()`` and ``Series.mode()`` aggregation (`323`_, contributed by `V1NAY8`_)

* Added support for ``pd.set_option("display.max_rows", None)``
(`308`_, contributed by `V1NAY8`_)

* Added Elasticsearch storage usage to ``df.info()`` (`321`_, contributed by `V1NAY8`_)

Removed
^^^^^^^

* Removed deprecated aliases ``read_es``, ``read_csv``, ``DataFrame.info_es``,
and ``MLModel(overwrite=True)`` (`331`_, contributed by `V1NAY8`_)

.. _336: https://github.com/elastic/eland/pull/336
.. _331: https://github.com/elastic/eland/pull/331
.. _323: https://github.com/elastic/eland/pull/323
.. _321: https://github.com/elastic/eland/pull/321
.. _308: https://github.com/elastic/eland/pull/308