Eland

---------------------

Added
^^^^^

* Added ``DataFrame.groupby()`` method with all aggregations
(`278`_, `291`_, `292`_, `300`_ contributed by `V1NAY8`_)

* Added ``es_match()`` method to ``DataFrame`` and ``Series`` for
filtering rows with full-text search (`301`_)

* Added support for type hints of the ``elasticsearch-py`` package (`295`_)

* Added support for passing dictionaries to ``es_type_overrides`` parameter
in the ``pandas_to_eland()`` function to directly control the field mapping
generated in Elasticsearch (`310`_)

* Added ``es_dtypes`` property to ``DataFrame`` and ``Series`` (`285`_)

Changed
^^^^^^^

* Changed ``pandas_to_eland()`` to use the ``parallel_bulk()``
helper instead of single-threaded ``bulk()`` helper to improve
performance (`279`_, contributed by `V1NAY8`_)

* Changed the ``es_type_overrides`` parameter in ``pandas_to_eland()``
to raise ``ValueError`` if an unknown column is given (`302`_)

* Changed ``DataFrame.filter()`` to preserve the order of items
(`283`_, contributed by `V1NAY8`_)

* Changed when setting ``es_type_overrides={"column": "text"}`` in
``pandas_to_eland()`` will automatically add the ``column.keyword``
sub-field so that aggregations are available for the field as well (`310`_)

Fixed
^^^^^

* Fixed ``Series.__repr__`` when the series is empty (`306`_)

.. _278: https://github.com/elastic/eland/pull/278
.. _279: https://github.com/elastic/eland/pull/279
.. _283: https://github.com/elastic/eland/pull/283
.. _285: https://github.com/elastic/eland/pull/285
.. _291: https://github.com/elastic/eland/pull/291
.. _292: https://github.com/elastic/eland/pull/292
.. _295: https://github.com/elastic/eland/pull/295
.. _300: https://github.com/elastic/eland/pull/300
.. _301: https://github.com/elastic/eland/pull/301
.. _302: https://github.com/elastic/eland/pull/302
.. _306: https://github.com/elastic/eland/pull/306
.. _310: https://github.com/elastic/eland/pull/310

--------------------

Added
^^^^^

* Added the ``predict()`` method and ``model_type``,
``feature_names``, and ``results_field`` properties
to ``MLModel`` (`266`_)


Deprecated
^^^^^^^^^^

* Deprecated ``ImportedMLModel`` in favor of
``MLModel.import_model(...)`` (`266`_)


Changed
^^^^^^^

* Changed DataFrame aggregations to use ``numeric_only=None``
instead of ``numeric_only=True`` by default. This is the same
behavior as Pandas (`270`_, contributed by `V1NAY8`_)

Fixed
^^^^^

* Fixed ``DataFrame.agg()`` when given a string instead of a list of
aggregations will now properly return a ``Series`` instead of
a ``DataFrame`` (`263`_, contributed by `V1NAY8`_)


.. _263: https://github.com/elastic/eland/pull/263
.. _266: https://github.com/elastic/eland/pull/266
.. _270: https://github.com/elastic/eland/pull/270

--------------------

Added
^^^^^

* Added support for Pandas v1.1 (`253`_)
* Added support for LightGBM ``LGBMRegressor`` and ``LGBMClassifier`` to ``ImportedMLModel`` (`247`_, `252`_)
* Added support for ``multi:softmax`` and ``multi:softprob`` XGBoost operators to ``ImportedMLModel`` (`246`_)
* Added column names to ``DataFrame.__dir__()`` for better auto-completion support (`223`_, contributed by `leonardbinet`_)
* Added support for ``es_if_exists='append'`` to ``pandas_to_eland()`` (`217`_)
* Added support for aggregating datetimes with ``nunique`` and ``mean`` (`253`_)
* Added ``es_compress_model_definition`` parameter to ``ImportedMLModel`` constructor (`220`_)
* Added ``.size`` and ``.ndim`` properties to ``DataFrame`` and ``Series`` (`231`_ and `233`_)
* Added ``.dtype`` property to ``Series`` (`258`_)
* Added support for using ``pandas.Series`` with ``Series.isin()`` (`231`_)
* Added type hints to many APIs in ``DataFrame`` and ``Series`` (`231`_)

Deprecated
^^^^^^^^^^

* Deprecated the ``overwrite`` parameter in favor of ``es_if_exists`` in ``ImportedMLModel`` constructor (`249`_, contributed by `V1NAY8`_)

Changed
^^^^^^^

* Changed aggregations for datetimes to be higher precision when available (`253`_)

Fixed
^^^^^

* Fixed ``ImportedMLModel.predict()`` to fail when ``errors`` are present in the ``ingest.simulate`` response (`220`_)
* Fixed ``Series.median()`` aggregation to return a scalar instead of ``pandas.Series`` (`253`_)
* Fixed ``Series.describe()`` to return a ``pandas.Series`` instead of ``pandas.DataFrame`` (`258`_)
* Fixed ``DataFrame.mean()`` and ``Series.mean()`` dtype (`258`_)
* Fixed ``DataFrame.agg()`` aggregations when using ``extended_stats`` Elasticsearch aggregation (`253`_)

.. _leonardbinet: https://github.com/leonardbinet
.. _V1NAY8: https://github.com/V1NAY8
.. _217: https://github.com/elastic/eland/pull/217
.. _220: https://github.com/elastic/eland/pull/220
.. _223: https://github.com/elastic/eland/pull/223
.. _231: https://github.com/elastic/eland/pull/231
.. _233: https://github.com/elastic/eland/pull/233
.. _246: https://github.com/elastic/eland/pull/246
.. _247: https://github.com/elastic/eland/pull/247
.. _249: https://github.com/elastic/eland/pull/249
.. _252: https://github.com/elastic/eland/pull/252
.. _253: https://github.com/elastic/eland/pull/253
.. _258: https://github.com/elastic/eland/pull/258

--------------------

Added
^^^^^

* Added the package to Conda Forge, install via
``conda install -c conda-forge eland`` (`209`_)
* Added ``DataFrame.sample()`` and ``Series.sample()`` for querying
a random sample of data from the index (`196`_, contributed by `mesejo`_)
* Added ``Series.isna()`` and ``Series.notna()`` for filtering out
missing, ``NaN`` or null values from a column (`210`_, contributed by `mesejo`_)
* Added ``DataFrame.filter()`` and ``Series.filter()`` for reducing an axis
using a sequence of items or a pattern (`212`_)
* Added ``DataFrame.to_pandas()`` and ``Series.to_pandas()`` for converting
an Eland dataframe or series into a Pandas dataframe or series inline (`208`_)
* Added support for XGBoost v1.0.0 (`200`_)

Deprecated
^^^^^^^^^^

* Deprecated ``info_es()`` in favor of ``es_info()`` (`208`_)
* Deprecated ``eland.read_csv()`` in favor of ``eland.csv_to_eland()`` (`208`_)
* Deprecated ``eland.read_es()`` in favor of ``eland.DataFrame()`` (`208`_)

Changed
^^^^^^^

* Changed ``var`` and ``std`` aggregations to use sample instead of
population in line with Pandas (`185`_)
* Changed painless scripts to use ``source`` rather than ``inline`` to improve
script caching performance (`191`_, contributed by `mesejo`_)
* Changed minimum ``elasticsearch`` Python library version to v7.7.0 (`207`_)
* Changed name of ``Index.field_name`` to ``Index.es_field_name`` (`208`_)

Fixed
^^^^^

* Fixed ``DeprecationWarning`` raised from ``pandas.Series`` when an
an empty series was created without specifying ``dtype`` (`188`_, contributed by `mesejo`_)
* Fixed a bug when filtering columns on complex combinations of and and or (`204`_)
* Fixed an issue where ``DataFrame.shape`` would return a larger value than
in the index if a sized operation like ``.head(X)`` was applied to the data
frame (`205`_, contributed by `mesejo`_)
* Fixed issue where both ``scikit-learn`` and ``xgboost`` libraries were
required to use ``eland.ml.ImportedMLModel``, now only one library is
required to use this feature (`206`_)

.. _200: https://github.com/elastic/eland/pull/200
.. _201: https://github.com/elastic/eland/pull/201
.. _204: https://github.com/elastic/eland/pull/204
.. _205: https://github.com/elastic/eland/pull/205
.. _206: https://github.com/elastic/eland/pull/206
.. _207: https://github.com/elastic/eland/pull/207
.. _191: https://github.com/elastic/eland/pull/191
.. _210: https://github.com/elastic/eland/pull/210
.. _185: https://github.com/elastic/eland/pull/185
.. _188: https://github.com/elastic/eland/pull/188
.. _196: https://github.com/elastic/eland/pull/196
.. _208: https://github.com/elastic/eland/pull/208
.. _209: https://github.com/elastic/eland/pull/209
.. _212: https://github.com/elastic/eland/pull/212

--------------------

Added
^^^^^

* Added support for Pandas v1.0.0 (`141`_, contributed by `mesejo`_)
* Added ``use_pandas_index_for_es_ids`` parameter to ``pandas_to_eland()`` (`154`_)
* Added ``es_type_overrides`` parameter to ``pandas_to_eland()`` (`181`_)
* Added ``NDFrame.var()``, ``.std()`` and ``.median()`` aggregations (`175`_, `176`_, contributed by `mesejo`_)
* Added ``DataFrame.es_query()`` to allow modifying ES queries directly (`156`_)
* Added ``eland.__version__`` (`153`_, contributed by `mesejo`_)

Removed
^^^^^^^

* Removed support for Python 3.5 (`150`_)
* Removed ``eland.Client()`` interface, use
``elasticsearch.Elasticsearch()`` client instead (`166`_)
* Removed all private objects from top-level ``eland`` namespace (`170`_)
* Removed ``geo_points`` from ``pandas_to_eland()`` in favor of ``es_type_overrides`` (`181`_)

Changed
^^^^^^^

* Changed ML model serialization to be slightly smaller (`159`_)
* Changed minimum ``elasticsearch`` Python library version to v7.6.0 (`181`_)

Fixed
^^^^^

* Fixed ``inference_config`` being required on ML models for ES >=7.8 (`174`_)
* Fixed unpacking for ``DataFrame.aggregate("median")`` (`161`_)

.. _mesejo: https://github.com/mesejo
.. _141: https://github.com/elastic/eland/pull/141
.. _150: https://github.com/elastic/eland/pull/150
.. _153: https://github.com/elastic/eland/pull/153
.. _154: https://github.com/elastic/eland/pull/154
.. _156: https://github.com/elastic/eland/pull/156
.. _159: https://github.com/elastic/eland/pull/159
.. _161: https://github.com/elastic/eland/pull/161
.. _166: https://github.com/elastic/eland/pull/166
.. _170: https://github.com/elastic/eland/pull/170
.. _174: https://github.com/elastic/eland/pull/174
.. _175: https://github.com/elastic/eland/pull/175
.. _176: https://github.com/elastic/eland/pull/176
.. _181: https://github.com/elastic/eland/pull/181

--------------------

Changed
^^^^^^^

* Changed requirement for ``xgboost`` from ``>=0.90`` to ``==0.90``

Fixed
^^^^^

* Fixed issue in ``DataFrame.info()`` when called on an empty frame (`135`_)
* Fixed issues where many ``_source`` fields would generate
a ``too_long_frame`` error (`135`_, `137`_)

.. _135: https://github.com/elastic/eland/pull/135
.. _137: https://github.com/elastic/eland/pull/137