Sparkly

Latest version: v2.8.2

Safety actively analyzes 629359 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 2 of 4

2.5.1

* Fix port issue with reading and writing `by_url`. `urlparse` return `netloc` with port, which breaks read and write from MySQL and Cassandra.

2.5.0

* Add `port` argument to `CassandraFixture` and `MysqlFixture`
* Add `Content-Type` header to `ElasticFixture` to support ElasticSearch `6.x`
* Update `elasticsearch-hadoop` connector to `6.5.4`
* Update image tag for elasticsearch to `6.5.4`

2.4.1

* Fix write_ext.kafka: run foreachPartition instead of mapPartitions because returned value can cause spark.driver.maxResultSize excess

2.4.0

* Respect PYSPARK_SUBMIT_ARGS if it is already set by appending SparklySession related options at the end instead of overwriting.
* Fix additional_options to always override SparklySession.options when a session is initialized
* Fix ujson dependency on environments where redis-py is already installed
* Access or initialize SparklySession through get_or_create classmethod
* Ammend `sparkly.functions.switch_case` to accept a user defined function for
deciding whether the switch column matches a specific case

2.3.0

* Overwrite existing tables in the metastore
* Add functions module and provide switch_case column generation and multijoin
* Add implicit test target import and extended assertEqual variation
* Support writing to redis:// and rediss:// URLs
* Add LRU cache that persists DataFrames under the hood
* Add ability to check whether a complex type defines specific fields

2.2.1

* `spark.sql.shuffle.partitions` in `SparklyTest` should be set to string,
because `int` value breaks integration testing in Spark 2.0.2.

Page 2 of 4

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.