Apache-iotdb

Improvements

* [IOTDB-4230] Modify the start script for flexible startup: add iotdb.in.sh, add pid file
* [IOTDB-4776] Change default storage group to root.__system in iotdb metric reporter
* [IOTDB-4946] Show Deadband Process Info
* [IOTDB-5025] Fix tag of metric in 0.13
* [IOTDB-5128] Perfect the usage of `-p pid`, `-v version` feature for start-server.sh
* [IOTDB-5232] Add Schema Memory Usage metric monitor


Bug Fix

* [IOTDB-3624] Fix meet error when inserting data in SQL without timestamp
* [IOTDB-4716] Fix the statistics of disk size in linux system in 0.13
* [IOTDB-4738] Fix TsFile damaged after writing empty value pages
* [IOTDB-4781] Fix the NOT LIKE value filter does not work
* [IOTDB-4842] Fix String number inserting bug
* [IOTDB-4938] Fix data are lost when meets exception `storage group not ready`
* [IOTDB-4967] Fix slidingTimeWindow stops when encounter an empty window
* [IOTDB-5136] NPE when use a null as username in session
* [IOTDB-5184] Fix CompressionType ArrayIndexOutOfBounds
* [IOTDB-5157] Fix "Invalid ID for ZoneOffset, invalid format:"
* [IOTDB-5206] Fix when target file is deleted in Compaction exception handler and recover
* [IOTDB-5225] Fix setting chunkPointNumLowerBound in IoTDBDescriptor
* [IOTDB-5236] DeleteData Python API cannot work
* [IOTDB-5253] Fix chunkMetadata mem metric
* [IOTDB-5266] Fix seq file may be lost when selecting cross compaction task
* [IOTDB-5281] Fix selecting deleted files in compaction
* [IOTDB-5286] Fix target files should not be select to participate in other compaction task
* [IOTDB-5307] Fix failed to get TsFile input of file: NoSuchFileException
* [IOTDB-5311] Fix RunTimeException and NoSuchFile Exception when selecting files causing compaction scheduled thread to get stuck
* [IOTDB-5346] Fix log error in MemtableFlushTask when recovering

Improvements

* [IOTDB-4525] Accelerate restart process
* [IOTDB-3164] Add compaction memory control
* [IOTDB-4364] Reduce read amplification in compaction
* [IOTDB-4424] Specify error message when time value of insert sql can not be correctly parsed
* [IOTDB-4492] Control total file size of cross space compaction task
* [IOTDB-4542] Optimize schema validate error message
* Optimize pattern matching in Regexp

Bug Fixes

* [IOTDB-3988] Fix reload problem of metric module and refactor metric module
* [IOTDB-4239] fix NPE to insert a null value into a TEXT timeseries
* [IOTDB-4318] Fix RESTAPI data type conversion failed
* [IOTDB-4320] Fix insert row with null cause recover throw NPE
* [IOTDB-4343] Fix NPE when using MQTT protocol
* [IOTDB-4357] fix start in windows, IOTDB_LOG_DIR_IS_UNDEFINED folder appears
* [IOTDB-4585] Incorrect query result after deleting data for aligned timeseries
* [IOTDB-4615] TTL adapts timestamp precision
* [IOTDB-4636] IndexOutOfBoundsException when compacting aligned series
* Fix PathAlreadyExistException during concurrent auto creating aligned timeseries

Improvements

[IOTDB-2669] Improve C++ client insertTablet performance
[IOTDB-3087] enlarge default value of avg_series_point_number_threshold
[IOTDB-3861] Enable insert null values in Session
[IOTDB-3996] REST API nonQuery support Continuous Query
[IOTDB-4120] Optimize memory allocation of Expression
[IOTDB-4190] update documents about nifi-iotdb-bundle
REST support "select into" in nonQuery API
Import-CSV supports specify data type and no need quotation for text value

Bug Fixes

[IOTDB-2736] DELETE_STORAGE_GROUP can not be granted to user (reporting 401)
[IOTDB-2760] Ordinary users do not grant any permissions, but can still show operations
[IOTDB-2769] Fix auth mapping of GRANT_ROLE_PRIVILEGE and GRANT_USER_ROLE
[IOTDB-3302] Without any authorization, ordinary users still have the right to query other user information
[IOTDB-4023] The C++ interface query result returns records error
[IOTDB-4047] Query NPE after change device alignment
[IOTDB-4096] Fix the names of metric pushed to Prometheus are inconsistency in micrometer and dropwizard
[IOTDB-4194] IOException happened in Compaction
[IOTDB-4215] Fix incorrect aggregate query results due to wrong unseq file traversal order
[IOTDB-4216] Fix execute create aligned timeseries but a non-aligned timeseries created
[IOTDB-4222] DeadLock when concurrently deleting and creating storage groups
[ISSUE-6774] Connection error when using DataGrip with JDBC driver
[ISSUE-6937] After restart, the aligned series turns to non-aligned
[ISSUE-6987] Fix select error when selecting a single quotation mark

New Features

* [IOTDB-2602] "Without null" supports filtering based on partial columns
* [IOTDB-3873] Aligned timeseries support single point fill query
* [ISSUE-6171] Support createTimeseriesOfTemplate in Session
* [IOTDB-3742] Support COUNT NODES by root.**

Improvements

* [IOTDB-2820] Update UserGuide SQL about Trigger
* [IOTDB-2837] Add check and sort for NumpyTablet to make sure timestamps are ordered
* [IOTDB-2838] Check and auto correct endian type for NumpyTablet
* [IOTDB-2873] provide json template for grafana
* [IOTDB-2888] Unary expression can followed by a constant
* [IOTDB-3747] Default Paging of Schema Query with limit 10000
* [IOTDB-3797] Print detailed info in session when connection fails
* [IOTDB-3851] C++ client method of tablet sorting optimization
* [IOTDB-3879] Modify document about the Programming-Cpp-Native-API
* [IOTDB-3901] C++ client method of insertRecordsOfOneDevice sorting optimization

Bug Fixes

* [IOTDB-2753] Insert a time series with a null value and report 500
* [IOTDB-2759] Result of "Show paths set schema template" or "using template" is not complete
* [IOTDB-2775] Fix throwing exception when query non-exist device in TsFileSequenceReader
* [IOTDB-2787] Fix aligned mem chunk concurrent problem
* [IOTDB-2826] Fix can not drop schema template
* [IOTDB-2828] Update system_version in system.properties after upgrading
* [IOTDB-2835] Fix empty page in selfcheck method of TsFileSequenceReader
* [IOTDB-2837] Add check and sort for NumpyTablet to make sure timestamps are ordered
* [IOTDB-2852] The import-csv tool can not import the data to nonaligned device
* [IOTDB-2859] Fix python tablet with None value is incorrect
* [IOTDB-2862] Fix SQL injection risks of grafana-connector
* [IOTDB-2864] Fix Read-only occurred when insert Text value to aligned timeseries
* [IOTDB-2882] Fix unary expression display bug
* [IOTDB-2902] Handling user privileges for aligned timeseries related features
* [IOTDB-2903] Fix show latest timeseries result does not sorted by time
* [IOTDB-2910] Fix count result not right after deleting storage group
* [IOTDB-2915] MLogTxtWriter error while parsing CreateAlignedTimeseriesPlan
* [IOTDB-2922] Fix NPE when compacting with files that contains zero device
* [IOTDB-2924] UDF Framework: index overflow while iterating sliding windows
* [IOTDB-2983] Serialization error in Partial insert
* [IOTDB-2999] Remove useless config and fix default value error.
* [IOTDB-3018] Fix compaction bugs on handling deleted target file
* [IOTDB-3029] The prefix path generated by the select into target sequence contains * and ** currently unchecked
* [IOTDB-3045] The query result contains some timeseries that have been deleted
* [IOTDB-3120] Print the tsfile name when meet IOException
* [IOTDB-3158] Fix NPE exception when use iotdb-reporter
* [IOTDB-3160] TsFile will be corrupted when flushing memtable appears OOM
* [IOTDB-3168] Forbid the path with * could be inserted
* [IOTDB-3171] Fix NPE when getting modification file
* [IOTDB-3219] Fix stop-server on windows
* [IOTDB-3247] Recover aligned sensors after deleting timeseries, query lost data
* [IOTDB-3301] Tag recover bug after IoTDB server restart
* [IOTDB-3364] Fix Query stucked with null valued aligned timeseries bug
* [IOTDB-3420] Fix show paths set schema template t1 error
* [IOTDB-3494] Fix TypeError in py-session
* [IOTDB-3523] Fix the count and COUNT not equal bug when querying with group by level
* [IOTDB-3645] Fix use statistics bug in aggregation query
* [IOTDB-3651] Stop compaction schedule when all compaction is disable
* [IOTDB-3709] Fix a loop occurred in TsFileResourceList, causing the query to fail and oom occurs
* [IOTDB-3730] Fix ArrayIndexOutOfBounds when flushing a memtable
* [IOTDB-3795] Remove setting read-only when handling compaction exception
* [IOTDB-3803] failed to insert data of TEXT by session
* [IOTDB-3816] Fix /zero bug in recover
* [IOTDB-3822] Fix cross compaction overlap bug
* [IOTDB-3826] Fix duplicate success when concurrent creating same timeseries
* [IOTDB-3858] IndexOutOfBoundsException: bitIndex < 0
* [ISSUE-5482] Fix Release zip files include incorrect version of Guava
* [ISSUE-5773] Aggregation result is not complete
* [ISSUE-5964] Fix bug of aligned timeseries time duplicated aggregation

New Features

* [IOTDB-924] Support insert multi rows in SQL
* [IOTDB-959] Add Create Storage Group Grammar
* [IOTDB-1037] set rpc\_compression as a parameter in JDBC URL
* [IOTDB-1059] Support sql statement insert without timestamp
* [IOTDB-1143] Support Continuous query
* [IOTDB-1199] Support aligned timeseries and schema template
* [IOTDB-1319] Support Trigger
* [IOTDB-1391] Add a new Aggregation function extreme (max absolute value)
* [IOTDB-1399] Add a session interface to connect multiple nodes
* [IOTDB-1400] Support arithmetic operations in SELECT clauses
* [IOTDB-1403] Dictionary encoding for TEXT
* [IOTDB-1490] Add built-in UDTFs: sinh, conh, tanh
* [IOTDB-1514] Support null in insertTablet
* [IOTDB-1524] Support SELECT ... INTO ... clause
* [IOTDB-1647] Nested Expressions in SELECT clauses
* [IOTDB-1673] CLI upgrade to JLine3
* [IOTDB-1739] Constant timeseries generating functions (const, pi and e)
* [IOTDB-1760] Support avg, count, extreme, first\_value, last\_value, max\_time, max\_value, min\_time, min\_value, sum aggregations in group by fill
* [IOTDB-1761] Add metric framework for IoTDB
* [IOTDB-1775] Add CAST function to convert data type
* [IOTDB-1823] Support group by multi level
* [IOTDB-1844] Query support timeseries prefix and suffix matching: root.\*sg\*
* [IOTDB-1859] Support REST API
* [IOTDB-1860] New Grafana plugin
* [IOTDB-1886] Support Constant Expressions in Select Clauses
* [IOTDB-1973] Supports aggregate queries, constants, and arithmetic nested expressions in SELECT clauses
* [IOTDB-1986] Support select UDF as alisa clauses
* [IOTDB-1989] Spark-IoTDB-connector support inserting data
* [IOTDB-2131] Support previous, linear, constant value fill funtion New fill
* [IOTDB-2593] Support using IoTDB with JDK17
* [ISSUE-3811] Provide a data type column for the last query dataset
* add rabbitmq example

Improvements

* [IOTDB-1280] Rewrite the Antlr grammar file
* [IOTDB-1372] Enhance management of TsFileResource
* [IOTDB-1428] Ask query threads to quit if query is timeout
* [IOTDB-1450] Deletion should only delete related time partitions
* [IOTDB-1463] Implement builder pattern for Session and SessionPool
* [IOTDB-1477] Optimize code logic of generateAlignByDevicePlan()
* [IOTDB-1559] Refactor the IT framework
* [IOTDB-1564] Make hearbeat and election timeout parameters be configurable
* [IOTDB-1581] Consider deletions when recovering tsFileResource of incomplete tsfile
* [IOTDB-1607] Optimize Tracing
* [IOTDB-1613] Recover mods file if write modification failed
* [IOTDB-1639] Refactoring the cluster class structure to make it consistent with the server module
* [IOTDB-1730] client-cpp, enhance session::insertTablet() etc.'s performance
* [IOTDB-1852] Accelerate queryies by using statistics
* [IOTDB-1857] Remove useless handle logic for CountPlan in executeNonQuery in cluster module
* [IOTDB-1884] Distinguish between zero and null values in sum aggregation
* [IOTDB-1924] Remove the operation of clearing the cache after the compaction is over
* [IOTDB-1950] Add Bloom Filter cache for query
* [IOTDB-2001] Remove redundant StorageGroupNotReadyException
* [IOTDB-2011] Update last cache while doing show latest timeseries query
* [IOTDB-2022] SessionDataSet implements AutoCloseable
* [IOTDB-2075] Accelerate the process of insertTablets by using thread pool
* [IOTDB-2119] IoTDB CSV export tools add timestamp accuracy setting function
* [IOTDB-2162] Simplify the recovery merge process
* [IOTDB-2176] Limit target chunk size when performing inner space compaction
* [IOTDB-2193] Reduce unnecessary lock operations of RaftLogManager to improve writing performance
* [IOTDB-2195] Control the concurrent query execution thread
* [IOTDB-2632] Set compaction_write_throughput_mb_per_sec to 16 by default
* [ISSUE-3445] New compaction strategy and compaction scheduling strategy
* [ISSUE-3856] refine exception handling logic in commitTo in RaftLogManager
* [Cluster] No need to shake hands with itself when one node restart


Incompatible changes

* [IOTDB-1026] Support wildcard \*\* in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
* [IOTDB-1620] Support backtick (\`) character and double quotes (") to quote identifiers
* [IOTDB-1650] Rename the sql command `move` to `unload`

Miscellaneous changes

* [IOTDB-1342] modify error message about LIMIT and OFFSET used in conjunction with the FILL clause
* [IOTDB-1372] delete devices field in FileTimeIndex
* [IOTDB-1531] Check tsfile creation time when recovering
* [IOTDB-1541] Change sequence of wal and memtable in insertion
* [IOTDB-1923] Separate the request unpacking and execution processing logic of TSServiceImpl
* [IOTDB-1969] Remove statistic resources of old metric framework from iotdb-server
* [IOTDB-2014] MicrometerPrometheusReportergetReporterType return null
* [IOTDB-2043] refactor: remove haveSorted param from Session
* [IOTDB-2154] add TsFileUtils.isTsFileComplete
* [IOTDB-2206] Rename StorageGroupProcessor to VirtualStorageGroupProcessor
* [IOTDB-2208] Reconstruct the process of generating resultset header of query
* [ISSUE-4047] Generic type in Statistics extend Serializable
* Add compaction version in cache key
* Add a constructor of IoTDBDescriptorHolder to prohibit instantiation

Bug Fixes

* [IOTDB-1266] Fix SHOW TIMESERIES will only display 2000 timeseries
* [IOTDB-1478] The whole IoTDB can not read/write if any one sg is not ready
* [IOTDB-1562] Fix incorrect exception processing in insertXXX() API
* [IOTDB-1583] Raft log failed to be committed in cluster version
* [IOTDB-1592] BugFix: SLimit Not effective in align by device
* [IOTDB-1736] Fix error code is not printed in TSServiceImpl's log
* [IOTDB-1749] sync-tool's lockInstance() dose not take effect
* [IOTDB-1758] sync-tool, empty uuid file cause tool can not auto-recovery
* [IOTDB-1848] Failed to initialize pool: Does not support setReadOnly in grafana-connector
* [IOTDB-1853] Fix bug about more than one TimeseriesMetadata in TsFile
* [IOTDB-2010] fix incomplete show timeseries result
* [IOTDB-2021] Fix Bloom Filter Cache doesn't take effect bug
* [IOTDB-2060] Fix NPE when using fill without a filter
* [IOTDB-2074] Fix NullPointerException when recovering compaction in MacOS
* [IOTDB-2077] Unexpected exception when executing raw data query without VF with limit clause
* [IOTDB-2088] Fix IndexOutOfBound exception in AggregationPlan.groupAggResultByLevel
* [IOTDB-2116] Fix all client connections are stuck bug caused by logback bug
* [IOTDB-2129] Wrong result returned when querying by nested expressions with aliases
* [IOTDB-2143] fix wrong precision in cluster mode
* [IOTDB-2153] [IOTDB-2157] fix incorrect path search space
* [IOTDB-2159] Fix wrong status code when inserting data to a read-only cluster
* [IOTDB-2163] Fix unexpected amount of columns in a cluster slimit query
* [IOTDB-2174] Fix Regexp filter serializing and deserializing error
* [IOTDB-2180] Fix show latest timeseries in cluster
* [IOTDB-2183] Fix the config problems in cluster mode
* [IOTDB-2185] Fix get an exception when parsing the header of CSV
* [IOTDB-2209] Fix logback CVE-2021-42550 issue
* [IOTDB-2251] [IOTDB-2252] Fix Query deadlock
* [IOTDB-2267] UDF: Error code 500 caused by user logic
* [IOTDB-2282] fix tag recovery bug after tag upsert
* [IOTDB-2290] Fix Incorrect query result in C++ client
* [IOTDB-2520] Fix `list user privilege root` returns empty
* [IOTDB-2544] Fix tag info sync error during metadata sync
* [IOTDB-2550] Avoid show timeseries error after alter tag on sync sender
* [IOTDB-2654] Fix Alias doesn't show when using group by level
* Fix CPP client could not be successfully built on windows
* Fix dead lock in setDataTTL method

Improvements
* [IOTDB-3515] Make the message clear when loading empty TsFile

Bug Fixes
* [IOTDB-2773] Fix overlapped data should be consumed first bug
* [IOTDB-3135] Fix parameter max_select_unseq_file_num_in_each_unseq_compaction doesn't work
* [IOTDB-3292] Fix chunk size overflow in level compaction
* [IOTDB-3392] Fix File doesn't exist when moving TsFile when virtual_storage_group > 1
* [IOTDB-3398] Remove the exception when query meet a device is not in TsFile, which is a normal case
* [IOTDB-3602] Skip empty TsFile when loading directory
* [ISSUE-4399] When non-root user get an empty query result, return no permission error message
* [ISSUE-5809] Fix bug: sum aggregation result return 0 when no data
* Fix data in seq files overlapped