Highlights
Major changes include:
* Dialect is now mandatory, either in command line, or in config **BREAKING CHANGE**
* Rename `spark3` dialect to `sparksql` **BREAKING CHANGE**
* L027 now checks tables references exist **BREAKING CHANGE**
* New rule L063 to allow Datatypes to have a different capitalisation policy from L010. **BREAKING CHANGE**
* Refactor and performance improvements of Delimited and L003
* Many dialect improvements and fixes
What’s Changed
* MySQL: Allow `JOIN`s in `UPDATE` expressions [3031](https://github.com/sqlfluff/sqlfluff/pull/3031) [zapion](https://github.com/zapion)
* Fix bug in patch generation for segments made of templated + literal fixes [3030](https://github.com/sqlfluff/sqlfluff/pull/3030) [barrywhart](https://github.com/barrywhart)
* Formatters code cleanup [3029](https://github.com/sqlfluff/sqlfluff/pull/3029) [barrywhart](https://github.com/barrywhart)
* Postgres better `CREATE USER`/`CREATE ROLE` support [3016](https://github.com/sqlfluff/sqlfluff/pull/3016) [mrf](https://github.com/mrf)
* SparkSQL: Add `MERGE` syntax [3025](https://github.com/sqlfluff/sqlfluff/pull/3025) [PhilippLange](https://github.com/PhilippLange)
* Remove Delimited workarounds [3024](https://github.com/sqlfluff/sqlfluff/pull/3024) [tunetheweb](https://github.com/tunetheweb)
* Add `exclude` option for `Ref` grammar [3028](https://github.com/sqlfluff/sqlfluff/pull/3028) [tunetheweb](https://github.com/tunetheweb)
* SparkSQL: Adding support for Delta Lake table schema updates [3013](https://github.com/sqlfluff/sqlfluff/pull/3013) [R7L208](https://github.com/R7L208)
* L027: Check table aliases exists [2998](https://github.com/sqlfluff/sqlfluff/pull/2998) [dmohns](https://github.com/dmohns)
* Snowflake: Added support for `REMOVE` statements [3026](https://github.com/sqlfluff/sqlfluff/pull/3026) [WittierDinosaur](https://github.com/WittierDinosaur)
* BigQuery: Support `WEEK` function with days of weeks [3021](https://github.com/sqlfluff/sqlfluff/pull/3021) [tunetheweb](https://github.com/tunetheweb)
* Sparksql quoted identifier in `STRUCT` [3023](https://github.com/sqlfluff/sqlfluff/pull/3023) [PhilippLange](https://github.com/PhilippLange)
* Force user to specify a dialect [2995](https://github.com/sqlfluff/sqlfluff/pull/2995) [barrywhart](https://github.com/barrywhart)
* BigQuery: Parse `CREATE TABLE` with trailing comma [3018](https://github.com/sqlfluff/sqlfluff/pull/3018) [dmohns](https://github.com/dmohns)
* Snowflake: Add `IS (NOT) DISTINCT FROM` test cases [3014](https://github.com/sqlfluff/sqlfluff/pull/3014) [kd2718](https://github.com/kd2718)
* BigQuery: Add support for column `OPTIONS` in `STRUCT` definitions [3017](https://github.com/sqlfluff/sqlfluff/pull/3017) [dmohns](https://github.com/dmohns)
* PostgreSQL: added support for `CREATE ROLE` and `DROP ROLE` statements [3010](https://github.com/sqlfluff/sqlfluff/pull/3010) [dnim](https://github.com/dnim)
* Separate slow CI job to it's own workflow [3012](https://github.com/sqlfluff/sqlfluff/pull/3012) [tunetheweb](https://github.com/tunetheweb)
* SparkSQL: Test cases for Delta Variation of Writing a table [3009](https://github.com/sqlfluff/sqlfluff/pull/3009) [R7L208](https://github.com/R7L208)
* Snowflake: Added support for `CLUSTER BY` and other `CREATE TABLE` improvements [3008](https://github.com/sqlfluff/sqlfluff/pull/3008) [WittierDinosaur](https://github.com/WittierDinosaur)
* Support `TRIM` function parameters [3007](https://github.com/sqlfluff/sqlfluff/pull/3007) [tunetheweb](https://github.com/tunetheweb)
* BigQuery: Support `AT TIME ZONE` in `EXTRACT` [3004](https://github.com/sqlfluff/sqlfluff/pull/3004) [tunetheweb](https://github.com/tunetheweb)
* BigQuery: Move some keywords to unreserved [3002](https://github.com/sqlfluff/sqlfluff/pull/3002) [tunetheweb](https://github.com/tunetheweb)
* BigQuery: Allow quoted variable names in `DECLARE` [3006](https://github.com/sqlfluff/sqlfluff/pull/3006) [dmohns](https://github.com/dmohns)
* T-SQL: allow optional `AS` keyword in parameters def [3001](https://github.com/sqlfluff/sqlfluff/pull/3001) [pguyot](https://github.com/pguyot)
* T-SQL: add support for `RETURNS var TABLE` syntax [3000](https://github.com/sqlfluff/sqlfluff/pull/3000) [pguyot](https://github.com/pguyot)
* T-SQL: add support for parenthesized nested joins [2993](https://github.com/sqlfluff/sqlfluff/pull/2993) [pguyot](https://github.com/pguyot)
* dbt: Read builtins from code [2988](https://github.com/sqlfluff/sqlfluff/pull/2988) [dmohns](https://github.com/dmohns)
* T-SQL: fix table type in `DECLARE` statements [2999](https://github.com/sqlfluff/sqlfluff/pull/2999) [pguyot](https://github.com/pguyot)
* T-SQL: allow leading `GO` [2997](https://github.com/sqlfluff/sqlfluff/pull/2997) [pguyot](https://github.com/pguyot)
* T-SQL: add support for assignment operators [2996](https://github.com/sqlfluff/sqlfluff/pull/2996) [pguyot](https://github.com/pguyot)
* BigQuery: Add more `MERGE` statement variants [2989](https://github.com/sqlfluff/sqlfluff/pull/2989) [dmohns](https://github.com/dmohns)
* L041: Fix bug when there is a newline after the select clause modifier [2981](https://github.com/sqlfluff/sqlfluff/pull/2981) [jmc-bbk](https://github.com/jmc-bbk)
* Rule L045 doesn't recognise CTE usage in a subquery when rule L042 is enabled [2980](https://github.com/sqlfluff/sqlfluff/pull/2980) [barrywhart](https://github.com/barrywhart)
* dbt: Make `is_incremental()` defaults consistent [2985](https://github.com/sqlfluff/sqlfluff/pull/2985) [dmohns](https://github.com/dmohns)
* Rename Grammars for consistency [2986](https://github.com/sqlfluff/sqlfluff/pull/2986) [tunetheweb](https://github.com/tunetheweb)
* Added support for MySQL `UPDATE` Statements [2982](https://github.com/sqlfluff/sqlfluff/pull/2982) [WittierDinosaur](https://github.com/WittierDinosaur)
* Redshift: Added `CREATE EXTERNAL SCHEMA`, bugfix in `PARTITION BY` [2983](https://github.com/sqlfluff/sqlfluff/pull/2983) [WittierDinosaur](https://github.com/WittierDinosaur)
* Added `ALTER INDEX` and `REINDEX` to Postgres, Some Grammar Cleaning [2979](https://github.com/sqlfluff/sqlfluff/pull/2979) [WittierDinosaur](https://github.com/WittierDinosaur)
* T-SQL grammar deduplication [2967](https://github.com/sqlfluff/sqlfluff/pull/2967) [tunetheweb](https://github.com/tunetheweb)
* L003 Refactor [2884](https://github.com/sqlfluff/sqlfluff/pull/2884) [OTooleMichael](https://github.com/OTooleMichael)
* Delimited Refactor [2831](https://github.com/sqlfluff/sqlfluff/pull/2831) [WittierDinosaur](https://github.com/WittierDinosaur)
* SparkSQL: Support for querying snapshots when reading data with Delta Lake [2972](https://github.com/sqlfluff/sqlfluff/pull/2972) [R7L208](https://github.com/R7L208)
* Fix bug in L063 for BigQuery `STRUCT` params [2975](https://github.com/sqlfluff/sqlfluff/pull/2975) [tunetheweb](https://github.com/tunetheweb)
* Fix assertion error in dbt templater when file ends with whitespace strip (`-%}`) [2976](https://github.com/sqlfluff/sqlfluff/pull/2976) [barrywhart](https://github.com/barrywhart)
* Pass dbt vars to dbt [2923](https://github.com/sqlfluff/sqlfluff/pull/2923) [tcholewik](https://github.com/tcholewik)
* BigQuery: Add support for column `OPTIONS` [2973](https://github.com/sqlfluff/sqlfluff/pull/2973) [dmohns](https://github.com/dmohns)
* BigQuery: Allow expressions in `OPTIONS` clauses [2971](https://github.com/sqlfluff/sqlfluff/pull/2971) [dmohns](https://github.com/dmohns)
* Bump black to 22.3.0 on pre-commit [2969](https://github.com/sqlfluff/sqlfluff/pull/2969) [pguyot](https://github.com/pguyot)
* T-SQL: Redefine `DatatypeIdentifierSegment` [2959](https://github.com/sqlfluff/sqlfluff/pull/2959) [alanmcruickshank](https://github.com/alanmcruickshank)
* T-SQL: Add support for `WAITFOR` statement [2968](https://github.com/sqlfluff/sqlfluff/pull/2968) [pguyot](https://github.com/pguyot)
* T-SQL: Add `WHILE` statement support [2966](https://github.com/sqlfluff/sqlfluff/pull/2966) [pguyot](https://github.com/pguyot)
* T-SQL: `INTO` is optional within `INSERT` statement [2963](https://github.com/sqlfluff/sqlfluff/pull/2963) [pguyot](https://github.com/pguyot)
* Add basic `IS (NOT) DISTINCT FROM` support in most dialects [2962](https://github.com/sqlfluff/sqlfluff/pull/2962) [tunetheweb](https://github.com/tunetheweb)
* SparkSQL: Create Table Delta Lake Variant [2954](https://github.com/sqlfluff/sqlfluff/pull/2954) [R7L208](https://github.com/R7L208)
* T-SQL: Add support for `CREATE`/`DROP`/`DISABLE` `TRIGGER` [2957](https://github.com/sqlfluff/sqlfluff/pull/2957) [tunetheweb](https://github.com/tunetheweb)
* Bug: L042 modifies parse tree even during "lint" [2955](https://github.com/sqlfluff/sqlfluff/pull/2955) [barrywhart](https://github.com/barrywhart)
* Allow multiple post function clauses in Postgres and Redshift [2952](https://github.com/sqlfluff/sqlfluff/pull/2952) [aviv](https://github.com/aviv)
* Fix bug in L022 for trailing comments in CTE [2946](https://github.com/sqlfluff/sqlfluff/pull/2946) [tunetheweb](https://github.com/tunetheweb)
* More dialect checking, fixes, inheritance cleanup [2942](https://github.com/sqlfluff/sqlfluff/pull/2942) [barrywhart](https://github.com/barrywhart)
* T-SQL: Support `OUTPUT` Params and `GOTO` Statements [2949](https://github.com/sqlfluff/sqlfluff/pull/2949) [tunetheweb](https://github.com/tunetheweb)
* BREAKING CHANGE: change existing dialect name from `spark3` to `sparksql` [2924](https://github.com/sqlfluff/sqlfluff/pull/2924) [R7L208](https://github.com/R7L208)
* Add Symend to SQLFluff In The Wild [2940](https://github.com/sqlfluff/sqlfluff/pull/2940) [HeyZiko](https://github.com/HeyZiko)
* Simplify segment creation and inheritance in dialects [2933](https://github.com/sqlfluff/sqlfluff/pull/2933) [barrywhart](https://github.com/barrywhart)
* Snowflake: Add `ALTER STREAM` support [2939](https://github.com/sqlfluff/sqlfluff/pull/2939) [HeyZiko](https://github.com/HeyZiko)
* T-SQL: Handle multiple nested joins [2938](https://github.com/sqlfluff/sqlfluff/pull/2938) [tunetheweb](https://github.com/tunetheweb)
* Snowflake: Add `CREATE STREAM` support [2936](https://github.com/sqlfluff/sqlfluff/pull/2936) [HeyZiko](https://github.com/HeyZiko)
* T-SQL: Support nested joins [2928](https://github.com/sqlfluff/sqlfluff/pull/2928) [tunetheweb](https://github.com/tunetheweb)
* To replace base dialect segment class, must subclass or provide same stuff [2930](https://github.com/sqlfluff/sqlfluff/pull/2930) [barrywhart](https://github.com/barrywhart)
* Add new rule L063 to allow separate capitalisation policy for Datatypes [2931](https://github.com/sqlfluff/sqlfluff/pull/2931) [tunetheweb](https://github.com/tunetheweb)
* Adds support for column definitions in table alias expressions [2932](https://github.com/sqlfluff/sqlfluff/pull/2932) [derickl](https://github.com/derickl)
* BigQuery: support numeric aliases in `UNPIVOT` clauses [2925](https://github.com/sqlfluff/sqlfluff/pull/2925) [tunetheweb](https://github.com/tunetheweb)
* T-SQL: Supported nested `MERGE` statements [2926](https://github.com/sqlfluff/sqlfluff/pull/2926) [tunetheweb](https://github.com/tunetheweb)
New Contributors
* [HeyZiko](https://github.com/HeyZiko) made their first contribution in [#2936](https://github.com/sqlfluff/sqlfluff/pull/2936)
* [aviv](https://github.com/aviv) made their first contribution in [#2952](https://github.com/sqlfluff/sqlfluff/pull/2952)
* [pguyot](https://github.com/pguyot) made their first contribution in [#2963](https://github.com/sqlfluff/sqlfluff/pull/2963)
* [dmohns](https://github.com/dmohns) made their first contribution in [#2971](https://github.com/sqlfluff/sqlfluff/pull/2971)
* [tcholewik](https://github.com/tcholewik) made their first contribution in [#2923](https://github.com/sqlfluff/sqlfluff/pull/2923)
* [jmc-bbk](https://github.com/jmc-bbk) made their first contribution in [#2981](https://github.com/sqlfluff/sqlfluff/pull/2981)
* [dnim](https://github.com/dnim) made their first contribution in [#3010](https://github.com/sqlfluff/sqlfluff/pull/3010)
* [kd2718](https://github.com/kd2718) made their first contribution in [#3014](https://github.com/sqlfluff/sqlfluff/pull/3014)
* [mrf](https://github.com/mrf) made their first contribution in [#3016](https://github.com/sqlfluff/sqlfluff/pull/3016)
* [zapion](https://github.com/zapion) made their first contribution in [#3031](https://github.com/sqlfluff/sqlfluff/pull/3031)