Mindsdb

**Features**
7141 Added finetune support for BYOM engine
7133 TimeGPT: Added handling for finetune steps, token validation, and data features
7122 Added PyPI Handler optimizations

**Documentation**
7240 Added technical docs for jobs
7227 Updated docs of JS SDK Connect
7208 Fixed instructions for generating slack tokens
7206 Added quickstart video
7200 Added SurrealDB documentation
7199 Added json_extract() method to docs
7194 updated git clone line on install docs
7192 Updated slack docs with permission information
7179, 7171 Updated documentation on Scylla and Cassandra
7173 Added Bot Scopes Explicitly to Slack Chatbot Tutorial
7168 Added Manual_QA for github_handler
7156 Typo fixes
7146 Added tutorial house sale example for TimeGPT
7145 Added monthly expenditure timegpt tutorial
7135 Added Apache ingnite form in documentation
7134 Fixed labels and description in clickhouse documentation
7128 Added docs for llm ui and slack
7125 Added docs for Email (connecting email account to MindsDB via 'email' engine)
7124 Added FAQ page for MindsDB with PHP
7119 Fixes for WebCrawler docs
7108 Updated AI Workflow section

**Fixes**
7239 Deleted obsolete methods
7237 Lightwood fix, alert on missing data when training.
7228 Fixed langchain parameter name, can use promp_template
7216 Fixed deprecated requirement sklearn, adding scikit-learn instead
7215 Fixed batch prediction via mongodb
7201 Added defensive failure when creating an ML_ENGINE with empty arguments
7197 Added support for non-grouped data in timegpt
7191 Fixed huggingface pandas requirement version
7178 Deleted docs_old directory
7176 Added docker dependencies for MonkeyLearn
7174 Langchain error messages used to only display first 50 characters, now will display whole message.
7163 TimeGPT fixes for frequency and levels
7161 Addeed missing autokeras requirement for the autokeras handler
7160 Deleted mindsdb and mindsdb-sql from handlers' requirements
7159 Rounding timestaps in the binance handler
7155 Predownloading nltk data in docker (punkt and stopwords)
7154 Added binance-connector to Docker
7152 Always return new model record when calling train or finetune
7151 Fix for rare issue raised when integration data is empty
7147 Support to convert datecolumn to datetime in timegpt_handler
7144 Fixed bug related to the target variable forcasted by StatsForcast in the Statsforcast's House Sale tutorial
7143 Removed restriction of only using SSH keys to clone MindsDB repo
7138 Added abillity for Webcrawler to fetch href attribute from an <a> tag
7136 Fixed typo in itnerfaces.database.projects
7130 Added duckdb to docker dependencies. (required for gogole sheet integration)
7117 Updated Walrus and redis requirements
7115 Fix to run 'predict' tasks in subprocess
7113 Allow Web Handler to Process URLS wrapped in single quotes
7111 Raise SlackAPIError in Select Methods
7102 Added improvements to email_Datasource handler, setting defaults to since_date and until_date
7098 Fixed error with web handler not getting content if link was without href
7093 Fixed Webhandler error when output dataframe is empty and doesn't contain error column
7241 Added Plaid and Confluence dependencies to docker image

New features, major changes:
- improved support of python 3.10 and 3.11
- Triggers engine, subscribe for mongodb 6819
- Chatbot engine unifies realtime and regular (select from table) chatbot interfaces 6790
- Added chatbots table 7038
- OpenAI embedding mode 7011
- Migrate to sqlalchemy 2.0 7014
- Huggingface - finetune 5993
- Run predictions in subprocess 6935

New handlers
- [ENH] Replicate Handler 6795
- Monkeylearn handler for Mindsdb 5543
- Handler for Nixtla's TimeGPT 7054
- Initial implementation of Webz Application Handler 6748
- PyPI Handler Integration 7026
- Handler for SentenceTransformers 7028

Improvements, optimizations:
- Improve CSV reader 7016
- Slack-OpenAI Integration: changes in Select and Delete statements 6808
- Improve error message when create ml engine 7040
- reading PDFs in web handler 7073
- Safely Fetch 'href' Attribute from 'a' Tag in Web Crawling 7138
- PyPI Handler Optimization 7122
- Update walrus and redis requirements 7117
- convert types to duck db friendly ones 7074
- Allow Web Handler to Process URLs Wrapped in Single Quotes 7113
- Raise SlackApiError in Select Method 7111
- Email Integration Improvements 7102
- Statsforecast: allow to create model without group column 7082

Bugs fixes
- fix typo Can't remove active version: f{model} 7018
- [Bug, Docker] Shopify bug fixed 6983
- Added attempt to parse as unicode 7032
- Fix for PG & sqlalchemy2 7041
- Don't fall if describe is not implemented 7020
- Fix same value error in batch for monkeylearn handler 7044
- Fixes for statsforecast 7050
- Fixed chatbot history limit 7062
- Fix for snowflake+sqlalchemy2 7084
- Fix for clickhouse+sqlalchemy2 7080
- Fix Database Connection Issue With Ignite by Ensuring Port is an Integer 7063
- Add Ignite dependencies to Docker 7065
- correct passing of finetuning args in lightwood 7067
- fix: convert date column to datetime (monthly expenditure fix)) 7147
- Fix web handler 7098
- WEB handler: no error if result is empty 7093

Updates of docs:
- New docs structure 7039
- Update shopify.mdx 7015
- Manual QA for Apache Ignite 6988
- Create twitterbot-mariadb-enterprise-server-skysql.md 7004
- Fixed: SurrealDB docs 6991
- Tested & Added docs for YouTube Integration 6947
- Slack-OpenAI Integration Tutorial 6808
- Changed Aplication to Application 5756
- Update install.mdx to new path for requirements.txt 7035
- misc docs updates 7042
- improve youtube documentation 7052
- updating twitter tutorial 7045
- updated a query 7046
- removed temp python sdk file 7047
- fixed links 7078
- fixed image in slack tutorial 7079
- twitter tutorial fix 7068
- added compressed gif 7075
- referenced git in twitter tutorial 7076
- fixed reference to a gif 7077
- added author link to twitter tutorial 7083
- fix for house sales statsforecast tutorial 7144
- added an faq page 7124
- added docs for llm ui and slack 7128
- added email docs 7125
- updated ai workflow section 7108
- docs for web crawler 7119
- Create Manual_QA.md 7049
- Create monkeylearn.mdx 7085
- Binance 7088

Features/Improvements:
- Added additional "conn_attrs" key to MySQL config, which is required for providing a partner name for the SingleStore Integration
- Improved error messages in cases where handler cannot be used, removing handlers in list when they cannot be imported, and deleting ability to create integration via config file.

Bug fixes:
- Fixing Writer Handler, adding column names to meta data, option to specify base_url directly for writer.
- Fixed type mismatch for models_versions table
- Added unknown argument validation for OpenAI Handler
- Fix for creating ML Engine without Params
- Fix to not rise an exception when table has more than two parts, will send select as is for an integration.
- Fixes for Oracle connection, adding "disable_oob" param to disable out-of-band breaks, "auth_mode" param for connecting with database privileged, and fixing get_table methods.
- Fix for Shopify integration's dependency, adding it to release file so it works with Docker
- Fix for file locking on some systems.
- Fix to be able to use statsforcast on docker.

Documentation:
- Microsoft MySQL Server - Updated docs to make process of setting up server simple.
- Updated "What is Mindsdb"
- Updated info field ind documentaiton for all apps
- Added Rest API Update View Page
- Added Manual QA Test for US Political Party tutorial
- Minor fix for list number in README
- Updated Manual QA testing for Shopify Handler
- Added ManualQA for supabase handler
- Added OpenAI Example to JS SDK Docs
- Fixing SurrealDB Docs, removing irrelevant documentation
- Added Shopify Docs to nav
- Updated ManualQA for Google BigQuery
- Updated Google BigQuery Docs for when to use service_account_keys versus service_account_json

Fixes:

- Fix building docker image

Updates/Improvements:
- Multi mode chatbot 6642 (works only locally)
- Decouple Langchain from OpenAI and add Claude Support 6912

Fixes:
- Fixed dremio_handler 6910
- Allow to upload CSV with single column 6896

Updates of documentation

Improvements
- Support embedding mode for lightwood models https://github.com/mindsdb/mindsdb/pull/6779
- Show error message from frappe api https://github.com/mindsdb/mindsdb/pull/6872
- Speedup deleting of models with multiple versions https://github.com/mindsdb/mindsdb/pull/6860
- `SHOW handlers` will show ML and Data handlers https://github.com/mindsdb/mindsdb/pull/6724
Bug fixes
- check project names case-independent https://github.com/mindsdb/mindsdb/pull/6842
- set predictor status to 'error' if can't get training data https://github.com/mindsdb/mindsdb/pull/6871
Other
- updates of documentation
- various improvements and bugfixes