Starwhale

🧭 What's Changed
* enhance(sdk): enhance model build sdk for searching modules (2182) tianweidut
* bug(client): fix the expired jobs.yaml for workdir mode in the swcli model run command (2184) tianweidut
* bug(sdk): fix predict inspect with decorators issue (2221) tianweidut
* bug(sdk): fix predict log plain mode error (2224) tianweidut
* chore(SDK): add kwargs for PipelineHandler init (2194) tianweidut
* chore(ci): disable codecov ci fail if error (2190) tianweidut
* chore(ci): disable release image build on push event (2178) tianweidut
* chore(ci): fix base and nodejs image rebuild issue (2174) tianweidut
* chore(ci):fix ut typo for useless owner (2237) tianweidut
* chore(client): add test case for needs on normal function (2180) tianweidut
* chore(client): add trace log function (2227) tianweidut
* chore(client): do not refine by default when parse uri str (2217) jialeicui
* chore(client): ignore conda env by default when building model (2232) jialeicui
* chore(controller): make status as running when pod condition contains PodScheduled (2173) jialeicui
* chore(datastore): batch put records with unknown type (2210) jialeicui
* chore(datastore): change map resp layout and add type for null value (2212) jialeicui
* chore(datastore): encodeWithType set to true by default (2200) jialeicui
* chore(e2e): fix conda runtime e2e test (2186) tianweidut
* chore(e2e): simplfy e2e test cases (2229) tianweidut
* chore(runtime): add pip compatible dependencies check for runtime build from existed env (2219) tianweidut
* chore(sdk): PipelineHandler support `predict` and `evaluate` function interface (2183) tianweidut
* chore(sdk): set predict replicas default to 1 (2198) tianweidut
* doc: add console project doc (2149) lijing-susan
* enhance(client): detect shell conda bin path at first (2215) tianweidut
* enhance(client): model build ignore venv and python cache files by default (2193) jialeicui
* enhance(client): tune client logger/console.print (2158) tianweidut
* enhance(client): tune copy dir performance without recording ignored files (2235) tianweidut
* enhance(client): tune model run output for runnable handlers (2211) tianweidut
* enhance(client): tune runtime activate for the broken restore (2216) tianweidut
* enhance(model): change current workdir to snapshot dir for model run command (2187) tianweidut
* enhance(model): provide snapshot for model run (2175) tianweidut
* enhance(runtime): make conda export for URL for pip packages installed from git (2164) tianweidut
* enhance(sdk): add more info into external for predict function (2209) tianweidut
* enhance(sdk): predict function accepts the multi input argument modes (2192) tianweidut
* enhance(sdk): support multi modes for predict result logging (2196) tianweidut
* enhance(sdk): tune predict log results for user-friendly viewer (2234) tianweidut
* feat(mode): add `model extract` command (2177) tianweidut
* feat(storage): add a head method that returns md5sum (2220) xuchuan
* feat(storage): support presigned url with put method (2197) xuchuan
* feat: add env support when running online evaluation and runtime dockerized (2223) anda-ren
* feature(datastore): add column value hints for query and scan (2108) xuchuan
* fix(client): model run with --runtime error when model has packaged rt (2236) goldenxinxing
* fix(client): parse full uri with type hint failed (2226) jialeicui
* fix(client): web for job info works (2225) jialeicui
* fix(client): wrong column hints response in local data store web api (2228) jialeicui
* fix(console): column wrong width when grid resize, job form wrong step when trigger raw type, etc (2199) waynelwz
* fix(controller): sync duration time to datastore job info table (2176) goldenxinxing
* fix(datastore): fix NPE when updating type of list element (2188) jialeicui
* fix(datastore): fix faking object response of encoded type value (2233) jialeicui
* fix(datastore): fix parsing type list to tuple (2231) jialeicui
* fix: canceled task is not released immediately (2218) anda-ren
* hotfix(ci): fix handler name typo for examples (2191) tianweidut
* hotfix(example): workaround pfp bbox type issue (2230) tianweidut
* refactor(client): deprecate URI (2185) jialeicui
* refactor(client): use uri components instead of URI (2124) jialeicui
* update(console): tooltip/datepicker/locales and new extend mode (2179) waynelwz
⚙️ Who Contributes
anda-ren, goldenxinxing, jialeicui, lijing-susan, tianweidut, waynelwz and xuchuan

🧭 What's Changed
* bug(client): fix model list too slow and unexpected model size issues (2129) tianweidut
* bug(client): fix model serve default yaml issue (2135) tianweidut
* bug(client): fix workdir path name resolve issue (2170) tianweidut
* bug(e2e): fix runtime build for e2e with no cache config (2138) tianweidut
* bug(example): fix speech command example to device typo (2130) tianweidut
* bug(server): fix dataset update sql error (2147) anda-ren
* chore(client): fix urllib3 version (2163) goldenxinxing
* chore(client): remove warning log when use event loop in thread pool (2161) goldenxinxing
* chore(controller): add job info in pod annotations (2139) jialeicui
* chore(controller): add project id into meta (2154) jialeicui
* chore(controller): add resource pool update api (2151) jialeicui
* chore(controller): ignore not found code when delete secret (2126) jialeicui
* chore(e2e): use model run command for e2e test (2140) tianweidut
* chore(runtime): runtime process live stream output for conda mode (2167) tianweidut
* enhance(client): simpify dataset info command with auto complete latest version (2150) tianweidut
* enhance(e2e): tune github e2e run logs (2125) tianweidut
* enhance(runtime): support `--runtime` run in the sys.argv subprocess mode (2133) tianweidut
* example(finetune): chatglm finetune (2117) anda-ren
* example(model): add tloen/alpaca-lora example (2136) anda-ren
* feat(client): support starwhale model package to embed starwhale runtime (2122) tianweidut
* feat(console): add job page (2128) waynelwz
* feat(console): datepicker ui (2168) waynelwz
* feat(console): extends route (2165) waynelwz
* feat(controller): support custom envs for steps (2105) goldenxinxing
* feat(dataset): support `--patch`/`--overwrite` options for dataset copy command (2141) tianweidut
* fix(client): ucf video mime type error (2169) waynelwz
* fix(console): columns transfer invisible not showing, parse bytes in panel (2171) waynelwz
* fix(console): model viewer not full size and support file size (2132) waynelwz
* fix(controller): return property of built image from api (2134) goldenxinxing
* fix(controller): the startup hangs when loading lots of wal files (2160) jialeicui
* fix(controller): unexpect file stream closed (2166) goldenxinxing
* fix(datastore): error when update table schema (2148) jialeicui
* hotfix(console): disable decode cache, for records with uncertain unique key (2152) waynelwz
* hotfix(console): gets columns as function in case of format by store (2131) waynelwz
* hotfix(console): step & editor status sync error (2153) waynelwz
* hotfix(e2e): tune e2e for the useless check (2172) tianweidut
* hotfix(runtime): fix runtime conda activate typo (2159) tianweidut
* refactor(console): make grid table scalable and modula, intergrate result compare in eval list (2127) waynelwz
⚙️ Who Contributes
anda-ren, goldenxinxing, jialeicui, tianweidut and waynelwz

🧭 What's Changed
* bug(model): fix generate dag bug for more than one out degree case (2101) tianweidut
* chore(client): always retry when updating remote data store (2110) jialeicui
* chore(controller): add metadata for annotation in resource pool (2111) jialeicui
* chore(controller): limit parameters for job create api (2115) goldenxinxing
* chore(e2e): keep mysql and minio when e2e success (2100) anda-ren
* chore(e2e): split e2e test phase into github steps (2106) tianweidut
* enhance(e2e): tune e2e speed (2109) tianweidut
* enhance(sdk): remove predict/evaluate decorator use limits (2112) tianweidut
* feat(client): support `swcli job` command (2104) tianweidut
* feat(client): support swcli model info command for output filter (2121) tianweidut
* feat(controller): add handler parameter for job create api (2114) goldenxinxing
* feat(controller): resource pool support tolerations (2079) jialeicui
* feat(dataset): add field transformer for dataset loader (2118) anda-ren
* feat(model): support multi handlers for model run (2071) tianweidut
* feature(datastore): return column hints for query and scan (2103) xuchuan
* fix(client): model run error when load modules with gradio (2120) goldenxinxing
* hotfix(console): use columnHints for type, use ignoreNonExistingTable as default (2107) waynelwz
⚙️ Who Contributes
anda-ren, goldenxinxing, jialeicui, tianweidut, waynelwz and xuchuan

🧭 What's Changed
* bug(client): fix too large indexed-size number for localFS make_file (2022) tianweidut
* bug(dataset): fix drop fp in the dataset build processing (2092) tianweidut
* bug(example): fix dataset examples for the localfs link issue (2040) tianweidut
* bug(example): fix examples(nmt/pfp) evaluation_store typo (2010) tianweidut
* bug(runtime): fix non-default runtime.yaml name build and info issue (2081) tianweidut
* chore(client): add ui assets to pkg data (2012) jialeicui
* chore(client): batch put data store records (2018) jialeicui
* chore(client): compress data store table file (2058) jialeicui
* chore(client): data store support deep copy when scan and insert (2063) jialeicui
* chore(client): make the internal options hidden in the `--help` output (2027) tianweidut
* chore(client): store diff only in the data store (2062) jialeicui
* chore(client): update cli `--project` option doc (2028) tianweidut
* chore(console): refine code editor language detection (1960) jialeicui
* chore(controller): add build runtime image result for api (2032) dreamlandliu
* chore(controller): change shared type to boolean (2044) dreamlandliu
* chore(controller): datastore query and scan api support revision (2021) jialeicui
* chore(controller): rename cmd's tag in the image build job yaml (1944) goldenxinxing
* chore(controller): update api of data store returns the revision (1988) jialeicui
* chore(dataset): sdk dataset head use internal data loader (1990) tianweidut
* chore(dataset): hidden get_data_loader and get_dataset_consumption low-level api (2025) tianweidut
* chore(dataset): new dataset entrypoint function for create or load in Python SDK (1971) tianweidut
* chore(dataset): refactor TabularRow `data` to `features` field (1946) tianweidut
* chore(dataset): remove `build_from_handler` interface for dataset python sdk (1964) tianweidut
* chore(dataset): remove dataset `create` argument in python sdk (1998) tianweidut
* chore(dataset): remove dataset origin field (2020) tianweidut
* chore(dataset): remove useless iterable builder (2019) tianweidut
* chore(dataset): remove user-raw dataset format support (2045) tianweidut
* chore(doc): refine some docs (2083) xuchuan
* chore(doc): refine some docs (2053) xuchuan
* chore(doc): refine some docs (2037) xuchuan
* chore(doc): refine some docs (2030) xuchuan
* chore(doc): refine some docs (2008) xuchuan
* chore(doc): refine some docs (2007) xuchuan
* chore(doc): refine some docs (1997) xuchuan
* chore(doc): remove the useless keep_files config for doc release ci (2002) tianweidut
* chore(doc): update doc release action for github pages publish (1954) tianweidut
* chore(docker): build nodejs image automatically and add make cmd in the image (1959) jialeicui
* chore(docs): set what-is-starwhale's routing path to the root (2005) xuchuan
* chore(e2e): refine e2e remove logic (1972) anda-ren
* chore(log): add admin server and client for monitor (1909) goldenxinxing
* chore(script):support multi arch for image sync (1995) goldenxinxing
* chore(test): make dataset sdk torch tests compatible with different pytorch versions (1974) tianweidut
* chore(test): move upgrade-test into apitest (2074) dreamlandliu
* e2e: down grade deployment after e2e success (2042) anda-ren
* example(GraphQuestions): add graph_questions_testing dataset (2075) anda-ren
* example(LLM): chat-gpt/bard model example (2059) anda-ren
* example(dataset): add ComplexWebQuestions Dataset (2088) anda-ren
* example(dataset): add GrailQA_v1.0 Dataset (2087) anda-ren
* example(dataset): add KQApro Dataset (2086) anda-ren
* example(dataset): add MKQA Dataset (2090) anda-ren
* example(dataset): add QALD-9 Dataset (2089) anda-ren
* example(dataset): add WebQuestions Semantic Parses Dataset (2084) anda-ren
* example(dataset): ds z bench (2070) anda-ren
* example(model): chatglm6b (2068) anda-ren
* feat(client): cli supports displaying eval on locally served web pages (1949) jialeicui
* feat(client): eval info --web support remote instance (1992) jialeicui
* feat(client): model building support eval layout yaml config (1987) jialeicui
* feat(console): i18n support for zh (2056) waynelwz
* feat(console): project sort by visit type and new project card (2024) waynelwz
* feat(console): shared api (dataset/runtime) with dynamic tree selector (2041) waynelwz
* feat(console): support model handler when job create & used Mono Font for version labels (2091) waynelwz
* feat(console): support showing the model builtin layout (1968) jialeicui
* feat(console): support simplified version of layout settings (1980) jialeicui
* feat(controller): diffrent sorting methods of project list (2006) dreamlandliu
* feat(controller): global runtime and dataset versions (2036) dreamlandliu
* feat(controller): support backup and restore for mysql (1978) goldenxinxing
* feat(controller): support share for model (2080) goldenxinxing
* feat(dataset) support mapping style dataset builder (1976) tianweidut
* feat(dataset): `MappingDatasetBuilder` supports to upload swds_bin to cloud instance (1996) tianweidut
* feat(dataset): add create mode argument for dataset sdk (2043) tianweidut
* feat(dataset): dataset sdk supports to use datastore revision for builder and loader (2026) tianweidut
* feat(dataset): new dataset constructive mechanism with versioning datastore table (1973) tianweidut
* feat(dataset): provide `uri`, `committed_version`, `changed` properties for SDK Dataset (1994) tianweidut
* feat(dataset): support dataset blob attributes (2034) tianweidut
* feat(dataset): support dataset row `features` CRUD operations with python sdk (1950) tianweidut
* feat(dataset):add more info into dataset summary (2029) tianweidut
* feat(datastore): datastore api support revision (2016) jialeicui
* feat(datastore): make table schemaless (1989) xuchuan
* feat(datastore): support merging local file with memory table (2052) jialeicui
* feat(fine-tune): support finetune and add example (2054) goldenxinxing
* feat(runtime): enable runtime activate command restore automatically (2057) tianweidut
* feat(runtime): support `swcli runtime build` from multi sources (2061) tianweidut
* feat(runtime): support output filter for runtime info (2065) tianweidut
* feat(sdk): support `DataRow.features` attribute-like usage (1948) tianweidut
* feat(sdk): support evaluation functional sdk (1900) tianweidut
* feat: upgrade for controller and e2e script (1986) dreamlandliu
* fix(ci): fix gradio build error (2038) anda-ren
* fix(client): extract python version when use runtime info (2077) goldenxinxing
* fix(client): fix unclosed file warning in model eval (1985) jialeicui
* fix(client): install console related dev tool for building ui (2013) jialeicui
* fix(client): remove the prefix slash when listing tables in the client (1966) jialeicui
* fix(client): ut without existing local config or default (2011) jialeicui
* fix(console): UCF video display error, add status label UI to evaluation, display table after query in panel, and fix deployment script. (2023) waynelwz
* fix(console): allow non-exist model builtin panel setting in the eval result page (1984) jialeicui
* fix(console): dataset selector filter error and make select default tree node when filter exsits with arrow keyboard (2047) waynelwz
* fix(console): delete the child widgets when deleting the panel (1965) jialeicui
* fix(console): e2e gradio build error (2046) waynelwz
* fix(console): eval pages works with project name in uri (1991) jialeicui
* fix(console): handler editable status not sync (2094) waynelwz
* fix(console): loop rerender when panel loading (1983) waynelwz
* fix(console): missing time format & popover ui (2096) waynelwz
* fix(console): online eval page can not show (1970) jialeicui
* fix(console): resolve the gitignore missing issue after yarn building (1982) jialeicui
* fix(console): set ploty auto layout when large mount of data (2009) waynelwz
* fix(console): sidebar invisible in admin page (2003) jialeicui
* fix(controller): admin server conflict with sw index page (1979) goldenxinxing
* fix(controller): agent save error when report empty node (1957) goldenxinxing
* fix(controller): job spec name typo (2031) goldenxinxing
* fix(controller): jobs from console with GPU requests can not work (1975) jialeicui
* fix(controller): optimize upgrade/cancel upgrading api and fix some bugs (2076) dreamlandliu
* fix(controller): the token may be treated as invalid if the pod is pending in eval (2095) jialeicui
* fix(datastore): check IOException when closing a parquet file (2060) xuchuan
* fix(datastore): fix rename issue after table file compress (2082) jialeicui
* fix(datastore): remove unnecessary testcases (2102) xuchuan
* fix(datastore): sort attributes by names when creating parquet types (2098) xuchuan
* fix(docker): buildx can not work without init in release workflow (1956) jialeicui
* fix(docker): do not use mirror when building nodejs multi-arch images (1961) jialeicui
* fix(e2e): fix e2e script error (2064) anda-ren
* fix(e2e): fix e2e won't fail (2085) anda-ren
* fix(e2e): remove possible ^M char in auth header (1999) anda-ren
* fix(e2e): torch 2.0 has no _six module (1981) anda-ren
* fix(e2e): tune e2e script (2014) anda-ren
* optimize(console): avoid rerender and use partical measure columns (1977) waynelwz
* refactor(client): datastore in standalone support version and mutable schema (2004) jialeicui
* refactor(console): support init state from editor props (1958) waynelwz
* refactor(console): support mixed type column (2072) waynelwz
* refactor(controller): add object storage for dataset (1993) anda-ren
* refactor(dataset): refactor dataset cli build and copy cmd with dataset versioning feature (2001) tianweidut
* refactor(dataset): use `features` word to explain data and annotations in the dataset row (1943) tianweidut
* tune(e2e): add fault tolerance for gradio build (2033) anda-ren
* update(console): adapt the API to the dataset revision version (2049) waynelwz
* update(console): fix button ui (1967) waynelwz
⚙️ Who Contributes
anda-ren, dreamlandliu, goldenxinxing, jialeicui, tianweidut, waynelwz and xuchuan

🧭 What's Changed
* bug(example): fix ucf101 makefile typo (1849) anda-ren
* bug(release): add cuda11.5 into the docker image sync list (1851) tianweidut
* chore(charts): refactor helm values for the hostpath pv (1888) tianweidut
* chore(charts): remove useless taskSplitSize argument in charts (1874) tianweidut
* chore(client): add a submit button for online eval (1889) jialeicui
* chore(client): build support swignore (1934) jialeicui
* chore(client): check if project is set when list bundle (1937) jialeicui
* chore(client): move auto-generated file to the .starwhale dir (1936) jialeicui
* chore(client): refine the behavior when copying resource from remote using url copied from the console (1870) jialeicui
* chore(client): remove useless config field in the model.yaml (1933) tianweidut
* chore(client): remove useless pkg_data/exclude_pkg_data field in the model.yaml (1939) tianweidut
* chore(client): show console url of the remote bundle after copy done (1923) jialeicui
* chore(console): online eval in server instance support examples (1846) jialeicui
* chore(controller): add remove permission to project maintainer (1887) dreamlandliu
* chore(controller): controller chart main storage supports local fs (1865) anda-ren
* chore(controller): make task info on task page richer (1892) anda-ren
* chore(controller): store manifest with oss instead of using db (1941) jialeicui
* chore(controller): update model_version.manifest to mediumtext (1935) jialeicui
* chore(controller): use version name of resource instead of using version id (1898) jialeicui
* chore(controller): validate and patch runtime resource when eval (1918) jialeicui
* chore(doc): add a temp flag to keep the first archived versions dir (1869) tianweidut
* chore(doc): redirect homepage to standalone quickstart doc (1857) tianweidut
* chore(doc): refactor docs organization by the product topic (1894) tianweidut
* chore(doc): tune git clone command in the docs (1872) tianweidut
* chore(docker): controller support multi-arch and refine Makefile (1922) jialeicui
* chore(e2e): enlarge pip timeout in e2e runtime (1879) anda-ren
* chore(example): add COLUMNS env for eval output in the jupyter notebook (1855) tianweidut
* chore(example): add stable diffusion example (1913) jialeicui
* chore(example): remove duplicate requirements in runtime (1873) anda-ren
* chore(log):add more visible log and status for evaluation (1826) goldenxinxing
* chore(sdk): refactor evaluation internal api (1895) tianweidut
* chore(sdk): remove useless single `__exit__` function in the evalution module (1911) tianweidut
* chore(test): make dataset loader test stable on the unordered rows (1931) tianweidut
* chore(ut):reduce datastore records num (1902) goldenxinxing
* doc(notebook): update readme and standalone quickstart doc, notebooks (1842) tianweidut
* e2e: e2e offline test support multiple python version (1837) anda-ren
* enhance(doc): support doc versioning (1862) tianweidut
* enhance(runtime): refactor runtime build with the shadow requirements-sw-lock.txt file (1878) tianweidut
* feat(client): auto reuse files of model (1906) jialeicui
* feat(console): add name confirm before project remove (1925) waynelwz
* feat(console): add virtual plugin to extend routes of out path (1901) waynelwz
* feat(console): dataset supports more flexible display, and fix various bugs (1893) waynelwz
* feat(console): show starting events in online eval loading page (1832) jialeicui
* feat(console): support multiple dataset (1924) waynelwz
* feat(controller): add runtime suggestion api (1805) jialeicui
* feat(eval): runtime support image build on server (1885) goldenxinxing
* feat(sdk): support model build sdk (1897) tianweidut
* feat(sdk): support track sdk for experiment (1696) tianweidut
* fix(client):dockerize cmd can not found tmpl and tmp file error (1886) goldenxinxing
* fix(console): coco with _show control (1903) waynelwz
* fix(console): show content directly in TextViewer (1930) jialeicui
* fix(console): support dataset without shape attribute (1850) waynelwz
* fix(console): update runtime version fetch param name (1863) jialeicui
* fix(controller): explicitlly set charset of offline log (1880) anda-ren
* fix(controller): fix bug with getting current user roles & e2e (1877) dreamlandliu
* fix(controller): fix eof issue when saving manifest to the oss (1947) jialeicui
* fix(controller): fix task offline emoji not show error (1875) anda-ren
* fix(controller): job status is still fail after resuming job (1884) anda-ren
* fix(controller): realtime log gibberish (1905) goldenxinxing
* fix(controller): tune controller jvm max heap size (1896) anda-ren
* fix(controller): update job status accord with k8s spec (1890) anda-ren
* fix(datastore): data lost when save table with error (1938) goldenxinxing
* fix(docker): fix wrong base image of the server (1929) jialeicui
* fix(docker): restore the missing build-release-base-server target (1926) jialeicui
* fix(e2e): fix server build issue in e2e (1927) jialeicui
* hotfix(console): extend folder should exsit for vite route plugin (1912) waynelwz
* hotfix(doc): add tag for doc versioning (1951) tianweidut
* hotfix(doc): fix docs copy path typo (1953) tianweidut
* refactor(controller): refactor of service & dao reference and fix bugs (1856) dreamlandliu
* update(console): remove accordin panel (1904) waynelwz
* update(console): set wrapper height to make sure resizer not null (1858) waynelwz
* update(console): ui optimize for process modal (1907) waynelwz
* update(docs): use default monospaced fonts (1861) waynelwz
⚙️ Who Contributes
anda-ren, dreamlandliu, goldenxinxing, jialeicui, tianweidut and waynelwz