Changelogs » Ngta



1. Change some arguments into multi-lines.
2. Update doc location to readthedocs.
3. Simply process name in log.
4. Add `_node_` attribute for module and class.
5. Provide additional kwargs for ngta.ext.http.bench.TestBench.
6. Add is_busy attribute for AmqpTestConsumer.


1. Import Mapping and Iterable from instead of collections.
2. Change agent default port to 7070.
3. Change ngta-agent linux service file to /etc/systemd/system.
4. Add .readthedocs.yml.
5. Return http 404 if there is no repo for /api/repo.


1. Add compare debug log for is_equal_to.
2. Fix bug: log traceback failed when exiting soft_assertions.


1. YAML `<testcase>` node support repeat-number.


1. Make errors more precisely.
2. Add more type annotations.
3. Fix bug: Some functions' locations are changed.
4. Add more type annotations.
5. Add more type annotations.
6. Replace type annotations -> None to -> NoReturn.


1. Export more classes and functions.
2. Add more comments.
3. Rename TestReport argument extras to props.
4. Rename TestProgram name.


1. Check node tag before iter its sub node name.
2. Modify testcase log name style.
3. Remove extra properties from TestResult.
4. Remove TestResultShelveInterceptor.
5. Save testbench dict into result.json in TestResultJsonInterceptor.
6. Support config result in yaml.
7. Define const CALLEE_KEY="()".
8. Update yaml sample config.
9. ErrorInfo support init with str.
10. Set TestBench, TestResult, TestCaseResultRecord, TestSuiteResultRecord, CheckPoint inherit from DictSerializable.
11. Move serialize related functions or classes to


1. Support construct instance when parsing agent and bench yaml config in ngta.agent.
2. Fix bug: Dump json failed if testcase args have annotation.
2. Remove useless interceptor.


1. Add more variables into constants.
2. Move !object:new and !object:locate from loader to config.
3. Construct instance when parsing yaml instead of ConfigFile.get_runners().


1. Add log_dir and log_name annotation for TestCase.
2. Mark merge_result not implemented.
3. Fix json schema for yml <report> node.
4. Fix bug when record error is string.
5. Change yaml <report> node format.
6. Merge !object:invoke into !
7. Support construct instance by () recursively if not specified !object:new.


1. Change default log format.
2. Add logger name truncate if log format include %(name)s. ye.yang 2019/7/26 13:15
3. Add FIXME note for
4. Rename argument class_ to clazz.


Fix bug: Don't save execution history into .cache file.
Update coupling version greater than and equal than 1.1.0.


1. Move cache from .ngta to .cache.
2. Update jsonschema version greater than or equal than 3.0.1.
3. json schema validate for simple yaml config.
4. Remove useless dependency.


1. parametrize support kwargs to generate data by cartesian product.
2. support json dump enum and unknown types.


1. Fix bug: when loading <testcase> and <cls-loader>, and not provide params, it can't load data from parameterize.
2. Default display parameters in test report.


1. yaml config node iterations support 2d-array.
2. add html block for testcase record extras.
3. log exception when running program.


1. Move log_name and log_handler to testcase level, otherwise when calling another testcase in testcase, it will release log handler failed.


1. Remove yaml config root node <config>
2. Support simply yaml config without <runner> and <context>
3. Remove fail-freeze feature, use listener on_testcase_failed instead of.
4. Rename yaml tag !object to !object:new.
5. Add yaml tag !object:invoke to call function.
6. Add yaml tag !object:locate to locate a object.


1. Move code in to resources folder.
2. Don't init etcd if there are no test benches.
3. Add a new api to get git repo info.
4. Remove api to get git commits.
5. When collect relevant call traceback, check testcase in work dir instead of `__traceback_hidden__`.
6. Remove useless sub parser.
7. No longer support xml config file.
8. Remove xml related document.
9. Change agent testbench config from xml to yml.
10. Change --includes, --excludes, and --tags to nargs format.
11. Provide error when calling result.add_failure method.
12. Change python requires to >=3.6.5.
13. Support rerun failed test cases from result.json.
14. Add document for yaml config.
15. Fix bug when run with multi processes.
16. Rename routing to route


1. Add assert_warn function.
2. Add assert_warn method for TestCase.
3. Add more unit tests.
4. Move record related code from TestResult to TestCase.
5. Add agent.service for systemd.
6. python -m ngta.agent support install service on windows and linux.


1. Add testbench property for TestCase.
2. Fix bug when upgrade pyyaml to latest version.
3. Fix bug when running http api testcase.


1. Use pypika instead of SqlQueryBuilder.
2. Add inspector for SqlAlchemyHelper.
3. Truncate assert message.
4. Don't check headers and text rule if not defined.
5. Don't convert json result to user defined list and dict.
6. Remove !exec.function.
7. Change rule as test method argument instead of constructor argument.
8. Make sure path of <api-loader> should be file or dir.


1. Add requirement records
2. Refactor database related code according to records library.
3. Add for building docs with multiple languages.
4. Update travis.yml for generating docs.
5. Support version and language in doc.
6. Fix bug for assert_raises.
7. Change some private methods to protected methods.
8. Fix bug for rename testcase's methods.
9. Rename to
10. When dump json for TestCaseResultRecord, should handle error is None.


1. Add constants for some dirs' base name.
2. Format code with PEP8.
3. Change case_dir to cases_dir for more accurate.
4. Add sphinx docs.
5. Add more comments.
6. Add deploy key for deploying automatically by travis.


1. Change github url.
2. Modify duration type from str to float.


1. Fix bug of LocateTestProgram.
2. Add attribute 'extras' for TestRecord.
3. Remove casting argument from `SqlAlchemyHelper.__init__`.


* Fix bug when getting value from etcd3 is None.
* Format code with PEP8.
* Add testcase list http restful api.
* Add LRU cache for /api/testcases and /api/testhierarchy.


* Update API test yaml sample.
* Add git related content for work dir in README.
* Format code with PEP8.
* Add node property for TestBench.
* Add bench info for TestRecord.
* Store testbenches as list into ETCD3.
* Add Http interface to get git commit logs.


* Handle test method starts with prefix test of TestCase.
* Support define tests as dict or list in config yaml when testing API.
* Agent: Add type for each node of get_hierarchy_by_module()
* Agent: Fix bug when get hierarchy with specified commit.
* Agent: Always reload module when get hierarchy.


* Add timezone for datetime string.
* Use etcd3-py instead of python-etcd3 because python-etcd3 will cause exception after http request.
* Add data attribute for TestCase.as_dict()
* Fix bug: name is always null in result of TestCase.as_dict()
* Don't auto delete amqp queue.


* Rename start_time to started_at, finish_time to stopped_at for TestResult and TestCaseResultRecord.
* Fix bug when init project.


* Use etcd instead of zookeeper.
* Use python-etcd3 instead of etcd3-py.
* Fix bug when consume test from RabbitMQ.
* Use absolute path instead of relative path when store log names in TestCaseResultRecord.


* Refactor agent code:
1. Remove requirement: sqlalchemy.
2. Only support consume tests from RabbitMQ.
3. Remove redundant workers.
* Remove setup_testbench function.
* Move AMQP related code to agent package.
* Add default params `__config__`, `__work_dir__`, `__output_dir__`, `__logs_dir__`, `__logs_dir___`.
* Replace pydoc.locate to locate.


* QueryResult.extract support callback.
* Implement filter and casting for QueryResult.


* Update Diff class.


* `<testsuite>` element support class attribute to specify the class path.
* Add get_status_name() method for TestCaseResultRecord.
* Change Http requests style in html report.


* Still compare list and dict if its type not matched.


* Update architecture.mdj, please open it by StartUML.
* Fix bug which can't find target http request.
* Currently don's use deepdiff to implement is_equal_to.


* Always print diff log.
* When using is_equal_to, accept kwargs and assign it to deepdiff.


Move diff log before raise exception.


* Move traceback block into <pre> in html report.
* Change is_equal_to method to use deepdiff library.


* Support display multiple http requests in html report.


* Don't call TestResultShelveInterceptor by default.
* Fix bug for function get_module_name_safely.
* Change resp.json() return dict type from AttrDict to AttrOrderedDict.


* Add operation which used to exec when testcase start and undo when testcase stop.
* Change inspect.getargspec to inspect.signature.


* Fix bug: check exc_info's value before print track traces.
* Remove useless code in


* Add web ui test package.
* Remove assert_soft.


* Fix bug: main() function can't load test from python file or dir, if its dir not same with current work dir.
* Update default argument in


* Store compares into checkpoint.record
* Refactor sql and cypher query result object.
* Add more type hints.


* Change DB helpers type from `AttrOrderedDict` to `Helpers`.
* Remove `db_helper` property from `api.http.bench.TestBench`.
* Add `testbench` property for ``.


* Fix bug: when there are multiple http request in on testcase, it display wrong request in test report.
* Enhance default assert message.
* Truncate traceback line and  length greater than 255 in html report.
* HelperFactory support register helper.
* DB helpers support first, one, one_or_none method to find a db helper.


* Fix bug: don't handle args and kwds not defined when using !assert:callback and !exec:function
* Add DatabaseHelperFactory.


* Fix bug: fix bug if testcase failed and there are no checkpoints.
* Change testcase log name from it real name to its class and method name.
* Add tag !assert, !assert:callback, !exec:function.
* Rename !sql to !exec:sql.
* Move all constants into


* Add pre_imports in .ngta file.


* Add validate-schema for <config>


* Fix bug: don't display request and response in html report.
* Fix bug: TestRecord.request.url is empty.
* Display full http url in html report.
* Add default message as checkpoint name if not provide description for AssertionBuilder.


* DB Helper support casting argument.
* DB Helper select result type change from list to ResultList. And its item type changes to AttrOrderedDict.
* requests.Response.json() default return ResultList with AttrOrderedDict.


* Remove postfix of testsuite log.
* Change TestCase, TestSuite and TestRunner's default id from str to UUID object.
* Change !sql:select to !sql.
* Use includes and excludes instead of columns.


* Remove title from TestCaseResultRecord.
* Support more features when configuring yaml for http api test.


* Remove concerns
* Support multi <runner> in xml or yml config.
* Support <include> and <exclude> under <columns> of sql:select in http api yml config.


* Add difflib.ndiff result in log.
* Add casting for !sql:select node in yaml.


* Add enum for http method in yaml api json schema.
* Add more schemas for yaml api config.
* Don't do auto map on database by default.


* Support filter columns in when using !sql:select in YAML API config.
* Add skip in YAML API config.
* Revise YAML API config schema.
* Move http package into api.
* Fix bug of


* print sqlalchemy log in console and file.
* Fix bug when executing SelectSQL.
* Store request and response into record in RequestsSession.
* Fix bug when init db_helpers.
* Change 'method' and 'path' not required in api yaml config.
* Re-structure sample.
* The style will be lost when sending report via email, so move table `<thead>` and `<tfoot>` background-color from .css file to html style,
* Fix bug which don't distribute rule.schema.json into python package.


* When using yaml config to test api, support validate by SQL.
* Add a new http api test report style.
* Move http extension from ngta.ext.api to ngta.


* Chart position and default display can be controlled now.
* Add Http API test package.


* Fix bug: check point status is not right when assert_that passed.
* Fix bug: when loading tests in WorkEnv.case_dir, using absolute path to check it.


* Fix bug: there is no finish_time in TestResult.
* Fix bug: should use absolute path when running
* Fix bug: only load tests in WorkEnv.case_dir


* Add more `__traceback_hidden__` for ignore more redundant traceback.
* Fix bug when using soft_assertions.


* Add message default value as empty string.
* Revise some tests.


* Don't compatible with unittest.
* For test method or test function, it should be marked by test.
* Merge class and method as path for <testcase> in xml or yaml config.
* simply parametrize.


* Move dispatch events for ON_TESTMETHOD_STARTED and ON_TESTMETHOD_STOPPED into `TestCase._wrap_testmethod`.
* The log-level and log-layout will be applied in sub-processes.
* Support is_async for TestEventHandler.
* Support generate empty report when running with multi-processes.




* Rename get_observers to get_event_handlers in TestContext.
* Fix bug when running when multi-processes.
* Set testsuite default name testsuite.
* Fix bug when decode exec_command's stdout in
* Fix bug of sample.listener which define multi-function with same name.
* Support specify testbench in command-line.


* Implement event handler for all fixture: setUp, testDown, setupClass, teardownClass, setUpModule, tearDownModule
* WorkEnv inherit from coupling.pattern.singleton.BaseSingleton
* Fix bug when test is ErrorHolder in
* Add some testcases for
* Fix bug when calling assertRaises, assertRaisesRegex, assertWarns, assertWarnsRegex, assertLogs


* Remove because it only be supported from python3.6.
* Rename <listeners> to <event-observable> in xml config.
* Rename <listener> to <observer> in xml config.
* Add event-observable node in yml config.
* Fix bug when discover testcases from dir.
* Refactor event mechanism.


* When define class, don't inherit from object.
* Rename BaseTestBench to TestBench.
* Rename BaseTestEventHandler to TestEventHandler.
* Add method on_any_event for TestEventHandler.
* Change on_* methods' argument to event in TestEventHandler.
* Move requests package from install_requires to tests_require in


* Add more help for command-line.
* Add event.listen to register callback function or listener instance.
* Add decorator event.listen_for to register callback function, listener instance or listener class.


* Fix bug when store json into database.


* Add argument 'testbench' for main() function.
* main() argument 'locate' support module, class and method.
* Add MARK_ON_TEST_CLASS and MARK_ON_TEST_METHOD class property for decorator.
* Remove setup_testbench.


* Add `__str__`, `uid`, `name` for TestSuite.
* Add `uid` for TestSuite.
* Remove `__enter__` and `__exit__` for TestBench and SetupTestBenchMark.
* Add type hints for


* For matching python require version, change super(C, self) to super().
* Remove TestResultXmlReport.
* Remove `_ts_records_` from base.html when generating html report.
* Change logger.warn to logger.warning.
* Change log file basename format.


* Fix bug when register runner into container.
* Default enable TestResultShelveInterceptor instead of TestResultXmlInterceptor.


* Rename WithTestBenchMark to SetupTestBenchMark.
* Add type hints.
* Support WorkEvn in ngta.agent.
* Remove package: six.


* Should provide arguments when calling generate_test_from_obj recursively.
* Support with context on setup_testbench.
* Require coupling>=0.8.16.


* Rename with_testbencn to setup_testbench
* Only handle method startswith test in generate_test_from_obj.


* Rename `<pkg-loader>` to `<obj-loader>`.
* Add `with_testbench` marked on TestBench class, so don't need to config testbench in xml or yml config file.
* Support `--locate` and relative arguments in command-line.
* Move `fail-fast`, `fail-freeze`, `process-count`, `disable-defaults-listeners` to root `<config>` element.
* `<obj-loader>` support package, module, class, method, file and dir now.
* Remove `<dir-loader>`.
* Change repeat-foreach default value to False.


* Fix bug: Always retry connect rabbitmq when creating PublishWorker.
* Add third-party package 'watchdog' to support hot patch module.
* Add http request handler for '/api/rest/testcases'.
* Add http request handler for '/api/rest/testcases/(.+)'.
* Refactor code for get_hierarchy_by_module.
* Fix bug: can't publish testcase record to RabbitMQ.
* Fix bug: don't generate task_execute.log and task_publish.log.
* Fix bug: can't construct correct object when using !!python/object in agent.yml.


* TestCaseResultRecord's enable_mock don't updated by testcase.
* Rename prepare_sys_path to prepare_sys_path_for_work_dir in
* When generating html from xml, still need to add cases dir and lib dir into sys.path.
* Fix bug: specify the cases_dir as root lib dir when getting testhierarchy.


* Highlight current showed status and parameters in report.


* Update sample and README.


* Change parametrize default type from str to object.
* Support validate json format with json schema when agent loading tests.


* Merge log_sample.yml into agent_sample.yml.
* Rename SrvSetting -> AgentSetting, ResSetting -> BenchSetting.


* Update tests.
* Rename
* srv.exe to agent.exe
* srv.xml to agent.xml
* srvconf_sample.yml -> agent_sample.yml
* resconf_sample.yml -> bench_sample.yml
* logconf_sample.yml -> log_sample.yml


* Fix bug when rerun inherited from base class.
* Remove useless samples.
* name support callback function without arguments.
* Fix bug when unregister non-exists TestContext.
* Add more tests.


* Add
* Fix bug when rerun's scope include SETUP or TEARDOWN.


* Fix bug: always skip testcase when using skipif.
* Rename _get_unique_mark to _get_single_mark.


* Add ignore_inherited_methods to ignore inherit test method from base class.
* Add ignore_inherited_marks to ignore inherit marks from base class.
* Remove ignore-inherited from xml and yaml configuration.


* Add block checkpoint_images in base.html.
* Always display checkpoint error if it is not empty.
* Use relative log path in html report.


* Add checkpoint images display in report.html
* Change coupling require greater than v0.8.10.


* Support read cases_dir from .ngta, if not provided, default value is `${WORKDIR}/cases`.
* Change html id 'chart-row' -> 'overview-chart-row', 'chart-div' -> 'overview-chart-div' in report.html.


* Add extras attribute for checkpoint.
* Add images and extras for pass_, warn_, fail_ and all assert methods.


* Change pie chart center to `['88%', '50%']`.
* When draw timeline chart, the scatter points should use middle time.


* Fix bug of find_work_dir function on Python2.7 in Windows.
* Add timeline chart.


* Draw chart only when click "Show Chart".
* Change echarts js to echarts.common.min.js.
* Remove python_requires from


* Move code from init/bin/ into ``.
* Update README.
* Change chart display position.
* Fix bug of total statistics of overview table.


* Change print to logger.debug.
* Change multi-processes run simplify.
* Remove redis require from
* Change coupling require greater than v0.8.9.


* When run with multi process, change QueueTestConsumerProcess id index startswith 1.
* Add pie chart in test report.


* Remove useless import and functions.
* Add argument images for TestCase methods: add_checkpoint, pass_, fail_, warn_.
* use signature instead of inspect.getargspec when wrap assert.
* For testcase log, use as postfix.
* Fix bug when running with multiple processes.


* Fix bug: When process killed abnormally, the report can't be generated because the result.start_time and result.finish_time is None.
* Add ${WORKDIR}/bin/cases into sys.path when running init/bin/
* Add images from TestCase.CheckPoint
* Add TestResultJsonInterceptor.


* When call listener in testsuite and testrunner, don't handle failfast argument.


* Update report base.html


* Add `` for agent.
* Default add bin/cases dir into sys.path.
* Remove,, and from init dir.
* Add .ngta into init dir for anchor and config.


* Add logs into html report.


* Change `<extra>` not required of `<html>`.
* Rename some html block name.
* TestCase.CheckPoint can ben replaced now.
* Add main function to run testcase simply.
* Add jinja2 block for checkpoints and checkpoint.


* Fix mock import bug when running on Python2.7
* Default enable default listeners even there is no `<context>` configured.


* Add log-level and log-layout for root element `<config>`.
* Change coupling version requirement to v0.8.7


* Add README.rst
* Rename method json() to as_dict()
* Rename mocking to patch, mocking.object to patch


* Change publishing test runner state from dict to TestRunnerPublishingState.
* Remove __json__ from CheckPoint and TestCaseResultRecord.
* Remove jmespath.
* Remove ngta.log.
* Add current_context() for, it is alias of TestContextManager.current_context()
* Fix bug of runner.json schema.


* Update mock sample.
* Update patch and patch.object decorator.


* Add more sample.
* Fix bug for rest api testhierarchy.
* Fix rerun bug when there are no arguments.
* Remove duplicated tags for testcase.
* Add mocking and mocking.object decorator.
* Remove --failfast and --failfreeze from command-line arguments.
* Add attribute enable-mock for `<testcase>` element.


* Change agent config file from srvconf.xml to srvconf.yml.
* Update SvnRepository and GitRepository.
* Add column revision for testrunner table.
* Support 'is_def' when parse testcase parameters json data.
* Change repeat to repeat-number
* Add attribute repeat-foreach for `<cls-loader>`, `<tag-loader>`, `<pkg-loader>`.


* Fix bug: For yaml config, don't handle failfast and failfreeze in command-line
* Fix bug: Pycharm debug not work when provide relative path for TestProgram.
* Fix bug: Change expire_on_commit as False when using sqlalchemy sessionmaker.


* Add long_description from by using pypandoc.
* Remove README.rst
* Fix bug: Don't handle failfast and failfreeze in command-line


* Move pattern attribute into base class ObjLoaderConfig.
* Insert lib dir at index 0 of sys.path.
* Don't overwrite logconf.yml, resconf.xml and srvconf.xml if exists.
* Support <dir-loader> in config file.
* Fix bug: Don't handle extend for <xml-loader> and <yml-loader>
* Using JSON schema to validate yaml config file.


* Update
* Update LICENSE.
* Change --output-dir not required in command-line args.
* Don't init vcs-worker if there is not config in srvconf.xml.
* Add prefix 'task_' for all task logs.
* Move logging config into logconf.yml
* Fix bug: Testbench class of sample is incorrect.
* Fix bug: ExecuteWorker object has no attribute 'runners'
* Fix bug: Should get worker interval as float.


* Initial release