<span id="english"> </span>
New: image recognition algorithm mstpl
We have added an improved version of the template matching recognition algorithm, which can greatly increase the success rate of the original template matching and better adapt to scenes with changing resolutions.
Update Airtest to version 1.2.0 or higher. If you are using AirtestIDE, please also update to version 1.2.10 or higher. The latest version has set this algorithm as the preferred algorithm.
Please refer to the following code example:
python
from airtest.core.api import *
touch(Template("search.png", scale_max=800, scale_step=0.005))
When the Template image object is initialized, the parameters related to image recognition are as follows:
- `threshold`: used to set the screening threshold of image recognition, the default is 0.8, so the recognition effect is required to be greater than 0.8 before the recognition is considered successful
- `rgb`: Whether to perform color recognition, the default is False, which means that airtest will process the picture as a grayscale picture by default, so if you need to add color three-channel recognition, you need to set it to True
- Newly added `scale_max`: new algorithm special parameter, used to adjust the maximum range of matching, **If the target UI to be searched is small, you can increase this value** appropriately, the default value is 800, the recommended value is 740, 800 , 1000 Value range [700, 2000]
- Newly added `scale_step`: a new algorithm dedicated parameter, used to control the search scale step size, **If the target UI to be found is small, you can appropriately reduce this value**, the default value is 0.005, the recommended value is 0.02, 0.005 , 0.001 Value range [0.001, 0.1]
If you encounter problems or abnormal errors during image recognition, please take a screenshot of the current interface and submit an issue together with the pictures used in the script.
Interface changes: No need to set parameters when connecting to a mobile phone
In Airtest 1.2.0 and above, we no longer want everyone to manually specify the connection parameters when connecting to the phone, including `cap_method`, `touch_method`, and `ori_method`.
Therefore, we have made some changes. When initializing the phone, if there is a problem, Airtest will automatically try to be compatible and select a backup plan to connect until it succeeds.
The important changes that need attention are as follows:
- **Currently Android12 is not yet well supported**, the screen effect is very poor
- Added `Android.screen_proxy`, which is similar to the previous touch_proxy and uses it as the call entry of the screenshot method.
- When initializing the Android() object, you can **no longer force to specify `cap_method="MINICAP"` or `"JAVACAP"`**, `screen_proxy` will automatically initialize in order until the screenshot method that can be successfully initialized is selected .
- For example, when connecting to the simulator before, you need to check `use javacap` on the IDE, or specify `cap_method=JAVACAP` when the script is running. Now it is not necessary to specify it, and it can be initialized successfully.
- Deleted the parameter `cap_method=MINICAP_STREAM`, please **do not pass the value of `MINICAP_STREAM` again**
- The priority order of initialization is `custom screenshot method` > `MINICAP` > `JAVACAP` > `ADBCAP`. If ADBCAP is called, a warning will be printed to tell the user that the screenshot efficiency is very low at this time.
- Keep some of the old call entry points, which is convenient to be compatible with the old code
- For example, `dev.minicap` will return the `dev.screen_proxy` object, and the `dev.minicap.get_frame_from_stream()` interface can still be used
- For example, `dev.cap_method` will return the name of the currently used screenshot method (such as `"MINICAP"`), and you can also use `dev.cap_method = "JAVACAP"` to force the specified screenshot method
- When acquiring pictures, there is no need to perform a lot of type judgment and then call the corresponding screenshot method
- Use `dev.screen_proxy.get_frame_from_stream()` directly
- That is, the previous code can be discarded:
-
if dev.cap_method == CAP_METHOD.MINICAP:
dev.minicap.get_frame_from_stream()
elif dev.cap_method == xxx:
...
- If you want to implement a new custom screenshot method: In `airtest.core.android.cap_methods.base_cap`, the base class `BaseCap`, which implements all screenshot methods, can be inherited and implement the interface `get_frame_from_stream`. A custom screenshot method can be added, and it can be used by registering it in ScreenProxy
- python
from airtest.core.android.cap_methods.base_cap import BaseCap
class TestCap(BaseCap):
def get_frame_from_stream(self):
return b"frame"
ScreenProxy.register_method("TESTCPY", TestCap)
The default priority is initialized to the custom TestCap
cap = ScreenProxy.auto_setup(dev.adb)
Other bug fixes
- `nbsp` added an `auto_kill` parameter, which can automatically kill the socket when the phone is disconnected to avoid memory leaks
- Solved several warnings that did not completely close the io stream at the end
- Compatible with / at the end of the script path passed in when the script is running
- The `SIFT` algorithm patent has been lifted, so the SIFT algorithm has been re-added as the default algorithm, but it is limited by the version number of OpenCV. Not all version numbers can be used directly https://github.com/AirtestProject/Airtest/commit/c8e9bba038b3bb1b5201bb336b5f99a1f5f7c71a
- Fixed an issue where the display_info size obtained in landscape mode was reversed on some new models of mobile phones. This issue would cause the screen to display incorrectly in landscape mode on the IDE https://github.com/AirtestProject/Airtest/pull/913/commits/9bcccdf06485b0121df2adf504f95693e9af3825
- Many models have been tested to fix this problem (such as vivo iQOO Neo5, Huawei Mate40Pro, etc.)
- If there are still models with the wrong display when the screen is rotated horizontally and vertically, please tell us the specific model