Pyocd

_As a reminder, support for Python 2 is deprecated and planned to be dropped from an upcoming release. If this is a problem for you moving forward, please raise your concerns in the [Dropping Python 2 support discussion](https://github.com/pyocd/pyOCD/discussions/1055)._

Additions

- SWO: When enabled with the new `swv_raw_enabled` option, the raw SWO data is served over TCP port 3443 by default. The port is changeable with `swv_raw_port` option. (Thanks patrislav1)
- CoreSight: Many ID table additions and renames, including adding A-profile IDs so meaningful information will be printed when connecting to an A+M multicore device.
- Probe server: Added `--allow-remote` argument to enable remote access to the probe server, as it now defaults to serve local-only, like the other TCP servers.
- Commander: `reset` command accepts optional reset type.
- Commander: `show aps` command to list DAP Access Ports.
- Architectural DAP reconnection support. `DebugPort.connect()` can be called multiple times and it will do the right thing.
- MEM-AP: Support for 64-256 bit transfers when the AP has the Large Data extension. Mostly only applies to A-profile and special devices.
- API: Added 64-bit memory transfers support to the memory interface API.
- Add a [Code of Conduct](CODE_OF_CONDUCT.md), adapted from the [Contributor Covenant](https://www.contributor-covenant.org/).

Boards and targets

- Add Arm Musca-S1 target and board. (Thanks mathias-arm)
- Add WIZwiki_W7500P board ID. (Thanks ArthurRichard)
- Add NXP i.MX RT1170 target and board. (Thanks jhqian)
- Fix NXP i.MX RT1015 memory map issue. (Thanks Kris-b50122)
- Updated NXP LPC5500 series with rev 1B debug unlock sequence. (Thanks ttt-t3r)
- Add family regex for NXP LPC5500 series, so the unlock logic will be applied to LPC5500 CMSIS-Packs.
- Correct NXP LPC5500 series flash erase sector size. Even though the device supports 512-byte erase sectors, the flash algo will always erase 32 kB at a time.
- For the NXP Kinetis family, ensure AP0 is a MEM-AP before attempting to use it to test device connectivity. (Thanks Hoohaha)
- Add STMicro NUCLEO-L432KC board. (Thanks trunet)
- Correct STMicro STM32F429xG flash size. (Thanks LONGZR007)
- Correct STMicro STM32F412xE flash size. (Thanks newbrain)

Changes and fixes

- CMSIS-DAP: Flush command queue before sending SWO commands.
- CMSIS-DAP: Improved `swo_read()` so it only locks and flushes when using the `DAP_SWO_Data` command instead of a dedicated SWO EP (which is supported by CMSIS-DAPv2).
- MEM-AP: Restructured `MEM_AP.init()` to use a single read and restore of the CSW register.
- CMSIS-DAP: Truncate unused bytes of a command buffer, so CMSIS-DAPv2 bulk packets only contain the command and no padding and prevent sending zero length packets to signal the end of a transfer. (Thanks kkitayam)
- Commander: Fixed selected core for gdbserver monitor commands to match the core associated with the gdbserver, and not have multiple gdbservers interfere with one another if the selected core is changed.
- Commander: Fixed "no-init" mode.
- Commander: Fix the `probeserver` command by wrapping the `DebugProbe` instance in the shared probe proxy.
- Commander: Fix `wreg` on peripheral registers.
- Always use the `DebugPort` object to perform resets, except for certain cases when the `DebugPort` doesn't exist yet (mostly "no-init" mode of Commander), so that pre- and post-reset notifications are sent.
- DP: Rename `DebugPort.init()` to `connect()`.
- AP: Property and method for detecting whether the AP is enabled.
- CoreSight discovery: Gracefully report failures to create CoreSight components.
- CoreSight discovery: Add `adi.v5.max_invalid_ap_count` session option.
- CoreSight discovery: fix a problematic error log message.
- FreeRTOS: Fix r12 offset for FPU-enabled normal frame.
- Numerous small fixes and code improvements to resolve issues identified by [LGTM](https://lgtm.com/projects/g/pyocd/pyOCD/?mode=list).
- Add `scripts/generate_flash_algo.py` tool, copied from the [FlashAlgo](/pyocd/FlashAlgo) project and simplified.

Documentation

- Some improvements to main readme.
- Terminology additions.
- Improve `adding_new_targets.md`.

Test

- Renamed `gdb_script.py` to `gdb_test_script.py`.
- Moved functional test output files to `test/output/` directory.
- Add `ElfReaderContext` test to `debug_context_test.py`.
- Add `commander_test.py`.
- Hex-encode XML-incompatible characters in test result XML.
- Fixed semihosting unit test issues.

Fixes

- Semihosting: fix Python 3 string/bytes issues and unit test problems.
- pyocd-gdbserver: fix regression preventing the server from starting.
- CMSIS-DAP: fix Atmel-ICE compatibility issue. (Thanks patrislav1)
- CMSIS-Packs: identify DFP memory regions by address space in addition to names. (Thanks DrTobe)
- CMSIS-DAP: update list of known CMSIS-DAP probes.
- J-Link: fix several issues that appeared when another process has a J-Link open.

Boards and targets

- Updated the flash algorithm for NXP i.MX RT1021xxxxx. (Thanks Hoohaha.)


Fixes

- Commander: Fix `quit`/`exit` command. (Thanks Hoohaha.)
- Commander: Removed (recently broken) support for using core register names as commands that print the register's value.
- Commander: Output from system commands is captured and written to the command context output. This only visibly affects running system commands using gdb's `monitor` command.
- Commander: Command parsing more closely matches shell command parsing. In particular, quotes are honoured within words, e.g. `hello"my"friend` is parsed as `hellomyfriend`. Backslash-escapes are also supported, but within quotes only quote and backslash characters can be escaped.
- CortexM: Fix early exit of range step loop by step hook. This appeared as the inability to interrupt a statement step over an infinite loop in gdb in certain cases (when gdb used a range step command).
- GDBServer: no longer send an unexpected response to the `k` (kill) command.

Fixes

- Resolved a regression where passing an ELF to Commander with the `--elf` argument caused an exception. 969

_**Important note:** Support for Python 2 is deprecated and planned to be dropped from an upcoming release. Existing releases of pyocd will, of course, continue to work with Python 2. If this is a major issue for you moving forward, please create a [new issue](https://github.com/mbedmicro/pyOCD/issues/new/choose) describing your concerns._

Additions

- Replaced the gdbserver remote monitor commands with the command set from the commander subcommand. This is the first step towards providing a full command mini-language integrated throughout pyocd.
- Added support to pyocd for RTOS and debug probe plugins. Third party packages on PyPI can now add features to pyocd.
- Debug probes can be shared over a TCP connection and accessed remotely. Documentation is provided in [here](https://github.com/mbedmicro/pyOCD/blob/master/docs/remote_probe_access.md).
- The gdbserver subcommand adds a command line argument `--probe-server` to cause the debug probe server to be started simultaneous with the gdbserver. For allows, for instance, to connect a pyocd commander to an already-running gdbserver.
- Some new core registers are exposed to the Python API and gdb: the NS versions of MSP and PSP, MSPLIM and PSPLIM for S and NS, and VPR.
- Command history file support, so the history is saved between runs of pyocd commander.
- By default the history is stored in `~/.pyocd_history`. The `PYOCD_HISTORY` environment variable will override this if set.
- The default history length is 1000 entries. The `PYOCD_HISTORY_LENGTH` environment variable or `commander.history_length` option allow you to change this. Setting either one to -1 will enable an infinite history (but beware history file size).
- The Arm SDC-600 Secure Debug Channel IP is supported in the internal Python API.
- The v8.1-M MVE extension is detected.

Boards and targets

- Added NXP MIMXRT1024-EVK, MIMXRT1060-EVK, and MIMXRT1064-EVK boards and i.MX RT1024, i.MX RT1060, and i.MX RT1064 targets (`mimxrt1024`, `mimxrt1060`, and `mimxrt1064`). (Thanks Hoohaha)
- Added new HDSC targets HC32F4A0xG, HC32F4A0xI and HC32M423 of HDSC MCU. (Thanks lennvn)
- Added OTP programming algorithm for the HDSC HC32F4A0. (Thanks lennvn)
- Added Nuvoton NuMaker-PFM-M487 board and M487JIDAE target, `m487jidae`. (Thanks ychsu-tf)
- Many fixes for NXP i.MX RT series devices related to reset and halt, as well as flash algorithms. (Thanks Hoohaha)
- Fixed HDSC HC32x120 flash algorithms. (Thanks lennvn)

Changes

- Command changes
- Prevent accessing core registers unless core is halted. An error is printed.
- Print core security state in status command, if the core has more than one security state.
- `reg` command accepts `all` or the name of a register group in addition to register names.
- `where` command prints the offset from the base address of the symbol.
- `show register-groups` prints a list of the named groups of core registers.
- Any prefixes of a command is accepted as long as it is unique. For example, "sym" for "symbol".
- New `probeserver` command to control the debug probe server.
- Vector catch supports SecureFault.
- A new `reset.halt_timeout` session option controls the timeout waiting for the core to recover from reset and halt.
- The standard Cortex-M system address space is used as the memory map if the target does not provide its own memory map (e.g. the `cortex_m` default target).
- The full name of a detected FPU is logged. These are the names such as "FPv4-SP-D16-M" or "FPv5-D16-M".
- An error will be reported if the config file (e.g., `pyocd.yaml`) does not have a yaml dictionary at its root. This helps users identify the cause of an otherwise obscure error message.
- Two session options are available too control RTOS awareness in the gdbserver. `rtos.enable` can be used to completely disable RTOS support, which is enabled by default. `rtos.name` can be set to the name of an RTOS plugin in order to limit RTOS awareness to that one RTOS (which can make startup faster).
- The `cache.read_code_from_elf` session option controls whether pyocd will read code from an ELF that is provided to it rather than read from the target (an optimisation that is enabled by default when an ELF is provided).
- The `cmsis_dap.deferred_transfers` session options controls whether the CMSIS-DAP layer will used deferred transfers. There is generally no reason to disable this, especially since it greatly affects performance, but it may help is special cases.
- The CMSIS-DAP pyusb/libusb backend will report a warning if pyusb fails to load libusb.
- The `cpu.step.instruction.timeout` can be used to enable a timeout when performing instruction stepping. By default the timeout is disabled.
- Handle the case where GDB sends an interrupt (Control-C) during a step or range step command.

Fixes

- Major fixes for ITM, SWO, and SWV support. (Thanks oxidecomputer)
- Another fix for CMSIS-Pack flash region creation. (Thanks menshiyu)
- USB HID detach kernel driver fix for FreeBSD. (Thanks cederom)
- Fixed case of multiple Cypress target types in the board ID. This did not prevent the targets from working, but caused incorrect data to be reported via the JSON boards list query. (Thanks mcgordonite)
- Corrected detection of the Security extension on v8.1-M cores such as the Cortex-M55.
- Fixed the name of the `scan_all_aps` session option in several places.
- Fixed hex dump generation on Python 2 (recent regression).

Documentation

- Added [command reference](https://github.com/mbedmicro/pyOCD/blob/master/docs/command_reference.md).
- Added [remote probe client/server documentation](https://github.com/mbedmicro/pyOCD/blob/master/docs/remote_probe_access.md).
- Fixed download address of the GNU Arm Embedded toolchain. (Thanks LONGZR007)
- A few smaller docs improvements.

Python API

- Major refactoring of command system used for the commander subcommand and (now) gdbserver monitor commands. See 955 for details.
- Major refactoring of the code for managing core registers. See 933 for details.
- Cores have an `extensions` property that is a list of enums from `pyocd.coresight.core_ids.CortexMExtensions`.
- Cores now have a `core_registers` property which is a `pyocd.core.core_registers.CoreRegisterIndex` object. This object gives you multiple ways to list and find core registers, such as by name, index, or group.
- Refactored `CoreSightTarget` so there is an `SoCTarget` superclass that implements the architecture-independent SoC-level features. Moved `CoreSightTarget` under `pyocd.coresight`.
- Replaced the `MemoryAccessError` exception only raised by the memory cache with `TransferFaultError`.
- The `Session` class now tracks instances of the gdbservers and probeserver.
- Added an optional hook callback parameter to the `Target.step()` API.

Tests

- Moved test binaries under `test/data`.
- Functional tests check for a `skip_test` session option. This would be set in per-probe options in the config file if you need to skip a probe during testing.
- `automated_test.py` gains `--list-tests`, `--include-tests`, and `--exclude-tests` command line arguments.
- Many functional test fixes and improvements.
- New unit test for fault and flash failure exceptions.
- Added a new functional test for commands.
- Removed the old and entirely unused `test_pyocd_tool.sh` script.
- Functional test for range step.

Fixes

- Fixed a regression in flash memory region creation from CMSIS-Packs that could result in an `AssertionError` being raised.