Problems debugging ESP32 using esp-prog on a Mac

Hello forum,

after extensive research, testing and reading posts here this is my last resort. Any help to fix my problems with debugging of my project using esp-prog from a Mac would be much appreciated. This is a long post because I’ve already spent more hours on this than I am willing to admit, tried many things and want to provide exhaustive information right off the bat.

I am using AZ-Delivery ESP-32 Dev Kit C V4 board together with ESP-Prog JTAG adapter. All software that I am running is latest and fully updated to date: VS Code 1.52.1, PIO Core 5.1.0, PIO Home 3.3.3, I’ve updated PlatformIO, libraries and platforms.

My OS is latest macOS Big Sur 11.1 and even though I previously experimented with manufacturer’s USB FTDI drivers I am currently running a fairly new clean install of macOS without those drivers installed.

I have installed libusb (brew install libusb).

I have been able to successfully run Debug using the example Blink project (Arduino flavor). Below is platformio.ini that I used and everything works fine. I am able to flash new firmware via JTAG by using “upload_protocol = esp-prog” option, run Debug, see CPU registers, variables, breakpoints work (both temporary and custom), serial monitor via the adapter - all of that works just fine. I think it’s a fair test that my JTAG adapter is working, that all wiring is correct and that nothing is fundamentally wrong with my setup. I am able to reproduce this successfully without much issues. It helps to stick to the workflow where I start Debug without any manual breakpoints, wait until CPU halts at the temporary breakpoint at setup() and then create my own breakpoint(s) (I read this on this forum). Both tasks “PIO Debug” and “PIO Debug (skip pre-Debug)” work for me.

; PlatformIO Project Configuration File
;
;   Build options: build flags, source filter, extra scripting
;   Upload options: custom port, speed and extra flags
;   Library options: dependencies, extra library storages
;
; Please visit documentation for the other options and examples
; http://docs.platformio.org/page/projectconf.html

[env:esp]
platform = espressif32
framework = arduino
board = az-delivery-devkit-v4
build_flags =
	-D LED_BUILTIN=2

monitor_speed = 115200
monitor_port = /dev/tty.usbserial-14201
monitor_filters = esp32_exception_decoder

upload_protocol = esp-prog

debug_tool = esp-prog
debug_init_break = tbreak setup

Now to my problems. Immediately after a successful test using the Blink app where everything worked I open the workspace with my moderately complex project that I coded earlier. The project builds, uploads to the ESP32 and passes verification. When I run serial monitor I see the serial output of my project, confirming it got uploaded over the previous Blink app. However, Debug looks like it started but it is not functional. Below is my project’s platformio.ini:

[env:esp]
platform = espressif32
board = az-delivery-devkit-v4
framework = arduino
board_build.partitions = partitions_custom.csv
lib_deps = 
	felis/USB-Host-Shield-20@^1.3.2
	lathoub/AppleMIDI@^2.1.0
	fortyseveneffects/MIDI Library@^5.0.2
	arduino-libraries/Ethernet@^2.0.0
	igorantolic/Ai Esp32 Rotary Encoder@^1.0
	blemasle/MCP23017@^2.0.0
	0xf6/TM74@^1.1.4
	luisllamasbinaburo/I2CScanner@^1.0.1
	lennarthennigs/Button2@^1.2.0

upload_protocol = esp-prog

monitor_port = /dev/tty.usbserial-14201
monitor_speed = 115200
monitor_filters = esp32_exception_decoder

debug_tool = esp-prog
debug_init_break = tbreak setup

This is the debug console output when I run “PIO Debug” for my project:

Processing esp (platform: espressif32; board: az-delivery-devkit-v4; framework: arduino)
--------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/az-delivery-devkit-v4.html
PLATFORM: Espressif 32 (3.0.0) > AZ-Delivery ESP-32 Dev Kit C V4
HARDWARE: ESP32 240MHz, 520KB RAM, 16MB Flash
DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
 - framework-arduinoespressif32 3.10004.210126 (1.0.4)
 - tool-esptoolpy 1.30000.201119 (3.0.0)
 - tool-openocd-esp32 2.1000.20201202 (10.0)
 - toolchain-xtensa32 2.50200.80 (5.2.0)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 35 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <USB-Host-Shield-20> 1.4.0
|   |-- <SPI> 1.0
|-- <AppleMIDI> 2.2.0
|   |-- <MIDI Library> 5.0.2
|-- <MIDI Library> 5.0.2
|-- <Ethernet> 2.0.0
|   |-- <SPI> 1.0
|-- <Ai Esp32 Rotary Encoder> 1.0.0
|-- <MCP23017> 2.0.0
|   |-- <Wire> 1.0.1
|-- <TM74> 1.1.4
|-- <I2CScanner> 1.0.1
|   |-- <Wire> 1.0.1
|-- <Button2> 1.5.3
|-- <ESP32 BLE Arduino> 1.0.1
|-- <WiFi> 1.0
|-- <ESPmDNS> 1.0
|   |-- <WiFi> 1.0
|-- <SPI> 1.0
|-- <Wire> 1.0.1
|-- <SPIFFS> 1.0
|   |-- <FS> 1.0
Building in debug mode
Retrieving maximum program size .pio/build/esp/firmware.elf
Checking size .pio/build/esp/firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=         ]  13.1% (used 69668 bytes from 532480 bytes)
Flash: [=====     ]  52.5% (used 1650805 bytes from 3145728 bytes)
========================= [SUCCESS] Took 5.56 seconds =========================
Reading symbols from /path/redacted/.pio/build/esp/firmware.elf...
done.
PlatformIO Unified Debugger -> http://bit.ly/pio-debug
PlatformIO: debug_tool = esp-prog
PlatformIO: Initializing remote target...
Open On-Chip Debugger  v0.10.0-esp32-20201202 (2020-12-02-17:38)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
adapter speed: 20000 kHz

WARNING: boards/esp-wroom-32.cfg is deprecated, and may be removed in a future release.
Info : FreeRTOS creation
adapter speed: 5000 kHz

Info : tcl server disabled
Info : telnet server disabled
Info : clock speed 5000 kHz
Error: JTAG scan chain interrogation failed: all ones
Error: Check JTAG interface, timings, target power, etc.
Error: Trying to use configured scan chain anyway...
Error: esp32.cpu0: IR capture error; saw 0x1f not 0x01
Warn : Bypassing JTAG setup events due to errors
Info : accepting 'gdb' connection from pipe
Warn : No symbols for FreeRTOS!
Error: Target not examined yet
Error executing event gdb-attach on target esp32.cpu0:

Error: Target not halted
Error: auto_probe failed
Error: Connect failed. Consider setting up a gdb-attach event for the target to prepare target for GDB connect, or use 'gdb_memory_map disable'.
Error: attempted 'gdb' connection rejected
Ignoring packet error, continuing...
warning: unrecognized item "timeout" in "qSupported" response
Ignoring packet error, continuing...

The CPU is not halted (serial monitor shows the code is running and outputs to serial), and Debug capabilities are therefore not available. After a while in this state I get “Failed to launch GDB: .pioinit:11: Error in sourced command file: Bogus trace status reply from target: timeout (from interpreter-exec console “source .pioinit”)” error message. I am working to get a screenshot of my situation at this point (for the record, I’ve seen an identical one before on this forum but can’t find it right now; unfortunately the solution did not seem to work for me).

I have tried to drop adapter speed to 5 MHz or even to 100 kHz to no avail.

After an unidentified number of attempts the JTAG adapter can no longer be found:

...
Building in debug mode
Retrieving maximum program size .pio/build/esp/firmware.elf
Checking size .pio/build/esp/firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=         ]  13.1% (used 69668 bytes from 532480 bytes)
Flash: [=====     ]  52.5% (used 1650805 bytes from 3145728 bytes)
========================= [SUCCESS] Took 4.24 seconds =========================
Reading symbols from /path/redacted/.pio/build/esp/firmware.elf...
done.
PlatformIO Unified Debugger -> http://bit.ly/pio-debug
PlatformIO: debug_tool = esp-prog
PlatformIO: Initializing remote target...
Open On-Chip Debugger  v0.10.0-esp32-20201202 (2020-12-02-17:38)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
adapter speed: 20000 kHz

WARNING: boards/esp-wroom-32.cfg is deprecated, and may be removed in a future release.
Info : FreeRTOS creation
adapter speed: 100 kHz

Info : tcl server disabled
Info : telnet server disabled
Error: libusb_claim_interface() failed with LIBUSB_ERROR_ACCESS
Error: unable to open ftdi device with vid 0403, pid 6010, description '*', serial '*' at bus location '*'
Error: no device found
Error: unable to open ftdi device with vid 0403, pid 6014, description '*', serial '*' at bus location '*'

Assertion failed: (jtag_trst == 0), function jtag_checks, file ../src/jtag/core.c, line 343.
.pioinit:11: Error in sourced command file:
Remote connection closed

I am completely at a loss trying to figure this one out.

If I understand it correctly that you were able to debug a simple Arduino blinky firmware but not your complex project although being on the exact same hardware (target board and apater), your project may be accessing GPIO pins and interfering with the JTAG communication.

See quirks.

You understand correctly, everything is the same and I did not even move or touch any HW or cabling between the test with Blinky and with my app. Even though I am aware of GPIO limitations of the ESP32 and avoid the use of bootstrapping pins I do have some pin definitions that include GPIO 12-14, but because that peripheral is not connected right now I forgot about them and considered them unused. When I think about it now what a mistake it seems. I think you may be on to something, I’ll remove/redefine them and report back.

I have removed all code that uses peripherals that I can miss for now and that previously referenced GPIO 12-14 to no avail. Removing all dependencies that became unnecessary made no difference. The remaining code does not use those pins, I checked the sources of dependencies (USB host shield, Button2, SPI, i2c drivers) to make sure they don’t default to those pins. I am back where I started.

One more update: I think I’ve found a golden path that seems to work. It goes as follows and any deviation results in a failure:

  • set ESP32 into bootloader mode (reset while holding down BOOT button)
  • compile and upload the project using PIO Upload task
  • reset ESP32 so that it runs the uploaded code
  • run PIO Debug (skip pre-Debug) task

I am in the process of adding removed code back and testing it to see where it breaks again to confirm the use of GPIOs 12-14 was indeed the culprit.

I’m happy to announce that the mystery has been solved. As @maxgerhardt pointed out I had defined GPIO 12-14 to be used in a periphery driver and forgot about it. Everything seems to work now and the conflict with JTAG pins explains a lot about the strange behavior I experienced, including the need to set my ESP32 to bootloader mode before flashing new firmware each time. That is no longer necessary now that the conflict is fixed.

Big thanks to @maxgerhardt for his hint.

1 Like