ESP32-Pico-V3 fails to upload/debug using esp-prog

I am trying to use esp-prog on a custom board using an ESP32-Pico-V3 SoC. Uploading fails with the following error:

Retrieving maximum program size .pio/build/esp32dev/firmware.elf
Checking size .pio/build/esp32dev/firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [====      ]  35.9% (used 117632 bytes from 327680 bytes)
Flash: [======    ]  58.4% (used 1836203 bytes from 3145728 bytes)
Configuring upload protocol...
AVAILABLE: esp-prog, espota, esptool, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa
CURRENT: upload_protocol = esp-prog
Uploading .pio/build/esp32dev/firmware.bin
Open On-Chip Debugger  v0.10.0-esp32-20190708 (2019-07-08-11:04)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
debug_level: 1
none separate
adapter speed: 20000 kHz
esp32 interrupt mask on
Error: cpu0: xtensa_write_memory (line 807): DSR (FFFFFFFF) indicates target still busy!
Error: cpu0: xtensa_write_memory (line 807): DSR (FFFFFFFF) indicates DIR instruction generated an exception!
Error: cpu0: xtensa_write_memory (line 807): DSR (FFFFFFFF) indicates DIR instruction generated an overrun!
Warn : esp32: Failed writing 4 bytes at address 0x3ff48000, data - 00, 20, 49, 9c, a6, 7e, 01, 88
Error: xtensa_write_uint32_list: error writing to 3ff48000
Warn : esp32_soc_reset xtensa_write_uint32_list (reg_value_pairs_pre) err=-4
in procedure 'program_esp32'

embedded:startup.tcl:480: Error: ** Unable to reset target **
in procedure 'program_esp32' 
in procedure 'program_error' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp32.cfg", line 119
at file "embedded:startup.tcl", line 480
Warn : Flash driver of esp32.flash does not support free_driver_priv()
Warn : Flash driver of irom does not support free_driver_priv()
Warn : Flash driver of drom does not support free_driver_priv()
*** [upload] Error 1

According to sprite from Espressif this indicates an outdated openocd version: “You possibly need to update your OpenOCD packages… the V3 ESP32 silicon has some small changes to the way JTAG works, and your OpenOCD doesn’t seem to take that into account.

I have tried manually updating the openocd version that comes with platformio, but that did not work at all.
Any ideas/suggestions?
thanks!

We really, really need a “solving ESP32 debugging problems” post. The steps to debugging this are scattered over at least 20 posts in this forum and most problems can be solved by this…

@valeros @ivankravets There is a surge of problems related to ESP32 debugging – likely due to the new ESP32 chip variants released by Espressif requiring a newer OpenOCD version. Any plans to update it to the latest version?

1 Like

Try the following troubleshooting steps in Cannot debug esp32 - #10 by maxgerhardt (that is, lower adapter speed, recheck all JTAG wiring, update tool-openocd-esp32 manually, use a different board_debug.openocd_board file in the platformio.ini, check flash voltage (3.3V vs 1.8V)).

“update openocd” issue: Update openocd · Issue #413 · platformio/platform-espressif32 · GitHub

Well, the wiring is 100% correct, if I try basically the same board but a revision that was still using the Pico-D4 debugging works fine out of the box. Just with the V3 silicon it refuses. I have not found thread related to V3 issues, so I opened a new thread, especially with the note I received from Sprite I found it worth creating a thread.

I have tried lower clock speed with no change of the symptom. I’ve also tried just replacing openocd with the version from july and now I get this error:

Retrieving maximum program size .pio/build/esp32dev/firmware.elf
Checking size .pio/build/esp32dev/firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [====      ]  35.9% (used 117632 bytes from 327680 bytes)
Flash: [======    ]  58.4% (used 1836203 bytes from 3145728 bytes)
Configuring upload protocol...
AVAILABLE: esp-prog, espota, esptool, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa
CURRENT: upload_protocol = esp-prog
Uploading .pio/build/esp32dev/firmware.bin
Open On-Chip Debugger  v0.10.0-esp32-20200709 (2020-07-09-08:54)
Licensed under GNU GPL v2
For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
debug_level: 1

adapter speed: 20000 kHz

WARNING: boards/esp-wroom-32.cfg is deprecated, and may be removed in a future release.
** Programming Started **
embedded:startup.tcl:449: Error: ** Programming Failed **
in procedure 'program_esp32' 
in procedure 'program_esp' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp32.cfg", line 64
in procedure 'program_error' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp_common.cfg", line 106
at file "embedded:startup.tcl", line 449
*** [upload] Error 1

edit: the result is the same regardless of wether board_debug.openocd_board = esp32-ethernet-kit-3.3v.cfg is present in platformio.ini or not

It’s still running at 20MHz, I think you have to change this in that file

(That also helped for a PICO-D4 according to an issue)

Since this issue seems to be actually new and unrelated to all other posts that was indeed the right thing to do.

Now the question here is if even the latest released esp32 openocd version from July 2020 is too old. Since I don’t have a Mac, I’m unable to compile the most recent version, and I also can’t seem to find officially released packages which are newer (or I have overlooked them).

Now to test this either someone can build for Mac or I’m building it for Linux and the thing has to be tested in a Linux VM with the USB debugger passed-through.

Yes because I reverted the change after it made no difference already 2 days ago, thus it is not reflected in this message. I did try and the error message did reflect the changes in adapter speed.

I’ll try to get it compiled for mac.

Ah, I just checked, the latest commit is from jul 6th and the release from jul 9th, so there is no point in compiling :expressionless:

Is the reason the latest openocd version fails maybe still related to a config issue?

Latest commits are from 9 days ago:

Ah I see my mistake, i was on the releases page and expected the current master branch when I clicked on code.
Trying to compile but I am presented with a bunch of compile errors :sob:

The folks at openocd say support for Eco3 silicon was added with v0.10.0-esp32-20191008.

They have also asked me to run OpenOCD with -d3 option to collect more verbose log. How would I be able to add this parameter using platformio?

I’ve run openocd manually via terminal from the platformio install to supply the folks at github with the log, and to my personal surprise (I did not check the log toroughly and posted it anyway) it worked fine.

So it seems the issue is not openocd. Any ideas?

Either see Redirecting... for using debug_tool = custom and debug_port = localhost:3333 etc for using a pre-opened openocd server, or modify this code locally on your machine

to say openocd_args = ["-d3"] (or any other debug level. This file should be in <home directory>/.platformio/platforms/esp32/builder/main.py.

Upload and stepping worked fine? Then there’s something with the timing. Running on -d3 adds a considerable amount of debug output and delay between individual executions of commands, which may influence things just in the right direction.

Seems familiar to my when you don’t use the --disable-werror option in the ./configure <options> command. But since it works on pre-compiled released versions, just a side note.

I only did an upload, assuming it would fail that’s why I posted the good log without checking :sweat_smile:

With your help I’ve managed to run openocd from within platformio with the d3 argument, and this is the output in the console: Uploading .pio/build/esp32dev/firmware.binOpen On-Chip Debugger v0.10.0-esp32 - Pastebin.com (too long to post, thus pastebin)

seems the problem has been identified: https://github.com/espressif/openocd-esp32/issues/125#issuecomment-716588856

Changing program_esp32 to program_esp in main.py at L374 and L380 fixes the issue with uploading.

It did not fix debugging though, I guess it is a similar if not the same issue though. Any pointers would be very appreciated!
Thank you for your fast help!

Edit: This is the output of the debug console:

Open On-Chip Debugger  v0.10.0-esp32-20200709 (2020-07-09-08:54)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
adapter speed: 20000 kHz

WARNING: boards/esp-wroom-32.cfg is deprecated, and may be removed in a future release.
Info : Configured 2 cores
Info : tcl server disabled
Info : telnet server disabled
Info : ftdi: if you experience problems at higher adapter clocks, try the command "ftdi_tdo_sample_edge falling"
Info : clock speed 20000 kHz
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : cpu0: Detected debug stubs @ 3ffd9834
Info : accepting 'gdb' connection from pipe
Error: No symbols for FreeRTOS
Info : Target halted. CPU0: PC=0x40127399 (active)
Info : Target halted. CPU1: PC=0x400F9DE0
Info : cpu0: Target halted, PC=0x40091856, debug_reason=00000001
Info : Flash mapping 0: 0x10020 -> 0x3f400020, 721 KB
Info : Flash mapping 1: 0xd0018 -> 0x400d0018, 966 KB
Info : cpu0: Target halted, PC=0x40091856, debug_reason=00000001
Info : Auto-detected flash bank 'esp32.flash' size 4096 KB
Info : Using flash bank 'esp32.flash' size 4096 KB
Info : cpu0: Target halted, PC=0x40091856, debug_reason=00000001
Info : Flash mapping 0: 0x10020 -> 0x3f400020, 721 KB
Info : Flash mapping 1: 0xd0018 -> 0x400d0018, 966 KB
Info : Using flash bank 'esp32.irom' size 968 KB
Info : cpu0: Target halted, PC=0x40091856, debug_reason=00000001
Info : Flash mapping 0: 0x10020 -> 0x3f400020, 721 KB
Info : Flash mapping 1: 0xd0018 -> 0x400d0018, 966 KB
Info : Using flash bank 'esp32.drom' size 724 KB
0x40127399 in spiffs_object_modify (fd=0x0, offset=0, data=0x3ffc84cc <Wire> "x\244G?", len=106) at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/spiffs/spiffs/src/spiffs_nucleus.c:1607
1607	/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/spiffs/spiffs/src/spiffs_nucleus.c: No such file or directory.
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : cpu0: Debug controller 0 was reset.
Info : cpu0: Core 0 was reset.
cpu0: Debug controller 0 was reset.
cpu0: Core 0 was reset.
Info : cpu0: Target halted, PC=0x500000CF, debug_reason=00000000
cpu0: Target halted, PC=0x500000CF, debug_reason=00000000
Info : esp32: Core 0 was reset.
esp32: Core 0 was reset.
Info : esp32: Debug controller 1 was reset.
Info : esp32: Core 1 was reset.
esp32: Debug controller 1 was reset.
esp32: Core 1 was reset.
Info : Target halted. CPU0: PC=0x40000400 (active)
Info : Target halted. CPU1: PC=0x40000400
Target halted. CPU0: PC=0x40000400 (active)
Target halted. CPU1: PC=0x40000400
** Programming Started **
** Programming Started **
embedded:startup.tcl:449: Error: ** Programming Failed **
in procedure 'program_esp32'
in procedure 'program_esp' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp32.cfg", line 64
in procedure 'program_error' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp_common.cfg", line 106
at file "embedded:startup.tcl", line 449
embedded:startup.tcl:449: Error: ** Programming Failed **
in procedure 'program_esp32'
in procedure 'program_esp' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp32.cfg", line 64
in procedure 'program_error' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp_common.cfg", line 106
at file "embedded:startup.tcl", line 449
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : cpu0: Debug controller 0 was reset.
Info : cpu0: Core 0 was reset.
cpu0: Debug controller 0 was reset.
cpu0: Core 0 was reset.
Info : cpu0: Target halted, PC=0x500000CF, debug_reason=00000000
cpu0: Target halted, PC=0x500000CF, debug_reason=00000000
Info : esp32: Core 0 was reset.
esp32: Core 0 was reset.
Info : esp32: Debug controller 1 was reset.
Info : esp32: Core 1 was reset.
esp32: Debug controller 1 was reset.
esp32: Core 1 was reset.
Info : Target halted. CPU0: PC=0x40000400 (active)
Info : Target halted. CPU1: PC=0x40000400
Target halted. CPU0: PC=0x40000400 (active)
Target halted. CPU1: PC=0x40000400
** Programming Started **
** Programming Started **
embedded:startup.tcl:449: Error: ** Programming Failed **
in procedure 'program_esp32'
in procedure 'program_esp' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp32.cfg", line 64
in procedure 'program_error' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp_common.cfg", line 106
at file "embedded:startup.tcl", line 449
embedded:startup.tcl:449: Error: ** Programming Failed **
in procedure 'program_esp32'
in procedure 'program_esp' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp32.cfg", line 64
in procedure 'program_error' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp_common.cfg", line 106
at file "embedded:startup.tcl", line 449
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : cpu0: Debug controller 0 was reset.
Info : cpu0: Core 0 was reset.
cpu0: Debug controller 0 was reset.
cpu0: Core 0 was reset.
Info : cpu0: Target halted, PC=0x500000CF, debug_reason=00000000
cpu0: Target halted, PC=0x500000CF, debug_reason=00000000
Info : esp32: Core 0 was reset.
esp32: Core 0 was reset.
Info : esp32: Debug controller 1 was reset.
Info : esp32: Core 1 was reset.
esp32: Debug controller 1 was reset.
esp32: Core 1 was reset.
Info : Target halted. CPU0: PC=0x40000400 (active)
Target halted. CPU0: PC=0x40000400 (active)
Info : Target halted. CPU1: PC=0x40000400
Target halted. CPU1: PC=0x40000400
** Programming Started **
** Programming Started **
embedded:startup.tcl:449: Error: ** Programming Failed **
in procedure 'program_esp32'
in procedure 'program_esp' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp32.cfg", line 64
in procedure 'program_error' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp_common.cfg", line 106
at file "embedded:startup.tcl", line 449
embedded:startup.tcl:449: Error: ** Programming Failed **
in procedure 'program_esp32'
in procedure 'program_esp' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp32.cfg", line 64
in procedure 'program_error' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp_common.cfg", line 106
at file "embedded:startup.tcl", line 449
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : cpu0: Debug controller 0 was reset.
Info : cpu0: Core 0 was reset.
cpu0: Debug controller 0 was reset.
cpu0: Core 0 was reset.
Info : cpu0: Target halted, PC=0x500000CF, debug_reason=00000000
cpu0: Target halted, PC=0x500000CF, debug_reason=00000000
Info : esp32: Core 0 was reset.
esp32: Core 0 was reset.
Info : esp32: Debug controller 1 was reset.
Info : esp32: Core 1 was reset.
esp32: Debug controller 1 was reset.
esp32: Core 1 was reset.
Info : Target halted. CPU0: PC=0x40000400 (active)
Info : Target halted. CPU1: PC=0x40000400
Target halted. CPU0: PC=0x40000400 (active)
Target halted. CPU1: PC=0x40000400
** Programming Started **
** Programming Started **
embedded:startup.tcl:449: Error: ** Programming Failed **
in procedure 'program_esp32'
in procedure 'program_esp' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp32.cfg", line 64
in procedure 'program_error' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp_common.cfg", line 106
at file "embedded:startup.tcl", line 449
embedded:startup.tcl:449: Error: ** Programming Failed **
in procedure 'program_esp32'
in procedure 'program_esp' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp32.cfg", line 64
in procedure 'program_error' called at file "/Users/wolfgang/.platformio/packages/tool-openocd-esp32/share/openocd/scripts/target/esp_common.cfg", line 106
at file "embedded:startup.tcl", line 449
Info : JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
JTAG tap: esp32.cpu0 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
JTAG tap: esp32.cpu1 tap/device found: 0x120034e5 (mfg: 0x272 (Tensilica), part: 0x2003, ver: 0x1)
Info : cpu0: Debug controller 0 was reset.
Info : cpu0: Core 0 was reset.
cpu0: Debug controller 0 was reset.
cpu0: Core 0 was reset.
Info : cpu0: Target halted, PC=0x500000CF, debug_reason=00000000
cpu0: Target halted, PC=0x500000CF, debug_reason=00000000
Info : esp32: Core 0 was reset.
esp32: Core 0 was reset.
Info : esp32: Debug controller 1 was reset.
Info : esp32: Core 1 was reset.
esp32: Debug controller 1 was reset.
esp32: Core 1 was reset.
Info : Target halted. CPU0: PC=0x40000400 (active)
Target halted. CPU0: PC=0x40000400 (active)
Info : Target halted. CPU1: PC=0x40000400
Target halted. CPU1: PC=0x40000400
Hardware assisted breakpoint 1 at 0x40117598: file /Users/wolfgang/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp, line 24.
PlatformIO: Initialization completed
PlatformIO: Resume the execution to `debug_init_break = thb app_main`
PlatformIO: More configuration options -> http://bit.ly/pio-debug
Note: automatically using hardware breakpoints for read-only addresses.
Warn : cpu0: No free slots to add HW breakpoint!
Info : cpu0: Target halted, PC=0x40091856, debug_reason=00000001
Info : cpu0: Detected debug stubs @ 3ffd9834
cpu0: Detected debug stubs @ 3ffd9834

It looks as if the command:

flash flash write_image erase .pio/build/esp32dev/firmware.bin 0x10000 verify

was used instead of:

flash write_image erase .pio/build/esp32dev/firmware.bin 0x10000 verify

And then the command file parsing fails (error -1404 is raised if OpenOCD cannot determine the image type based on the filename).

If so, the problem could be in PlatformIO or in one of the config files.

Well, if you read the issue on github and this thread it seems rather clear to me that the problem is with platformio.

If by any chance you find a quick fix I can apply to get debugging working I’d be incredibly thankful! Uploading works now, which was not a big issue because I could always upload via serial. Spent all day tracking this down to get debugging working. I am glad it’s not a hardware issue with my prototypes though!

What have you changed? Why does uploading suddenly work?

We’ve just updated openOCD to the latest version but you will need the latest PlatformIO Core Dev and special branch of ESP32 dev-platform. If this works for you, we merge it into develop branch when release PlatformIO Core 5.0.2.

How to try?

  1. Please open PlatformIO Core CLI in VSCode (terminal icon on the blue bottom bar) and type:
pio upgrade --dev
  1. Use this line in your platformio.ini:
[env:yourenv]
platform = https://github.com/platformio/platform-espressif32#feature/openocd-update

Does it work?