Erase FLASH before upload in debug mode

niels · September 29, 2021, 4:13pm

I have noticed that when I do an upload in release mode the FLASH is erased:

.
.
Configuring flash size…
Auto-detected Flash size: 4MB
Flash will be erased from 0x00001000 to 0x00007fff…
Flash will be erased from 0x00008000 to 0x00008fff…
Flash will be erased from 0x0000d000 to 0x0000efff…
Flash will be erased from 0x00010000 to 0x0010afff…
.
.

Is it possible to do the same when uploading in debug mode, maybe inserting a ‘line’ in some debug script?

When I switch from release to debug mode I need to erase the FLASH manually for the debugger to work and I tend to forget that (very annoying).

maxgerhardt · September 29, 2021, 6:08pm

So upload_flags = --erase-all does not help (docs)?

>C:\Users\Max\.platformio\packages\tool-esptoolpy\esptool.py write_flash --help
optional arguments:
  -h, --help            show this help message and exit
  --erase-all, -e       Erase all regions of flash (not just write areas) before programming

niels · September 29, 2021, 6:22pm

@maxgerhardt Sounds like it could work, I just don’t know where to put it (I am not familiar with the PIO/VSCode integration.
I am using the ‘Start Debugging’ (F5) from the ‘Run’ Menu and my thought was that maybe it could be integrated into this command/shortcut? The ‘Erase Flash’ from the PIO menu is working fine in my case, I just have to remember to use it before I press F5.

[EDIT#1]
Are you saying that I should just put it into the platformio.ini (sounds too easy )?

[EDIT#2]
At a first glance it seems that it has solved my problem - thanks.

niels · September 30, 2021, 6:38pm

I was a bit too fast with the:

So upload_flags = --erase-all does not help

when reverting to the release mode I got this error:

usage: esptool [-h]
[–chip {auto,esp8266,esp32,esp32s2,esp32s3beta2,esp32s3beta3,esp32c3,esp32c6beta}]
[–port PORT] [–baud BAUD]
[–before {default_reset,usb_reset,no_reset,no_reset_no_sync}]
[–after {hard_reset,soft_reset,no_reset,no_reset_stub}]
[–no-stub] [–trace] [–override-vddsdio [{1.8V,1.9V,OFF}]]
[–connect-attempts CONNECT_ATTEMPTS]
{load_ram,dump_mem,read_mem,write_mem,write_flash,run,image_info,make_image,elf2image,read_mac,chip_id,flash_id,read_flash_status,write_flash_status,read_flash,verify_flash,erase_flash,erase_region,merge_bin,version,get_security_info}
…
esptool: error: unrecognized arguments: --erase_all
*** [upload] Error 2
============================================================================= [FAILED] Took 32.74 seconds =============================================================================
The terminal process “C:\Users\Niels.platformio\penv\Scripts\platformio.exe ‘run’, ‘–target’, ‘upload’” terminated with exit code: 1.

I seems that the ‘upload_flags’ does not recognize the ‘–erase-all’.
I then tried with ‘upload_flags = write_flash --erase-all’, but that just gave me this error:

usage: esptool [-h]
[–chip {auto,esp8266,esp32,esp32s2,esp32s3beta2,esp32s3beta3,esp32c3,esp32c6beta}]
[–port PORT] [–baud BAUD]
[–before {default_reset,usb_reset,no_reset,no_reset_no_sync}]
[–after {hard_reset,soft_reset,no_reset,no_reset_stub}]
[–no-stub] [–trace] [–override-vddsdio [{1.8V,1.9V,OFF}]]
[–connect-attempts CONNECT_ATTEMPTS]
{load_ram,dump_mem,read_mem,write_mem,write_flash,run,image_info,make_image,elf2image,read_mac,chip_id,flash_id,read_flash_status,write_flash_status,read_flash,verify_flash,erase_flash,erase_region,merge_bin,version,get_security_info}
…
esptool: error: argument operation: invalid choice: ‘write_flash --erase-all’ (choose from ‘load_ram’, ‘dump_mem’, ‘read_mem’, ‘write_mem’, ‘write_flash’, ‘run’, ‘image_info’, ‘make_image’, ‘elf2image’, ‘read_mac’, ‘chip_id’, ‘flash_id’, ‘read_flash_status’, ‘write_flash_status’, ‘read_flash’, ‘verify_flash’, ‘erase_flash’, ‘erase_region’, ‘merge_bin’, ‘version’, ‘get_security_info’)
*** [upload] Error 2

So I am back to square one.

maxgerhardt · September 30, 2021, 6:54pm

You’re right, the problem is that it injects the flag way too early. “Verbose Upload” shows

“c:\users\max\appdata\local\programs\python\python38\python.exe” “C:\Users\Max.platformio\packages\tool-esptoolpy@1.30000.201119\esptool.py” --erase-all --before default_reset --after hard_reset --chip esp8266 --port “COM4” --baud 115200 write_flash 0x0 .pio\build\nodemcuv2\firmware.bin

but the --erase-all must come after write_flash.

Which means the only way I can see it work is by duplicating the original upload logic using an extra_script that injects the flag in the right position.

If I do

[env:nodemcuv2]
platform = espressif8266
board = nodemcuv2
framework = arduino
upload_protocol = custom
extra_scripts = erase_before_upload.py

With the erase_before_upload.py file being

from SCons.Script import (COMMAND_LINE_TARGETS, AlwaysBuild,
                          Builder, Default, DefaultEnvironment)
from os.path import join
Import("env")

platform = env.PioPlatform()

env.AutodetectUploadPort()


def get_esptoolpy_reset_flags(resetmethod):
    # no dtr, no_sync
    resets = ("no_reset_no_sync", "soft_reset")
    if resetmethod == "nodemcu":
        # dtr
        resets = ("default_reset", "hard_reset")
    elif resetmethod == "ck":
        # no dtr
        resets = ("no_reset", "soft_reset")

    return ["--before", resets[0], "--after", resets[1]]
    
env.Replace(
    UPLOADER=join(
        platform.get_package_dir("tool-esptoolpy") or "", "esptool.py"),
    UPLOADERFLAGS=[
        "--chip", "esp8266",
        "--port", '"$UPLOAD_PORT"',
        "--baud", "$UPLOAD_SPEED",
        "write_flash",
        "--erase-all"
    ],
    UPLOADCMD='"$PYTHONEXE" "$UPLOADER" $UPLOADERFLAGS 0x0 $SOURCE'
)
for image in env.get("FLASH_EXTRA_IMAGES", []):
    env.Append(UPLOADERFLAGS=[image[0], env.subst(image[1])])

if "uploadfs" in COMMAND_LINE_TARGETS:
    env.Replace(
        UPLOADERFLAGS=[
            "--chip", "esp8266",
            "--port", '"$UPLOAD_PORT"',
            "--baud", "$UPLOAD_SPEED",
            "write_flash",
            "--erase-all",
            "$FS_START"
        ],
        UPLOADCMD='"$PYTHONEXE" "$UPLOADER" $UPLOADERFLAGS $SOURCE',
    )

env.Prepend(
    UPLOADERFLAGS=get_esptoolpy_reset_flags(env.subst("$UPLOAD_RESETMETHOD"))
)

upload_actions = [
    env.VerboseAction(env.AutodetectUploadPort,
                      "Looking for upload port..."),
    env.VerboseAction("$UPLOADCMD", "Uploading $SOURCE")
]

I get as a verbose upload

"c:\users\max\appdata\local\programs\python\python38\python.exe" "C:\Users\Max\.platformio\packages\tool-esptoolpy@1.30000.201119\esptool.py" --before default_reset --after hard_reset --chip esp8266 --port "COM4" --baud 115200 write_flash --erase-all 0x0 .pio\build\nodemcuv2\firmware.bin
esptool.py v3.0
Serial port COM4
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 60:01:94:0d:d6:c2
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Erasing flash (this may take a while)...
Chip erase completed successfully in 9.0s
Compressed 264240 bytes to 194515...
Writing at 0x00000000... (8 %)
Writing at 0x00004000... (16 %)
Writing at 0x00008000... (25 %)
Writing at 0x0000c000... (33 %)
Writing at 0x00010000... (41 %)
Writing at 0x00014000... (50 %)
Writing at 0x00018000... (58 %)
Writing at 0x0001c000... (66 %)
Writing at 0x00020000... (75 %)
Writing at 0x00024000... (83 %)
Writing at 0x00028000... (91 %)
Writing at 0x0002c000... (100 %)
Wrote 264240 bytes (194515 compressed) at 0x00000000 in 17.9 seconds (effective 118.3 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
============================================= [SUCCESS] Took 32.63 seconds =============================================

looks like it’s working.

niels · September 30, 2021, 8:33pm

Wow, you are way ahead of me.

Is it correctly understod that you have copied parts of the main.py for the ESP8266 to your new ‘erase_before_upload.py’ script? And in which folder did you place this new script?

Looking at the ESP32 main.py (which I use) it seems somewhat different than the ESP8266 main.py. So what part(s) will I need to copy from this ESP32 script? (I guess it shows that I am not familiar with the VSCode IF and the scripting in PIO).

maxgerhardt · September 30, 2021, 8:35pm

The path is relative to the project’s root folder (same level at which the platformio.ini lives).

Woops, I did not read the tags of the topic correctly and assumed ESP8266. But actually there should a much shorter and simpler way of doing things – let me try.

maxgerhardt · September 30, 2021, 8:46pm

Much much easier, and works for both ESP8266 and ESP32: Just dynamically manipulate env["UPLOADERFLAGS"] last-minute so that the --erase-all is injected after write_flash.

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino
extra_scripts = erase_before_upload.py

[env:nodemcuv2]
platform = espressif8266
board = nodemcuv2
framework = arduino
extra_scripts = erase_before_upload.py

With erase_before_upload.py in the root project folder

Import("env")

old_uploaderflags = env["UPLOADERFLAGS"]
#print("Old uploaderflags: " + str(old_uploaderflags))
    
index_write_flash = old_uploaderflags.index("write_flash")
if index_write_flash != -1: 
    new_uploaderflags = old_uploaderflags[::]
    new_uploaderflags.insert(index_write_flash + 1, "--erase-all")
    #print("Replaced with: " + str(new_uploaderflags))
    env.Replace(UPLOADERFLAGS=new_uploaderflags)

Note that you do also have an explicit “Erase Flash” button (uses this project task)

grafik

So if you use that at the correct time there shouldn’t be a need to do an --erase-all at every upload actually.