Esp32-s3 debugger fails to mount littlefs

ruprup · March 4, 2025, 9:58pm

Hi there,

I have a project which uses a littleFS partition and systematically fails to mount the partition when I run the debugger (but the partition mounts correctly the rest of the time)
It’s like if the partition cannot be accessed whenever the debugger is running.
Note that after a debug attempt the partition can mount again properly whithout re-flashing anything. So the issue only occurs during debugging…

Thanks in advance if you have any lead, this prevents me for a long time to debug a growing project and therefore a growing struggle …

I get the following serial output during a debug attempt:

E (4778) esp_littlefs: ./components/esp_littlefs/src/littlefs/lfs.c:1367:error: Corrupted dir pair at {0x0, 0x1}

E (4781) esp_littlefs: mount failed,  (-84)
E (4784) esp_littlefs: Failed to initialize LittleFS
E (4814) esp_littlefs: Failed to format filesystem
[ 15381][E][LittleFS.cpp:123] format(): Formatting LittleFS failed! Error: -1
[ 15392][E][LittleFS.cpp:98] begin(): Mounting LittleFS failed! Error: -1

This occurs when debugging this minimal reproducible example:

#include "FS.h"
#include <Arduino.h>
#include <LittleFS.h>

void setup() {
  Serial.begin(115200);
  delay(3000);
  if (!LittleFS.begin(true)) {
    Serial.println("LittleFS Mount Failed");
    return;
  }
  Serial.println("LittleFS Mount Success");
  Serial.print("Used filesystem bytes: ");
  Serial.println(LittleFS.usedBytes());
}

void loop() {}

With

[env:esp32-s3-devkitc-1]
platform = espressif32
board = esp32-s3-devkitc-1
build_type = debug
framework = arduino
board_build.filesystem = littlefs
board_build.partitions = partitions_custom.csv
monitor_speed = 115200
build_flags = -DARDUINO_USB_CDC_ON_BOOT=1

sivar2311 · March 5, 2025, 4:10am

To be able to reproduce this, please tell the platform version used and the content of your partitions_custom.csv.

ruprup · March 5, 2025, 8:11am

Thanks for looking at it,

Here you go:

Versions:
Pio core 6.1.17
framework-arduinoespressif32 @ 3.20017.0 (2.0.17)

partition file is:

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x5000,
otadata,  data, ota,     0xe000,  0x2000,
app0,     app,  ota_0,   0x10000, 0x200000,
app1,     app,  ota_1,   0x210000,0x200000,
spiffs,   data, spiffs,  0x410000,0x300000,

sivar2311 · March 5, 2025, 8:41am

This could be anyone from 6.8.0 up to 6.10.0. Which one is it exactly?
Please check the installed espressif32 platform version or specify the exact version in your platformio.ini

Also check this list for available arduino and espressif32-platform versions

sivar2311 · March 5, 2025, 9:07am

Not reproducible (platform = espressif32 @ 6.10.0.)

Normal output:

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x4bc
load:0x403c9700,len:0xbd8
load:0x403cc700,len:0x2a0c
entry 0x403c98d0
E (90) esp_core_dump_flash: No core dump partition found!
E (90) esp_core_dump_flash: No core dump partition found!
LittleFS Mount Success
Used filesystem bytes: 8192

Output while debugger is running:

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x15 (USB_UART_CHIP_RESET),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40378be4
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x4bc
load:0x403c9700,len:0xbd8
load:0x403cc700,len:0x2a0c
entry 0x403c98d0
E (90) esp_core_dump_flash: No core dump partition found!
E (90) esp_core_dump_flash: No core dump partition found!
LittleFS Mount Success
Used filesystem bytes: 8192

I’m using the ESP32-S3-DevKitC-1-N16R8 and for simplicity the UART-Port for logging:

[env:esp32-s3-devkitc-1]
platform = espressif32 @ 6.10.0
board = esp32-s3-devkitc-1
framework = arduino

board_build.arduino.memory_type = qio_opi
board_build.flash_mode = qio
board_build.psram_type = opi
board_upload.flash_size = 16MB
board_upload.maximum_size = 16777216
build_flags = 
  -DBOARD_HAS_PSRAM

board_build.filesystem = littlefs
board_build.partitions = partitions_custom.csv
debug_init_break = tbreak setup

build_type = debug

monitor_port = COM9
monitor_speed = 115200

Did you try a full clean or erase flash?

ruprup · March 5, 2025, 2:38pm

Platform version was : Espressif 32 (6.9.0) (I pasted the wrong line sorry)

Given the versions list you provided (very informative by the way), this version is very close. So I would not expect the platform version to be the source of the issue compared to your working attempt but let’s keep it on the radar.

I also indeed tried to completely clean the flash. No effect.

The only significant differences I could see with your attempt are:

I am running the debugger on the USB port (I can’t access the UART on this board).
My chip is an ESP32-S3-MINI-1-N8 (but I expect it to have the same config as the ESP32-S3-DevKitC-1-N8 which I am using as board definition)

But I can’t reproduce your success under the exact same conditions. So there is something else.
Did you set breakpoints to have the debugger actually interfering with the code ? or are you sure the debugger was really running ? because I have an additional line in the output that you do not have…
“pro cpu reset by JTAG” which I interpret as the debugger is running.
Below the output obtained with the N16R8 board I have at hand with your platformio.ini config and platfrom 6.10.0.

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x3 (RTC_SW_SYS_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x4037aa6e
pro cpu reset by JTAG
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x4bc
load:0x403c9700,len:0xbd8
load:0x403cc700,len:0x2a0c
entry 0x403c98d0
E (97) esp_core_dump_flash: No core dump partition found!
E (97) esp_core_dump_flash: No core dump partition found!
E (20846) esp_littlefs: ./components/esp_littlefs/src/littlefs/lfs.c:1367:error: Corrupted dir pair at {0x0, 0x1}
....

Sometimes I also get

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x3 (RTC_SW_SYS_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x4037aa6e
pro cpu reset by JTAG
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x4bc
load:0x403c9700,len:0xbd8
load:0x403cc700,len:0x2a0c
entry 0x403c98d0
E (97) esp_core_dump_flash: No core dump partition found!
E (97) esp_core_dump_flash: No core dump partition found!
Guru Meditation Error: Core  1 panic'ed (Unhandled debug exception). 
Debug exception reason: BREAK instr 
Setting breakpoint at 0x00000000 and returning...
Guru Meditation Error: Core  1 panic'ed (InstrFetchProhibited). Exception was unhandled.

Setting breakpoint at 0x40374400 and returning...

(Note I first do upload then F5 to debug)

sivar2311 · March 5, 2025, 5:59pm

The debugger here is running via native USB port, too! The only difference is the output of Serial.print statements. For this I use the UART port.

There should be no difference. At least no which would cause the issues you have.

I think this is the root cause! But I don’t know where this comes from.

Here is a video of my test: https://youtu.be/TNTZlv8jQhA
On the right you see Putty with a serial connection via UART to the DevKit.
Uploading and Debugging happens via the native USB port.

ruprup · March 6, 2025, 12:35am

It’s really curious we don’t endup with the same results while doing the exact same thing. (note that I am on macOS)

I see during your debugging attempt the code actually does not stop at the breakpoint you set, which might indicate something behaves different.
To get the debugger to stop as expected, I followed the instruction from the pio docs here
It is instructed to put a breakpoint in the underlying arduino looptask: " 2. WARNING! Please set a breakpoint at void loopTask(void *pvParameters) (line 13 in the screenshot below - this line can change between releases)"

Would you mind trying this to see if it stops at the breakpoint, and then still successfully mount the partition ?

sivar2311 · March 6, 2025, 4:39am

Sure! Here is the result: https://youtu.be/XKEFmy9oAts

ruprup · March 6, 2025, 11:11am

Thanks a lot for counter checking.

Turns out I could reproduce your mount success at last. This was with the exact same breakpoint setup as yours (one in the background app_main() and none in the main file).

However, as soon as I put a breakpoint anywhere in the main file, like in setup but even in the loop, then the partition fails to mount. Also tried to create a seperate freertos task to not put the breakpoint in the same task as the partition mounting, but same happens, no breakpoint ok, breakpoint causes a fail.
We are progressing

So either the debugger interferes with the partition mount… or this is specific to my debugger install…
Could you confirm one more time if adding an additional break point in the setup for eg causes a fail or not ?

sivar2311 · March 6, 2025, 12:03pm

There is an “invisible” breakpoint in setup() as in my platformio.ini is

debug_init_break = tbreak setup

Check the video at 0:41

ruprup · March 6, 2025, 1:54pm

Yes. And like you I confirm there is no issue when doing so.
The issue occurs when I put breakpoints within the functions, which is what you would do to debug a program
Maybe the fact the “tbreak setup” works but not breakpoints added throug vscode interface is a lead to the issue if this is confirmed.

sivar2311 · March 6, 2025, 8:41pm

You’re right! If I add a breakpoint to setup() or loop() I’ve got an error:

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x3 (RTC_SW_SYS_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x4037aa6e
pro cpu reset by JTAG
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3808,len:0x4bc
load:0x403c9700,len:0xbd8
load:0x403cc700,len:0x2a0c
entry 0x403c98d0
E (95) esp_core_dump_flash: No core dump partition found!
E (95) esp_core_dump_flash: No core dump partition found!
E (7067) esp_littlefs: ./components/esp_littlefs/src/littlefs/lfs.c:1367:error: Corrupted dir pair at {0x0, 0x1}

E (7070) esp_littlefs: mount failed,  (-84)
E (7073) esp_littlefs: Failed to initialize LittleFS
E (7155) esp_littlefs: Failed to format filesystem
[ 16866][E][LittleFS.cpp:123] format(): Formatting LittleFS failed! Error: -1
[ 16877][E][LittleFS.cpp:98] begin(): Mounting LittleFS failed! Error: -1
LittleFS Mount Failed

I have checked it with Espressif Arduino 3.1.3 and the error does not occur.
So it seems that there was a bug in the littlefs component of ESP-IDF that must have been fixed.
A possible solution for you could be an update to a newer Espressif Arduino 3.x.

ruprup · March 7, 2025, 12:33pm

Great ! thanks for confirming the issue.
Any idea whether it might come from platformio side or arduino core itself ?

Good that it works in a more recent release, but lots of people like me might have not migrated large projects from 2.x to 3.x yet (especially with no official support to 3.x)

So it might be interesting that I report it on the platformio git ?

sivar2311 · March 7, 2025, 12:38pm

Neither of them, but ESP-IDF!
It seems to have been fixed sometime between ESP-IDF 4.4.7 and 5.3.2.

Since the cause lies in ESP-IDF (and this has already been fixed in newer versions of ESP-IDF), this makes no sense - The bug has already been fixed.

Use pioarduino to get the latest Arudino 3.x and a more recent ESP-IDF version:

platform = https://github.com/pioarduino/platform-espressif32/releases/download/53.03.13/platform-espressif32.zip