ESP32 SPIFFS .... hard hard hard times

All this begun just after trying to save data to SPIFFS with VSCode.

I don’t see how that relates to anything in this topic.

If I try to read back the partition table after having upload a normal firmware to the application, it works just fine.

>pio run -t upload -v 
[..]
"c:\users\max\appdata\local\programs\python\python38\python.exe" "C:\Users\Max\.platformio\packages\tool-esptoolpy\esptool.py" --chip esp32 --port "COM4" --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0x1000 C:\Users\Max\.platformio\packages\framework-arduinoespressif32\tools\sdk\bin\bootloader_dio_40m.bin 0x8000 C:\Users\Max\temp\esp32_blink_regr\.pio\build\esp32dev\partitions.bin 0xe000 C:\Users\Max\.platformio\packages\framework-arduinoespressif32\tools\partitions\boot_app0.bin 0x10000 .pio\build\esp32dev\firmware.bin
esptool.py v3.1
Serial port COM4

Reading out the flash

C:\Users\Max\temp\esp32_blink_regr>python C:\Users\Max\.platformio\packages\tool-esptoolpy\esptool.py read_flash 0x8000 0xc00 ptable.img
esptool.py v3.1
Found 2 serial ports
Serial port COM4
Connecting........_
Detecting chip type... ESP32
Chip is ESP32-D0WDQ6 (revision 0)
Features: WiFi, BT, Dual Core, Coding Scheme None
Crystal is 40MHz
MAC: 24:0a:c4:04:26:38
Uploading stub...
Running stub...
Stub running...
3072 (100 %)
3072 (100 %)
Read 3072 bytes at 0x8000 in 0.3 seconds (81.7 kbit/s)...
Hard resetting via RTS pin...

Hexdumping the content to verify that the magic bytes aa 50 are there

C:\Users\Max\temp\esp32_blink_regr>hexdump -C ptable.img
00000000  aa 50 01 02 00 90 00 00  00 50 00 00 6e 76 73 00  |ªP.......P..nvs.|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000020  aa 50 01 00 00 e0 00 00  00 20 00 00 6f 74 61 64  |ªP...à... ..otad|
00000030  61 74 61 00 00 00 00 00  00 00 00 00 00 00 00 00  |ata.............|
00000040  aa 50 00 10 00 00 01 00  00 00 14 00 61 70 70 30  |ªP..........app0|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000060  aa 50 00 11 00 00 15 00  00 00 14 00 61 70 70 31  |ªP..........app1|
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000080  aa 50 01 82 00 00 29 00  00 00 17 00 73 70 69 66  |ªP.‚..).....spif|
00000090  66 73 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |fs..............|
000000a0  eb eb ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |ëëÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
000000b0  28 f1 4c 09 45 01 77 60  a1 07 06 5d b9 7a 25 07  |(ñL        E.w`¡..]¹z%.|
000000c0  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
*
00000c00

Having it decode the table

C:\Users\Max\temp\esp32_blink_regr>python C:\Users\Max\.platformio\packages\framework-espidf\components\partition_table\gen_esp32part.py ptable.img
Parsing binary partition input...
Verifying table...
# ESP-IDF Partition Table
# Name, Type, SubType, Offset, Size, Flags
nvs,data,nvs,0x9000,20K,
otadata,data,ota,0xe000,8K,
app0,app,ota_0,0x10000,1280K,
app1,app,ota_1,0x150000,1280K,
spiffs,data,spiffs,0x290000,1472K,

And obviously it’s the same as default.csv

since that’s what my application has setup (defaults).

[env:esp32dev]
platform = espressif32
board = esp32dev
frmework = arduino

Again, reading out the partition table will not help you other than verifying that the toolchain (PlatformIO and esptool.py) have written the partition table correctly to the chip. There is no bug in there. The tools work exactly as you configure it.

The first time you helped me with that trick of grounding GPIO0 during flasing, I could save there a small code there. And it has run, printing millis() via serial.
Then, I’ve tried to save files again on SPIFFS and now:

MacBook-Pro-de-Ciro:ESP32_PIO_Teste02 cirobruno$ platformio device monitor --baud 115200
— Available filters and text transformations: colorize, debug, default, direct, esp32_exception_decoder, hexlify, log2file, nocontrol, printable, send_on_enter, time
— More details at Redirecting...
— Miniterm on /dev/cu.SLAB_USBtoUART 115200,8,N,1 —
— Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H —
ets Jun 8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun 8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun 8 2016 00:22:57

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun 8 2016 00:22:57

But is there no way to look at what is there in the chip?

And how did you do that exactly? Looks like you’ve overwritten the wrong part of flash there.

I just demonstrated that the above method works. If there is a partition table at that point in flash. Since it’s 0xff for you, it looks like it’s overwritten. Since you’re not telling how you’ve written the firmware or the files on the device, I can’t tell you what’s wrong with it.

That’s what I’m trying to do:

and then
“Build Filesystem Image”, then “Upload Filesystem Image”

image

I’ll try it again now.

Ending on:

You have not shown the platformio.ini.

Also, are you sure you want to start with a complicated two-data partitions example, one SPIFFS and one FAT formatted? Try the regular way first with a known-good partition table, as e.g. linked above.

PlatformIO doesn’t even handle FAT formatted partitions. It only accepts SPIFFS.

Here you are:

That’s a good advice. I’ll try it.

But now, taking a look at my platformio.ini, it looks like it has built according to default.csv.

Okay that’s a ton of libraries in there.

I can also tell you that adding the arduino-libraries/SD library to an ESP32 project is wrong. The Arduino-ESP32 core has a built-in SD library that will be picked up automatically if #include <SD.h> is detected. You should delete it from lib_deps.

What does the following code output of you overwrite your old code?


#include "FS.h"
#include "SPIFFS.h"

/* You only need to format SPIFFS the first time you run a
   test or else use the SPIFFS plugin to create a partition
   https://github.com/me-no-dev/arduino-esp32fs-plugin */
#define FORMAT_SPIFFS_IF_FAILED false
	
void listDir(fs::FS &fs, const char * dirname, uint8_t levels){
    Serial.printf("Listing directory: %s\r\n", dirname);

    File root = fs.open(dirname);
    if(!root){
        Serial.println("- failed to open directory");
        return;
    }
    if(!root.isDirectory()){
        Serial.println(" - not a directory");
        return;
    }

    File file = root.openNextFile();
    while(file){
        if(file.isDirectory()){
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if(levels){
                listDir(fs, file.path(), levels -1);
            }
        } else {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("\tSIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}


void setup(){
    Serial.begin(115200);
	delay(500);
    if(!SPIFFS.begin(FORMAT_SPIFFS_IF_FAILED)){
        Serial.println("SPIFFS Mount Failed");
        return;
    }
    
    listDir(SPIFFS, "/", 0);
}

void loop() {}

(Upload + Monitor task)

I didn’t know that. Indeed this location points to a header containing:
“// Copyright 2015-2016 Espressif Systems (Shanghai) PTE LTD
// …” :slight_smile:

The particular reason for using this specific lib is:
“public:
SDFS(FSImplPtr impl);
bool begin(uint8_t ssPin=SS, SPIClass &spi=SPI, uint32_t frequency=4000000, const char * mountpoint=”/sd", uint8_t max_files=5);"

Because it has to communicate to SD via HSPI, while VSPI is used by LoRa module. I must call HSPI and VSPI individually.

I’m trying to clean it up. I’ve been working with them in Arduino IDE and everything was going very smooth. Now I’m trying to identify which versions of each library were working there, so that I can use them in this project.

That’s what I mean:

Back to Arduino Pro IDE, it looks like the device is still working.

Something wrong happened.

“/Users/cirobruno/Documents/Arduino/Testes ESP32/ESP32 FS plugin 00/ESP32 FS plugin 00.ino:29:34: error: ‘class fs::File’ has no member named ‘path’
listDir(fs, file.path(), levels -1);”

To check where was the problem.
I was afraid the procedure had messed up the whole flash.

If I had obtained the same result you did, I’d at least figure out that my partition scheme hadn’t been applied. Remember I had left “default.csv” in platformio.ini?

Both times the same way.
Just like this tutorial:

Before starting VSCode, I’ve been doing similar procedure in Arduino IDE 1.8.xx

image

My bad it should have been

#include "FS.h"
#include "SPIFFS.h"

/* You only need to format SPIFFS the first time you run a
   test or else use the SPIFFS plugin to create a partition
   https://github.com/me-no-dev/arduino-esp32fs-plugin */
#define FORMAT_SPIFFS_IF_FAILED false


void listDir(fs::FS &fs, const char * dirname, uint8_t levels){
    Serial.printf("Listing directory: %s\r\n", dirname);

    File root = fs.open(dirname);
    if(!root){
        Serial.println("- failed to open directory");
        return;
    }
    if(!root.isDirectory()){
        Serial.println(" - not a directory");
        return;
    }

    File file = root.openNextFile();
    while(file){
        if(file.isDirectory()){
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if(levels){
                listDir(fs, file.name(), levels -1);
            }
        } else {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("\tSIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}

void setup(){
    Serial.begin(115200);
        delay(500);
    if(!SPIFFS.begin(FORMAT_SPIFFS_IF_FAILED)){
        Serial.println("SPIFFS Mount Failed");
        return;
    }

    listDir(SPIFFS, "/", 0);
}


void loop(){}

Do not use absolute paths like that. Just reference the library via #include <WiFi.h>.

The library dependency finder will pick it up after you build the project at least once.

@ciro_bruno the initial mentioning of “menuconfig” led me to believe you are on ESP-IDF framework. But no, you are firmly on Arduino framework! If I am not mistaken, the menuconfig command is relevant only to ESP-IDF!

Moving to PlatformIO from any other Arduino-type IDE requires climbing a steep hill, as I had to realize. However, once you have done it, there is no going back, ever. And while A-IDE2.0 will be much improved, it won’t still be on the level of pio. They even explicitly stated wanting to dumb it down…

But be gentle to yourself on the start. Your list of libs looks a bit … weird. Was this by any chance an automated import by pio? Begin with something a whole lot simpler!

Re the flash: there are two more generic commands, which I find helpful:

esptool.py flash_id
esptool.py erase_flash

The first gives info on the flash, the second erases the complete flash. The next upload in pio will completely rebuild the flash (except for any data in the file system). Whenever things seem strange to me as perhaps a result of playing with partitions and file systems, I erase the flash (and also “clean” pio).

Re the partitions: I stopped using the generic code words for various partition layouts, and always use an explicit definition. Easy to do on pio: create a file named partitions.csv in the root of your pio project, and announce it in pio’s ini:

in platformio.ini:
   board_build.partitions = partitions.csv

content of file partitions.csv (example):
   #Espressif ESP32 Partition Table
   #Name,       Type,   SubType,    Offset,     Size,      Flags
   nvs,        data,   nvs,        0x9000,     0x5000,
   otadata,    data,   ota,        0xe000,     0x2000,
   app0,       app,    ota_0,      0x10000,    0x160000,
   app1,       app,    ota_1,      0x170000,   0x160000,

   # in this case SPIFFS is inactive, FFat is active!
   #spiffs,    data,   spiffs,     0x2D0000,   0x130000,
   ffat,       data,   fat,        0x2D0000,   0x130000,

The space below 0x9000 is used by this:

Type                          Address    Size[B] Size[MB]    
SecureBoot                   0x000000  0x0001000    0.004
Bootloader                   0x001000  0x0007000    0.027
Partition Table              0x008000  0x0001000    0.004

In particular you see, the partition table is at 0x8000 and has size 0x1000, which you can read out with esptool.py.