ESP32 esp-idf: cannot increase spifss and app memory

Development Platforms
,
1

Hi,
I have a strange issue with not being able to increase the spiffs memory alocation using custom partition table. Let me give you the details on what works and what not.

My platformio.ini:

[env]
platform = espressif32@6.3.2
board = esp32dev
framework = espidf
platform_packages = framework-espidf
upload_speed = 921600
monitor_speed = 115200
monitor_raw = true
board_build.partitions = partitions_4mb_spiffs_not_working.csv

Using the following partitions_4mb_spiffs_not_working.csv:

# Name,   Type, SubType, Offset, Size,    Flags
nvs,      data, nvs,     0x9000, 0x6000,
phy_init, data, phy,     0xf000, 0x1000,
factory,  app,  factory,       , 1500K,
storage,  data, spiffs,        , 2M,

I get the issue that spiffs cannot be mounted, no matter what I do. I already enabled CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y via config menu.

E (625) SPIFFS: mount failed, -10025
E (625) SPIFFS: Init failed (ESP_FAIL)

The following partition table works without issues:

# Name,   Type, SubType, Offset, Size,    Flags
nvs,      data, nvs,     0x9000, 0x6000,
phy_init, data, phy,     0xf000, 0x1000,
factory,  app,  factory,       , 1M,
storage,  data, spiffs,        , 1M,

Am I doing something wrong here?

2

Did you take care to also change the partition name to partitions_4mb_spiffs_not_working.csv in the ESP-IDF menuconfig options? This needs to be double-declared.

3

Just tried by changing the name of the file to default partitions.csv. I checked that sdkconfig has:

#
# Partition Table
#
# CONFIG_PARTITION_TABLE_SINGLE_APP is not set
# CONFIG_PARTITION_TABLE_SINGLE_APP_LARGE is not set
# CONFIG_PARTITION_TABLE_TWO_OTA is not set
CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions.csv"
CONFIG_PARTITION_TABLE_FILENAME="partitions.csv"
CONFIG_PARTITION_TABLE_OFFSET=0x8000
CONFIG_PARTITION_TABLE_MD5=y
# end of Partition Table

Unfortunatelly, the same issue.

4

Note that when you change the partition table, you change the address at which the SPIFFS is stored too here. So if you had previously written data to the SPIFFS, that was then at a different address compared to what it is with the new partition table. So I would understand that it does not mount anymore due to there not yet being a new SPIFFS filesystem. But, you should be able to use some SPIFFS.format() call to create it anew.

5

That is true. So every time I change the partition table I do re-build and Upload Filesystem Image. That finishes sucessfully and looks that is writing to correct adress.

image

Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
Configuring flash size...
Flash will be erased from 0x00190000 to 0x0038ffff...
Compressed 2097152 bytes to 31732...
Writing at 0x00190000... (50 %)
Writing at 0x001b7f14... (100 %)
Wrote 2097152 bytes (31732 compressed) at 0x00190000 in 8.9 seconds (effective 1891.6 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
============================== [SUCCESS] Took 18.42 seconds ============================== *  Terminal will be reused by tasks, press any key to close it.

Edit: I also tried format and still fails. I can redo and give you the output. Very strange…

6

Can you add some code in the main firmware to print the address of the partition table that ESP-IDF calculates? There might be an address mismatch between what PlatformIO calculates as target address and what ESP-IDF reads from (there have already been such issues before.)

Looking at this, something like

#include "esp_partition.h"

esp_partition_subtype_t subtype = ESP_PARTITION_SUBTYPE_DATA_SPIFFS;
const esp_partition_t* partition = esp_partition_find_first(ESP_PARTITION_TYPE_DATA,
                                   subtype, NULL);
if(!partition) {
  LOGI("SPIFFS partition not found\n");
} else {
  LOGI("SPIFFS partition at 0x%08x size %u\n", partition->address, partition->size);
}
7

I have just run the code you suggested. Result:

I (596) MAIN: SPIFFS partition at 0x187000 size 2097152

That seems indeed different compared to where the platformio Uploads th system file image 0x00190000 to 0x0038ffff

8

Well that’s a 0x3000 (12 kByte) difference :frowning:

In fact,

> hex(0xf000+0x1000 + (1500*1024))
0x187000

when calculating it from the values of the partition table too. And yet PIO insists to flash to 0x190000? Mh.

Can you try the following:

  1. Open a CLI
  2. Do a verbose filesystem upload by doing pio run -t uploadfs -v
  3. Copy the command but modify the target address to 0x187000
  4. Use the full path to the python interpreter and esptool.py executable (for first see pio system info, for second see C:\Users\<user>\.platformio\packages\tool-esptoolpy)
  5. Reboot the board without reflashing the firmware. Does it mount the SPIFFS now?

If yes, that confirms that its offset calculation is wrong and an issue at Issues · platformio/platform-espressif32 · GitHub should be opened if not already existing.

9

My escuses, after running step 2. I guess this is the command

C:\Users\eferu\.platformio\penv\Scripts\python.exe" "C:\Users\eferu\.platformio\packages\tool-esptoolpy\esptool.py" --chip esp32 --port "COM24" --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size 4MB 1638400 .pio\build\main\spiffs.bin

but I don’t see where to modify the target address.

10

That’s the address, just given in decimal.

>>> hex(1638400)
'0x190000'

You should be able to use a hex value in your modified command though, i.e., 0x187000 (decimal 1601536)

11

I now have the command

PS C:\git\firmware> C:\Users\EFeru\.platformio\penv\Scripts\python.exe" "C:\Users\EFeru\.platformio\packages\tool-esptoolpy\esptool.py" --chip esp32 --port "COM24" --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size 4MB 0x187000 
.pio\build\main\spiffs.bin       
>>   
>>

seems like is stuck in the python interpreter >> any idea how to poperly execute it?

12

There’s one quote " at the end of ´python.exe` that wasn’t started at the beginning.

13

It should have been without quotes, my mistake

PS C:\git\firmware> C:\Users\EFeru\.platformio\penv\Scripts\python.exe C:\Users\EFeru\.platformio\packages\tool-esptoolpy\esptool.py --chip esp32 --port "COM24" --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size 4MB 0x187000 .pio\build\main\spiffs.bin
esptool.py v4.5.1
Serial port COM24
Connecting....................
Chip is ESP32-D0WD-V3 (revision v3.0)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 921600
Changed.
Configuring flash size...
Flash will be erased from 0x00187000 to 0x00386fff...
Compressed 2097152 bytes to 31732...
Wrote 2097152 bytes (31732 compressed) at 0x00187000 in 9.1 seconds (effective 1845.7 kbit/s)...
Hash of data verified.

And yes, it works without problems now

I (806) SPIFFS: Init OK
I (806) SPIFFS: size: 1920401 B, used: 112699 B

Edit: What would you say is the next step? To wait for new release or maybe is there a temporary workaround.

14

Then the address calculation is confirmed wrong in the platform-espressif32. Please create an issue with your platformio.ini, partition table, wrong address and right address and a link to this thread.

In the meanwhile you might be be able to supress the error in PlatformIO by explictly stating the offset in the partition table.

# Name,   Type, SubType, Offset, Size,    Flags
nvs,      data, nvs,     0x9000, 0x6000,
phy_init, data, phy,     0xf000, 0x1000,
factory,  app,  factory,       , 1500K,
storage,  data, spiffs, 0x187000, 2M,
15

Issue created:

As always, thank you very much for your excellent support!