PlatformIO Community

Download or view esp8266 file system

I know the ide has a file system upload command but I can’t find a corresponding download. Sometimes my script will create files and I’d like to view or download the file system contents for debugging. Is this possible?

If not, does anyone have script code that can display the file sytem using serial out?

A conversation about that already occurred in ESP8266FS- file uploader for ESP (a hint to the https://github.com/kash4kev/vscode-esp8266fs tool)

Are you using the SPIFFS or LittleFS filesystem?

LittleFS

****************** line to oveerride stupid 20 char rule in this forum

I see. Well it should be fairly easy to write an extra_script for it that takes the information of where the filesystem would be uploaded to and then use esptool.py to download that memory region and mklittlefs (or mkspiffs) to unpack it. This can then also be proposed as a direct extension to the platform-espressif8266 (or espressif32 for that matter) platform so that it’s directly a built-in “platform task”.

I’ll have a look at that.

I’ve created an extractor plugin script at https://github.com/maxgerhardt/pio-esp32-esp8266-filesystem-downloader. I’ve tested with ESP8266 (SPIFFS + LitteFS) and ESP32 (SPIFFS) and it works in all tested cases. Downloads the memory content from the chip and extracts / decodes the filesystem content.

Can you give that a test?

@Monsterthews @maxromanovsky for your interest.

1 Like

FYI I’ve opened https://github.com/platformio/platform-espressif8266/issues/238 and https://github.com/platformio/platform-espressif32/issues/460 so that this might some day be included directly in the platform code.

@maxgerhardt Thank you for your plugin script. I’ve tried it on macOS & pio 5.0.4 (which embeds Python 3.9.1).
It doesn’t work due to the fact that subprocess.call expects a list instead of string.

Here's an error
pio run -t downloadfs -v

[...]

command_download_fs(["downloadfs"], [])
Entrypoint
Retrieving filesystem info for ESP8266.
FS_START: 0x300000
FS_END: 0x3fa000
FS_PAGE: 0x100
FS_BLOCK: 0x2000
Recognized LittleFS filesystem.
Parsed FS info: FS type FSType.LITTLEFS Start 0x300000 Len 1024000 Page size 256 Block size 8192 Tool: /Users/max/.platformio/packages/tool-mklittlefs/mklittlefs
Executing flash download command.
"/usr/local/Cellar/platformio/5.0.4/libexec/bin/python" "/Users/max/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp8266 --port "/dev/cu.SLAB_USBtoUART" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "/Users/max/Documents/projects/foo/downloaded_fs_0x300000_0xfa000.bin"
scons: *** [downloadfs] "/usr/local/Cellar/platformio/5.0.4/libexec/bin/python" "/Users/max/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp8266 --port "/dev/cu.SLAB_USBtoUART" --baud 115200 --before default_reset --after hard_reset read_flash 0x300000 0xfa000 "/Users/max/Documents/projects/foo/downloaded_fs_0x300000_0xfa000.bin": No such file or directory

I’ve fixed in in my fork (just for esp8266 & littlefs): https://github.com/maxromanovsky/pio-esp32-esp8266-filesystem-downloader/commit/c9d85edfe3670215bfd30c78ce7c59daaf144cc1

However, even after that fix extraction fails

Read 1024000 bytes at 0x300000 in 92.4 seconds (88.7 kbit/s)...
Hard resetting via RTS pin...
Downloaded filesystem binary.
Download was okay: True. File at: /Users/max/Documents/projects/foo/downloaded_fs_0x300000_0xfa000.bin
Executing extraction command: ['/Users/max/.platformio/packages/tool-mklittlefs/mklittlefs', '-b', '8192', '-p', '256', '--unpack', 'unpacked_fs', '/Users/max/Documents/projects/foo/downloaded_fs_0x300000_0xfa000.bin']
Assertion failed: (block < lfs->cfg->block_count), function lfs_bd_read, file littlefs/lfs.c, line 84.

I’ve opened the file with hex editor and saw that there were leading bytes before magic string littlefs: 01 00 00 00 F0 0F FF F7. I’ve tried removing it, and mklittlefs now fails with the other error:

lfs error:969: Corrupted dir pair at 0 1
Assertion failed: (block != 0xffffffff), function lfs_bd_read, file littlefs/lfs.c, line 41.

Anyway, I am able to see the contents of littlefs in the hex editor.

Thank you, and have a great New Year!

Okay path escaping for other OSes needs some more work in this script then, I’ve tested in on Windows. (Or if you’ve rewritten it to a list, I guess that automatically takes care of it).

What’s the full platformio.ini you’re working with? Is it the from my original repository? And the firmware has executed succesfully on the chip and placed the file in the LittleFS, according to the serial output?

I will. This is something a lot of people clamor for.

You see it on Facebook.

@maxgerhardt I’ve tried it previously on my own project by adding extra_scripts = download_fs.py as described in https://github.com/maxgerhardt/pio-esp32-esp8266-filesystem-downloader#using-in-a-different-project

Gave it a try today using your project (default_envs = esp8266_littlefs, with upload_speed = 400000 commented out), still received the same error:

pio run -t downloadfs -v
Processing esp8266_littlefs (platform: espressif8266; board: nodemcuv2; framework: arduino; monitor_speed: 74880; board_build.filesystem: littlefs; build_flags: -D DO_LITTLEFS; extra_scripts: download_fs.py; custom_unpack_dir: unpacked_esp8266_littlefs)
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/nodemcuv2.html
PLATFORM: Espressif 8266 (2.6.2) > NodeMCU 1.0 (ESP-12E Module)
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES:
 - framework-arduinoespressif8266 3.20704.0 (2.7.4)
 - tool-esptool 1.413.0 (4.13)
 - tool-esptoolpy 1.20800.0 (2.8.0)
 - toolchain-xtensa 2.40802.200502 (4.8.2)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 29 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <LittleFS(esp8266)> 0.1.0 (/Users/max/.platformio/packages/framework-arduinoespressif8266/libraries/LittleFS)
Building in release mode
Auto-detected: /dev/cu.usbserial-0001
command_download_fs(["downloadfs"], [])
Entrypoint
Retrieving filesystem info for ESP8266.
FS_START: 0x300000
FS_END: 0x3fa000
FS_PAGE: 0x100
FS_BLOCK: 0x2000
Recognized LittleFS filesystem.
Parsed FS info: FS type FSType.LITTLEFS Start 0x300000 Len 1024000 Page size 256 Block size 8192 Tool: /Users/max/.platformio/packages/tool-mklittlefs/mklittlefs
Executing flash download command.
['/usr/local/Cellar/platformio/5.0.4/libexec/bin/python', '/Users/max/.platformio/packages/tool-esptoolpy/esptool.py', '--chip', 'esp8266', '--port', '/dev/cu.usbserial-0001', '--baud', '115200', '--before', 'default_reset', '--after', 'hard_reset', 'read_flash', '0x300000', '0xfa000', '/Users/max/Downloads/pio-esp32-esp8266-filesystem-downloader/downloaded_fs_0x300000_0xfa000.bin']
esptool.py v2.8
Serial port /dev/cu.usbserial-0001
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: a0:20:a6:08:08:96
Uploading stub...
Running stub...
Stub running...
1024000 (100 %)
1024000 (100 %)

Read 1024000 bytes at 0x300000 in 92.2 seconds (88.9 kbit/s)...
Hard resetting via RTS pin...
Downloaded filesystem binary.
Download was okay: True. File at: /Users/max/Downloads/pio-esp32-esp8266-filesystem-downloader/downloaded_fs_0x300000_0xfa000.bin
Executing extraction command: ['/Users/max/.platformio/packages/tool-mklittlefs/mklittlefs', '-b', '8192', '-p', '256', '--unpack', 'unpacked_esp8266_littlefs', '/Users/max/Downloads/pio-esp32-esp8266-filesystem-downloader/downloaded_fs_0x300000_0xfa000.bin']
Assertion failed: (block < lfs->cfg->block_count), function lfs_bd_read, file littlefs/lfs.c, line 84.
Unpacked filesystem.
Extracted 0 file(s) from filesystem.
===================================================================== [SUCCESS] Took 94.55 seconds =====================================================================

Environment       Status    Duration
----------------  --------  ------------
esp8266_spiffs    IGNORED
esp8266_littlefs  SUCCESS   00:01:34.551
esp32dev          IGNORED
====================================================================== 1 succeeded in 00:01:34.551 ======================================================================

Can you erase the flash the same way I did? esptool.py erase_flash.

That’s really strange. Can you upload the extracted /Users/max/Downloads/pio-esp32-esp8266-filesystem-downloader/downloaded_fs_0x300000_0xfa000.bin file?

Maybe there really is a mismatch between the configured block count of the downloaded filesystem versus what’s passed into mklittlefs and we think is used.

@maxgerhardt Just tried, but without any luck. Still the same error. And again, tried removing 8 bytes before littlefs magic string and got the same error.
LittleFS reads the file just fine from the firmware.