ESP32 Upload stops after 25KB transfer

PlatformIO Core
,
1

Upload to my ESP32 board was working fine yesterday, I had made no changes but suddenly uploads are not working fine, the firmware file is around 720KB but upload halts at 25KB transfer and throws a flash verify error:

Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Auto-detected Flash size: 2MB
Compressed 25504 bytes to 14987...

Writing at 0x00001000... (100 %)
Wrote 25504 bytes (14987 compressed) at 0x00001000 in 0.3 seconds (effective 597.8 kbit/s)...
File  md5: 3ae26d559994cd531bc0bf49f23046a3
Flash md5: e666720ed1c825f6bbbee63fd458910c
MD5 of 0xFF is c14df0edbd07e54190878c348d133622

I tried upload manually via(same esptool version 2.6): python3 esptool.py -b 460800 write_flash 0x10000 firmware.bin and upload is successful.

Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Auto-detected Flash size: 2MB
Compressed 730656 bytes to 460406...
Wrote 730656 bytes (460406 compressed) at 0x00010000 in 10.5 seconds (effective 555.6 kbit/s)...
Hash of data verified.

Here is my Platform.ini file

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = espidf
monitor_speed = 115200
;upload_speed = 115200
; set frequency to 80MHz
board_build.f_flash = 80000000L
board_build.f_cpu = 40000000L
board_build.flash_mode = dio
board_upload.flash_size= 2MB
board_upload.maximum_size=2097152
board_build.partitions = partitions_example.csv

Partitions file:

# Name,   Type, SubType, Offset,  Size, Flags
# Note: if you have increased the bootloader size, make sure to update the offsets to avoid overlap
nvs,      data, nvs,     0x9000,  0x6000,
phy_init, data, phy,     0xf000,  0x1000,
factory,  app,  factory, 0x10000, 1M,
storage,  data, spiffs,  ,        0xF0000,

Partiions configured in sdkconfig:

CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions_example.csv"
CONFIG_PARTITION_TABLE_FILENAME="partitions_example.csv"


framework-espidf 3.40001.200521 (4.0.1) 
tool-cmake 3.16.4 
tool-esptoolpy 1.20600.0 (2.6.0) 
tool-mkspiffs 2.230.0 (2.30) 
tool-ninja 1.7.1 
toolchain-esp32ulp 1.22851.191205 (2.28.51) 
toolchain-xtensa32 2.80200.200827 (8.2.0)

Can anyone please help with this, I have to manually copy the firmware file and use the esptool command every time when upload is required.

2

These are old versions. The latest platofrm-espressif32 it as ESP-IDF 4.2 and esptoolpy 3.0. But you’ve demonstrated that it works with esptool 2.6.

A major difference I see is that with PlatformIO’s upload you get

but with your command

These are completely different filesizes. I think PlatformIO will try to burn the bootloader or the partitions file first. What is the full output of “Advanced → Verbose Upload” project task?

3

Below is Verbose Upload output:

Building in release mode
<lambda>(["checkprogsize"], [".pio/build/esp32dev/firmware.elf"])
MethodWrapper(["checkprogsize"], [".pio/build/esp32dev/firmware.elf"])
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [=         ]  13.1% (used 42844 bytes from 327680 bytes)
Flash: [=======   ]  69.7% (used 730544 bytes from 1048576 bytes)
.pio/build/esp32dev/firmware.elf  :
section                                                size         addr
.rtc.text                                                 0   1074528256
.rtc.dummy                                                0   1073217536
.rtc.force_fast                                           0   1073217536
.rtc_noinit                                               0   1342177280
.rtc.force_slow                                           0   1342177280
.iram0.vectors                                         1024   1074266112
.iram0.text                                           93833   1074267136
.dram0.data                                           14724   1073414144
.noinit                                                   0   1073428868
.dram0.bss                                            28120   1073428872
.flash.rodata                                        107132   1061158944
.flash.text                                          513831   1074593816
.debug_frame                                          90480            0
.debug_info                                         2983259            0
.debug_abbrev                                        268998            0
.debug_loc                                           710118            0
.debug_aranges                                        35416            0
.debug_ranges                                         40224            0
.debug_line                                         1643904            0
.debug_str                                           267323            0
.comment                                                217            0
.xtensa.info                                             56            0
.xt.prop._ZTISt9exception                                12            0
.xt.prop._ZTISt9bad_alloc                                12            0
.xt.prop._ZTVN10__cxxabiv120__si_class_type_infoE        12            0
.xt.prop._ZTVN10__cxxabiv117__class_type_infoE           12            0
Total                                               6798707

Warning! Please install `99-platformio-udev.rules`. 
More details: https://docs.platformio.org/page/faq.html#platformio-udev-rules

<lambda>(["upload"], [".pio/build/esp32dev/firmware.bin"])
AVAILABLE: esp-prog, espota, esptool, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa
CURRENT: upload_protocol = esptool
MethodWrapper(["upload"], [".pio/build/esp32dev/firmware.bin"])
Auto-detected: /dev/ttyUSB0
"/home/abhijeet/.platformio/penv/bin/python" "/home/abhijeet/.platformio/packages/tool-esptoolpy/esptool.py" --chip esp32 --port "/dev/ttyUSB0" --baud 460800 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size detect 0x1000 /home/abhijeet/Documents/PlatformIO/Projects/CUE2/.pio/build/esp32dev/bootloader.bin 0x8000 /home/abhijeet/Documents/PlatformIO/Projects/CUE2/.pio/build/esp32dev/partitions.bin 0x10000 .pio/build/esp32dev/firmware.bin
esptool.py v2.6
Serial port /dev/ttyUSB0
Connecting........_____....._____.....__
Chip is ESP32D2WDQ5 (revision 1)
Features: WiFi, BT, Dual Core, Embedded Flash, VRef calibration in efuse, Coding Scheme None
MAC: 30:ae:a4:a5:1d:94
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Auto-detected Flash size: 2MB
Compressed 25504 bytes to 14987...

Writing at 0x00001000... (100 %)
Wrote 25504 bytes (14987 compressed) at 0x00001000 in 0.3 seconds (effective 596.5 kbit/s)...
File  md5: 3ae26d559994cd531bc0bf49f23046a3
Flash md5: e666720ed1c825f6bbbee63fd458910c
MD5 of 0xFF is c14df0edbd07e54190878c348d133622

A fatal error occurred: MD5 of file does not match data in flash!
*** [upload] Error 2
4

Yes the first binary is definitely the bootloader.

What happens when you use your esptool 2.6 version to write the bootloader file at 0x1000 when using the filepahts above?

5

On a sidenote, are you sure this is correct? 80MHz flash frequency instead of the standard 40MHz one? Your flash chip can handle that?

6

Yes i get the same error using esptool 2.6. I tried changing flash frequency to 40Mhz, still upload via pio is not working. Tried to upload using esptool 3.0 i get the same error.

Yesterdday, I wanted to test upload on a windows machine, I installed python and pyserial on a win7 VM and uploaded the firmware at 0x10000 directly without bootloader upload. I think this is cause for the issue.

I got a new device with me now which uploads perfectly fine via PIO.

I have erased entire flash for old device, still i get the error. Any suggestions on what can be performed next?

7

Try erasing the chip again with esptool, then try some things from Flashing problem · Issue #33 · m5stack/M5Stack · GitHub.

Specifically a lower upload speed and normal flash frequency.

upload_speed = 115200
; this line removed or commented out
;board_build.f_flash = 80000000L

You might also want to try different flash modes (DIO, QIO, DOUT, QUOT) if that could be a problem.

1 Like
8

Tried multiple erases, change upload speed and flash modes, got no success. I have observed that uploading works fine with arduino IDE, and arduino uploads bootloader at 0x1000 followed by firmware. I tried manually flashing bootloader_dio_40m.bin and it works fine( for both esptool v2.6 & v3):

python3 esptool.py --chip esp32 --baud 115200 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0x1000 bootloader_dio_40m.bin
esptool.py v2.6
Found 1 serial ports
Serial port /dev/ttyUSB0
Connecting........_
Chip is ESP32D2WDQ5 (revision 1)
Features: WiFi, BT, Dual Core, Embedded Flash, VRef calibration in efuse, Coding Scheme None
MAC: 30:ae:a4:a5:1d:94
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 2MB
Flash params set to 0x0210
Compressed 17104 bytes to 11193...
Wrote 17104 bytes (11193 compressed) at 0x00001000 in 1.0 seconds (effective 138.4 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

erased flash and retired with PIO bootloader, i get the same error:

python3 esptool.py --chip esp32 --baud 115200 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0x1000 /home/abhijeet/Documents/PlatformIO/Projects/CUE2/.pio/build/esp32dev/bootloader.bin
esptool.py v2.6
Found 1 serial ports
Serial port /dev/ttyUSB0
Connecting........__
Chip is ESP32D2WDQ5 (revision 1)
Features: WiFi, BT, Dual Core, Embedded Flash, VRef calibration in efuse, Coding Scheme None
MAC: 30:ae:a4:a5:1d:94
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 2MB
Compressed 25504 bytes to 14987...
Wrote 25504 bytes (14987 compressed) at 0x00001000 in 1.3 seconds (effective 154.6 kbit/s)...
File  md5: 8dc64b6aa3745cde2eced2e04f06d89f
Flash md5: 3549ee74a8ad1c2c51e12119232c0dfd
MD5 of 0xFF is c14df0edbd07e54190878c348d133622

A fatal error occurred: MD5 of file does not match data in flash!

In the first case, there is Flash params set to 0x0210 in output, is the absence of this causing issue with upload in 2nd case, then I am not sure why upload works for other new device

9

Can you try if this still occurrs with all the latest versions and an Arduino project? Open a CLI, do a pio platform update espressif32, then start a new Arduino project for the ESP32 dev module with any code and see if it uploads.

(If you need to develop with the old ESP-IDF 4.0.1, you can still do it with platform = espressif32@1.12.4, as the is the latest release featuring that version.

10

Updated espressif32 package, created 2 new projects one on ESP and the other on Arduino Framework. Issue exists on esp while Arduino framework does fine and bootloader gets uploaded.
This has taken a lot of time and effort for me and I am now settling with a workaround.

To ensure the application works fine, I uploaded Arduino’s bootloader and flashed partiton.bin and firmware from esp. This would be one time, moving forward, I’ll just run esptool to update firmware at 0x10000 without touching other partitions, considering there is no change on them.

Thanks @maxgerhardt for your assistance on this.

11

Same problem have OS X 10.14.6. had this problem month before, fix with removing the chineese driver and used the one apple provides.
Now after some mont I started a new project, updates pio first to latest fevuary version. Now the problem starts again come to 100% uploading 1 in 10 ore more tries. tried different cables, no change. Did not try other upload speed than 115200 should not bring anything reading a lot of posts. I also tried different ESPboard I have a handfull of these ESP 12E Lolin boards no change.
There was time when I used Arduino, mostly ESP projects, everthing worked fine, than I changed to pio autum 2019 with a smart home project. everything worked fine. with arduin nano. In autum 2020 I started a project with ESP and the problems started. And not only these discussion show that a lot ogf people have this problem, so it needs to be fixed.

12

How did you remove the non-Apple drivers? Did you just run kextunload or did you also delete the files (rm -rf)? In the former case, they haven’t been removed and will return after a reboot.

13

@mauelbl
I do not remember which tutorial I followed, but befor I had two cu drivers unde /dev and afterward I just had only one. I rechecked it adetr rebooot and still have only one, so I think its OK