PlatformIO Community

Trying to reduce espressif32 and espressif8266 build size to match arduino studio

Hi,
This is my 3rd attempt to move from Arduino Studio to Platform.io for my own IoT project and I have made significant progress (I finally have it compiling for all my targets from platformio). However, the resulting binary size for a d1_mini (esp8266) is 437,041 bytes from Arduino Studio but 645,205 bytes from Platformio. That is such an enormous difference that I am unable to deploy it to some of the same hardware and none of them are able to have OTA updates anymore which is a deal-breaker for devices embedded in walls for obvious reasons.

I’m using both espressif32 and espressif8266 platforms with multiple boards each but I’m happy to focus on just a wemos D1 mini and solve that one.

my ini file is

[env]
monitor_speed = 115200
framework = arduino
build_flags = 
	-D__VERSION="3.0.0.0"
	-w
lib_deps = 
	ArduinoJson
	StreamUtils
	makuna/NeoPixelBus@^2.6.9
	me-no-dev/ESP Async WebServer@^1.2.3
	slocomptech/QList@^0.6.7
extra_scripts = pre:rename_bin.py

[env:d1_mini]
board = d1_mini
platform = espressif8266
board_build.filesystem = littlefs
board_build.ldscript = eagle.flash.4m3m.ld
build_flags = 
	${env.build_flags}
	-DBOARD_VARIANT="d1_mini"
	-DBOARD_FLASH_SIZE="4M"
	-D__HARDWARE_GENERATION="LOLIN(WEMOS) D1 R2 & mini"
lib_deps = 
	${env.lib_deps}
	me-no-dev/ESPAsyncTCP@^1.2.2

I figure that the difference is going to be with either the core platform being included, or the compiler flags being different, or both. I tried a verbose build out of platformio to try and catch the compiler flags but the terminal window doesn’t have enough scrollback to get them.
size
I’m not sure if 136kb of “unknown” is normal with this build system.

So really I’m looking for hints and suggestions for how to find the cause of my oversized binaries. I desparately need a more capable build system than Arduino Studio can offer but obviously the requirement that my code fit on the board is more important.

The project is about 3700 lines, so I have opted to not include the code here. I did convert the .ino file to main.cpp.

Here is a chunk of the build log (not verbose) which might be useful

CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/d1_mini.html
PLATFORM: Espressif 8266 (3.2.0) > WeMos D1 R2 and mini
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES:
 - framework-arduinoespressif8266 3.30002.0 (3.0.2)
 - tool-esptool 1.413.0 (4.13)
 - tool-esptoolpy 1.30000.201119 (3.0.0)
 - toolchain-xtensa 2.100300.210717 (10.3.0)
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 41 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <ArduinoJson> 6.19.4
|-- <StreamUtils> 1.6.2
|   |-- <EEPROM> 1.0
|-- <NeoPixelBus> 2.6.9
|   |-- <SPI> 1.0
|   |-- <I2S> 1.0
|-- <ESP Async WebServer> 1.2.3
|   |-- <ESPAsyncTCP> 1.2.2
|   |-- <Hash> 1.0
|   |-- <ArduinoJson> 6.19.4
|   |-- <ESP8266WiFi> 1.0
|-- <QList> 0.6.7
|-- <ESPAsyncTCP> 1.2.2
|-- <LittleFS> 0.1.0
|-- <ESP8266HTTPClient> 1.2
|   |-- <ESP8266WiFi> 1.0
|-- <ESP8266httpUpdate> 1.3
|   |-- <ESP8266HTTPClient> 1.2
|   |   |-- <ESP8266WiFi> 1.0
|   |-- <ESP8266WiFi> 1.0
|-- <ESP8266WiFi> 1.0
Building in release mode

Any ideas?
Thanks

What version of Arduino-ESP8266 does it use? What version of xtensa-lx106-elf-gcc does it use? Have 100% equialized the version of all used libraries? Have you got access to a verbose compilation log from Arduino Studio?

Thanks for your reply.

Not sure. I’m using esp8266 v3.0.2 from the boards manager. How do I find my xtensa version?
EDIT: in the Arduino log I see “10.3-1757bed” which matches the toolchain version listed against the Espressif 8266 platform.

As far as I can tell, yes. There are a lot fewer libraries in this than I thought.

C:\Program Files (x86)\Arduino\arduino-builder -dump-prefs -logger=machine -hardware C:\Program Files (x86)\Arduino\hardware -hardware C:\Users\Someone\AppData\Local\Arduino15\packages -tools C:\Program Files (x86)\Arduino\tools-builder -tools C:\Program Files (x86)\Arduino\hardware\tools\avr -tools C:\Users\Someone\AppData\Local\Arduino15\packages -built-in-libraries C:\Program Files (x86)\Arduino\libraries -libraries D:\Someone\Documents\Arduino\libraries -fqbn=esp8266:esp8266:d1_mini:xtal=80,vt=flash,exception=disabled,stacksmash=disabled,ssl=all,mmu=3232,non32xfer=fast,eesz=4M3M,ip=lm2f,dbg=Disabled,lvl=None____,wipe=none,baud=921600 -vid-pid=1A86_7523 -ide-version=10813 -build-path C:\Users\Someone\AppData\Local\Temp\arduino_build_730264 -warnings=none -build-cache C:\Users\Someone\AppData\Local\Temp\arduino_cache_431172 -prefs=build.warn_data_percentage=75 -prefs=runtime.tools.xtensa-lx106-elf-gcc.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.0.4-gcc10.3-1757bed -prefs=runtime.tools.xtensa-lx106-elf-gcc-3.0.4-gcc10.3-1757bed.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.0.4-gcc10.3-1757bed -prefs=runtime.tools.mklittlefs.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\mklittlefs\3.0.4-gcc10.3-1757bed -prefs=runtime.tools.mklittlefs-3.0.4-gcc10.3-1757bed.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\mklittlefs\3.0.4-gcc10.3-1757bed -prefs=runtime.tools.python3.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\python3\3.7.2-post1 -prefs=runtime.tools.python3-3.7.2-post1.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\python3\3.7.2-post1 -prefs=runtime.tools.mkspiffs.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\mkspiffs\3.0.4-gcc10.3-1757bed -prefs=runtime.tools.mkspiffs-3.0.4-gcc10.3-1757bed.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\mkspiffs\3.0.4-gcc10.3-1757bed -verbose D:\Someone\Documents\github\core\ESP_IoT_Core\ESP_IoT_Core.ino
C:\Program Files (x86)\Arduino\arduino-builder -compile -logger=machine -hardware C:\Program Files (x86)\Arduino\hardware -hardware C:\Users\Someone\AppData\Local\Arduino15\packages -tools C:\Program Files (x86)\Arduino\tools-builder -tools C:\Program Files (x86)\Arduino\hardware\tools\avr -tools C:\Users\Someone\AppData\Local\Arduino15\packages -built-in-libraries C:\Program Files (x86)\Arduino\libraries -libraries D:\Someone\Documents\Arduino\libraries -fqbn=esp8266:esp8266:d1_mini:xtal=80,vt=flash,exception=disabled,stacksmash=disabled,ssl=all,mmu=3232,non32xfer=fast,eesz=4M3M,ip=lm2f,dbg=Disabled,lvl=None____,wipe=none,baud=921600 -vid-pid=1A86_7523 -ide-version=10813 -build-path C:\Users\Someone\AppData\Local\Temp\arduino_build_730264 -warnings=none -build-cache C:\Users\Someone\AppData\Local\Temp\arduino_cache_431172 -prefs=build.warn_data_percentage=75 -prefs=runtime.tools.xtensa-lx106-elf-gcc.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.0.4-gcc10.3-1757bed -prefs=runtime.tools.xtensa-lx106-elf-gcc-3.0.4-gcc10.3-1757bed.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\xtensa-lx106-elf-gcc\3.0.4-gcc10.3-1757bed -prefs=runtime.tools.mklittlefs.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\mklittlefs\3.0.4-gcc10.3-1757bed -prefs=runtime.tools.mklittlefs-3.0.4-gcc10.3-1757bed.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\mklittlefs\3.0.4-gcc10.3-1757bed -prefs=runtime.tools.python3.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\python3\3.7.2-post1 -prefs=runtime.tools.python3-3.7.2-post1.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\python3\3.7.2-post1 -prefs=runtime.tools.mkspiffs.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\mkspiffs\3.0.4-gcc10.3-1757bed -prefs=runtime.tools.mkspiffs-3.0.4-gcc10.3-1757bed.path=C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\tools\mkspiffs\3.0.4-gcc10.3-1757bed -verbose D:\Someone\Documents\github\core\ESP_IoT_Core\ESP_IoT_Core.ino
Using board 'd1_mini' from platform in folder: C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2
Using core 'esp8266' from platform in folder: C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2
Detecting libraries used...
"C:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\tools\\xtensa-lx106-elf-gcc\\3.0.4-gcc10.3-1757bed/bin/xtensa-lx106-elf-g++" -D__ets__ -DICACHE_FLASH -U__STRICT_ANSI__ -D_GNU_SOURCE "-IC:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.0.2/tools/sdk/include" "-IC:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.0.2/tools/sdk/lwip2/include" "-IC:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.0.2/tools/sdk/libc/xtensa-lx106-elf/include" "-IC:\\Users\\Someone\\AppData\\Local\\Temp\\arduino_build_730264/core" -c -w -Werror=return-type -Os -g -free -fipa-pta -mlongcalls -mtext-section-literals -fno-rtti -falign-functions=4 -std=gnu++17 -ffunction-sections -fdata-sections -fno-exceptions -DMMU_IRAM_SIZE=0x8000 -DMMU_ICACHE_SIZE=0x8000 -w -x c++ -E -CC -DNONOSDK22x_190703=1 -DF_CPU=80000000L -DLWIP_OPEN_SRC -DTCP_MSS=536 -DLWIP_FEATURES=1 -DLWIP_IPV6=0 -DARDUINO=10813 -DARDUINO_ESP8266_WEMOS_D1MINI -DARDUINO_ARCH_ESP8266 "-DARDUINO_BOARD=\"ESP8266_WEMOS_D1MINI\"" "-DBOARD_VARIANT=\"d1_mini\"" "-DBOARD_FLASH_SIZE=\"4M\"" -DFLASHMODE_DIO -DESP8266 "-IC:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.0.2\\cores\\esp8266" "-IC:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.0.2\\variants\\d1_mini" "C:\\Users\\Someone\\AppData\\Local\\Temp\\arduino_build_730264\\sketch\\ESP_IoT_Core.ino.cpp" -o nul -DARDUINO_LIB_DISCOVERY_PHASE

...

"C:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\tools\\python3\\3.7.2-post1/python3" -I "C:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.0.2/tools/elf2bin.py" --eboot "C:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.0.2/bootloaders/eboot/eboot.elf" --app "C:\\Users\\Someone\\AppData\\Local\\Temp\\arduino_build_730264/ESP_IoT_Core.ino.elf" --flash_mode dio --flash_freq 40 --flash_size 4M --path "C:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\tools\\xtensa-lx106-elf-gcc\\3.0.4-gcc10.3-1757bed/bin" --out "C:\\Users\\Someone\\AppData\\Local\\Temp\\arduino_build_730264/ESP_IoT_Core.ino.bin"
Creating BIN file "C:\Users\Someone\AppData\Local\Temp\arduino_build_730264/ESP_IoT_Core.ino.bin" using "C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2/bootloaders/eboot/eboot.elf" and "C:\Users\Someone\AppData\Local\Temp\arduino_build_730264/ESP_IoT_Core.ino.elf"
"C:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\tools\\python3\\3.7.2-post1/python3" -I "C:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.0.2/tools/signing.py" --mode sign --privatekey "D:\\Someone\\Documents\\github\\core\\ESP_IoT_Core/private.key" --bin "C:\\Users\\Someone\\AppData\\Local\\Temp\\arduino_build_730264/ESP_IoT_Core.ino.bin" --out "C:\\Users\\Someone\\AppData\\Local\\Temp\\arduino_build_730264/ESP_IoT_Core.ino.bin.signed" --legacy "C:\\Users\\Someone\\AppData\\Local\\Temp\\arduino_build_730264/ESP_IoT_Core.ino.bin.legacy_sig"
"C:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\tools\\python3\\3.7.2-post1/python3" -I "C:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\3.0.2/tools/sizes.py" --elf "C:\\Users\\Someone\\AppData\\Local\\Temp\\arduino_build_730264/ESP_IoT_Core.ino.elf" --path "C:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\tools\\xtensa-lx106-elf-gcc\\3.0.4-gcc10.3-1757bed/bin" --mmu "-DMMU_IRAM_SIZE=0x8000 -DMMU_ICACHE_SIZE=0x8000"
Executable segment sizes:
ICACHE : 32768           - flash instruction cache 
IROM   : 397196          - code in flash         (default or ICACHE_FLASH_ATTR) 
IRAM   : 28777   / 32768 - code in IRAM          (IRAM_ATTR, ISRs...) 
DATA   : 1528  )         - initialized variables (global, static) in RAM/HEAP 
RODATA : 9540  ) / 81920 - constants             (global, static) in RAM/HEAP 
BSS    : 26032 )         - zeroed variables      (global, static) in RAM/HEAP 
Using library LittleFS at version 0.1.0 in folder: C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2\libraries\LittleFS 
Using library ArduinoJson at version 6.19.4 in folder: D:\Someone\Documents\Arduino\libraries\ArduinoJson 
Using library ESPAsyncTCP at version 1.2.2 in folder: D:\Someone\Documents\Arduino\libraries\ESPAsyncTCP 
Using library ESPAsyncWebServer at version 1.2.3 in folder: D:\Someone\Documents\Arduino\libraries\ESPAsyncWebServer 
Using library ESP8266WiFi at version 1.0 in folder: C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2\libraries\ESP8266WiFi 
Using library Hash at version 1.0 in folder: C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2\libraries\Hash 
Using library QList at version 0.6.7 in folder: D:\Someone\Documents\Arduino\libraries\QList 
Using library NeoPixelBus_by_Makuna at version 2.6.9 in folder: D:\Someone\Documents\Arduino\libraries\NeoPixelBus_by_Makuna 
Using library SPI at version 1.0 in folder: C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2\libraries\SPI 
Using library ESP8266HTTPClient at version 1.2 in folder: C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2\libraries\ESP8266HTTPClient 
Using library ESP8266httpUpdate at version 1.3 in folder: C:\Users\Someone\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\3.0.2\libraries\ESP8266httpUpdate 
"C:\\Users\\Someone\\AppData\\Local\\Arduino15\\packages\\esp8266\\tools\\xtensa-lx106-elf-gcc\\3.0.4-gcc10.3-1757bed/bin/xtensa-lx106-elf-size" -A "C:\\Users\\Someone\\AppData\\Local\\Temp\\arduino_build_730264/ESP_IoT_Core.ino.elf"
Sketch uses 437041 bytes (41%) of program storage space. Maximum is 1044464 bytes.
Global variables use 37100 bytes (45%) of dynamic memory, leaving 44820 bytes for local variables. Maximum is 81920 bytes.

Unfortunately the log is too long so i snipped out the middle. Apparently the full log exceeds the limits of a post here and on pastebin.
This is basically my first embedded project so I’m still working things out as I go.

Some of the output from platform.io verbose build

section             size     addr
.data               2276     1073643520
.noinit             56       1073645796
.text               468      1074790400
.irom0.text         597252   1075843088
.text1              28473    1074790868
.rodata             16736    1073645856
.bss                30472    1073662592
.comment            5066     0

vs Arduino stuido

Executable segment sizes:
ICACHE : 32768           - flash instruction cache 
IROM   : 397196          - code in flash         (default or ICACHE_FLASH_ATTR) 
IRAM   : 28777   / 32768 - code in IRAM          (IRAM_ATTR, ISRs...) 
DATA   : 1528  )         - initialized variables (global, static) in RAM/HEAP 
RODATA : 9540  ) / 81920 - constants             (global, static) in RAM/HEAP 
BSS    : 26032 )         - zeroed variables      (global, static) in RAM/HEAP 

I was expecting the command line with compiler flags to come out of platform.io’s verbose build but all i saw was the parsed ini file.

I had a look at the binaries coming out in a hex editor and just at a cursory glance it looks like all the libraries are being build with debug symbols and other non-optimised build flags. is there a way to specify or control the building of libraries loaded in platform.io’s UI?

So I googled some of the spurious strings and I found them here

Random snipped of the kind of stuff that was in the platformio build and not the arduino studio one.

cannot create shim for unknown locale::facet�uninitialized __any_string�*NSt13__facet_shims12_GLOBAL__N_113numpunct_shimIcEE�*NSt13__facet_shims12_GLOBAL__N_112collate_shimIcEE�*NSt13__facet_shims12_GLOBAL__N_115moneypunct_shimIcLb1EEE�*NSt13__facet_shims12_GLOBAL__N_115moneypunct_shimIcLb0EEE�*NSt13__facet_shims12_GLOBAL__N_114money_get_shimIcEE�*NSt13__facet_shims12_GLOBAL__N_114money_put_shimIcEE�*NSt13__facet_shims12_GLOBAL__N_113messages_shimIcEE�*NSt13__facet_shims12_GLOBAL__N_113numpunct_shimIwEE�*NSt13__facet_shims12_GLOBAL__N_112collate_shimIwEE�*NSt13__facet_shims12_GLOBAL__N_115moneypunct_shimIwLb1EEE�*NSt13__facet_shims12_GLOBAL__N_115moneypunct_shimIwLb0EEE�*NSt13__facet_shims12_GLOBAL__N_114money_get_shimIwEE�*NSt13__facet_shims12_GLOBAL__N_114money_put_shimIwEE�*NSt13__facet_shims12_GLOBAL__N_113messages_shimIwEE�NSt6locale5facet6__shimE�*NSt13__facet_shims12_GLOBAL__N_113time_get_shimIcEE�*NSt13__facet_shims12_GLOBAL__N_113time_get_shimIwEE�%s: __pos (which is %zu) > this->size() (which is %zu)�basic_string::insert�basic_string::erase�%.*Lf�%H:%M�NSt7__cxx117collateIcEE�NSt7__cxx118numpunctIcEE�NSt7__cxx1110moneypunctIcLb1EEE�NSt7__cxx1110moneypunctIcLb0EEE�NSt7__cxx118messagesIcEE�NSt7__cxx119money_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEEE�NSt7__cxx119money_putIcSt19ostreambuf_iteratorIcSt11char_traitsIcEEEE�NSt7__cxx118time_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEEE�*NSt13__facet_shims12_GLOBAL__N_113numpunct_shimIcEE�*NSt13__facet_shims12_GLOBAL__N_112collate_shimIcEE�*NSt13__facet_shims12_GLOBAL__N_115moneypunct_shimIcLb1EEE�*NSt13__facet_shims12_GLOBAL__N_115moneypunct_shimIcLb0EEE�*NSt13__facet_shims12_GLOBAL__N_114money_get_shimIcEE�*NSt13__facet_shims12_GLOBAL__N_114money_put_shimIcEE�*NSt13__facet_shims12_GLOBAL__N_113messages_shimIcEE�*NSt13__facet_shims12_GLOBAL__N_113numpunct_shimIwEE�*NSt13__facet_shims12_GLOBAL__N_112collate_shimIwEE�*NSt13__facet_shims12_GLOBAL__N_115moneypun

What the heck is going on?

There’s 95kb of “/include/bits” in my binary. Is this normal? That’s where those random strings are from anyway (locale_facets_nonio.tcc etc) so Arduino Studio doesn’t include all of this, if any.

bits

Did you ever find a solution to this? I have run into a similar problem and a looking for solutions.