ESP32 dev: difference between "framework = espidf", "framework = arduino" and "framework = arduino, espidf"

So when developing for the ESP32 with platformIO, one has three framework options (or more?) to work with:

framework = espidf (example)
framework = arduino (Since I’m a new user I can only put two links, example for this can be found at the same place as other two)
framework = espidf, arduino (example)

Can someone give (or point to) an indepth explanation of what the dependencies are in the different cases, which toolchain is used in which case and what the advantages and disadvantages of them are?

Toolchain is in both cases the toolchain-xtensa32 package. Toolchain is just the compiler (and linker).

  • for framework = espidf: Just the ESP-IDF framework.
  • for framework = arduino: Just the Arduino-ESP32 framework.
  • for framework = espidf, arduino: Special case of “Arduino as ESP-IDF component” (see here). The base framework here is again ESP-IDF, with Arduino core API implementations added as a component.

The ESP-IDF framework has an API aimed at more technically skilled users (see examples). Arduino is aimed at the beginner level (though you can achieve the same thing in every framework with enough of your own code). Arduino-as-ESP-IDF-component enables common Arduino libraries to run in an ESP-IDF environment by providing the needed Arduino APIs, so you can use the Arduino libs but also still use advanced ESP-IDF functions; compatibility mode, so to say.

2 Likes

But is it using the actually same toolchain files and version in all cases? I have xtensa-esp32-elf toolchain in at least four places on my computer currently:
C:\Users\User.espressif\tools\xtensa-esp32-elf\esp32-2019r1-8.2.0\xtensa-esp32-elf
C:\Users\User.espressif\tools\xtensa-esp32-elf\1.22.0-80-g6c4433a5-5.2.0\xtensa-esp32-elf
C:\Users\User.platformio\packages\toolchain-xtensa32\xtensa-esp32-elf
C:\Users\User\AppData\Local\Arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0\xtensa-esp32-elf

I’ve read that readme regarding arduino as an esp-idf component already - but how does that apply for a platformIO project? do I also need to make a components folder in my platformio project and gitclone arduino framework into it? Seems not since it compiles also without having done that…

And is it supposed to compile the arduino framework in the “framework = espidf, arduino” case while using precompiled version in the “framework = arduino” case?

PIO will only ever use its compiler from its downloaded package and ignores everything else. So C:\Users\User\.platformio\packages\toolchain-xtensa32\xtensa-esp32-elf-* is used in all PIO compilations for ESP32, regardless of the framework.

No, the buildscripts of PlatformIO already handle the building in a fully-automatic way. See build code and related code. Just cloning the folder platform-espressif32/examples/espidf-arduino-blink at develop · platformio/platform-espressif32 · GitHub and executing pio run it will compile the firmware, no other steps needed.

Both framework are compiled from source and put into the firmware. (Durung linking of course, the precompiled libraries for each framework are added as always). See the build log.

Processing esp32dev (platform: espressif32; framework: arduino, espidf; board: esp32dev)
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html
PLATFORM: Espressif 32 1.11.1 > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: toolchain-xtensa32 2.50200.80 (5.2.0), framework-arduinoespressif32 2.10004.191002 (1.0.4), framework-espidf 3.30300.190916 (3.3.0), tool-esptoolpy 1.20600.0 (2.6.0)
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 26 compatible libraries
Scanning dependencies...
No dependencies
Building in release mode
[.. ESP-IDF is compiled from source from packages\framework-espidf ..]
xtensa-esp32-elf-gcc [..] C:\Users\Maxi\.platformio\packages\framework-espidf\components\bootloader_support\src\bootloader_clock.c
[.. Arduino-ESP32 is compiled from source from packages\framework-arduinoespressif32..]
xtensa-esp32-elf-g++ [..] C:\Users\Maxi\.platformio\packages\framework-arduinoespressif32\cores\esp32\WString.cpp
[ final linker command contains ESP-IDF and Arduino]
xtensa-esp32-elf-g++ -o .pio\build\esp32dev\firmware.elf -T esp32_out.ld -nostdlib -Wl,-static -u call_user_start_cpu0 -Wl,--undefined=uxTopUsedPriority -Wl,--gc-sections -u __cxa_guard_dummy -u esp_app_desc -u ld_include_panic_highint_hdl -T esp32.project.ld -T esp32.rom.ld -T esp32.peripherals.ld -T esp32.rom.libgcc.ld -T esp32.rom.spiram_incompatible_fns.ld -u __cxx_fatal_exception .pio\build\esp32dev\src\main.cpp.o -LC:\Users\Maxi\.platformio\packages\framework-espidf\components\esp32 -LC:\Users\Maxi\.platformio\packages\framework-espidf\components\esp32\ld -LC:\Users\Maxi\.platformio\packages\framework-espidf\components\esp32\ld\wifi_iram_opt -LC:\Users\Maxi\.platformio\packages\framework-espidf\components\esp32\lib -LC:\Users\Maxi\.platformio\packages\framework-espidf\components\bt\lib -LC:\Users\Maxi\.platformio\packages\framework-espidf\components\newlib\lib -L.pio\build\esp32dev -L.pio\build\esp32dev -Wl,--start-group .pio\build\esp32dev\libapp_update.a .pio\build\esp32dev\libbt.a .pio\build\esp32dev\libcoap.a .pio\build\esp32dev\libconsole.a .pio\build\esp32dev\libcxx.a .pio\build\esp32dev\libdriver.a .pio\build\esp32dev\libesp-tls.a .pio\build\esp32dev\libespcoredump.a .pio\build\esp32dev\libesp_adc_cal.a .pio\build\esp32dev\libesp_event.a .pio\build\esp32dev\libesp_https_ota.a .pio\build\esp32dev\libesp_https_server.a .pio\build\esp32dev\libesp_http_client.a .pio\build\esp32dev\libesp_http_server.a .pio\build\esp32dev\libesp_ringbuf.a .pio\build\esp32dev\libethernet.a .pio\build\esp32dev\libexpat.a .pio\build\esp32dev\libfatfs.a .pio\build\esp32dev\libfreemodbus.a .pio\build\esp32dev\libjsmn.a .pio\build\esp32dev\libjson.a .pio\build\esp32dev\liblog.a .pio\build\esp32dev\libmbedtls.a .pio\build\esp32dev\libmdns.a .pio\build\esp32dev\libmicro-ecc.a .pio\build\esp32dev\libmqtt.a .pio\build\esp32dev\libnewlib.a .pio\build\esp32dev\libnghttp.a .pio\build\esp32dev\libnvs_flash.a .pio\build\esp32dev\libopenssl.a .pio\build\esp32dev\libprotobuf-c.a .pio\build\esp32dev\libpthread.a .pio\build\esp32dev\libsdmmc.a .pio\build\esp32dev\libsmartconfig_ack.a .pio\build\esp32dev\libspiffs.a .pio\build\esp32dev\libtcpip_adapter.a .pio\build\esp32dev\libtcp_transport.a .pio\build\esp32dev\libulp.a .pio\build\esp32dev\libvfs.a .pio\build\esp32dev\libwear_levelling.a .pio\build\esp32dev\libxtensa-debug-module.a .pio\build\esp32dev\libasio.a .pio\build\esp32dev\libsoc.a .pio\build\esp32dev\libefuse.a .pio\build\esp32dev\libaws_iot.a .pio\build\esp32dev\libapp_trace.a .pio\build\esp32dev\libspi_flash.a .pio\build\esp32dev\libesp32.a .pio\build\esp32dev\libbootloader_support.a .pio\build\esp32dev\libunity.a .pio\build\esp32dev\liblwip.a .pio\build\esp32dev\libprotocomm.a .pio\build\esp32dev\libheap.a .pio\build\esp32dev\libfreertos.a .pio\build\esp32dev\libsodium.a .pio\build\esp32dev\libwpa_supplicant.a .pio\build\esp32dev\libwifi_provisioning.a -lbtdm_app -lhal -lcoexist -lcore -lnet80211 -lphy -lrtc -lpp -lwpa -lwpa2 -lespnow -lwps -lsmartconfig -lmesh -lc -lm -lgcc -lstdc++ .pio\build\esp32dev\libFrameworkArduinoVariant.a .pio\build\esp32dev\libFrameworkArduino.a -Wl,--end-group

DATA:    [          ]   3.8% (used 12532 bytes from 327680 bytes)
PROGRAM: [==        ]  16.8% (used 176612 bytes from 1048576 bytes)
1 Like

This is a great answer, thank you very much!

So in all cases sources are only loaded from within

C:\Users\User.platformio…

And any installation of ESP-IDF (which installs to C:\Users\User.espressif) and Arduino with ESP32 package (which installs to C:\Users\User\AppData\Local\Arduino15\packages\esp32…) is ignored in the compilation process?

But I see that within C:\Users\User.platformio… some components are there once as precompiled lib like for example in
C:\Users\User.platformio\packages\framework-arduinoespressif32\tools\sdk\lib\libwear_levelling.a
and once as sourcecode
C:\Users\Users.platformio\packages\framework-espidf\components\wear_levelling\

whereas some seem to be only available precompiled such as
C:\Users\User.platformio\packages\framework-arduinoespressif32\tools\sdk\lib\libesp32-camera.a

in which of the three cases does it use the precompiled libs and how would I go about if I wanted for example libesp32-camera to be compiled from source as well?

Yes, PIO does not depend on an Arduino IDE installation.

Indeed the Arduino-ESP32 contains a precompiled version of components/wear_leveling, the object files

$ xtensa-esp32-elf-nm libwear_levelling.a | grep ".o"
wear_levelling.o:
WL_Flash.o:
crc32.o:
WL_Ext_Safe.o:
SPI_Flash.o:
Partition.o:
WL_Ext_Perf.o:

match the sourcefiles from ESP-IDF

[..] framework-espidf\components\wear_levelling>ls 
CMakeLists.txt  include          SPI_Flash.cpp       WL_Ext_Safe.cpp
component.mk    Kconfig          test                WL_Flash.cpp
crc32.cpp       Partition.cpp    test_wl_host
crc32.h         private_include  wear_levelling.cpp
doc             README.rst       WL_Ext_Perf.cpp

The linker command however uses the version compiled from source.

The included static libraries are added via the LIBS variable in the build script plus some extra. Everything else is compiled from source, is my intuition. If you have a request for clarification, Issues · platformio/platform-espressif32 · GitHub would be a place to ask.

1 Like