Migrating from Arduino IDE to PlatformIO

I am trying to migrate my project from Arduino IDE to PlatformIO. I am new with PlatformIO. My project runs on an ESP32-C6. I am trying my best to learn the quirks and features of PlatformIO but I find it hard to find the information I am looking for within the documentation. In a way, this post sums up my struggles.

My main source of information is the following:

  • https ://docs.platformio.org/en/latest/platforms/espressif32.html#espressif-32
  • https ://docs.platformio.org/en/latest/frameworks/arduino.html#framework-arduino
  • and this forum.

My issue I am having is mostly with how to configure platformio.ini properly so that compilation behaves like with Arduino IDE. When building with Arduino IDE, I get the following output:

Sketch uses 703230 bytes (53%) of program storage space. Maximum is 1310720 bytes.
Global variables use 33620 bytes (10%) of dynamic memory, leaving 294060 bytes for local variables. Maximum is 327680 bytes.

But when compiling with PlatformIO, the binary is too large and does not fit in a 4MB flash:

RAM:   [==        ]  17.6% (used 57728 bytes from 327680 bytes)
Error: The program size (1328424 bytes) is greater than maximum allowed (1310720 bytes)
Flash: [=====*** [checkprogsize] Explicit exit, status 1
=====]  101.4% (used 1328424 bytes from 1310720 bytes)

Where does the jump from 703kb to 1328kb comes from ? Clearly the platformIO project compiles more stuff or links more stuff. Maybe I am compiling in debug mode ? I don’t really know.

The source code is the same in both IDE/projects.

My project settings in Arduino IDE are the following:

  • Boards: ESP32C6 Dev Module
  • Debug Core Level: Info
  • Flash Frequency: 80Mhz
  • Flash mode: QIO
  • Flash size: 4MB
  • Partition Scheme: Zigbee 4MB with spiffs
  • Upload speed 921600
  • Zigbee Mode: Zigbee ED (end device)

My platformio.ini file is the following:

; PlatformIO Project Configuration File
;
;   Build options: build flags, source filter
;   Upload options: custom upload port, speed and extra flags
;   Library options: dependencies, extra library storages
;   Advanced options: extra scripting
;
; Please visit documentation for the other options and examples
; https://docs.platformio.org/page/projectconf.html

[env:esp32-c6-devkitc-1_arduino]
platform = https://github.com/pioarduino/platform-espressif32.git
board = esp32-c6-devkitc-1
framework = arduino
board_build.arduino.upstream_packages = no
lib_deps = 
	adafruit/Adafruit NeoPixel@^1.15.4
	end2endzone/SoftTimers@^2.1.0
	lennarthennigs/Button2@^2.5.0

build_flags = 
  -D ZIGBEE_MODE_ED
  -D CORE_DEBUG_LEVEL=3

; PARTITION SCHEME:
; https://community.platformio.org/t/board-build-partitions-which-option-is-the-right-one/37201/3
; https://github.com/espressif/arduino-esp32/tree/master/tools/partitions
; https://esp32.jgarrettcorbin.com/
; https://raw.githubusercontent.com/espressif/arduino-esp32/refs/tags/3.3.7/boards.txt
; [...]
; According to https://raw.githubusercontent.com/espressif/arduino-esp32/refs/tags/3.3.7/boards.txt,
; to emulate selecting menu option `Zigbee 4MB with spiffs`, I need to define the following:
;     esp32c6.menu.PartitionScheme.zigbee.build.partitions=zigbee
;     esp32c6.menu.PartitionScheme.zigbee.upload.maximum_size=1310720
; Which maps to the following:
board_build.partitions=zigbee.csv
board_upload.maximum_size=1310720

; https://community.platformio.org/t/how-to-change-partition-scheme-for-esp32/29687/5
; https://community.platformio.org/t/moving-from-arduino-ide-to-platformio-ide/42372/15
; https://community.platformio.org/t/how-to-change-partition-scheme-for-esp32/29687/6
board_build.flash_mode = qio
board_build.f_flash = 80000000L

; ESP32 Core Debug Levels:
; https://docs.platformio.org/en/latest/platforms/espressif32.html#debug-level
; https://community.platformio.org/t/how-to-set-up-log-level-to-be-able-to-debug-the-esp32/8278
; Set macro `CORE_DEBUG_LEVEL` to one of the following values:
;   NONE    0
;   ERROR   1
;   WARNING 2
;   INFO    3
;   DEBUG   4
;   VERBOSE 5
; For example:
;   -D CORE_DEBUG_LEVEL=2


; Zigbee Modes:
; https://github.com/espressif/arduino-esp32/blob/3.3.7/boards.txt
;  -D ZIGBEE_MODE_ED
;  -D ZIGBEE_MODE_ZCZR


; ERASE FLASH
; https://docs.platformio.org/en/latest/platforms/espressif32.html#erase-flash
; `pio run --target erase`

I’ve also found hints in other posts stating that I need to find my Arduino IDE settings in https://raw.githubusercontent.com/espressif/arduino-esp32/refs/tags/3.3.7/boards.txt and then “maps” them to values in the ini file. I do not understand how I am supposed to do this. For example, in Arduino IDE, i’Ve selected Upload Speel 921600, this matches the following lines:
```
esp32c6.menu.UploadSpeed.921600=921600
esp32c6.menu.UploadSpeed.921600.upload.speed=921600
```
But then I do not know the expected parameter names in the ini. The examples in https ://docs.platformio.org/en/latest/projectconf/index.html are not giving me hints of what it could be.

Same for when I configure my project settings as a Zigbee ED (end device). It matches the following lines in boards.txt:

esp32c6.menu.ZigbeeMode.ed=Zigbee ED (end device)
esp32c6.menu.ZigbeeMode.ed.build.zigbee_mode=-DZIGBEE_MODE_ED
esp32c6.menu.ZigbeeMode.ed.build.zigbee_libs=-lesp_zb_api.ed -lzboss_stack.ed -lzboss_port.native

but without my previous c/c++ experience I would not recognise that -DZIGBEE_MODE_ED is a macro definition which must be added to build_flags. But how about -lesp_zb_api.ed -lzboss_stack.ed -lzboss_port.native? These looks like linking libraries but I do not know what to do with these. How can I specify “additionnal linking library directories” (I come from Visual Studio 2019+) ?

I’m sorry. I do not want to sound angry, like I am ranting over this but I feel really lost and confused. I’ve seens many videos on youtube showing how easy it is to write arduino projects using PlatformIO. Now that VSCode’s Arduino extensions is down, I think PlatformIO is the right choice as my main IDE for Arduino dev. Any help a much appreciated.

(sorry for the messeup links. I wrote those badly on purpose. As a new user, I can only have 2 links in a post).

I hear you. Migration to a new build environment can be frustrating.

I switched from ArduinoIDE a couple years ago. Still learning, but in the end it’s very much worth it.

Two things. The structure and options for the platformio.ini file are still best documented on this PlatformIO site. Spending a few hours there to understand how the .ini file directs the build is needed.

The second thing is understanding the options when building specifically for the ESP32 series of SoC’s, where now we are discussing using pioarduino as the build system.For that best to start at the pioarduino GitHub: GitHub - pioarduino/platform-espressif32: Espressif 32: pioarduino community platform compatible with PlatformIO · GitHub which will point you to the pioarduino Wiki: pioarduino/platform-espressif32 | DeepWiki

The default options for a in board are in the pioarduino json file for that board. Those are at /Users/<user_name>/.platformio/platforms/espressif32/boards/ (on macOS systems).

Decoding the ArduinoIDE build string and dropping those options into the platformio.ini file is not needed.

Here’s a platformio.ini file I use for the -C6.

; PlatformIO Project Configuration File

; Build:
;   pio run -e xiao_c6              ; Build for XIAO ESP32-C6
;   pio run -e xiao_c6 -t upload    ; Build and upload
;   pio run -e xiao_c6 -t monitor   ; Serial monitor
; or use the pio widgets in the pioarduino GUI in VSCode

[env]
framework = arduino
monitor_speed = 115200
monitor_echo = no
monitor_filters =
esp32_exception_decoder
time
upload_protocol = esptool

; ============================================================================
; XIAO ESP32-C6 Hardware Build
; ============================================================================
[env:xiao_c6]
platform =
; pioarduino latest stable release
;   https://github.com/pioarduino/platform-espressif32/releases/download/stable/platform-espressif32.zip
; arduino-esp32 core 3.3.2
;   https://github.com/pioarduino/platform-espressif32/releases/download/55.03.32/platform-espressif32.zip
; arduino-esp32 core 3.3.7
https://github.com/pioarduino/platform-espressif32/releases/download/55.03.37/platform-espressif32.zip
board = seeed_xiao_esp32c6

lib_deps =
; add the external librairies

build_type = debug
build_flags =
-DCORE_DEBUG_LEVEL=3
-DARDUINO_USB_CDC_ON_BOOT=1
; Zigbee ENABLED - End Device mode
-DUSE_ZIGBEE=1
-DZIGBEE_MODE_ED

; Zigbee partition table (adds zb_storage + zb_fct for Zigbee NVM)
board_build.partitions = zigbee.csv

; USB CDC for serial output (XIAO ESP32-C6 uses native USB)
; below line is optional if you wan to target a board on a specific port
monitor_port = /dev/cu.usbmodem14340

1 Like

It can take some effort to switch toolchains. Not only because PlatformIO works in a certain way, but also because the Arduino IDE has started so many people off in a completely different direction. I’m not blaming either one - just pointing out that both toolchain environments have such a different approach.

After many years, my main reference for PIO is still “platformio.ini” (Project Configuration File) — PlatformIO latest documentation - everything in there is worth exploring (it’s a lot to take in, but there’s a “search” box to help out).

Another way to get going is to look at existing projects in various git repositories. Perhaps you can find some using the same platform and framework, which already have a working PIO setup.

Lastly: the build size increase is somewhat surprising. Note that by default, PIO includes all the source files in your src/ folder. The only other reason I can think of is that some defaults differ, pulling in more code or enabling more debug settings than what you had in the Arduino IDE.

I haven’t used the Arduino IDE for ages (nor ESP32-C6) unfortunately, so this is all I can think of.

Thank you to both of you for the kind words and for you helpful respond. I had more time to take a look at other sources of information.

Since then, i’ve tried multiple ways to properly configure the project. My board is actually a nanoESP32-C6 (https ://github.com/wuxx/nanoESP32-C6/blob/master/README_en.md) which is based on the ESP32-C6-WROOM-1 board. I’ve created a custom board.json file to match it. It is based on ESP32-C6-WROOM-1 but it adds Zigbee connectivity. I’ve also updated my platformio.ini file with the following:

[platformio]
boards_dir = .
[...]
board = nanoesp32_c6
[...]
build_flags =
	-DCORE_DEBUG_LEVEL=3
	-DUSE_ZIGBEE=1
	-DZIGBEE_MODE_ED
	-DARDUINO_USB_MODE=1
	-DARDUINO_USB_CDC_ON_BOOT=0

I also added build log traces from both Arduino IDE and PlatformIO if anyone is interrested to look at them and can quickly spot the differences. The logs are:

  • https ://raw.githubusercontent.com/end2endzone/daikin-zigbee-bridge/refs/heads/platformio/traces/compilations/arduinoide-v2.3.7_arduino-esp32-v3.3.7.log
  • https ://raw.githubusercontent.com/end2endzone/daikin-zigbee-bridge/refs/heads/platformio/traces/compilations/platformio.log

But I still have not figured out how to reduce the binary size to fit in a 4MB esp32-c6. There is too much code that is being linked with the final binary. Code that (I guess) I do not need for my project. I think this is because platformio/pioarduino uses prebuild libraries. I do not know how to change that or make the project link with other sets of prebuild libraries. I also do not know which set of libraries would be appropriate.

I also tries to use Claude AI into the mix to see if it could spot the differences. It noticed the following:

The 90% size increase (703 KB → 1,328 KB) isn’t a code difference — it’s a build configuration mismatch that causes PlatformIO to link against a different, larger variant of the prebuilt ESP-IDF libraries.

It suggested to link with esp32c6/lib/qio_qspi which is a smaller library compared to esp32c6/lib/qio_qspi. As far as I know, this change is properly implemented in my board.json file with build.arduino.memory_type=qio_qspi.

I also think that Arduino IDE links with esp32c6-libs while PlatformIO project links with arduinoespressif32-libs. This is kind of strange to me that changing platform also changes the sets of library used for the final binary.

It also notived that Arduino IDE and PlatformIO may not use the same build of OpenThread. From the information I was able to gather, OpenThread can be build in 2 flavors:

  • FTD (Full Thread Device)
    • Acts as a router, leader, or border router. Its radio is always on, maintaining network topology and routing messages.
    • Build flags: -DOT_FTD=ON
  • MTD (Minimal Thread Device)
    • Acts only as an end device (child). It forwards all messages to its parent. It can be a Minimal End Device (MED) with the transceiver always on, or a Sleepy End Device (SED) that sleeps to conserve power.
    • Build flags: -DOT_MTD=ON

So I tried using build_flags = -DOPENTHREAD_FTD=0 -DOPENTHREAD_CONFIG_THREAD_FTD=0 -DOPENTHREAD_CONFIG_THREAD_MTD=1 -DOT_MTD=ON but it did not changed the final binary size. I guess OpenThread is also prebuild for PlatformIO/pioarduino.

Claude also identified that I may also be linking with code for Matter support…
I tried to disable some libraries as explained in https ://deepwiki.com/pioarduino/platform-espressif32#development-workflow. For example:

; Ignore more build libraries
; Does not seems to have an efect on the final binary size.
lib_ignore = Matter

But it did not changed the final binary size.

Claude also identified in platformio build logs that I may have TensorFlow Lite Micro compiled based on :

-iwithprefix/espressif__esp-tflite-micro
-DTF_LITE_STATIC_MEMORY

I am unfamiliar with that. Do not know how to disable/remove this. IT might also be precompiled statically linked in other mandatory library.

The last thing Claude mentionned is that I am not linking with the same sets of libraries at all.
Arduino IDE links with esp32c6-libs @ 3.3.7 while PlatformIO links with arduinoespressif32-libs @ 5.5.0. This might the the actual main issue.

As a beginner with PlatformIO/pioarduino trying to make the switch from Arduino IDE, changing the list of libraries is way out of my league. This platform is not as declarative as I am used when working with Visual Studio 2019 or CMake. There are a lot of implicitly linked libraries or automation.

I now kind of reach a dead end. It seems migrating to PlatformIO/pioarduino forces my project to switch from a 4MB esp32c6 to a 8MB one. This is a bit anoying.

I guess the best way to reach people to get help for a certain topic is through this forum and not by raising an issue on a github repo such as https ://github.com/pioarduino/platform-espressif32/issues.

I’ll keep the post open hopping that the build logs traces might reveal something that someone else might catch.

Thank you.