How to setup PlatformIO for Framework developement (arduino-esp32)?

I would like to contribute to the development of the Arduino framework for the ESP32. How do I have to set up my system (VSCode + PlatformIO @ Windows) for this?

So far I have only developed projects or libraries.
To contribute to library development I just cloned the source code via git and used the local folder “c:\username.platformio\lib”. This way I am able to develop, use and test the library at the same time.

How do I have to set up the system to be able to contribute to the Arduino framework for the ESP32? As far as I know, I can’t use the Espressif-Arduino32 repository directly. Neither can I use the platformio/platform-espressif32 repository since it is built from the espressif/arduino-esp32 repository.
Which repository do I need to use here?

Furthermore PlatformIO has to be configured to accept a local folder as source for a platform. Is the platform-package the right approach here? If so, how do I need to configure this?

It would be very nice if someone could give me a little guidance.

I develop the platform and Arduino framework for GigaDevice GD32, and the way I develop the framework is quite simple: First, fork the core on Github (so GitHub - espressif/arduino-esp32: Arduino core for the ESP32 in your case). Then, create a new firmware project that uses this fork, by using platform_packages.

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino
platform_packages =
   framework-arduinoespressif32@https://github.com/yourusername/arduino-esp32.git

after building it once (which better be successfull, otherwise the core is broken), the downloaded fork will appear in <home folder>\.platformio\packages\framework-arduinoespressif32@src-<some hash>.

Just drag-and-drop that folder (through the file explorer) into the VSCode files sidebar and choose “Add Folder to Workspace”. Then, to get that good intellisense, copy the .vscode folder from the firmware project into the root of the framework-arduinoespressif32@src-.. folder (but don’t push that into the git).

You can now change source code at will, recompile the firmware project to test the changes, and also open a new terminal in the framework folder and do git commands to push the changes et cetara (or use the VSCode GUI for that).

There are more than one ways to do this though. If you want to use a local folder, you can also tell PlatformIO to source the framework from a symlink://. It’s important to not use file:// here (docs), because that would just cause PlatformIO to copy it into its pacakges folder and not use the folder in-place.

platform_packages =
   framework-arduinoespressif32@symlink://C:\dev\myfork
1 Like

Thanks @maxgerhardt for this detailed description and the extra tips! :ok_hand:
This has helped me a lot. :+1:

Advanced question:
The arduinoespressif32 framework is based on the espressif-idf. Via a (for me complicated) process the ESP-IDF is integrated in parts into the arduino framework. The ESP-IDF itself is also partly taken from external repositories which are integrated into the ESP-IDF (e.g. the Rainmaker repository GitHub - espressif/esp-rainmaker: ESP RainMaker Agent for firmware development).

Is it possible to map the whole route / process from an external IDF repository (Rainmaker) through the ESP-IDF to the arduino-espressif32-framework locally?

1 Like

Huh, I was just about to write how PlatformIO in general has this capability of using an ESP-IDF base project and adding Arduino-ESP32 on top of it as a (ESP-IDF) component, but that this isn’t working at the moment because of non updated packages (Support for IDF 4.4 release · Issue #710 · platformio/platform-espressif32 · GitHub), but just 17 hours ago that was actually fixed, looking at platform-espressif32/examples/espidf-arduino-blink at develop · platformio/platform-espressif32 · GitHub.

With this fixed you can use the standard ESP-IDF 4.4.1 package and the Arduino-ESP32 package to truly build the firmware from the ground up.

Can you download the above repository but use as the platformio.ini

[env:esp32dev]
platform = https://github.com/platformio/platform-espressif32.git
framework = arduino, espidf
board = esp32dev
build_flags =
    -D CONFIG_BLINK_GPIO=2
monitor_speed = 115200

(this needs the bleeding-edge version of platform-espressif32 to work).

Once you can build it with the default packages, you can again use platform_packages to change framework-arduinoespressif32 and framework-espidf at will, while adding any components you like.

Well technically GitHub - espressif/esp-rainmaker: ESP RainMaker Agent for firmware development isn’t ESP-IDF, it’s a component for ESP-IDF, so you would add that into the components folder of the PlatformIO project like the example would and reference it in the CMakeLists.txt.

Another note would that, when you want to modify Arduino-ESP32 with changed ESP-IDF settings without building ESP-IDF on every build, you have to use GitHub - espressif/esp32-arduino-lib-builder to regenerate those static libraries that Arduino-ESP32 is using. When using the framework = arduino, espidf combination, those libraries are ignored.

1 Like

Wow! Once again, thank you for your very competent and fast reply! :ok_hand:
There are so many information contained that I have to absorb and process.
This will certainly bring me a good step further.
It looks like I have a lot to try and test :star_struck:

Sonnige Grüße aus Schleswig-Holstein!