Using latest esp-idf (4.3.1) in platformio.ini

I am trying to use the latest esp-idf 4.3.1 by tweaking my platformio.ini file which currently picks up esp-idf 4.3.0 by default. I am not familiar with the necessary steps and runes to do so.

My failing platformio.ini file:

[platformio]
src_dir = main

[env:esp32dev]
;platform = espressif32
platform = https://github.com/platformio/platform-espressif32.git                                                                                                                                                                                                                                                           
platform_packages = framework-espidf @ https://github.com/espressif/esp-idf#v4.3.1                                                                                                                                                                                                                                          
board = esp32dev
board_build.partitions = partitions.csv
framework = espidf
monitor_speed = 115200

I was hoping that this would pull esp-idf 4.3.1 and compile it. But I end up with an early build error since the framework-espidf directory is not fully populated:

$ pio run 
Processing esp32dev (platform: https://github.com/platformio/platform-espressif32.git; board: esp32dev; framework: espidf)
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html
PLATFORM: Espressif 32 (3.3.2+sha.1e0f147) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-espidf 1.1.0+sha.2e74914 
 - tool-cmake 3.16.4 
 - tool-esptoolpy 1.30100.210531 (3.1.0) 
 - tool-ninja 1.7.1 
 - toolchain-esp32ulp 1.22851.191205 (2.28.51) 
 - toolchain-xtensa32 2.80400.210211 (8.4.0)
Reading CMake configuration...
-- Configuring incomplete, errors occurred!
See also "/home/ov/work/CO2Ampel/sensor-devices/esp32/espidf_co2/bleprph_wifi_coex/.pio/build/esp32dev/CMakeFiles/CMakeOutput.log".

CMake Error at CMakeLists.txt:5 (include):
  include could not find load file:

    /home/ov/.platformio/packages/framework-espidf/tools/cmake/project.cmake

Do I need to override anything else? Or is it not that trivial to achieve this?

It is missing an ESP-IDF file here. This is most likely due to the fact that you can’t just specify the source of framework-espidf to be from their github repo. In order for something to be usable as a package (in platform_packages), it needs to have a package.json metainfo file at the top level. See here for a more in-depth explanation.

Try forking the ESP-IDF repo, going to the v4.3.1 branch and adding a package.json to it, akin to the one found in https://github.com/maxgerhardt/esp-idf/blob/master/package.json but with corrected version info. All that can be comfortably done via the Github GUI (switch branch, “Add file → Upload file”)

@maxgerhardt I tried your suggested method, and I get the following error:

Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html
PLATFORM: Espressif 32 (3.5.0) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES: 
 - framework-espidf 1.1.0+sha.bb29e93 
 - tool-cmake 3.16.4 
 - tool-esptoolpy 1.30100.210531 (3.1.0) 
 - tool-ninja 1.7.1 
 - toolchain-esp32ulp 1.22851.191205 (2.28.51) 
 - toolchain-riscv32-esp 8.4.0+2021r2-patch2 
 - toolchain-xtensa-esp32 8.4.0+2021r2-patch2 
 - toolchain-xtensa-esp32s2 8.4.0+2021r2-patch2
Reading CMake configuration...
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring incomplete, errors occurred!
See also "/home/name/Documents/ThingsboardCode/ESP32_ENC28J60_Static_IP/.pio/build/esp32dev/CMakeFiles/CMakeOutput.log".

CMake Error at CMakeLists.txt:3 (include):
  include could not find load file:

    /home/name/.platformio/packages/framework-espidf/tools/cmake/project.cmake

I’ve added the package.json to my fork of the compiler. Is there something I am missing?

Thanks

I will post a new example targeting ESP-IDF v4.4 later.

I would appreciate that, thanks.