Is there a step-by-step for ESP32arduino with ESPIDF?

I’ve been working at this for a couple of days. My project runs under ESP32/Arduino. Was easy to setup and update as needed. Now I need to make changes to the underlying ESP-IDF config so trying to setup Arduino running on ESP-IDF.

I had no problems setting up VSCode/ESP-IDE with Arduino but my project has so many libraries and dependencies and the CMakeFiles is just too complicated and rigid.

I’ve spent a couple of days trying to just get a simple project to compile, but just feel clueless. This is what PlatformIO is supposed to eliminate. Could someone point me at or give me a step by step procedure for creating this environment? Everything I see is years old.

UPDATE: This post was somewhat helpful in that I added the version to my platform spec and the errors changed.

So, now I get several exceptions/errors that I have no idea how to resolve:

> Processing esp32dev (platform: espressif32@5.3.0; board: esp32dev; framework: arduino, espidf)
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via `-v, --verbose` option
> CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32dev.html
> PLATFORM: Espressif 32 (5.3.0) > Espressif ESP32 Dev Module
> HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
> DEBUG: Current (cmsis-dap) External (cmsis-dap, esp-bridge, esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
> PACKAGES:
>  - framework-arduinoespressif32 @ 3.20006.221224 (2.0.6)
>  - framework-espidf @ 3.40403.0 (4.4.3)
>  - tool-cmake @ 3.16.4
>  - tool-esptoolpy @ 1.40400.0 (4.4.0)
>  - tool-idf @ 1.0.1
>  - tool-mconf @ 1.4060000.20190628 (406.0.0)
>  - tool-ninja @ 1.9.0
>  - toolchain-esp32ulp @ 1.23500.220830 (2.35.0)
>  - toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5
> Installing ESP-IDF's Python dependencies
> Collecting idf-component-manager~=1.0
>   Using cached idf_component_manager-1.2.3-py3-none-any.whl (116 kB)
> Collecting future (from idf-component-manager~=1.0)
>   Using cached future-0.18.3-py3-none-any.whl
> Collecting packaging (from idf-component-manager~=1.0)
>   Using cached packaging-23.1-py3-none-any.whl (48 kB)
> Collecting requests<3 (from idf-component-manager~=1.0)
>   Using cached requests-2.31.0-py3-none-any.whl (62 kB)
> Collecting urllib3<2 (from idf-component-manager~=1.0)
>   Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
> Collecting requests-file (from idf-component-manager~=1.0)
>   Using cached requests_file-1.5.1-py2.py3-none-any.whl (3.7 kB)
> Collecting requests-toolbelt (from idf-component-manager~=1.0)
>   Using cached requests_toolbelt-1.0.0-py2.py3-none-any.whl (54 kB)
> Collecting schema (from idf-component-manager~=1.0)
>   Using cached schema-0.7.5-py2.py3-none-any.whl (17 kB)
> Collecting six (from idf-component-manager~=1.0)
>   Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
> Collecting tqdm<5 (from idf-component-manager~=1.0)
>   Using cached tqdm-4.65.0-py3-none-any.whl (77 kB)
> Collecting colorama (from idf-component-manager~=1.0)
>   Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB)
> Collecting pyyaml>5.2 (from idf-component-manager~=1.0)
>   Using cached PyYAML-6.0-cp311-cp311-win_amd64.whl (143 kB)
> Collecting cachecontrol[filecache]>0.12.6 (from idf-component-manager~=1.0)
>   Using cached cachecontrol-0.13.1-py3-none-any.whl (22 kB)
> Collecting contextlib2>0.6.0 (from idf-component-manager~=1.0)
>   Using cached contextlib2-21.6.0-py2.py3-none-any.whl (13 kB)
> Collecting click (from idf-component-manager~=1.0)
>   Using cached click-8.1.3-py3-none-any.whl (96 kB)
> Collecting msgpack>=0.5.2 (from cachecontrol[filecache]>0.12.6->idf-component-manager~=1.0)
>   Using cached msgpack-1.0.5-cp311-cp311-win_amd64.whl (60 kB)
> Collecting filelock>=3.8.0 (from cachecontrol[filecache]>0.12.6->idf-component-manager~=1.0)
>   Using cached filelock-3.12.2-py3-none-any.whl (10 kB)
> Collecting charset-normalizer<4,>=2 (from requests<3->idf-component-manager~=1.0)
>   Using cached charset_normalizer-3.1.0-cp311-cp311-win_amd64.whl (96 kB)
> Collecting idna<4,>=2.5 (from requests<3->idf-component-manager~=1.0)
>   Using cached idna-3.4-py3-none-any.whl (61 kB)
> Collecting certifi>=2017.4.17 (from requests<3->idf-component-manager~=1.0)
>   Using cached certifi-2023.5.7-py3-none-any.whl (156 kB)
> Installing collected packages: msgpack, urllib3, six, pyyaml, packaging, idna, future, filelock, contextlib2, colorama, charset-normalizer, certifi, tqdm, schema, requests, click, requests-toolbelt, requests-file, cachecontrol, idf-component-manager
>   Attempting uninstall: msgpack
>     Found existing installation: msgpack 1.0.5
>     Uninstalling msgpack-1.0.5:
>       Successfully uninstalled msgpack-1.0.5
>   Attempting uninstall: urllib3
>     Found existing installation: urllib3 1.26.16
>     Uninstalling urllib3-1.26.16:
>       Successfully uninstalled urllib3-1.26.16
>   Attempting uninstall: six
>     Found existing installation: six 1.16.0
>     Uninstalling six-1.16.0:
>       Successfully uninstalled six-1.16.0
>   Attempting uninstall: pyyaml
>     Found existing installation: PyYAML 6.0
>     Uninstalling PyYAML-6.0:
>       Successfully uninstalled PyYAML-6.0
>   Attempting uninstall: packaging
>     Found existing installation: packaging 23.1
>     Uninstalling packaging-23.1:
>       Successfully uninstalled packaging-23.1
>   Attempting uninstall: idna
>     Found existing installation: idna 3.4
>     Uninstalling idna-3.4:
>       Successfully uninstalled idna-3.4
>   Attempting uninstall: future
>     Found existing installation: future 0.18.3
>     Uninstalling future-0.18.3:
>       Successfully uninstalled future-0.18.3
>   Attempting uninstall: filelock
>     Found existing installation: filelock 3.12.2
>     Uninstalling filelock-3.12.2:
>       Successfully uninstalled filelock-3.12.2
>   Attempting uninstall: contextlib2
>     Found existing installation: contextlib2 21.6.0
>     Uninstalling contextlib2-21.6.0:
>       Successfully uninstalled contextlib2-21.6.0
>   Attempting uninstall: colorama
>     Found existing installation: colorama 0.4.6
>     Uninstalling colorama-0.4.6:
>       Successfully uninstalled colorama-0.4.6
>   Attempting uninstall: charset-normalizer
>     Found existing installation: charset-normalizer 3.1.0
>     Uninstalling charset-normalizer-3.1.0:
>       Successfully uninstalled charset-normalizer-3.1.0
> ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'C:\\Users\\Bob\\.platformio\\penv\\Lib\\site-packages\\~.arset_normalizer\\md.cp311-win_amd64.pyd'
> Check the permissions.
> 
> *** Error 1
> Warning! Arduino framework as an ESP-IDF component doesn't handle the `variant` field! The default `esp32` variant will be used.
> Reading CMake configuration...
> -- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.41.0.windows.1") 
> -- The C compiler identification is GNU 8.4.0
> -- The CXX compiler identification is GNU 8.4.0
> -- The ASM compiler identification is GNU
> -- Found assembler: C:/Users/Bob/.platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/xtensa-esp32-elf-gcc.exe
> -- Check for working C compiler: C:/Users/Bob/.platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/xtensa-esp32-elf-gcc.exe
> -- Check for working C compiler: C:/Users/Bob/.platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/xtensa-esp32-elf-gcc.exe -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Detecting C compile features
> -- Detecting C compile features - done
> -- Check for working CXX compiler: C:/Users/Bob/.platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/xtensa-esp32-elf-g++.exe
> -- Check for working CXX compiler: C:/Users/Bob/.platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch5/bin/xtensa-esp32-elf-g++.exe -- works
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> -- Building ESP-IDF components for target esp32
> -- Configuring incomplete, errors occurred!
> See also "C:/Users/Bob/SynologyDrive/Github/ESP32Arduino/.pio/build/esp32dev/CMakeFiles/CMakeOutput.log".
> 
> fatal: not a git repository (or any of the parent directories): .git
> CMake Error at C:/Users/Bob/.platformio/packages/framework-espidf@3.40403.0/tools/cmake/build.cmake:452 (message):
>   C:\Users\Bob\.platformio\penv\Scripts\python.exe: Error while finding
>   module specification for 'idf_component_manager.prepare_components'
>   (ModuleNotFoundError: No module named 'idf_component_manager')
> 
> Call Stack (most recent call first):
>   C:/Users/Bob/.platformio/packages/framework-espidf@3.40403.0/tools/cmake/project.cmake:384 (idf_build_process)
>   CMakeLists.txt:3 (project)
> 
> 
> 
> ================================================================================================ [FAILED] Took 17.41 seconds ================================================================================================
>  *  The terminal process "C:\Users\Bob\.platformio\penv\Scripts\platformio.exe 'run'" terminated with exit code: 1. 
>  *  Terminal will be reused by tasks, press any key to close it.

The OSError installing charset-normalizer is a mystery. The directory and file have normal permissions. I deleted the file and reran and get the same error. Could it have anything to do with the double \ between each node?

I’m getting:
Warning! Arduino framework as an ESP-IDF component doesn't handle the variantfield! The defaultesp32 variant will be used.
No idea what that refers to.
And the final nail in the coffin:
fatal: not a git repository (or any of the parent directories): .git
CMake Error at C:/Users/Bob/.platformio/packages/framework-espidf@3.40403.0/tools/cmake/build.cmake:452 (message):

C:\Users\Bob\.platformio\penv\Scripts\python.exe: Error while finding
  module specification for 'idf_component_manager.prepare_components'
  (ModuleNotFoundError: No module named 'idf_component_manager')

Does that mean anything to anyone?

Oh, and my .ini file:

[env:esp32dev]
platform = espressif32@5.3.0
board = esp32dev
framework = arduino, espidf

Sounds like Creating new ESP-IDF project fails with (ModuleNotFoundError: No module named 'idf_component_manager') message and PlatformIO fails to install python dependencies - #3 by maxgerhardt to me.

Thanks. I had not found those posts. Like the others, I deleted everything and started over. Managed to get 5.1.0 working. Not sure how or what to apply the patch for the others, but 5.1.0 is fine for now.