Esp_idf: Menuconfig fails with CodecRegistryError

Hi PlatformIO Community,

I wanted to start a project where I use esp-idf directly without going through Arduino. When trying to setup Bluetooth, I ran into the obnoxious problem of the copmiler not finding bt.h. The solution quoted on the forum is to run platformio run -t menuconfig and enable the respective configuration options. Unfortunately, running Menuconfig fails for me with the following output (German Windows 10 Pro, PlatformIO upgraded & updated to whatever is latest):

PS C:\Users\Damian\Documents\GitHub\WeatherEsp32> & C:\Users\Damian\.platformio\penv\Scripts\platformio.exe run -t menuconfig
Processing esp32doit-devkit-v1 (platform: espressif32; board: esp32doit-devkit-v1; framework: espidf)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif32/esp32doit-devkit-v1.html
PLATFORM: Espressif 32 1.12.4 > DOIT ESP32 DEVKIT V1
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
 - framework-espidf 3.40001.200521 (4.0.1)
 - tool-cmake 3.16.4
 - tool-esptoolpy 1.20600.0 (2.6.0)
 - tool-idf 1.0.1
 - tool-mconf 1.4060000.20190628 (406.0.0)
 - tool-ninja 1.9.0
 - toolchain-esp32ulp 1.22851.190618 (2.28.51)
 - toolchain-xtensa32 2.80200.200226 (8.2.0)
Reading CMake configuration...
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 0 compatible libraries
Scanning dependencies...
No dependencies
Building in release mode
Running menuconfig...
[0/1] CMakeFiles\menuconfig-5f29641.bat bd8a8efa19fdd265
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = 'C:\Users\Damian\.platformio\penv\Scripts\platformio.exe;c:\users\damian\.platformio\penv\scripts\python27.zip;c:\users\damian\.platformio\penv\DLLs;c:\users\damian\.platformio\penv\lib;c:\users\damian\.platformio\penv\lib\plat-win;c:\users\damian\.platformio\penv\lib\lib-tk;c:\users\damian\.platformio\penv\scripts;C:\Users\Damian\.platformio\python27\Lib;C:\Users\Damian\.platformio\python27\DLLs;C:\Users\Damian\.platformio\python27\Lib\lib-tk;c:\users\damian\.platformio\penv;c:\users\damian\.platformio\penv\lib\site-packages'
  program name = 'python'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = 'C:\\Program Files (x86)\\Python38-32\\python.exe'
  sys.base_prefix = 'C:\\Program Files (x86)\\Python38-32'
  sys.base_exec_prefix = 'C:\\Program Files (x86)\\Python38-32'
  sys.executable = 'C:\\Program Files (x86)\\Python38-32\\python.exe'
  sys.prefix = 'C:\\Program Files (x86)\\Python38-32'
  sys.exec_prefix = 'C:\\Program Files (x86)\\Python38-32'
  sys.path = [
    'C:\\Users\\Damian\\.platformio\\penv\\Scripts\\platformio.exe',
    'c:\\users\\damian\\.platformio\\penv\\scripts\\python27.zip',
    'c:\\users\\damian\\.platformio\\penv\\DLLs',
    'c:\\users\\damian\\.platformio\\penv\\lib',
    'c:\\users\\damian\\.platformio\\penv\\lib\\plat-win',
    'c:\\users\\damian\\.platformio\\penv\\lib\\lib-tk',
    'c:\\users\\damian\\.platformio\\penv\\scripts',
    'C:\\Users\\Damian\\.platformio\\python27\\Lib',
    'C:\\Users\\Damian\\.platformio\\python27\\DLLs',
    'C:\\Users\\Damian\\.platformio\\python27\\Lib\\lib-tk',
    'c:\\users\\damian\\.platformio\\penv',
    'c:\\users\\damian\\.platformio\\penv\\lib\\site-packages',
    'C:\\Program Files (x86)\\Python38-32\\python38.zip',
    'C:\\Program Files (x86)\\Python38-32\\DLLs',
    'C:\\Program Files (x86)\\Python38-32\\lib',
    'C:\\Program Files (x86)\\Python38-32',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
  File "c:\users\damian\.platformio\penv\lib\encodings\__init__.py", line 123
    raise CodecRegistryError,\
                            ^
SyntaxError: invalid syntax

Current thread 0x00001e84 (most recent call first):
<no Python frame>
Batch file failed at line 3 with errorcode 1
FAILED: CMakeFiles/menuconfig 
CMakeFiles\menuconfig-5f29641.bat bd8a8efa19fdd265
ninja: build stopped: subcommand failed.
Error: Couldn't execute 'menuconfig' target.
scons: *** [menuconfig] Explicit exit, status 1
=============================================================================================================== [FAILED] Took 8.13 seconds ===============================================================================================================

This error occurs no matter what shell I am using to run it. Options are PowerShell, cmd.exe, Git Bash. I found no mentions of the problem related to PlatformIO. However, general Google results for the CodecRegistryError point towards the wrong python installation being used. I do indeed have another systemwide Python 3.8 installation I use for other projects. When I run python from the command line, the system-wide installation is used.

I know that PlatformIO comes with its own Python 2.7 distribution. I cannot find that distribution on my PATH and frankly would like to keep it that way. Things run from the general command line should run using the system-wide installation under my personal control, not the separate Python distribution that is used to make PlatformIO work. However, I would have expected calls made through platformio.exe to use the “correct” Python enviornment - whatever Platformio.exe determines what that would be. From the above log, it appears to me that Platformio.exe is making subsequent calls using my systemwide python 3.8, despite listing the PlatformIO-integrated Python paths just a few lines above.

What can I do to fix this problem and make run -t menuconfig work?

Regards
Damian