But PIO project configurator error

cosmin.todor · May 13, 2024, 3:50pm

I’m trying to run the following configuration:

[env]
platform = espressif32
platform_packages =
framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32#3.0.0-rc1
framework-espidf @ ~3.50101.0
board = arduino_nano_esp32
framework = arduino
build_type = debug

But it seems there is something wrong:

KeyError: 'framework-arduinoespressif32-libs':
  File "C:\Users\myuser\.platformio\penv\Lib\site-packages\platformio\builder\main.py", line 173:
    env.SConscript("$BUILD_SCRIPT")
  File "C:\Users\myuser\.platformio\packages\tool-scons\scons-local-4.7.0\SCons\Script\SConscript.py", line 612:
    return _SConscript(self.fs, *files, **subst_kw)
  File "C:\Users\myuser\.platformio\packages\tool-scons\scons-local-4.7.0\SCons\Script\SConscript.py", line 279:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "C:\Users\myuser\.platformio\platforms\espressif32\builder\main.py", line 368:
    target_elf = env.BuildProgram()
  File "C:\Users\myuser\.platformio\packages\tool-scons\scons-local-4.7.0\SCons\Util\envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "C:\Users\myuser\.platformio\penv\Lib\site-packages\platformio\builder\tools\piobuild.py", line 61:
    env.ProcessProgramDeps()
  File "C:\Users\myuser\.platformio\packages\tool-scons\scons-local-4.7.0\SCons\Util\envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "C:\Users\myuser\.platformio\penv\Lib\site-packages\platformio\builder\tools\piobuild.py", line 121:
    env.BuildFrameworks(env.get("PIOFRAMEWORK"))
  File "C:\Users\myuser\.platformio\packages\tool-scons\scons-local-4.7.0\SCons\Util\envs.py", line 251:
    return self.method(*nargs, **kwargs)
  File "C:\Users\myuser\.platformio\penv\Lib\site-packages\platformio\builder\tools\piobuild.py", line 347:
    SConscript(env.GetFrameworkScript(name), exports="env")
  File "C:\Users\myuser\.platformio\packages\tool-scons\scons-local-4.7.0\SCons\Script\SConscript.py", line 676:
    return method(*args, **kw)
  File "C:\Users\myuser\.platformio\packages\tool-scons\scons-local-4.7.0\SCons\Script\SConscript.py", line 612:
    return _SConscript(self.fs, *files, **subst_kw)
  File "C:\Users\myuser\.platformio\packages\tool-scons\scons-local-4.7.0\SCons\Script\SConscript.py", line 279:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "C:\Users\myuser\.platformio\platforms\espressif32\builder\frameworks\arduino.py", line 36:
    SConscript(
  File "C:\Users\myuser\.platformio\packages\tool-scons\scons-local-4.7.0\SCons\Script\SConscript.py", line 676:
    return method(*args, **kw)
  File "C:\Users\myuser\.platformio\packages\tool-scons\scons-local-4.7.0\SCons\Script\SConscript.py", line 612:
    return _SConscript(self.fs, *files, **subst_kw)
  File "C:\Users\myuser\.platformio\packages\tool-scons\scons-local-4.7.0\SCons\Script\SConscript.py", line 279:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "C:\Users\myuser\.platformio\packages\framework-arduinoespressif32\tools\platformio-build.py", line 40:
    FRAMEWORK_LIBS_DIR = platform.get_package_dir("framework-arduinoespressif32-libs")
  File "C:\Users\myuser\.platformio\penv\Lib\site-packages\platformio\platform\_packages.py", line 32:
    pkg = self.get_package(name)
  File "C:\Users\myuser\.platformio\penv\Lib\site-packages\platformio\platform\_packages.py", line 29:
    return self.pm.get_package(spec or self.get_package_spec(name))
  File "C:\Users\myuser\.platformio\penv\Lib\site-packages\platformio\platform\_packages.py", line 21:
    owner=self.packages[name].get("owner"),

Do you have any idea what could be wrong?
Thank you.

sivar2311 · May 13, 2024, 4:45pm

What’s the reason for this?

If it is a standard Arduino project: remove these lines.

cosmin.todor · May 13, 2024, 5:04pm

I set framework-arduinoespressif32 = arduino-esp32#3.0.0-rc1 because of this issuse:

github.com/espressif/esp-idf

heap_caps_check_integrity doesn’t find overflows (IDFGH-11050)

opened 03:10PM - 12 Sep 23 UTC

closed 07:44AM - 25 Oct 23 UTC

MattiasTF

Type: Bug Status: Done Resolution: NA

### Answers checklist. - [X] I have read the documentation [ESP-IDF Programmi…ng Guide](https://docs.espressif.com/projects/esp-idf/en/latest/) and the issue is not addressed there. - [X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there. - [X] I have searched the issue tracker for a similar issue and not found a similar issue. ### IDF version. v4.4.5-400-gc57d30e1df ### Operating System used. Linux ### How did you build your project? VS Code IDE ### If you are using Windows, please specify command line type. None ### Development Kit. Custom Board ### Power Supply used. USB ### What is the expected behavior? I expected the family of heap_caps_check_integrity functions to detect heap corruption, such as buffer overflows, when CONFIG_HEAP_POISONING_LIGHT=y is set. ### What is the actual behavior? Heap buffer overflows are not detected by any heap_caps_check_integrity function and only a free() will trigger a check that aborts the program. ### Steps to reproduce. My project contains the following code to deliberately cause a buffer overflow on the heap: ``` uint32_t *ptr = static_cast<uint32_t *>(heap_caps_malloc(1 * sizeof(uint32_t), MALLOC_CAP_DEFAULT)); printfln("array at %p", static_cast<void *>(ptr)); printfln("canaries before corruption: 0x%08x 0x%08x", ptr[-2], ptr[1]); ptr[-2] = 0xdeaddead; ptr[ 0] = 0xff1234ff; ptr[ 1] = 0xdeaddead; printfln("canaries after corruption: 0x%08x 0x%08x", ptr[-2], ptr[1]); bool valid = heap_caps_check_integrity_all(true); printfln("heap_caps_check_integrity_all returned valid=%i", valid); heap_caps_free(ptr); ``` This is the output when run: ``` array at 0x3f80dca0 canaries before corruption: 0xabba1234 0xbaad5678 canaries after corruption: 0xdeaddead 0xdeaddead heap_caps_check_integrity_all returned valid=1 CORRUPT HEAP: Bad head at 0x3f80dc98. Expected 0xabba1234 got 0xdeaddead assert failed: multi_heap_free multi_heap_poisoning.c:259 (head != NULL) ``` * Allocate memory for one uint32_t. * Deliberately over- and underflow the buffer. * heap_caps_check_integrity_all claims that the heap is valid. * heap_caps_free() detects the corruption. Printing the memory before and after the buffer shows that the canaries are present and free() detects the corruption, which means that the heap is properly poisoned and the detection works, but heap_caps_check_integrity_all() does not. ### Debug Logs. _No response_ ### More Information. `multi_heap_poisoning.c` contains `multi_heap_internal_check_block_poisoning`, which has code to check the poisoning of in-use blocks. In `multi_heap.c`, this function is used in the `tlsf_check_hook`. However, in `heap_tslf.c`, this hook is only called while checking the free lists and never for any in-use blocks. Therefore, corruption in in-use blocks is not detected. The `tlsf_check_hook` can be added to the `integrity_walker` to check in-use blocks, similar to how it is called in `tlsf_check`. The check will fail, however, because the address of the block to check is incorrectly calculated in `multi_heap_internal_check_block_poisoning`. `sizeof(poison_head_t)` is added to the start address even though the start address is already positioned after the poison head. This was probably not found because the code never runs. If the start address is directly passed to `tlsf_check_hook` without the offset, the heap validation works as expected and both the buffer under- and overflow from the test code are detected. My project is incompatible with IDF 5.x, so I cannot check if the issue is still present there, but looking at the relevant files on master shows that `tlsf_check_hook` is also called only during the free list verification and the calculation of the block address also contains the probably incorrect offset, so I expect both issues to be present there as well. [Here is a proof-of-concept patch](https://github.com/espressif/esp-idf/files/12674361/heap_caps_check_integrity.patch.gz) that includes both changes mentioned above. Edit: The proof-of-concept patch has been updated to print more useful information when a corruption was found.

and of this comment

sivar2311 · May 13, 2024, 5:27pm

Try the tasmota fork. This has Arduino 3.x implemented. See

cosmin.todor · May 13, 2024, 6:46pm

Txs. I’ll try.

But now there is another problem.
I’ve clean folder “c:\Users\myuser.platformio\packages” and created a dummy project from scratch.

[env:arduino_nano_esp32]
platform = espressif32
board = arduino_nano_esp32
framework = arduino

and I’m not able to build it:

c:/users/myuser/.platformio/packages/toolchain-xtensa-esp32s3/bin/../lib/gcc/xtensa-esp32s3-elf/12.2.0/../../../../xtensa-esp32s3-elf/bin/ld.exe: .pio/build/arduino_nano_esp32/FrameworkArduinoVariant/dfu_callbacks.cpp.o:(.literal._ZN12_GLOBAL__N_110MD5Builder8getCharsEPc+0x0): undefined reference to `(anonymous namespace)::HEXBuilder::bytes2hex(char*, unsigned int, unsigned char const*, unsigned int)'
c:/users/myuser/.platformio/packages/toolchain-xtensa-esp32s3/bin/../lib/gcc/xtensa-esp32s3-elf/12.2.0/../../../../xtensa-esp32s3-elf/bin/ld.exe: .pio/build/arduino_nano_esp32/FrameworkArduinoVariant/dfu_callbacks.cpp.o:(.literal._ZN12_GLOBAL__N_110MD5Builder12addHexStringEPKc+0x0): undefined reference to `(anonymous namespace)::HEXBuilder::hex2bytes(unsigned char*, unsigned int, char const*)'
c:/users/myuser/.platformio/packages/toolchain-xtensa-esp32s3/bin/../lib/gcc/xtensa-esp32s3-elf/12.2.0/../../../../xtensa-esp32s3-elf/bin/ld.exe: .pio/build/arduino_nano_esp32/FrameworkArduinoVariant/dfu_callbacks.cpp.o: in function `(anonymous namespace)::MD5Builder::getChars(char*)':
C:/Users/myuser/.platformio/packages/framework-arduinoespressif32/cores/esp32/md5builder.cpp:92: undefined reference to `(anonymous namespace)::HEXBuilder::bytes2hex(char*, unsigned int, unsigned char const*, unsigned int)'      
c:/users/myuser/.platformio/packages/toolchain-xtensa-esp32s3/bin/../lib/gcc/xtensa-esp32s3-elf/12.2.0/../../../../xtensa-esp32s3-elf/bin/ld.exe: .pio/build/arduino_nano_esp32/FrameworkArduinoVariant/dfu_callbacks.cpp.o: in function `(anonymous namespace)::MD5Builder::addHexString(char const*)':
C:/Users/myuser/.platformio/packages/framework-arduinoespressif32/cores/esp32/md5builder.cpp:39: undefined reference to `(anonymous namespace)::HEXBuilder::hex2bytes(unsigned char*, unsigned int, char const*)'

Also on “Problem” window, PlatformIO reports:

Cannot find “C:/Users/myuser/.platformio/packages/framework-arduinoespressif32/tools/esp32-arduino-libs/esp32s3/include/esp_system/port/soc”

sivar2311 · May 13, 2024, 6:53pm

But that’s not the Tasmota Fork.
Didn’t you require Arduino 3.x ?

I haven’t tried it because I’ll wait for official support.

cosmin.todor · May 13, 2024, 7:08pm

“Didn’t you require Arduino 3.x ?” Yes, but there where some build issues and I tried to make a simple project using default settings.

I managed to build the “dummy” project with

[env:arduino_nano_esp32]
platform = espressif32
board = arduino_nano_esp32
framework = arduino

only after I cleared the

c:\Users\myuser.platformio.cache
c:\Users\myuser.platformio\packages
folders.

cosmin.todor · May 13, 2024, 7:14pm

For the same dummy project I’ve changed to

platform=https://github.com/tasmota/platform-espressif32.git#Arduino/IDF5

and there is the same link error:

c:/users/myuser/.platformio/packages/toolchain-xtensa-esp32s3/bin/…/lib/gcc/xtensa-esp32s3-elf/12.2.0/…/…/…/…/xtensa-esp32s3-elf/bin/ld.exe: .pio/build/arduino_nano_esp32/FrameworkArduinoVariant/dfu_callbacks.cpp.o:(.literal._ZN12_GLOBAL__N_110MD5Builder8getCharsEPc+0x0): undefined reference to `(anonymous namespace)::HEXBuilder::bytes2hex(char*, unsigned int, unsigned char const*, unsigned int)' c:/users/myuser/.platformio/packages/toolchain-xtensa-esp32s3/bin/../lib/gcc/xtensa-esp32s3-elf/12.2.0/../../../../xtensa-esp32s3-elf/bin/ld.exe: .pio/build/arduino_nano_esp32/FrameworkArduinoVariant/dfu_callbacks.cpp.o:(.literal._ZN12_GLOBAL__N_110MD5Builder12addHexStringEPKc+0x0): undefined reference to ` (anonymous namespace)::HEXBuilder::hex2bytes(unsigned char*, unsigned int, char const*)’
c:/users/myuser/.platformio/packages/toolchain-xtensa-esp32s3/bin/…/lib/gcc/xtensa-esp32s3-elf/12.2.0/…/…/…/…/xtensa-esp32s3-elf/bin/ld.exe: .pio/build/arduino_nano_esp32/FrameworkArduinoVariant/dfu_callbacks.cpp.o: in function `(anonymous`.............

cosmin.todor · May 13, 2024, 7:20pm

Now, even if I rollback the platform to platform=espressif32 the project dependencies still point to Arduino3.x

see

sivar2311 · May 13, 2024, 7:24pm

As far as I know, not everything is implemented in the Tasmota fork.
I wait for the official support or use the ArduinoIDE until then.

Sure, it will use the latest available version, which is still the Tasmota fork on your system.

Either you provide a specific version
platform = espressif32 @ 6.6.0
or you remove the platforms except Espressif32 version 6.6.0:

Click on PlatformIO Icon and choose “PIO Home / Platforms”
Then remove all Espressif32 frameworks except Version 6.6.0