Pio remote agent: "Python.h: No such file or directory"

This error is generated as I try to run the “pio remote agent start” command on a Raspberry Pi 3B+ running DietPi (i.e. debian 11.1) in 32-bit mode:

    building 'twisted.test.raiser' extension
    creating build/temp.linux-armv7l-3.9
    creating build/temp.linux-armv7l-3.9/src
    creating build/temp.linux-armv7l-3.9/src/twisted
    creating build/temp.linux-armv7l-3.9/src/twisted/test
    arm-linux-gnueabihf-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/build/python3.9-jS0VHk/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/build/python3.9-jS0VHk/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/jcw/.platformio/penv/include -I/usr/include/python3.9 -c src/twisted/test/raiser.c -o build/temp.linux-armv7l-3.9/src/twisted/test/raiser.o
    src/twisted/test/raiser.c:4:10: fatal error: Python.h: No such file or directory
        4 | #include "Python.h"
          |          ^~~~~~~~~~
    compilation terminated.
    error: command '/usr/bin/arm-linux-gnueabihf-gcc' failed with exit code 1

The next two errors complain about missing "ffi.h" and "opensslv.h" headers.
And last but not least: the Rust compiler must be present (a 295 MB download, incl dependencies).

This can all be solved by running the following command:

sudo apt install libpython3-dev libffi-dev libssl-dev rustc cargo

… but I thought I’d mention it here. Maybe a note could be added to the documentation (or maybe I overlooked it?).

These dependencies all seem to originate from the cryptography package:

If you want to compile cryptography yourself you’ll need a C compiler, a Rust compiler, headers for Python (if you’re not using pypy ), and headers for the OpenSSL and libffi libraries available on your system.

Lots of “Skipping wheel build for XYZ, due to binaries being disabled for it.” messages - presumably because there were no pre-compiled packages for this setup? Since there was a lot of compilation involved on the 32-bit RasPi3 board, the entire build took quite a long time: well over an hour.

Whoops … spoke too soon:

Successfully installed idna-3.3
WARNING: Target directory /home/jcw/.platformio/packages/contrib-pysite/idna-3.3.egg-info already exists. Specify --upgrade to force replacement.
WARNING: Target directory /home/jcw/.platformio/packages/contrib-pysite/idna already exists. Specify --upgrade to force replacement.
Error: Traceback (most recent call last):
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/platformio/__main__.py", line 129, in main
    cli()  # pylint: disable=no-value-for-parameter
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/platformio/commands/__init__.py", line 44, in invoke
    return super(PlatformioCLI, self).invoke(ctx)
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/platformio/commands/remote/command.py", line 58, in remote_agent_start
    from platformio.commands.remote.client.agent_service import RemoteAgentService
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/platformio/commands/remote/client/agent_service.py", line 24, in <module>
    from platformio.commands.remote.client.base import RemoteClientBase
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/platformio/commands/remote/client/base.py", line 30, in <module>
    from platformio.commands.remote.factory.ssl import SSLContextFactory
  File "/home/jcw/.platformio/penv/lib/python3.9/site-packages/platformio/commands/remote/factory/ssl.py", line 16, in <module>
    from OpenSSL import SSL  # pylint: disable=import-error
  File "/home/jcw/.platformio/packages/contrib-pysite/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import crypto, SSL
  File "/home/jcw/.platformio/packages/contrib-pysite/OpenSSL/crypto.py", line 17, in <module>
    from OpenSSL._util import (
  File "/home/jcw/.platformio/packages/contrib-pysite/OpenSSL/_util.py", line 6, in <module>
    from cryptography.hazmat.bindings.openssl.binding import Binding
  File "/home/jcw/.platformio/packages/contrib-pysite/cryptography/hazmat/bindings/openssl/binding.py", line 170, in <module>
    _verify_package_version(cryptography.__version__)
  File "/home/jcw/.platformio/packages/contrib-pysite/cryptography/hazmat/bindings/openssl/binding.py", line 159, in _verify_package_version
    raise ImportError(
ImportError: The version of cryptography does not match the loaded shared object. This can happen if you have multiple copies of cryptography installed in your Python path. Please try creating a new virtual environment to resolve this issue. Loaded python version: 3.1.1, shared object version: b'3.4.8'

Did you install PlatformIO via the installer script method or via a global sudo -H pip3 install platformio?

Ah, that could be it - it’s been a while, I see that $HOME/.platformio/penv/bin/pio is where it currently lives, with my id as owner - so my guess would be the installer script.

Perhaps get rid of pio and reinstall from scratch? Which option should I choose?

PS. I did see that two different versions of cryptography were both being installed when pio remote agent start was installing stuff. I can do a fresh run and report the full log here, if that’s useful.