Cannot get custom test runner to work - running tests on native platform

I am working on a project and I’ve ran into a “weird” problem trying to follow the tutorial at

Every time I try to run the tests, I get
UserSideException: Could not find custom test runner by this path -> <path_to_file>/
where <path_to_file> is the actual path to file that exists. I’ve triple-checked, placed it in different locations of the test hierarchy, the file exists and can be opened in editor if I follow the link in VSCode terminal. I get the exact same error when trying the same from a command line.

My setup is a multi-environment ESP32 project, where the “default” ESP32 environment runs on-chip C++ tests without a problem (other than occasional error in the test or code under test, that is) and I’ve created a second project environment, native platform, with the hope that I could run python pytest tests against the device (the device, among other things, communicates SCPI via serial, so I could use PyVisa to check if I get the expected values and responses). The computer environment is a Fedora 38 (Linux) with Python 3.11 and PlatformIO 6.1.11.

So far I’ve managed to track my problem down to platformio-core/platformio/test/runners/ where the exception is raised (it’s either FileNotFoundError (unlikely) or an ImportError if load_python_module method from platformio-core/platformio/ does not succeed).

Before trying to debug the situation further, I thought I’d better ask if there’s something obvious I’m doing wrong, anybody has any experience with this kind of setup or if there’s a better way to do the thing I want to do - running tests other than C/C++ (pytest) against the device from within Platformio?

Ouch, my fault.

After printing the whole exception in it turns out the problem is that pytest is not accessible from the environment and once that dependency is removed, the gets loaded fine.

Not that I have it all sorted out yet, but at least I know where the current problem with my setup is :slight_smile: