Missing SConscript when using github ci

PlatformIO Core
1

Hi there,

I recently get an error from my ci pipeline in github.
The message is

*** missing SConscript file 'custom_hwids.py'

File "/opt/hostedtoolcache/Python/3.12.1/x64/lib/python3.12/site-packages/platformio/builder/main.py", line 167, in <module>

========================= [FAILED] Took 24.37 seconds =========================

Error: Process completed with exit code 1.

it is started with
run: platformio ci --lib="." --project-conf=examples/knx-usb/platformio-ci.ini examples/knx-usb/src/main.cpp

I have this line in my plattformio.ini
extra_scripts = pre:custom_hwids.py
and this file is in the projects root side-by-side to the ini.

Here is a link to the repo:

when I build it locally, everything is fine.
Until some weeks ago, this worked also on github actions, but suddently (without a change in my repo) not anymore.

I tried to move the custom_hwids.py’ into a “shared” folder but that has no effect…

I also tried to use this path but:

*** missing SConscript file 'examples/knx-usb/custom_hwids.py'
File "/opt/hostedtoolcache/Python/3.12.1/x64/lib/python3.12/site-packages/platformio/builder/main.py", line 167, in <module>
========================= [FAILED] Took 25.58 seconds =========================
Error: Process completed with exit code 1.

Any ideas?

2

Hm, with pio ci you basically create a new temporary project, in your case with a copied platformio.ini and copied main.cpp. Everything else is not copied. That means that temporary project really has no custom_hwids.py in it. But, if you say it worked a few days ago, maybe it’s a regression or behavior change in the PlatformIO core… (https://github.com/platformio/platformio-core)

3

Thanks max,

I tracked it down - 3weeks ago ( I hate github for not giving exact timestamps…) it was still working, 2 weeks ago the first fail with this error.

It seems that there has been changes in this relevant time to something linked with SConscript…

4

Mhh but you’re not using any SCons variable references in your platformio.ini…

If it’s the core that has a regression, try installing an older one to prove it. You have

which would install the latest. Do e.g.

pip install "platformio==6.1.11"

instead. (release history on pypi)

5

Thanks again max !

Now I see:

previously, there ws just a warning that this file was missing. Now its an error.

Transition by adding must_exist=False to SConscript calls.
Missing SConscript 'custom_hwids.py'

but I wonder why - isn’t ci --lib="." copying the whole project into that temp folder?

6

Aha, so you’re actually running into a consequence of a general SCons update, and it never ever executed the .py script in the CI build.

That’s not what it’s documented to do. It would be copied to the lib folder of the new temp project, as the documentation says.

7

yes, but I would really be interested in fixing this issue, so that this extra_script is really executed also in the ci…

where would I specify this “must_exist=False” ?
Its not the ini not an pio ci argument… ? is it even possible?

8

Easiest way would be to change pio ci to pio run -d <example project path>, then everything will stay in-place. I don’t actually see a way of copying arbitrary files to the root of the new temp project with the pio ci command.

This would have been something that would need to be added in the PIO core, aka, the place where the potentially missing SCons script is called, in the constructor of the SConsScript object… not externally changeable.

9

I’m pretty sure my workflow YAML is correct but one of the projects won’t build properly when using the regular platformio.ini, the github page is either private or deleted for the FlashStorage library:

See https://github.com/maxgerhardt/knx/blob/openknx/pipeline/test_dom_2024_01_29/.github/workflows/platform-io.yml

10

this lib is not needed anymore. I’ll remove it.