Pio painfully slow for several same microcontrollers with slightly different code on Raspberry Pi

I have built my own IoT framework for teaching (ulnoiot), based on platformio. The system runs in class on raspberry pis and manages 4-10 Wemos D1 minis (based on esp8266) all with one different configuration cpp-file. Compiling each of these takes initially 5 minutes, totaling easily 30 minutes class time lost (with explanations and coaching this is more like several hours of class time lost).

As there is only a difference in code of less than 2%, I was wondering how I could re-use or pre-compile some of the used libraries or equivalent C+±files. The only things I could avoid was downloading the libraries for each time in sym-linking the .piolibdeps to a pre-compiled node-folder. Other experiments with sym-links, copies, or extra-libs build options all fail and require a complete re-built.

How can I avoid compiling the same code for the same platform/microcontroller again and again?

Can anybody point me into the right direction for this?

I also have a project where I change a few settings in a .h file, recompile and upload the project. PIO is smart enough that it will only re-compile a minimal set of source files and then upload the project. Recompiling takes about a half to one miniute (depending on how long the LDF takes).

Does it in your case recompile the entire project? Do you modify the platformio.ini?

I am not changing platformio.ini, I use separate projects for each of my nodes. So everything is in different directories. I was just wondering, if I could create some kind of shared pre-compiled libraries.

So your suggestion is keeping everything in one directory and change the one file different before compiling? Hmm, I could eventually do that. This will tough never allow me to do the compilation of all nodes at once.

If you specify multiple environments in the [platformio] section, it will build (and upload) them sequentially (docs).

Maybe parallel builds are something for the future, @ivankravets? :slight_smile:

BTW since you have full control over linker and building flags, you could indeed do some complicated stuff to pre-compile a non-changed library once, then -l into the binary. But within the same project, PIO will already optimize that. That only gives you gains when you use it accross multiple different projects.

1 Like

PlatformIO Build System is based on the powerful SCons tool. You can extend our build environment using SCons instruments. What do you need?

  1. PRE script => Redirecting...
  2. Setting env.CacheDir('/path/to/cache/dir') => SCons 4.4.0

So, you will have 1 global cache directory per different project/build environments.

Does it work?

I implemented @maxgerhardt’s first suggestion. Compile-time is down to one minute (which is still a bit slow but maybe the minimum I can achieve on the raspberry pi). So I just need a lock now to prevent multiple compilations at once. I will look into your suggestions (especially the chap-caching) at a later point. Thanks for your help.