PlatformIO Community

Something like `make -jN` for building?

I know this is an old thread but its 2019, I am using PIO 3.6.4 on Visual Studio, and compiling ESP-IDF projects is insanely slow and painful as it still compiles every single source file individually, how do I enable parallel building on this version?

I even screen captured the compile time, 1 min 53 seconds

1 Like

PlatformIO automatically detects a number of processors and enables parallel builds. You can open the system Task manager and see it in action.

To check how many CPUs are available for a build process, please open terminal and type:

python -c "from multiprocessing import cpu_count; print(cpu_count())"

So your saying I just have to live with 2 min build times when build clean? I have 8 cores on my system ;).

What’s the native ESP-IDF performance with make -j8 for the same project configuration?

How do you add the -j command using platformio?

No, I mean, the native ESP-IDF. From their build system, as instructed in https://github.com/espressif/esp-idf/ adn https://docs.espressif.com/projects/esp-idf/en/latest/get-started/index.html#installation-step-by-step

1 Like

No idea, the native build system is totally different then PlatformIO’s, and my project is very massive and doesn’t have the component files that native IDF uses.

This is so fast 2 minutes for 800Mbytes of source files. I’ve never seen so massive SDK as ESP-IDF. Windows XP distribution has less size :metal:

1 Like

Have you run up a task manager to check what the processor core usage is? Unless you fiddle with the platformio.ini file, does platformio not only rebuild modified files? Meaning that subsequent builds are much, much faster?

I can confirm looking at task manager it is compiling in parallel, but the annoying part is that any new file, removing of a file, or even changing platformio.ini will cause a clean compile, and re-compile the entire IDF when really I see no reason that the IDF should have to be compiled unless you downloaded an update or modified something in the core. Most other make tools for ESP do this, you only get a single compile of the IDF libraries which is then linked against.

Changing the platformio.ini always triggers a re-compile, so it’s best to avoid doing that where possible. As to why adding/removing other files would trigger a re-compile I don’t know the reasoning for, but it does happen. If you simply remove the #include of another file it won’t do a complete re-compile, so it seems adding/removing files is considered a significant enough change to trigger a rebuild… whether it is needed or not. It would be nice to see if that could be made a bit smarter.

Hi,
I have the same problem and observation. I’m rather sure PlatformIO use one core in my compilation. I modified ini file and added this:

build_flags =
j=8

This option, as I expected, dramaticaly increase speed of compilation. But proces crashes during library linking. So two steps forward and one back.

See Something like `make -jN` for building?

How many CPUs does Python print?

python -c "from multiprocessing import cpu_count; print(cpu_count())"

Python said that 8……………

hmmm… But it was on different computer. I’ll check once again it.

Hi, how i can set only one job (-j1) in a multihread build?

i have some error in the code but i cant find witch fail when build it in pio because is builded in miltithread mode (40threads in my machine). i think set “-j1” when build i can able depure the code

any hint? I cant find it in the documentation

greetings

Implemented in PlatformIO Core 4.0. See http://docs.platformio.org/en/latest/userguide/cmd_run.html#cmdoption-platformio-run-j

Hi, that is implemented in the rc releases?, Because i use platformio rc builds (used rc3 or rc4 when wrote the post)

Greetings

PlatformIO Core 4.0 has been just released => https://github.com/platformio/platformio-core/releases

Please switch to the stable version via pio upgrade.

oh!. great news

greetings