I’m using the latest platformio core (6.1.16) and vscode extension versions (v3.3.3). I have an ESP32-S3 project setup using the ESP-IDF framework. Right now it’s still a pretty bare project, just a main function printing some stuff. Anytime I change anything in the platformio.ini file, the configure obviously needs to update but it takes forever to finishing updating the configuration. I’m talking like 45-60+ seconds each time, which really slows down my workflow. Is there anyway to fix this?
I share your experience that PlatformIO is just unpleasantly slow. Steps that helped my relationsip with PIO (actually, that strained relationship is why I come here):
- Wire up ccache
- Use a build cache. Add this to p*ini: build_cache_dir = .pio/build_cache
-
- See if you can share that amongst developers and presubmit Ideall
- Build your include trees responsibly and use tools and design patterns like iwyu and PIMPL to reduce friction by excessive coupling.
- just avoid interacting with pio as much as you can; potentially just spinning compile_commands.jsons and doing builds in other systems and increasing your use of TDD so you’re compiling tiny programs most of the time and the big program as little as you can.
- Don’t feel the need to ‘git pull’ from your team multiple times a day, even if everything IS conflict-free and functional. Just take the hit of build builds less often.
- Interact less with pio. Script 'PLATFORMIO_SERIAL_PORT=blah pio run -e foo`
Steps that did not help:
- Raging at scons and other stupid python code that takes longer to compute dependencies and - even more embarrassingly - retrieve a cached buidl object than just to build it again.
- If you’re using an OS that allows random people to run code on your computer and thus are spending computer power on antivirus, building code tends to be just torture on them. Disabled it as allowed by local policy.
- Screaming at the screen when it spends .5s saying “Using cached copy of foo” when it takes .1 to build.
- Looking at the process tree (top or equiv) and wondering why python is taking way more time than gcc.
- Thinking that a do-nothing build does nothing. On my ESP32 project, I just ran
Environment Status Duration
------------- -------- ------------
FOO SUCCESS 00:00:42.822
================================== 1 succeeded in 00:00:42.822 ==================================
pio run -e demo 24.34s user 7.71s system 72% cpu 44.259 total
The first time is “real,” and I’m not including it. The second time does no useful work. The objects exist, are cached, are built, are linked, and are not being senf over a connection. My MINIMUM cost to touch pio run is between 37 and 44 seconds. It’s interesting that our floors are so close.
Currently under study: adding --disable-auto-clean to the default build flags. I saw it in a recent post. I didn’t really take the time to get my head around it. On one system, there was a positive change in build times, but I just repeated my “null build” case and got 39.348 seconds. I don’t add flags like that lightly as one incorrect build will cost me more time than saving 5% on every other build of the day.
You can throw in a -v flag in the ‘pio run -v build -e’ and see where the time goes
Scanning dependencies… No file has changed. You have nothing to scan, silly!
Building in release mode You’re not building anything! What are you DOING for ten seconds?
My “build everything” case takes about 50 minutes on an M1 mini to ssd, but I’ll accept tha a lot of that 50 minutes is fetching 39 copies of the same files, then being shocked that it has to recompile everything 39 minutes. The 39 seconds above is on a single -e env.
The build system is easily the #1 reason I can’t make friends with platformio. I’ve considered implementing a sidecar build system with cmake/ninja that’s actually fast and keeping the platformio build system for new users not living in a code; build; test loop.
I will, of course, be watching this thread to see results of others.
This kind of defeats the purpose of why I use platformio to begun with. I’ve lost countless hours trying to spin up my own build system and it’s never worth the time and effort
Not relevant? I mentioned this is an empty project already
I’ll give your cache thing a go. I don’t have that much a problem with builds being slow. They are mostly slow on first go or if you change anything in the platformio.ini file because it rebuilds allllllll of the framework dependencies which is a lot for ESP-IDF (and I have no idea why it does this) but that much I can live with. What I can’t live with is the 1-2 minute delay every time I change something in that platformio.ini file where it’s doing…something I actually can’t tell what it could possibly be doing to take that long but it does really bother me
Fair point. I was just reaching for ways to reduce the number of builds.
We can theorize why it rebuilds ESP-IDF. It doesn’t keep a list of symbolds that you might be changing that would impact ESP-IDF, so it runs around rebuilding timers, interrupt handlers, bluetooth stacks, freertos, and a zillion other things any time you change platformio.ini. Add -v ro the build and look at those -I paths—there’s a terrifying amount of symbol leakagae betweens “apps” and “OS” in these builds.
The 44 second null build I showed has literally zero files changed and was ‘time pio run -e foo ; time pio run -e foo’ issued in one command. Even if the code wasn’t using OS facilities of the last 40 years to get notifications of a file change and had to do an ftw and a zillion stats, it should see the dependency tree is fully resolved and do … exactly nothing. Still, it takes 44 seconds to do nothing.
It’s just slow.