Always full rebuild allegedly after V4.0.0 - is my platformio.ini file deficient?

Hmmmm, prompted by your success, I pulled the repo to a new directory and it now builds correctly there!

There are other non source files and directories floating about that are not in in the repo.
Is it possible platformio is looking at things other than the src, lib and data directories?

I’ll try pruning some stuff to elsewhere for now…

1 Like

I removed the solution as it is still not clear why the full rebuild takes place.
What is unsettling platform IO it is surely more concerning, and if I can determine that I will certainly then post a true solution. :slight_smile:

Well that didn’t last long, now even the new clone is exhibiting the same behaviour.

One thing I did do was add a new library to the /lib directory:
async-mqtt-client & AsyncTCP

Back doing full builds again :frowning:

Hmmmm, perhaps this is more a VScode issue.

In my cloned repo, that I started working with as it was building properly for a while, I noted that were some of the new library files opened in the tabs. I closed them, and now the rebuild is once again as expected.

Sadly I now cannot replicate by re-opening them - PIO is building as expected again.
One was “half opened” - the italic title - ie not a sticky tab, a single click from the file list.
Have done both the single and double click of a couple lib files but not upsetting PIO yet.

So, perhaps VScode is doing a save every time I use Upload despite the files not actually being edited, causing the full rebuild?

I’ll see if I can note when it goes wrong again.

Thanks for your continued patience :slight_smile:

I would have thought it would only trigger a rebuild of that library, and possibly others it touches though… or at least that has been my experience when altering library files… it doesn’t seem to trigger a total rebuild… just the affected files, which is understandable… as even if it was a ‘no changes save’… the timestamp has still been modified…

I have found a scenario that definitely leads to a full rebuild on my machine for an Upload build.

If the COM port used for esptool is in use during the dependency check, a full rebuild gets undertaken.

You can abort that with CTRL-C in the console, close the terminal package (Putty) using the COM port, then retry the Upload Build.
A few files will be rebuilt as they were likely underway during the prior aborted build, but then only the true files are built, linked and upload proceeds.

And obviously if Putty was closed prior - a normal build process works as well!

Hope that is useful to the developers :slight_smile:

When you say putty, do you mean miniterm? As that’s the serial monitor used by platformio.

Does this happen if you have say there serial monitor open (i.e. Monitor task) and then hit upload? As the monitor task should be halted, and then the build as needed + upload occur, and finally the serial monitor re-opened. Or does it need the serial port to be locked by an external application - i.e. putty.

In other words, is it something going wrong with platformio itself, or is this interaction from a external tool?

It is an external putty client. I find the internal one frustrating to use.

1 Like

But why should a terminal COM port being in use affect the build of source files?
I get the failed upload, but file rebuilds?

1 Like

That I can’t tell you… but now at least this can be tested with the serial port locked by another program to see why that is interacting with the build process. Even with the upload process, you would only expect the upload to fail, not for it to interact with the build stage. Plus the failed upload should leave the files in a state where they don’t need to be rebuilt on a retry anyway.

I don’t have any ESP32s free at present, and don’t normally use the ESP-IDF, but did quickly check doing a couple of builds of the ESP-IDF blink example, and doing several builds and uploads with and without the upload_port in use or not didn’t seem to do anything unexpected… i.e. no rebuilds were triggered. more something like a 7 second delay whilst the source files were scanned for changes, and that was it. That’s about all I can do unless you think it’s somewhat related to your project, and that should what’s compiled…

It seems too much of a co-incidence, but I setup PIO on a laptop recently and found it was using Python 3.7 and was working well.
So I updated my PC to Python 3.7 and ever since then PIO has rebuilt correctly every time…

So is the Python version critical to this issue? Obviously 3.7 is the way of the future…

Hi rljonesau,
I am also experiencing the same error as you. How to fix them
Thanks

Same problem, I updated to Python 3.8 but no change.
A full rebuild of mbed is triggered for the smallest change.

My observations :

  • Full rebuild is only triggered by “upload” and “debug”, not by “build”.
  • Even if no files are modified a full rebuild is triggered on “upload” and “debug”, in this case the checksum doesn’t change

VS Code v1.40.1 / Windows 10

CONFIGURATION: Redirecting...
PLATFORM: ST STM32 5.7.0 > ST 32F746GDISCOVERY
HARDWARE: STM32F746NGH6 216MHz, 320KB RAM, 1MB Flash
DEBUG: Current (stlink) On-board (stlink) External (blackmagic, jlink)
PACKAGES: toolchain-gccarmnoneeabi 1.70201.0 (7.2.1), framework-mbed 5.51401.191023 (5.14.1), tool-stm32duino 1.0.2, tool-openocd 2.1000.190707 (10.0), tool-dfuutil 1.9.190708
Collecting mbed sources…
LDF: Library Dependency Finder → Library Dependency Finder (LDF) — PlatformIO latest documentation
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 9 compatible libraries
Scanning dependencies…
Dependency Graph
|--
Building in release mode

Platformio.ini

[platformio]
description = My project
[env:stm32f746ng]
platform = ststm32
board = disco_f746ng
framework = mbed
;test_ignore = test_local
;build_flags = -D PIO_FRAMEWORK_MBED_RTOS_PRESENT
;[env:local]

;platform = native
;test_ignore = test_device

pio --version

PlatformIO, version 4.1.0

pio update

Updating contrib-piohome @ 3.0.1 [Up-to-date]
Updating contrib-pysite @ 2.38.191019 [Up-to-date]
Updating tool-pioplus @ 2.6.0 [Up-to-date]
Updating tool-unity @ 1.20403.0 [Up-to-date]
Updating tool-scons @ 3.30101.0 [Up-to-date]

Platform Manager

Platform Native

Updating native @ 1.1.2 [Up-to-date]

Platform ST STM32

Updating ststm32 @ 5.7.0 [Up-to-date]
Updating toolchain-gccarmnoneeabi @ 1.70201.0 [Up-to-date]
Updating framework-mbed @ 5.51401.191023 [Up-to-date]
Updating tool-stm32duino @ 1.0.2 [Up-to-date]
Updating tool-openocd @ 2.1000.190707 [Up-to-date]
Updating tool-dfuutil @ 1.9.190708 [Up-to-date]

Platform Windows x86

Updating windows_x86 @ 1.1.3 [Up-to-date]
Updating toolchain-gccmingw32 @ 1.50100.0 [Up-to-date]

Library Manager

Library Storage:

This is normal behaviour. We re-build source with debug information if you try to debug board. You can control this, see Build Configurations — PlatformIO latest documentation

1 Like

@ivankravets thanks but is a full build of mbed necessary for two consecutive uploads ?
I can understand if I switch from upload to debug, there needs to be a rebuild.

If I upload twice with no changes to the code it still does a full rebuild.
The build time of mbed is heavy, so you can understand my frustration

This is a bug. Could you provide a simple project to reproduce this issue? @valeros will help.

2 Likes

I am having the same problem. Here’s a script I created to help demonstrate:

#!/bin/sh

TEST_PROJECT_NAME=x

rm -rf "${TEST_PROJECT_NAME}"

mkdir "${TEST_PROJECT_NAME}"
cd "${TEST_PROJECT_NAME}"

# use 'pio init' to create a new project
pio init --board nucleo_l432kc

# change the framework to mbed
cp platformio.ini platformio.ini.SAVE
sed 's/arduino/mbed/g' platformio.ini.SAVE >platformio.ini

# create a minimal main.cpp
echo "#include <mbed.h>"		>src/main.cpp
echo "int main() {}"			>>src/main.cpp

# compile the first time - expect to recompile everything
pio run

After running the script, if I do the following…

cd x
pio run

Sometimes it recompiles all the files and sometimes it doesn’t. If it doesn’t (like I’d expect), all I have to do is execute

pio run

once or twice more before it will eventually recompile all the files.

FYI / FWIW: Although I don’t use it often, I haven’t run into this problem under the Arduino framework.

FYI - I did an analysis of all file changes in the directory.

First, every ‘pio run’ (whether it’s recompiling or not) changes ./.pio/build/nucleo_l432kc/.sconsign.py37.dblite . I’m assuming this is expected.

Then, to see better what else was changing without showing all the recompiled files (*.o, *.d, etc.), I temporarily changed my compiler executables to do nothing (script with ‘exit 0’). When I did ‘pio run’ and it chose to recompile everything, all the *.o files (but not the *.d files) disappeared. I don’t know if this somehow happened before the recompile decision which would understandably cause a recompilation. I suppose it could be, on the other hand, a normal step in the recompilation phase (delete the *.o file then run the compiler).

PS - No other files or timestamps changed. (Except for the directory timestamps where the *.o files were deleted.)

FYI - I also watched all the files listed in the generated *.d files and they weren’t changing either.

The solution is here : Build compiles all source files every time - #3 by nmathon
Hope it helps