PlatformIO Community

Always full rebuild allegedly after V4.0.0 - is my platformio.ini file deficient?

Hi There,

Platform IO used to always work as expected, only building the files that were altered or dependent upon an edited file.
Of late it is always re-building all my source files despite only changing some lines of source in one file.

The only real change I can think of is the new V4.0.0 core (since updated to V4.0.2).

Is my platform IO file deficient in some area, causing the full rebuild?

; PlatformIO Project Configuration File ; ; Build options: build flags, source filter ; Upload options: custom upload port, speed and extra flags ; Library options: dependencies, extra library storages ; Advanced options: extra scripting ; ; Please visit documentation for the other options and examples ; https://docs.platformio.org/page/projectconf.html

[env:esp32dev]
platform = espressif32
lib_extra_dirs = ~/Documents/Arduino/libraries
board = esp32dev
framework = arduino
board_build.partitions = min_spiffs.csv
upload_speed = 921600
upload_port = COM16
upload_protocol = esptool
monitor_speed = 115200
extra_scripts = post:add_CRC.py
; replace shitty Arduino millis with a linear time version
build_flags =
-Wl,–wrap,millis

I don’t see anything there that would trigger a rebuild. And I’m not seeing that behaviour on 4.0 or later (also running 4.0.2 atm)… although I’m mainly doing espressif8266 stuff atm, not espressif32.

Just to be clear, you aren’t making changes to platformio.ini when this happens? As that is also a trigger for a full rebuild. And what OS/IDE is this with?

What do you do here? if you generate random or different build flags, PlatformIO will rebuild these files because build environment changes.

Hi Ivan,

I’m adding a CRC across the generated .bin file so when it gets uploaded by others there is an extra level of confirmation the file is really meant for the device.

No I am no changing platformIO.ini, its file date is always older than the source files.

I’m using Windows10, VSCode

Thanks, Ray

FWIW, I have also pulled my project to another PC, just in case it was something to do with my PC.
Still exhibits the same rebuild of all the project files - noting the IDF only gets done once.
Yes I did try the build twice :slight_smile:

The timestamp on platformIO.ini does not change.
I also tried commenting out my post: - made no difference.

I have read the other instances of full rebuild in the issues and cannot understand why my problem is persisting. :frowning:

Good! See how we calculate a project checksum https://github.com/platformio/platformio-core/blob/develop/platformio/project/helpers.py#L168

Could you help us to debug this issue? Please open PlatformIO IDE Terminal and type:

python -c "from platformio.project import helpers; print(helpers)"

So, you will see a path where is located helpers.py file (not PYC). So, add before this line https://github.com/platformio/platformio-core/blob/develop/platformio/project/helpers.py#L193

        if WINDOWS:  # case insensitive OS
            chunks_to_str = chunks_to_str.lower()
        checksum.update(hashlib_encode_data(chunks_to_str))

    print("Checksum = %s" % checksum.hexdigest())
    return checksum.hexdigest()

Now, try to build project a few times. Does checksum change?

Hello Ivan,

It appears the checksum stays the same, always.
One thing I have just noticed though is the Build: (platformio.exe run) seems to behave OK.
Whereas Upload: (platformio.exe run --target upload) is always rebuilding my files, but not the IDF.

THEN, the next time I did Build:, it rebuilt all my source files, then was happy again after that first build.
Hopefully that is useful to you :slight_smile:

I have pasted the checksums of a few builds, and commented against each line:

Build: Checksum = 69f678a8d4a118cea67084fca5087c4d8af52468 - rebuilt everything, including IDF files
Build: Checksum = 69f678a8d4a118cea67084fca5087c4d8af52468 - did not rebuild after dependency check!
Build: Checksum = 69f678a8d4a118cea67084fca5087c4d8af52468 - same, no rebuild
.
Changed some source
.
Build: Checksum = 69f678a8d4a118cea67084fca5087c4d8af52468 - rebuilt changed file!
Upload: Checksum = 69f678a8d4a118cea67084fca5087c4d8af52468 - rebuilt ALL MY source files - not IDF
Upload: Checksum = 69f678a8d4a118cea67084fca5087c4d8af52468 - rebuilt ALL MY source files - not IDF - AGAIN!
Upload: Checksum = 69f678a8d4a118cea67084fca5087c4d8af52468 - rebuilt ALL MY source files - not IDF - AGAIN!
.
Build: Checksum = 69f678a8d4a118cea67084fca5087c4d8af52468 - rebuilt ALL MY source files - not IDF - AGAIN!
Build: Checksum = 69f678a8d4a118cea67084fca5087c4d8af52468 - did not rebuild after dependency check!

Could you comment extra_scripts = post:add_CRC.py in platformio.ini. Does it work now?

1 Like

It is already commented out :smiley:

Could you try this simple example https://github.com/platformio/platform-espressif32/tree/develop/examples/arduino-blink ? Do you have issue with it?

No, that appears to do the right thing, only rebuilds if I edit blink.cpp.

My project does have have quite a few source files.

Could you comment other additional options in platformio.ini just to check which triggers project rebuilding? For example, board_build.partitions

Sadly no.

I have reduced it down to the following, any more and I cannot use upload!
Still rebuilds all the files when I use Upload.
As I mentioned before, works OK when I just Build.

[env:esp32dev]
platform = espressif32
;lib_extra_dirs = ~/Documents/Arduino/libraries
board = esp32dev
framework = arduino
;board_build.partitions = min_spiffs.csv
upload_speed = 921600
;upload_port = 192.168.20.40
;upload_protocol = espota
upload_port = COM16
upload_protocol = esptool
;upload_flags =
; --port=3232
;monitor_speed = 115200
;extra_scripts = post:add_CRC.py
; replace shitty Arduino millis with a linear time version
;build_flags =
; -Wl,–wrap,millis

Could you upload your project somewhere?

Hello Valeros,

it is the same project you looked at in regards to “the linker line is too long” from me a couple of months back.
Have since implemented you suggestions and streamlined the directory structures.

Ray

OK, I’ve found your project on Gitlab and didn’t notice any problems: the project is compiled exactly once then I can build, upload without recompiling. BTW, what is your OS?

Windows 10, VS Code.

Hmmmm, prompted by your success, I pulled the repo to a new directory and it now builds correctly there!

There are other non source files and directories floating about that are not in in the repo.
Is it possible platformio is looking at things other than the src, lib and data directories?

I’ll try pruning some stuff to elsewhere for now…

1 Like

I removed the solution as it is still not clear why the full rebuild takes place.
What is unsettling platform IO it is surely more concerning, and if I can determine that I will certainly then post a true solution. :slight_smile:

Well that didn’t last long, now even the new clone is exhibiting the same behaviour.

One thing I did do was add a new library to the /lib directory:
async-mqtt-client & AsyncTCP

Back doing full builds again :frowning: