I am trying to compile Arduino code, but my compiler seems to run out of memory. I’ve googled this issues and others have had it with Qt. But I’ve found no solution for PlatformIO.
512MByte is not much when I have 64GByte available.
How can I solve this?
The error I get:
cc1plus.exe: out of memory allocating 536875007 bytes
Please post the output of pio system info on the PIO CLI as well as the list of packages that is output at the start of compilation.
Okay. That was the first half of what I wanted to know.
I am running out of memory because I am compiling a large firmware that contains lots of images, fonts and much else. I understand that part of it. What I am seeking is the ability to increase the 512MByte that seems to be the limit in this case. There are solutions for Qt online but I’ve found nothing for PlatformIO.
I cannot post a public list of all dependencies I use, that will reveral to much about my project.
I am asking for the packages list, not all library dependencies. The toolchain specifically. E.g.,
PACKAGES:
- framework-arduinoespressif32 @ 3.20011.230801 (2.0.11)
- tool-esptoolpy @ 1.40501.0 (4.5.1)
- toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch5 <--- THIS
Ah my mistake!
PACKAGES:
- framework-arduinoteensy @ 1.158.0 (1.58)
- tool-teensy @ 1.158.0 (1.58)
- toolchain-gccarmnoneeabi-teensy @ 1.110301.0 (11.3.1)
Per https://api.registry.platformio.org/v3/packages/platformio/tool/toolchain-gccarmnoneeabi-teensy the same package is used on 32-bit and 64-bit Windows
So the compiler itself must be a 32-bit executable. And indeed the arm-none-eabi-gcc.exe etc. does have a PE<nul><nul>L in it, so it’s 32-bit. That means that the program can’t even see your 64GB overkill amount of RAM, it has a 4GByte address space, 2 of which is mapped by the Windows kernel, effectively leaving less than 2GB for actual allocations. For big compiler or linking operations, that could run out indeed. (Or your program is heavily malformed, or you are forcing it to link against tons of objects files or precompiled libraries that have humungous size.)
2 ways:
1.) You might be able increase the allocatable space to 3GB using this. (large address aware). But this is a bandaid and will likely fail again in the future.
2.) Force it to use the actual 64-bit toolchain to make full use of your RAM. This makes more sense.
Problem: The Teensy package index (https://www.pjrc.com/teensy/package_teensy_index.json) actually does not list a 64-bit Windows toolchain, only i686-mingw32 (=Windows 32-bit). So Teensyduino falls short here by not providing it.
From experience however, their “teensy” toolchain really is just a Arm GNU arm-none-eabi-gcc (11.3.1) plus the floating point libs for Cortex-M4 etc.
y>C:\Users\Max\Downloads\toolchain-gccarmnoneeabi-teensy-windows-1.110301.0.tar\bin\arm-none-eabi-gcc.exe --version
arm-none-eabi-gcc.exe (Arm GNU Toolchain 11.3.Rel1) 11.3.1 20220712
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Surely, they would have a native 64-bit version of the toolchain? Well, looking at https://developer.arm.com/downloads/-/arm-gnu-toolchain-downloads:
Nope they don’t 
Only 32-bit.
At this point there are two branches:
After very painful experiences lessons in compiler building and working around a few build bugs, I managed to compile the Arm Gnu toolchain natively for 64-bit Windows. It can successfully compile a Teensy Arduino sketch.
I’ve setup an A/B test project at https://github.com/maxgerhardt/pio-teensy-64bit-toolchain.
You have to download https://www.dropbox.com/scl/fi/qot0wtsxh2xk5xzlw6yvp/gnuarm.zip?rlkey=iwl13bs04aank3lrmmxn7wgza&dl=0, extract it somewhere, then adapt the path in the above project’s platformio.ini to the extracted folder.
Does the firmware work the same in both envrionments? If yes, does using the 64-bit toolchain the same way as in the reference project fix your problem?
Wow that was a really comprehensive response, I wasn’t expecting that.
First of all, many thanks for taking the time to do this, I’m impressed!
I tested it and this is what I got:
Note that this is an even smaller firmware, it does actually compile with the ordinary compiler, I can add more images to make it not work like in my original post in this thread.
I can clearly hear my water cooled monste of a PC spin the fans a little faster when I compile with your compiler, that did not happen before, so clearly it’s using more power, which is exactly what we want! 
@maxgerhardt, Any news on this? 
Oh I didn’t notice you were editing your previous answer; that didn’t notify me at all, only new posts do.
4.66 Megabytes of firmware? Okay so first of all, it seems to have generated a .hex file. That’s good at least, that’s the main compiler output after the .elf. Now some encryption tool seems to be failing or detecting that the .hex is malformed.
Then, I don’t see the PlatformIO-provided platform at https://github.com/platformio/platform-teensy/ generating an .ehex at all. Is this custom added? By https://github.com/platformio/platform-teensy/pull/98 maybe?
Max, yes the .ehex is custom added. Perhaps that’s the problem here?
It’s hard to say without seeing the actual files which step goes wrong: Whether the .elf file is generated wrongly, or only the .hex conversion is wrong, or whether there’s an actual bug in the encryption tool that generates the .ehex file.
Question again: Does this compile normally on Linux? If just the 64-bit compiler is the problem, your project should compile fine in a Linux VM or similiar.
I don’t have Linux unfortunately.
I’ll try to disable the .ehex stuff and see.
