No, the SSL client tries to allocate its working memory on the heap, which lives in RAM. Flash is for constant, unchanging data – a firmware ususally never writes into Flash during runtime (needs special re-programming routine anyways), except in the cases of Flash-based filesystems or non-volatile storage where they have to store the data in persistent memory.
For the ESP32 yes, you can buy an ESP32 module which has a built-in PSRAM (pseudo-static RAM) chip extension, such as a ESP32-WROVER-B (adafruit, amazon).
The Preferences library uses a the NVS (non-volatile storage) partition of the flash. It is a pre-allocated section of flash as determined by the partition table (default.csv). The application lives in a separate partition than NVS, so by writing things to NVS, you do not decrease the amount of Flash available to the application by the number of bytes written to the NVS. You do however cause a slight increase in flash usage since you use the Preferences library, and its program code will be stored in Flash.
If you use this different partition table for your application, it shifts the available space for the application, but it does not solve the problem that you’ve run out of RAM.
This behavior is extremely indicative of a memory leak. You can add as much RAM as you physically can, it will just delay the moment in time that this problem occurs. The source of your problem is your code.
It is extremely critical that you clean-up / free resources after using them. If e.g. an object allocates memory using the new opperator or malloc(), but after it’s done using it (or when the object is destroyed) it does not delete or free() the memory again, that chunk of memory will be forever locked up as “used” although no one can possible use the memory again. These errors accumulate over time, say if for every telegram message a connection is made, which dynamically allocates 20 kBytes of memory, but the client is never stopped so it never gives back that 20 kBytes, then you’re losing 20 kByte of dynamic memory in every message. At ~320 kBytes of RAM, a few connection attempts more and your ESP32 is seemingly “out of memory” because of this 20kbyte per message memory leak.
For a SSL/TLS connection with the WiFiClientSecure library it’s imperative that you call the .stop() function on it so that it releases its internal resources.
Depending on what library you’re using, you might either be forgetting to clean up its resources by forgetting to call a function in there (or using the objects of the library in a completely wrong way), or the library itself might have a bug. Check the library’s github page for possible known issues.
You can verify that you have a memory leak by monitoring the amount of free RAM on the heap. Just have your sketch periodically print or send the value of ESP.getFreeHeap() (ESP32 Arduino: Getting the Free Heap - techtutorialsx).
If you see a strictly decreasing curve over time, your system has a memory leak and is slowly starving out memory.
If you can accelerate operation of your firmware (e.g., sending more messages per second or triggering the bot periodically), you should be able to starve it faster / see an effect faster.
Dear maxgerhardt a lot of thanks for your comprehensive answer!
It really helps me to understand what is going on inside of ESP32 memory.
Now I detect ESP.getFreeHeap(), ESP.getMaxAllocHeap(), ESP.getMinFreeHeap() each hours and see that MaxAllocHeap constantly decreases.
Seems I’ve already found the reason for the memory leak.
The script below shows how even a little string formatted with *storePrintf() function eats all RAM
Great, you’ve pin-pointed the exact point of the memory leak! I’m however unsure of the entire urpose of the storePrintf() function… since it just calls printf in a really more complicated way, why is storePrintfnot equivalent to just printf()? If it does not have additional logic besides printing, the function implementation should be removed and there should just be a macro
#define storePrintf printf
instead of the function declaration char *storePrintf(const char *fmt, ...);.
Oh oh sorry I have to rethink that – there are multiple things wrong with that
Since the variable is returned no way you can deallocate it. However, when the function returns a char* and the left side of the expression is a String, that should have… never worked.
The tricky piont here is that you dynamically allocate a string of the required size and put it into a String object, and so when the String object is destroyed it must free that initially malloced memory. The problem is that whatever const char* you give into a String object it always wants to do make an internal copy of it which is again mallocd (see sourcecode) – you can’t initialize a String object from a verbatim pointer that it will again free in the destructor.
You should also be able to change it to
String storePrintf(const char *fmt, ...) {
va_list arg;
va_start(arg, fmt);
size_t sz = snprintf(NULL, 0, fmt, arg);
char *buf = (char *) malloc(sz + 1);
vsprintf(buf, fmt, arg);
va_end(arg);
// copy into string object
String s(buf);
//now the buf content have been duplicated inside the String, destroy it
free(buf);
return s;
}
although your std::string collection also looks valid if you’ve validated it…
With the return types its still extremely funny though, because you return a const char*, that was created from a std::string, which is somehow auto-converted into a String, although all these 3 things are different. But okay.
Probably because there’s a String constructor that works with const char* but not std::string. If you have verified working code, it should be it good though.