How can I set and keep the Arduino Stack size at increased size?

1

I need the size of Arduino’s loopTask stack larger than the default. What I currently do is to change the main.cpp file (~/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp) like this:

#ifndef CONFIG_ARDUINO_LOOP_STACK_SIZE
#define CONFIG_ARDUINO_LOOP_STACK_SIZE 8192
#endif
// mod by ullix
#define CONFIG_ARDUINO_LOOP_STACK_SIZE 16384
// end mod

The definition of CONFIG_ARDUINO_LOOP_STACK_SIZE is found in file ~/.platformio/packages/framework-arduinoespressif32/tools/sdk/sdkconfig:

#
# Automatically generated file; DO NOT EDIT.
# Espressif IoT Development Framework Configuration
#
.... 
#
# Arduino Configuration
#
...
CONFIG_ARDUINO_LOOP_STACK_SIZE=8192

So, it would be prudent to modify the setting there, but as it says at the beginning, it is auto-generated and will be overwritten, and it really is.

What I could not find out: who is the over-writer? Can I somehow command him to keep my stack setting with anything I can do in pio?

2

Already discussed at ESP32 Stack configuration (reloaded)

3

Impressive to see what kind of hacks can be done with pio. However, given the overall effort this takes, I will keep with simply manually changing the value in file sdkconfig.h. Lasts until the next update :-/

But given that it says: ‘Automatically generated file; DO NOT EDIT.’ I am still wondering about what part of the software does this generation? I searched all code within pio, but couldn’t find any hints.

Is this file even generated within pio, or is it already contained in the package obtained by pio?

4

It’s not generated by PlatformIO, but ESP-IDF. The Arduino-ESP32 core is an extension of ESP-IDF (precompiled ESP-IDF libs in tools/sdk), and the ESP-IDF libraries have been generated with a specific sdkconfig and using the tool GitHub - espressif/esp32-arduino-lib-builder.

For the most part, editing the sdkconfig.h will have no effect since the macros are used inside the ESP-IDF libraries, which are not recompiled during a normal Arduino-ESP32 build. That CONFIG_ARDUINO_LOOP_STACK_SIZE macro however is used in the source code that is recompiled, so the warning can be ignored.

5

Aaah, there it is. Many thanks.

At least good to know I can rely on that little config change (until the next update).

6

Are you sure you don’t just want to solve the problem purely in firmware code? Just create a new FreeRTOS task with the desired bigger stack size and delay(1000); in the old one. Doesn’t need any config file mods and the resulting code will also be compilable for Arudino IDE and PlatformIO users that don’t have this local mod. It wastes a bit of RAM since the old task is now mostly unused (from the very big RAM anyways), but otherwise it’s a quite general solution.

See tutorial.

7

At first glance this sounded like the Sorcerer’s Stone, but it is not.

I am using tasks already, so it was easy to create an extraLoop() which took all the content of loop(), which now got nothing but the delay(1000);. With a few more changes to setup(); it compiled with a loopTask sized to 8k.

However, the loopTask is this:

void loopTask(void *pvParameters)
{
    setup();
    for(;;) {
        if(loopTaskWDTEnabled){
            esp_task_wdt_reset();
        }
        loop();
        if (serialEventRun) serialEventRun();
    }
}

So, setup() is part of the loopTask. And once setup() fishes, its share of stack is released, and becomes available for loop(). But now with loop() being empty, and all its former content being in extraLoop(), the 8k for loopTask stack are blocked and inaccessible!

Originally I set CONFIG_ARDUINO_LOOP_STACK_SIZE to 16k and all ran fine. Now I need 8k for loopTask(), plus 11k for extraLoop().

In addition, to squeeze my setup() into 8k, I also had to remove some SafeStrings from local in setup() to global. Those take another 4k of memory. Not sure how to free this memory.

So, in total I now use 8k + 11k + 4k = 23k compared to the previous 16k. Doesn’t look like progress?

8

Not if you vTaskDelete(NULL); in loop() to kill the original loopTask (API ref). That kills the task and the task’s stack (which was originally allocated on the heap) is returned back to the heap.

Of course the

        if(loopTaskWDTEnabled){
            esp_task_wdt_reset();
        }

logic has to be done in the new loop task then too.

9

I did that, but deleted loopTask from within the newly created extraLoop task. Got some strange responses and went back. It feels like pulling the rug from under me.

Now you say to do it in loop(), as the first and only ever executed command in loop()? The new extraLoop task has a life of its own and will not be touched by this in any way?

Thanks for the reminder for the WDT!

10

But you’d have to do that with the task handle for the loopTask task then as an argument to vTaskDelete, right?

Exactly. I haven’t tested it yet, but it should work that way, given that the watchdog is being kept fed.

11

Sure, I did use the proper task handle. I believe, I have now figured out, what problem I have had.

It is unfortunate that when you do a vTaskDelete the handle itself is NOT NULLed, but continues to live and points to nowhere. When you then use some of the RTOS functions on this handle, you are lucky when your program crashes, but it may continue delivering nonsense numbers! As, I think, it was in my case.

With a vTaskDelete(NULL) you can’t execute anything after this command, so you’d have to NULL the handle before, like in this example (from the RTOS site) :

TaskADelete(){
/* Remember the value of the variable as it will be set to NULL. */
TaskHandlet xTask = taskAHandler;

vTaskSuspendAll();
if( taskAHandler != NULL ){
    /* The task is going to be deleted.    Set the handle to NULL. */
    taskAHandler = NULL;
    /* Delete using the copy of the handle. */
    vTaskDelete( xTask );
}
xTaskResumeAll();
}

Note the suggested use of Suspend and Resume. If one wanted to use them, one could never delete a task from within the task, but must always do it from outside, because the resume would otherwise never be executed!

I my case, I would have to do the task deletion in the newly created extraLoop(), and neither in setup() nor in loop().

12

If I try this

#include <Arduino.h>
#include "esp_task_wdt.h"

extern bool loopTaskWDTEnabled;
xTaskHandle loopTask2Handle;
extern TaskHandle_t loopTaskHandle;
#define LOOPTASK_STACK_SIZE (64 * 1024)

void printStack()
{
  char *SpStart = NULL;
  char *StackPtrAtStart = (char *)&SpStart;
  UBaseType_t watermarkStart = uxTaskGetStackHighWaterMark(NULL);
  char *StackPtrEnd = StackPtrAtStart - watermarkStart;
  Serial.printf("=== Stack info ===\n");
  Serial.printf("Free Stack near start is:  %d \r\n", (uint32_t)StackPtrAtStart - (uint32_t)StackPtrEnd);
}

void setup2()
{
  Serial.println("New setup2() with bigger stack reached!");
  printStack();
}

void loop2()
{
  delay(1000);
  Serial.println("Looping in loop2()..");
}

void loopTask2(void *pvParameters)
{
  //remap in case other tasks want to refernece us :)
  loopTaskHandle = loopTask2Handle;
  enableLoopWDT();
  setup2();
  for (;;) {
    if (loopTaskWDTEnabled) {
      esp_task_wdt_reset();
    }
    loop2();
    if (serialEventRun)
      serialEventRun();
  }
}

void setup()
{
  Serial.begin(115200);
  Serial.println("Firmware start!");
  printStack();
}
void loop()
{
  Serial.println("Executing loop()!");
  //create new task before killing..
  xTaskCreateUniversal(loopTask2, "loopTask2", LOOPTASK_STACK_SIZE, NULL, 1, &loopTask2Handle, CONFIG_ARDUINO_RUNNING_CORE);
  Serial.println("Killing thread now.");
  disableLoopWDT();
  vTaskDelete(NULL);
  //everything beneath this will never be reached.
}

with the standard

[env:esp32]
platform = espressif32
board = esp32dev
framework = arduino
monitor_speed = 115200

it works just fine, as I thought.

Firmware start!
=== Stack info ===
Free Stack near start is:  7524
Executing loop()!
Killing thread now.
New setup2() with bigger stack reached!
=== Stack info ===
Free Stack near start is:  65012
Looping in loop2()..
Looping in loop2()..
Looping in loop2()..
Looping in loop2()..
Looping in loop2()..
Looping in loop2()..
Looping in loop2()..
Looping in loop2()..
Looping in loop2()..

Edit: Added code to remap original task handle.

13

It wasn’t quite clear from my statement, but I referred to the case where you wanted to use vTaskSuspendAll and xTaskResumeAll; then you’d need the kill from outside. May be relevant only by more complex cases, though, not here.

Duplicating setup() too, darn, good things come in small packages. This frees me from using global definitions, and all can be left on the stack and reused by loop2. Great. I’ll try it.

Can you explain the purpose of serialEventRun?

14

The Arduino core has a mechanism that firmwares can implement that function and in there check for incomming serial data. See https://www.arduino.cc/en/Tutorial/BuiltInExamples/SerialEvent. But it’s really not any different from doing that as the first thing inside loop(). By default, the function is weak and not defined (aka, nullptr). I’ve only included it because the original loopTask() does the same.

15

My code is running well now, with setup() being completely empty, and loop() containing nothing but the creation of the new task with a big stack, plus the deletion of the original loopTask.

But one thing needs to be observed: you better do NOT activate the WDT for the looptask! In Arduino main.cpp the flag loopTaskWDTEnabled is set to false and remains so. The functions enableLoopWDT() and disableLoopWDT() are found in file esp32-hal-misc.c, but those are used nowhere within the whole Arduino framework!

extern "C" void app_main()
{
    loopTaskWDTEnabled = false;
    initArduino();
    xTaskCreateUniversal(loopTask, "loopTask", CONFIG_ARDUINO_LOOP_STACK_SIZE, NULL, 1, &loopTaskHandle, CONFIG_ARDUINO_RUNNING_CORE);
}

And sure enough, when you enable the WDT, your better have a short loop. In my case WiFi activation, taking ~2.5 sec, takes long enough to always trigger WDT crashes. Not sure what the WDT setting is; doesn’t seem to be CONFIG_TASK_WDT_TIMEOUT_S 5 in sdkconfig.

Overall, a somewhat surprising solution to my original question :-). Thanks.

But I am wondering whether I have now created a new problem: the original looptask was created first, then the new one, then the old deleted. Does it not mean that I have now created a hole in the heap, which - without garbage collection - may be haunting me later?

16

I don’t think so because the (normal) task creation and execution is pretty much the last step in the initialization, so it will have allocated the block, then de-allocated it when the task was killed, and a the new task stack was allocated on the heap. When nothing else allocated something in between the start address of the new stack will be the same as the old one. One can also check that approximately by outputting the stackpointer (aka the address of a variable close to the beginning of the stack) and seeing where they are (in my example printing SpStart with %p within printStack()).

And even if something did come in between and allocated more memory, there will initially be an 8KByte hole, but the heap allocated wil still use that for objects <= 8kByte, so it will be filled again.

17

I stumbled over this discovery, and I am surprised because I thought to have understood from the previous discussion that this is impossible, namely to define an sdkconfig.h value from a pio ini setting:

build_flags =
    	-D CONFIG_ARDUINO_LOOP_STACK_SIZE=33333

Though there is a catch: when I keep the setting in the sdkconfig.h file like this:

#define CONFIG_ARDUINO_LOOP_STACK_SIZE 8192

the compiler floods me with warning: "CONFIG_ARDUINO_LOOP_STACK_SIZE" redefined, but at the end keeps the original setting and ignores the ‘redefinition’.

When I comment out this line in the sdkconfig.h file, the setting in pio’s ini is used. But now the use of CONFIG_ARDUINO_LOOP_STACK_SIZE is marked with a squiggly line as identifier "CONFIG_ARDUINO_LOOP_STACK_SIZE" is undefined (I guess by the language server?).

So one can set parameters in sdkconfig.h from the ini file?

18

If it’s defined in the build flags it should not be marked with a squiggly line. Try rebuilding the IntelliSense with Ctrl+Shift+P → Rebuild IntelliSense.

19

Too bad I can’t test the rebuilding, because I had to restart vscode. This, however, seems to have the same effect. The squiggles are gone. There will be a next time to test …

Looks like the build_flags are actually defined first in the whole chain, and the sdkconfig.h comes later. Which is why the compiler warns everytime the Arduino.h is included, and overwrites the build_flag setting.

So, one has to modify the sdkconfig.h to either delete the respective entry, or set it to only define it if it does not yet exist. If you forget, the compiler warns you! Not too bad.