Hi community! I’m still quite new in PIO use, so maybe I just overlooked something.
I’m trying to base my project on FreeRTOS using library FreeRTOS_SAMD21 by BriscoeTech. The problem is, that always after first tick I get stuck in HardFault handler, caused by invalid address in rtosSystick_Handler function pointer. This pointer is supposed to be 0 from start (and therefore not executed) and initialised only later within xPortStartScheduler() to point to xPortSysTickHandler:
Implementation of function sysTickHook is inside file port.c delivered with the library:
void (*rtosSysTick_Handler)(void);
int sysTickHook(void)
{
if (rtosSysTick_Handler)
{
rtosSysTick_Handler();
}
return 0; // return zero to keep running the arduino default handler!
}
In debug, I can make it work by forcing address of that pointer to 0 on first hit of sysTickHook().
When I tried Basic_RTOS_Example compiled and uploaded by Arduino IDE, it worked nice. When I import such project to platformio, problem with HardFault caused by invalid rtosSysTick_Handler() starts happening.
Since the FreeRTOS port defines the weak sysTickHook now, it will get called from the SysTick_Handler. It itself then looks if there are systick hooks defined by the rtosSysTick_Handler function pointer.
A possible problem I see is the time between the Arduino core enabling the systick and the user sketch calling the xPortStartScheduler (or vTaskStartScheduler) that sets the rtosSysTick_Handler variable. There is a timespan where the systick handler function runs but the rtosSysTick_Handler is not yet valid.
I’m not sure why it does not get a 0 value, it should be zero-initialized by the startup file . Can you try setting it to the NULL value explicitly? Change
Even setting it explicitly to 0 doesn’t help, I don’t understand why. Maybe some issue with the startup file you mentioned? Or it could be something with debugger, that doesn’t allow proper initialization? But the issue occurs also when I just upload it without debugger using bootloader.
I get the feeling that it’s being memory corrupted by something. I can suggest two things:
make the example more minimal with only one thread and a more minimal amount of task memory used for it
set a data breakpoint / watchpoint on the rtosSysTick_Handler variable using the GDB shell exposed in the “Debug Console”. See c++ - Can I set a breakpoint on 'memory access' in GDB? - Stack Overflow. Set the debug_init_break to break Reset_Handler to stop the core at the earliest possible point, execute the watch rtosSysTick_Handler command in the debug console and let the code keep running. The first hit should be right in the Reset_Handler loop where it copies the initial values for the variables from Flash to RAM (and the written value should be 0) here. If you let the chip run then (continue) and there are any other writes to this variable that is not the rtosSysTick_Handler = &xPortSysTickHandler; line but in some other random place, there’s a memory corruption happening, possibly due to a FreeRTOS task using too much stack memory while not being given enough in the xTaskCreate function.
Thanks for your suggestions, I will follow it in the evening and let you know the results.
However, I found out that once I set the rtosSysTick_Handler to 0 by debugger, followed by proper initialisation by xPortStartScheduler, it stays set to the value from xPortStartScheduler also after restart. Without power cycle, application works also after upload over boot loader. When I try to start debugging afterwards, it still set to the address from xPortStartScheduler (but not to 0). Only power cycle make the address random again. So this makes me think that it could be not initialised at all on startup. But anyway, I will follow your recommendations regarding simplified example and watching the memory from earliest entry point in Reset_Handler.
EDIT: One more discovery: It seems that sysTick_Hook is called even before the Reset_Handler() can finish initialisation. It was still inside the for loop when the interrupt jumped into sysTick_Hook():
You need to watch writes to the variable, the “Watches” windows in the GUI is just a number of variables whose values are read when the chip is already halted, it doesn’t trigger the chip to halt when a read or write occurs.
Oh that is critical. How do you reset the chip? Pressing the reset button or cutting the power?
It may that your power-cycle method keeps the interrupt for SysTick enabled when it shouldn’t be enabled at all yet? The Reset_Handler should not be interrupted by anything.
Power-cycle method: When uploaded using boot loader, I just disconnect and reconnect the micro usb. Afterwards, it’s not working. When debugging, I guess that debugger does some kind of software reset, but there is my knowledge quite limited yet.
I will make simplified FreeRTOS example as you recommended to see better what’s going on. I will focus on Reset_Handler completeness vs sysTick_Hook call. I will try to check interrupts configuration/status as well, especially SysTick interrupt.
Is there any significant capacitance on the board that could lead to a slow brownout of the supply voltage of the chip and that sort of unclean reset?
The only other thing I can recommend is to 1. modify the Reset_Handler to disable all interrupts as the very first line of executed code, so that it’s not interrupted and 2. add a special variable that needs to have a particular value before the sysTick_Hook function goes further, as in
#define MCU_STARTUP_DONE_CANARY 0x52ae19f1
volatile uint32_t startup_val = 0;
void Reset_Handler() {
//disable all interrupts
startup_val = 0;
//copy loop and BSS zero-init loop
startup_val = MCU_STARTUP_DONE_CANARY;
// calling into main etc
}
void sysTickHook() {
if(startup_val != MCU_STARTUP_DONE_CANARY) {
return; // called too early
}
// rest
}
of course these are workarounds. You need to find out how in the world the MCU can reset but not disable all interrupts an startup.
It seems that disabling sysTick is enough for now to make things work, although it could be better to disable all interrupts.
The strange thing is that even if I let the board disconnected for a minute, it doesn’t start without this workaround (to check if some capacity is the issue). Same with the reset button, it won’t help. I will definitely try to find a root cause and let you know.
For now, I’m at least able to finish first demo application for customer, even with workaround. However, understanding why this is happening is important for getting my trust into MKR WIFI 1010 and to use it as reliable prototyping board.
Thank you very much for your detailed explanations and analysis, without it I won’t move forward!
Last tip: If you can reproduce this behavior with FreeRTOS crashing / SysTick called too early after a reset in the plain Arduino IDE, the people at GitHub - arduino/ArduinoCore-samd: Arduino Core for SAMD21 CPU should accept that as an issue and investigate it for you, as it is also in their interest.
