Adding to this solution.
In my case it was a multi step process.
I was trying to debug an ELEGOO ARDUINO UNO R3 following the avr-stub on my windows machine.
I encountered this issue first and then this other issue right away.
My solution was to change the Arduino COM port from 5 to 100 like explained on this post.
And then, in the platformio.ini file, I had to configure the debug port like this debug_port = \\.\COM100.
Then closed all open PlatformIO windows (and serial monitors) ; un-plugged + re-plugged Arduino into USB and tried to debug.
Worked like a charm.