Have you tried it? If you just overwrite the old files (overwrite, do not delete the old content, especially not framework-zephyr/scripts/platformio/), it could work.
A cleaner way to override packages is explained in How to use latest zephy build / rc.
I’ve also opened issue Update Zephyr to 2.6.0 · Issue #541 · platformio/platform-ststm32 · GitHub so that PlatformIO will update, so this shouldn’t be a problem for too long.