Problems like these have been obseved with the JLink + USB-enabled microcontrollers for a long time, e.g., Problems starting debug session with jlink on feather M0 and Hang Starting nRF52840 Feather Express Under Segger Debugger - #10 by geeksville.
As far as I know, nobody ever figured out why that happens in this configuration. Other microcontroller combinations with JLink do not have that problem.