Something fishy is going on in the USB adapter. You are powering the target ESP board separately from the debug probe right? The debug probe probably can’t power it on it’s own… Also try another USB port as Manuel said. Further, there are adapter speed settings which can influence stability. See topic ESP32 + J-link debugger does not start - #12 by Yourigh.