Issue Resolved.
My 6 pin USB to Serial card has a DTR signal. (This is used to good effect on the HV UPDI programmer to give a 12v pulse to the UPDI pin to kick it into HV mode - but it is not applicable here with this ISP programmer).
In this ISP configuration, the DTR signal is dropping very briefly the first time a connected session sends an AVRDUDE command and my Pro Mini does a reset. Doh! It must miss a bunch of the AVRDUDE traffic during this reset.
When it gets going, it send some responses but they are mis formed and AVRDUDE throws an error and aborts.
If a subsequent AVRDUDE command is sent, DTR does not drop so there is no reset and the AVRDUDE interaction happens successfully.
My solution was to slug the reset pin on the Pro Mini with a large value electrolytic capacitor (33uF) so the very brief drop in DTR would not have time to bring the reset pin low enough to trigger a reset.
I also found I needed a similar sized capacitor on the Vcc supply to the target devices for decoupling to guard against incorrect fuse readings. It can’t be too big (mine is 33uF) or the specified Vcc power on time for HV programming will not be met. Life is a compromise. (BTW - I do a lot of ‘select on test’).
But now I need to press the reset button on my Pro Mini briefly to reprogram this - maybe that’s a good thing to stop it accidentally being re-programmed as something else.
But time now to join the Christmas celebrations…