How to debug a Arduino mega 2560 with ATMEL ICE

PlatformIO IDE
, ,
1

Hi all,
I have a problem debugging the Arduino mega 2560 with an external ATMEL-ICE debugger. I have previously had success -with the help of maxgerhardt- to setup a custom tool for use with the Arduino Due, for the ATSAM3x8E microcontroller. However, the same setup fails with the Atmega 2560, for which I have simply selected “avr32” as a chip name (see platform.ini below). However, the OCD tool seems to expect SWD communication and not JTAG, and it complains about settings for the transport protocol (see terminal error messages down at the bottom of the page).
Could you please help me to set that up? I have managed to upload the code onto the board using the avr-stub, but the debugging is impossible as it complains about the (auto-detected) COM16 port which the debug software does not seem to understand.
Best regards,
Clemens

; PlatformIO Project Configuration File (ATMEL - ICE mode) 

[env:megaatmega2560]
platform = atmelavr
board = megaatmega2560
framework = arduino

debug_tool = custom
upload_protocol = atmel-ice
debug_init_break = tbreak setup
debug_server = 	/Users/Thinkpad/.platformio/packages/tool-openocd/bin/openocd
              	-d2 
              	-s 
              	/Users/Thinkpad/.platformio/packages/tool-openocd/scripts
              	-f 
              	interface/cmsis-dap.cfg 
              	-c "set CHIPNAME avr32" 
              	-c  "source [find target/avr32.cfg]"

***Error message from the terminal, ATMEL-ICE debugger connected

xPack OpenOCD x86_64 Open On-Chip Debugger 0.11.0+dev (2021-10-16-21:19)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
debug_level: 2

avr32 Info : auto-selecting first available session transport "swd". To override use 'transport select <transport>'.embedded:startup.tcl:128: Error: session transport is "swd" but your config requires JTAG

; PlatformIO Project Configuration File (avr-stub mode)
debug_tool = avr-stub

debug_port = COM16
lib_deps = jdolinay/avr-debugger@^1.5
 ;use serial1

build_flags =
   -DAVR8_SWINT_SOURCE=6
   -DAVR8_UART_NUMBER=1
   -AVR8_BREAKPOINT_MODE=1

***Error message from the terminal, avr-stub mode connected

======================== [SUCCESS] Took 11.18 seconds =========================
Reading symbols from g:\Shared drives\Technical Development\Engineering\Development\ePATH Display\Git Repositories\Z1002 Tester\.pio\build\megaatmega2560\firmware.elf...
done.
PlatformIO Unified Debugger -> https://bit.ly/pio-debug
PlatformIO: debug_tool = avr-stub
PlatformIO: Initializing remote target...
.pioinit:13: Error in sourced command file:
COM16: No such file or directory.
2

I don’t think OpenOCD is capable of 8-bit AVR chips at all? AVR32 is a 32-bit architecture, not applicable for the Mega2560.

AVR hardware debugging with open-source tools is a hard topic because those tools aren’t provided by Microchip and the public ones are not guaranteed to work and still have problems.

I happen to be the CI maintainer for AVaRICE. The program should be capable of using a Atmel ICE to connect to a ATMega2560, at least per code and text notes. I’m not sure whether that then happens via JTAG or the ISP protocol / debugWire though.

You can download the latest built binaries for Windows per the latest CI run. I suggest you download + extract that and try to see if avarice can connect at all. Open a command terminal in the folder you downloaded it and execute

avarice.exe -4 --read-fuses --debug

and see what that say, at all. There are also different debugger types supported (-1 to -4, see --help).

3

Irregardless of that, avr-stub stub a different story and approach and should work without problems. That’s the GDB stub running on the Mega2560 itself, no Atmel ICE needed, but it hogs the Serial object all for itself. Make sure you cerate a minimal project first that exactly follows the documentation.

No, COM ports beyond 9 must be given with the \\.\ prefix. It should say

debug_port = \\.\COM16

there.

4

Thank you very much, this looks like some UNC-type of path notation that I would have never ever connected with a COM port name. Thank you for your help, this works now.

5

Thank you very much for your response, that was quite helpful.

I think that I have had a part-success with the command line as one of the commands (command 6) failed. Does it mean that the ICE can be used in connection with an 8 or 16 bit ATMEL- MCU?

Cheers,
Clemens

C:\Users\Thinkpad\Downloads\AVaRICE Windows 64-bit\usr\bin>avarice.exe -4 --read-fuses --debug
AVaRICE version 2.14svn20200906, Jun 19 2023 12:42:35

Defaulting JTAG bitrate to 250 kHz.

Found HID PID:VID 0x03eb:0x2141, serno J42700009315
Probing for HID max. packet size
Setting max. packet size to 512 from DAP_Info
JTAG config starting.
HID thread started

command “sign-on” [0x01, 0x10]
0E 00 00 00 01 10 00
Received 0x81 0x11 0x00 0x06 0x0e 0x00
read: 0e 00 00 01 80 00

Got message seqno 0 (command_sequence == 0)
response: 01 80 00

command “get info (serial number)” [0x00, 0x00]
0E 00 01 00 00 00 00 81
Received 0x81 0x11 0x00 0x12 0x0e 0x01
read: 0e 01 00 00 81 00 4a 34 32 37 30 30 30 30 39 33 31 35

Got message seqno 1 (command_sequence == 1)
response: 00 81 00 4A 34 32 37 30 30 30 30 39 33 31 35
Found a device, serial number: J42700009315

command “get parameter” [0x01, 0x02]
0E 00 02 00 01 02 00 00 00 05
Received 0x81 0x11 0x00 0x0c 0x0e 0x02
read: 0e 02 00 01 84 01 00 01 29 89 00 00

Got message seqno 2 (command_sequence == 2)
response: 01 84 01 00 01 29 89 00 00
ICE hardware version: 0
ICE firmware version: 1.41 (rel. 137)

command “set parameter” [0x12, 0x01]
0E 00 03 00 12 01 00 00 00 01 02
Received 0x81 0x11 0x00 0x06 0x0e 0x03
read: 0e 03 00 12 80 00

Got message seqno 3 (command_sequence == 3)
response: 12 80 00

command “set parameter” [0x12, 0x01]
0E 00 04 00 12 01 00 00 01 01 02
Received 0x81 0x11 0x00 0x06 0x0e 0x04
read: 0e 04 00 12 80 00

Got message seqno 4 (command_sequence == 4)
response: 12 80 00

command “set parameter” [0x12, 0x01]
0E 00 05 00 12 01 00 01 00 01 04
Received 0x81 0x11 0x00 0x06 0x0e 0x05
read: 0e 05 00 12 80 00

Got message seqno 5 (command_sequence == 5)
response: 12 80 00

command “AVR sign-on” [0x12, 0x10]
0E 00 06 00 12 10 00 00
Received 0x81 0x11 0x00 0x07 0x0e 0x06
read: 0e 06 00 12 a0 00 20

Got message seqno 6 (command_sequence == 6)
response: 12 A0 00 20
initJtagBox() failed: no answer from target

command “sign-off” [0x01, 0x11]
0E 00 07 00 01 11 00
Received 0x81 0x11 0x00 0x06 0x0e 0x07
read: 0e 07 00 01 80 00

Got message seqno 7 (command_sequence == 7)
response: 01 80 00

6

This looks to me like the Atmel-ICE itself responded properly but the Mega2560 that it is (supposed to be) connected to didn’t anwer via JTAG (or ISP)?

I assume you did test your wireup in Microchip Studio and there it was programming and debugging the chip fine?

Also it looks like this issue is at least known to one other person

7

I have checked the connection with the Microchip Studio programmer and unfortunately, this is an ISP connection, so -by definition- debugging is physically impossible. I had some problems running the AVR stub, so for me, it is actually easiest to forget about any of the simpler Arduino series and stick to the Arduino Due only, where my current VS Code /PlatformIO setup works really fine for me.