Summary

gdbserver is available in the Ubuntu repository, but not actively tested or used in Ubuntu or Linaro. We want to integrate this into Linaro:

  • identify Linaro images which should include gdbserver by default
  • verify cross-architecture remote debugging functionality with Ubuntu packages
  • make recommendations for ease of installation of cross-architecture debug symbols packages on the host system
  • provide a wiki guide with recommendations for setting up and running gdbserver

Rationale

Remote/cross debugging is useful for headless or resource-constrained target devices

Use Cases

  • remote debugging of headless or resource-constrained devices

Scope

Mostly of interest to linaro, particularly with the headless image.

Design

Summary

Required features:

  • Include gdbserver in all linaro images by default,

  • Create a mechanism for getting the right debug symbols onto the host. The process doesn't need to be completely automated, but the user should be able to get the debug symbols for a package and all its dependencies with a few simple commands. So that this benefits native debugging too, we need a modular approach, keeping the two main tasks separate:
    • determine the correct set of debug packages to install
    • install the debug packages (for native debugging, this is a matter of installing .ddebs, whereas for cross debugging, we want to unpack the .ddeb contents under ~).

Desirable features:

  • Make sure that gdb can verify that the debug images match the target binaries (e.g., using .note.gnu.build-id or similar)

    • If gdb does not already check this for native debugging either, we should aim to patch upstream to support this, if we do implement the checks for cross-debugging.

  • Make sure that gdb<->gdbserver architecture and version mismatches are detected.

  • Make sure that the user gets a sane warning if the ptrace interface is disabled on the target, and instructions on how to enable it (as currently implemented in gdb).

    • Someone needs to investigate what warning (if any) is currently implemented first.

Rationale

  • gdbserver is small and doesn't run unless explicitly launched; so footprint impact will be minimal.

  • Getting the right debug symbols installed is a laborious process - we need to make this easier. This work can benefit Ubuntu too,

Scope and Use Cases

As documented in previous sections

Implementation Plan

Action items are defined in the "Action Items" section at the end of this page.

Implementation

Documentation

Rough Notes

  • be default GDB will read debug info from binary executables / debuginfo files found on the host system (where gdb runs).
  • You can configure gdb to read those files via the remote protocol from the target system
  • Installation on the host is "sort of easier" however when you are cross architecture you'd need to be able to handle install of cross architecture packages (interesting problem without some sort of multi-arch support)
  • acceptance of distribution specific features upstream unlikely. Hook mechanism would have a better chance.

Outstanding Issues

BoF agenda and discussion

gdbserver is available in the Ubuntu repository, but not actively tested or used in Ubuntu or Linaro. We want to integrate this into Linaro:

  • identify Linaro images which should include gdbserver by default
    • it's small, so install it everywhere except on images that have size limits?
  • verify cross-architecture remote debugging functionality with Ubuntu packages
  • make recommendations for ease of installation of cross-architecture debug symbols packages on the host system
  • provide a wiki guide with recommendations for setting up and running gdbserver
  • version compatibility between host and target - between 6 and 7 there was a discontinuity
    • treat this as a bug in the newer gdb
  • not an issue right now for us because this is an initial release
  • need a cross-gdb in Ubuntu
    • qemu has gdbserver functionality internally, could also use this
    • could be built either as part of the existing cross-toolchain metapackage, or as a standalone cross build
  • should gdb be buildable with support for multi-targets at the same time?
    • follow up with Ulrich Weigand about this
  • ask doko to add a message to gdb to let users know about the ptrace sysctl
  • debug symbols: ddebs available for Ubuntu packages
    • armel was missing many ddeb packages - solved now?
    • on ARM, if you don't have *all* the ddebs installed, you may have a garbage backtrace
  • cross-gdb library lookup location should have a default that's appropriate for ddeb downloads
  • 'I want to debug package x'?
    • look up dependencies in the cross-arch packages file and resolve
  • 'I want the debug symbols for this set of loaded objects'?
  • need to make sure we cleanup what we've installed - the wrapper tool should remove everything it's downloaded?
    • no - better to have this as a manual step
  • possible to use a VFS layer to automatically grab only the used symbols files - but this is fragile and not a target right now
  • cache - organized by gnu build id?
  • matching - how to verify that the target binaries and debug symbols match
    • (including being the right arch etc.)?
    • can build-id be used for this?
  • fetch sources?
    • some challenges - source packages may not match state of the source tree after build, etc.
  • testing - need to defined tests for:
    • availability ot ddebs
    • workingness of xgdb<->gdbserver in some scenarios

    • use gdb testsuite with xdgdb?
  • do we need to run the gdb testsuite for the gdb-cross build?
    • requires a remote target to test against, so we can't run this as part of the package build
      • run the test suite by hand?
      • put together an emulated server for the test suite?

Action items

  • (linaro foundations) add the gdbserver package to all linaro seeds

  • (linaro foundations) investigate how apport determines the list of ddebs to download
  • (linaro foundations) define an appropriate per-user cache layout that can be shared by gdb and perftools, preferably using GNU build-id in the heirarchy so that it can be shared between multiple versions of the same package * (linaro foundations) implement a mechanism to facilitate fetching and installing of ddebs to match the target images when cross-debugging.
    • the emphasis here is on reducing the pain - complete automation isn't necessary, but I would want to run a command on the host such as install-cross-ddebs --arch=armel --include-all-dependencies firefox and have it work. This task is dependent on the two previous tasks. (i.e., for debugging multiple targets that are running different releases)

  • (linaro toolchain) check the presence gdbserver protocol support for verifying that the target architecture, protocol version and the debugged binary (or binaryies) are correct (i.e. they match the local debug binaries on the gdb host)
    • this may require implementation work depending on what gdbserver currently supports
    • ld --build-id can potentially be used as part of a mechanism for verifying matching binaries. For recent Ubuntu/linaro releases, gcc passes this option to ld by default, so we should rarely/never have to deal with images that don't have a build ID.

  • (linaro toolchain) patch gdbserver to warn if ptrace() is disabled, and explain how to enable it


CategorySpec

Platform/DevPlatform/Specs/GdbserverIntegration (last modified 2011-01-27 07:32:56)