User Space Memory Management for Multimedia

*** Work Items tracked under the Graphics Working Group Blueprints ***

Summit Minutes: https://wiki.linaro.org/Events/2011-06-MMWG/MemoryManagement

Summit Actions and Closure Notes

  • ACTION: establish how much are people using EGLImage already?
    • TI: not currently for generic linux
    • ST-E: TBD
    • Freescale: TBD
    • Samsung: TBD
  • ACTION: Does using EGLImage require a full GL(ES) stack on the target?
    • TI: currently part of (closed source) pvr/gl stack
    • others: ??
  • ACTION: why not use a /dev directory to share the FDs
  • ACTION: Come up with a diagram for what the buffer sharing model would look like [Done]
  • ACTION: List pieces of software that would be updated to cope with this
    • GStreamer:
      • 0.11: makes it easier to pass around a handle.
        • plugins that need to support GstUmmMemory:

          • v4l2 (camera)
          • vendor specific decoders: gst-ducati, etc
          • sink elements?
      • 0.10: possibly could be done but only in a hacky way so considering 0.11 as a requirement
  • ACTION: Discuss what needs to be changed in key components

User Space UMM Recommendations

Research for methods and considerations

VDAPU

VAAPI

EGLVideoSink

Publish recommendation

User Space API (low level)

Proposal about what the kernel (ie. ioctl() level) interface for a UMM buffer.. this is independent of whatever higher level interface(s) may be supported on top (ie. VAAPI, eglImage, etc).

(maybe move this somewhere else?)

On kernel side, a file descriptor is created as the handle to a buffer (ie. with anon_inode_getfile()), with it's own struct file_operations.

offsets

Multi-planar formats would have one offset per plane. Ie. I420 would have a Y offset, U offset, and V offset. User could seek to these offsets and use read() or write() to access the data. Or optionally mmap() that offset to map the buffer into userspace.

Buffers may not be accessible (or example, read() or mmap() could fail). For example DRM (Digital Rights Management) protected content.

ioctls

A set of ioctls are defined to query information about the buffer (format, offsets of a particular color plane, etc). These ioctls are performed directly on the buffer file descriptor, no need to know (or even have open) the original device that created the buffer.

..TBD.. define ioctls..

User Space Reference Implementation

NOTE: implementing a GstUmmMemory to handle UMM buffers would be a good way to flesh out the needed ioctl's to query buffer formats, etc.

Test Framework/Testcases

WorkingGroups/Middleware/Multimedia/Specs/1111/UserSpaceUMM (last modified 2011-07-09 17:44:51)