OpenMAX Integration TR

Blueprint Specifications

Summit Minutes: https://wiki.linaro.org/Events/2011-06-MMWG/OpenMaxIntegration

Summit Actions and Closure Notes

  • ACTION: collect source code
    • TI gstreamer components
    • Freescale Android OMX Components
      • Note that Freescale doesn't use Stagefright directly, but their own Graph Manager framework
      • Graph Manager itself is an IL client, and interfaces with OMX components
      • Do implement complete OMX graph (parsers, clock, etc)
    • Look at Android OMX standard glue
  • ACTION (Kiko): identify OpenMAX representatives from members
    • Nikhil (TI)
    • Thierry (ST-Ericsson)
    • Jim (Nvidia)
    • Bryan Murray (Freescale)
    • (ARM; no OMX rep)
      • (Samsung)
  • ACTION (Kiko): set up call with QC/Nvidia to talk about VDPAU/VAAPI as an alternative
  • ACTION (Rob, Kurt): set up wikipage to collect input for API; collect survey data here
  • ACTION (Kurt, Kiko): share initial set with Google, Linaro TSC and collect input
  • ACTION: Jim/Kurt/Thierry call with Neil to define collaboration model
  • ACTION: study how to formally communicate to OMX IL WG Tuesday meeting
  • ACTION: prepare reference for OpenMAX F2F meeting (Phoenix, September)

Vendor Survey

1) Create and populate a vendor code analysis repository

  • ST-Ericsson StageFright source code:

    • Add ST-E OMX video encoder/decoder components.
    • ST-E follow OMX specifications for buffer pre-announcement: large patches to support buffer negotiation (number, size, allocation).
    • ST-E use specific YUV format.
    • ST-E add new quirks to retrieve additional information from OMX components (mainly for buffer management).
  • TI gst-ducati source code: here

  • TI gst-openmax source code: here

  • TI StageFright source code: here (gingerbread branch)

    • add video and audio codecs for OMAP4 and OMAP3.
    • add custom properties (quirks) for TI OMX components.
    • hack buffer allocation function for thumbnail uses case and manage buffer alignment requirement.
  • FreeScale source code:

    • Add new media type definitions as flac, sorensen, ac3 etc.
    • Add new component parameters like seekable, duration for parser components
    • Add buffering event for support streaming
    • Add resource manager for buffer management (vir/phy address)
  • Samsung source code:
    • Version 2.02 opencore used with Eclair and Fryo.
    • Added Real media Codec support
    • Used the mp3 generic component and did modifications for Real Media support

2) Analysis, summary and publication

Compliance verification

Publish findings

  • vanilla OpenCore use a "Buffer Allocator Object" to allocate buffers for OMX components

    • This class manage the buffer refcounting.
  • vanilla StageFright use MediaBuffer and MediaBufferGroup to allocate buffers for OMX components

    • A refcount method is implemented.
  • only ST-Ericsson component take care of buffer pre-announcement like describe in OMX IL specification.
  • most of the additional OMX components have specific "quirks".
  • expect in ST-Ericsson source code there is no majors changes in StageFright behavior.

  • StageFright uses only 3 states of OMX: idle, loaded, executing.

Survey conclusions

  • StageFright and OpenCore seem to be the main open source implementations of OpenMax IL client.

  • StageFright and OpenCore implement buffer refcounting methods.

  • Other non-free OpenMax IL clients exist.

  • buffer pre-annoucement isn't implemented by most the SoC vendors.
  • buffer allocation strategies are different for each SoC vendor.
  • lot of additional custom properties has been added on SoC OpenMax components.

Reference Implementation

API definition

The goal of a new API definition (OpenMax Lite) is to resolve the issues listed during Austin summit:

  • No buffer pre-announcement.
  • Simplest state machine.
  • Provide a way to query what bitstream format is supported by component.
  • Buffer refcounting (and callback when it is freed).
  • Synchronous API:
    • Synchronization is done at StageFright level, for example OMXCodec::init() state change is made synchronous.

  • Error codes simplification.

OpenMax survey presentations

Summarize OpenMax survey key messages.

OpenMax suvey pdf or ppt

WorkingGroups/Middleware/Multimedia/Specs/1111/OpenMaxIntegration (last modified 2011-09-14 07:40:49)