Miscellaneous Topics

Open Test Content

  • Idea: to assemble a variety of test content for benchmarking and/or evaluating performance

gstreamer 1.0 (Alessandro)

  • Topics
    • Summary of what is being done
    • Is there work we should do to contribute?
  • Next version of gstreamer (0.11 and 1.0 based on that)
  • Improvements
    • Dynamic strides on buffers, avoid memcpy when aligment differs - using metadata on pipelined data (keeping metadata extensible under a standardized interface)
    • Standard API for buffer metadata; allows for customizable attributes to be added
    • Support for handles within pipelines, and use an explicit API to manipulate handle to extract address
    • Buffer pool API consolidation; elements can negotiate the size of the buffer pool, buffer pool is managed by the source (can manage flushing all the buffers associated with a specific pool)
      • Benjamin: I have already provide patches for that feature in x[v]imagesink
      • Allows unblocking of buffers without sending a flush through
    • Caps negotiation is slightly simplified
  • Under consideration
    • Separate planes for YUV; looking for feedback on importance of this (breaks compatibility so good reasoning is needed).
  • Would like Memory management feedback from Linaro MMWG
    • provide in next few weeks
    • Metadata around the buffer which allows accessing buffer memory layout; would use the fd
    • Question is whether a standard interface needs to be provided by gstreamer to map/unmap addresses, or whether this will be done elsewhere
      • Having the ability to request a per-plane pointer should avoid the need to memcpy
  • 0.11 will break ABI and API, so applications will need to be relinked
    • Vendors will need to provide both 0.10 and 0.11 plugin versions for example
    • Same API is kept for playbin2, so minimal application bustage
  • Beta around end of June, so that's a good time for packages to be updated

Benjamin: a porting guide from 0.10 to 0.11 with a list of API changes is available here: http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/random/porting-to-0.11.txt?h=0.11


gstreamer element ranking

  • Proposal provided by David S. to bug report
  • Codecs from external sources should be able to get ranked higher
    • To cope with purchased swcodecs or hardware blocks
    • Parsers should be ranked higher still so they are always in the pipeline
    • Steve: in low-power situations, you don't want pass-through at all because it keeps waking up the CPU
  • ACTION: go ahead and implement David's suggestion; agrees that a config file is overkill for this problem
  • One of the pre-conditions is that parsers are all good quality
    • Have been quality issues in the past, but 0.10 rewrite should have addressed these issues
    • Probably just crank up the rank and see if there are problems

EGL Video Sink

  • Need recap from ? missed "secret handshake" discussion
  • After OXM IL Decoder, Vendor specific GST video sink vs EGL passthrough GST to EGL sink
  • Create EGL Image from file descriptor
  • Need GST video sink that can output EGL videosink for using EGL for buffer passing
  • Using EGL Image as opaque handle
  • EGL Stream gaining traction in openmax and google
  • Jesse: No way for application to elicit what color formats the EGLImage can support
    • ACTION: follow-on call with NVidia/Jesse/Rob to define what this should look like
    • for example, an example of shader.. but this implies the application knows that it is using a YUV texture (and from a particular vendor):

   1 # ifdef GL_IMG_texture_stream2
   2 # extension GL_IMG_texture_stream2 : enable
   3 # endif
   4 varying mediump vec2 texcoord;
   5 uniform samplerStreamIMG streamtexture;
   6 void main(void)
   7 {
   8     gl_FragColor = textureStreamIMG(streamtexture, texcoord);
   9 }

  • ACTION: pass action item over to Jesse to hash out an initial EGLImage API spec; set up call and figure out next implementation step


  • What to use for camera API on Linux
    • GStreamer camerabin2
    • Don't use V4L2 - lower level api

  • Use cases
    1. Video capture
    2. Still capture
    3. Raw camera access (for Image analysis, effects, etc)
  • Nice-to-have would be implementing an app on camerabin2 (perhaps a mobile app, or a Cheese port)
  • ACTION follow up call with NVidia Jim/Kurt/Rob

Test Content

  • Generate freely redistributable content for validation and benchmarking
    • Can use this to drive testcases
    • Go through boards and enumerate formats supported
  • Motivation is to exercise variety of codecs
    • testing content for correctness and benchmarking may vary
  • ACTION: identify location to put this content
    • Who should be able to write to this?
    • How much information is this?
  • ACTION: document matrix of encoding, platforms, test cases (and results)
  • ACTION: packaging of test content in a format which can be installed on Android/Ubuntu
  • Sources of content

Managing Vendor Binaries

  • ACTION - Identify all vendor binaries, licensing
    • TI: most is available publically
      • Missing: camera ISP bits missing; VC1 codecs;
    • Freescale
      • Binary codec pack
  • ACTION: identify legal contact for redistribution conversation
  • ACTION - make sure that all binaries can be used with the latest Linaro kernel
  • ACTION - handle vendor binary distribution
  • Needs to include areas outside MMWG, including graphics, drivers, etc
  • Test builds for binaries, what combinations of binaries are needed per environment


  • ACTION - follow up call Rob - prototype - code dive libva
    • Alternative to openMAX Dependent on the mem mgmt proposals and kernel statr What is the advantage to do any prototyping:
    • e.g. openMAX looks good in theory but when going into details there are issues - same principle here
    • possibility to support GLES : could benefit all Linaro participants

VAAPI: Desktop apps already support these interfaces, so providing support for those interfaces the apps could work out of the box. Example apps: VLC, clutter, ffmpeg, flash player (gnash), gstreamer, mythtv (work in progress), realplayer, mplayer etc. Also quite a good selection of HW support it. VAAPI: http://www.freedesktop.org/wiki/Software/vaapi Same license as x server - BSD/MIT

Kiko: if we used this, we'd be up to three API's. GST->openmax, GST->direct, and this

Events/2011-06-MMWG/MiscTopics (last modified 2011-06-14 13:08:29)