The opengl module requires the movit library, which is not typically included with distributions as it is still very new (as of February, 2013). git clone http://git.sesse.net/movit/ in order to get the movit source code. It might say that you need some things like gtest, libpng, and SDL, but not really for our needs. We only need Eigen3 and GLEW, which are very common. Then, you can run “CXXFLAGS-fPIC ./autogen.sh” with whatever additional parameters you like such as perhaps –prefix. It is important to put -fPIC into the CXXFLAGS because we are going to be linking the lib into a shared library as a MLT plugin. Finally, “make libmovit.a install” to build only the lib and not the unit tests and demo program. Install is important because it will add a pkg-config file and put some runtime shader programs into $prefix/share (datadir). At this point, MLT’s configure should be able to detect libmovit.
movit.convert, movit.crop, movit.resample, and movit.resize are not intended to be used directly. They are automatically added normalizing filters. Use the regular crop filter to pass the crop parameters to movit.crop. Use the new movit.rect filter to affect movit.resample and movit.resize in order to have more control over scaling and padding. This works well in conjunction with movit.overlay to accomplish effects similar to the traditional CPU-based composite or affine services. In other words, the geometry parameter lives on a discrete movit.rect filter and movit.overlay simply composites. There is nothing at this time that does rotation or skewing.
Use “melt -query” to get more information about these at this time including properties.
Do not use mlt_image_opengl - that is legacy, deprecated and basically equivalent to rgba24.
You will want to use !OpenGL as a display mechanism in your app to get the most benefit from this. That means you need to create an !OpenGL context for your display and another one that “shares” with it to use with MLT as the “render context.”
You must allocate a “glsl.manager” filter to determine first if the MLT build includes the opengl module. Next, you need to fire the MLT event “init glsl” on this filter to let MLT test for GL extensions. However, it is important that you fire this event when the render context is valid and current on the thread on which rendering actually takes place! Therefore, there is a new MLT consumer event “consumer-thread-started” that you must handle. It is within this handler that you should make the render context current and fire “init glsl” on the glsl.manager. See src/modules/qimage/consumer_qglsl.cpp for a simple example.
After firing “init glsl” you can get the int-as-boolean property “glsl_supported” to determine if the !OpenGL subsystem on this computer is adequate. Lastly, you should have a consumer “consumer-frame-show” event handler that calls mlt_frame_get_image() or Mlt::Frame::get_image() on the supplied frame argument. You can give it any mlt_image_format, but if you want to integrate a GL texture rendered by MLT, then use mlt_image_glsl_texture. The returned image image pointer must be cast to a pointer to GLuint, which contains the texture name. If you are using real_time = 1 or -1, then the app’s display GL context can be current, but if you are using real_time = 0, then the render context must be current.
Just as the “init glsl” event must be fired on the thread on the glsl.manager filter on the rendering thread, you should also fire “close glsl” on the rendering thread before destroying the !OpenGL context. This will make it deallocate any frame buffer, pixel buffer, and texture objects that it has created. Typically, you do this inside of a “consumer-thread-stopped” MLT event listener so that it is performed on the rendering thread. Firing this is particularly important if your app is stopping and restarting the consumer. If the app is just going to close after stopping, then it is not needed because the glsl.manager destructor will take care of the cleanup.
Additionally, the qglsl consumer registers a MLT event “consumer-cleanup” that you can fire before closing producers. You see, when producers are closed, it may go into movit destructors that will use GL functions to release GL resources. Those should be performed on the context on which they were allocated - the render context. Howver, often that closure occurs on a separate thread such as the app main event thread. So, the qglsl consumer will make the render context current in the thread on which you will be closing the producers.
As of January 1, 2014 there are two new events on the consumer to override the creation and cleanup of threads. Basically, they let you use another thread implementation, such as Qt QThread, for managing threads. This is handy if you need to create a context on the Qt main thread, but you need to move the context to another thread, which only works with QThreads, not pthreads.
The first is “consumer-thread-create” to create and start a thread. It
provides the following additional arguments in addition to the standard
first two MLT event handler arguments:
void** handle, int* priority, thread_function_t, void* data
handle is a return variable that is the thread context object pointer equivalent to pthread_t or QThread.
Priority is a the POSIX sched_priority
thread_function_t is a function pointer with the following signature:
void* f(void *data)
Data is an opaque pointer passed to the function pointer.
Your new thread must run function(data).
The second event is “consumer-thread-join” on the consumer object. The
event handler should wait for the thread to complete (pthread_join) and
thencleanup the thread resources including destructing the handle. The
event provides one additional argument in addition to the standard first
two MLT event handler arguments:
As of January 12, 2014, the opengl module no longer calls glFinish() due to unwanted side effects. If you are using rendering to GL textures using mlt_image_glsl_texture and using them from another thread - for example, for display - then you need to synchronize on a GL fence sync object. Basically, after you call mlt_frame_get_image() or Mlt::Frame::get_image(), and before trying to use the texture ID with a GL function, then you need to get the sync object pointer from the frame’s movit.convert.fence data property and call glClientWaitSync() on it:
GLsync sync = (GLsync) frame.get_data("movit.convert.fence"); if (sync) glClientWaitSync(sync, 0, GL_TIMEOUT_IGNORED);