OnSite Broadcast Requirements

Phase 1

• New MLT module: Firewire input
  • Should work in conjunction with the avformat producer for decoding.
  • Uses libavc1394 and libiec61883
  • Should have separate thread and buffering to prevent dropped packets in kernel and hardware FIFO
  • Use lessons learned from dvgrab
• Melted: add feature to switch command to switch back upon end-of-stream
• Melted: add feature to define behaviour at beginning of playlist item: continue (current behaviour), wait for N seconds, wait until system time N
• Melted: add feature to define a playlist item as normal producer (current behaviour), previous item (i.e. repeat), blank, xfer from unit N (live)
• Rugen: bring over to mltframework.org and update for changes in mlt/melted
• Rugen: consider porting to Python and possible integration with Python/GTK-based OpenShot editor for future requirements

-- DanDennedy - 31 Aug 2009

There should only be one melted server per machine - melted can run and manage multiple MLT "units" or pipelines. -- DanDennedy - 01 Sep 2009

Discussion

For every of the multiple (=4) pipelines of broadcasted live programm on the GUI-client we need to be able to build a playlist of clips and to start the next clip in the playlist by pushbutton or timer. We need to be able to define at every clip if we switch back to live or start the next clip in the playlist or replay the same clip. If there is no input coming from the the grabber, a stream of black.screen needs to be generated by MLT to keep the DVB-T pipeline running. -- EdmundHumenberger - 04 Sep 2009

OK, it is good that you mention these requirements. Please describe more as time allows and you recall them. Melted does currently support a playlist for each consumer (output). However, it currently lacks the ability to change behaviour at the end of each clip as you describe. Right now, it simply plays the next thing in the list, but I think what you want is very feasible. Also, Melted is already configurable to define what should appear when there is "dead air." It can be anything - color generator, image file, video file, or even live source. -- DanDennedy - 04 Sep 2009

The needed features for the interface to manage the live signal, the prerecorded clips and graphics and text will develop with further client projects. I will keep an eye on it and will edit here. -- EdmundHumenberger - 04 Sep 2009

Problem Constant Bitrate

FFMPEG is not generating a sufficient constant bitrate for a single MPEG2 stream. Under DVB-T this is a massive problem. Under DVB-T we need a constant Bitrate of the MPTS to feed the modulator. Currently we handle the Bitrate problem by stuffing/dropping in the resulted MPTS. When we would encode to MPEG2 on MLT and REMUX with FFMPEG, we do not have as much controll over the Bitrate during the REMUXING. Our programmer is much more into the details of this problem as he is working on it since February. He prefers to get DV out of MLT (over UDP?). -- EdmundHumenberger - 04 Sep 2009

I understand. There is a standard for DV/RTP/UDP, but it is not common. I think FFmpeg can send and receive raw DV over UDP as well without RTP. But DV encoding is still a bit heavier than MPEG-2, and it is certainly a higher datarate, which is going to stress the userspace/kernel interface and network stack. In fact, FFmpeg is even more efficient at encoding and decoding MPEG-4 than either DV or MPEG-2. Your MPTS muxer can then transcode instead of remux - as it does today - but from MPEG-2 TS, MPEG-4 TS, or DV/UDP. We have options here, but my experience is suggesting MPEG-4 in TS will be the most robust for both Melted and the Modulator running the ffmpeg MPTS muxer. Furthermore, since there are options, I do not believe this is something that we need to finalize now. -- DanDennedy - 04 Sep 2009

The question is how much quality suffers by the 3 encodings (from analog to DV to MPEG4 to MPEG2)? We can transcode 4 streams from MPEG4 to MPEG2 form on the same box where we modulate the signal. For the transcoding we would definitely need another box -- EdmundHumenberger - 05 Sep 2009

Yes, the multiple encodings is not highly desirable. It is possible to generate very high quality, high bitrate MPEG-4. Also, there is still an option to do analog->DV->DV->MPEG2 if that proves to be better. If there are modifications you can make to ffmpeg libavcodec's mpeg2video encoder to make the output suitable for mpts remuxing, but without mpts in MLT, that would be ideal. -- DanDennedy - 08 Sep 2009

Phase 2

• Rugen/OpenShot: ability to apply text and/or graphic overlay. Needs to support the following high level functions:
  • Author
  • Preview
  • Browse and/or search
  • Select
  • Apply
• Text/graphics track overlay (http://www.openshotvideo.com/2009/06/video-title-editor-for-linux.html)

Discussion

Phase 3

• Cut effects
• timeline for graphics
• templates for graphics and datasource

-- DanDennedy - 31 Aug 2009

Thoughts about final architecture

• in grabberbox box there is
  • multiple Firewire live grabbing, DV decoding
  • graphics rendering
  • playout from file
  • overlay, cut effects
  • compression to multiple MPEG-4 single program transport streams (SPTS) in separate MLT pipelines
  • sending of multiple SPTS over UDP to multiplexer box

• in multiplexerbox there is
  • reception of multiple SPTS over UDP
  • transcoding of multiple SPTS from MPEG4 into one DVB-T compatible Multiple Program Transport Stream (MPTS) by FFmpeg
  • Buffer management to avoid overrrun/underrun (done)
  • sending of MPTS over UDP to modulatorbox (done)

• in modulator box
  • receiption of MPTS over UDP (done)
  • modulation to OFDM (done)

-- EdmundHumenberger - 01 Sep 2009

Discussion

Are you sure you want program stream and not transport stream between the two boxes? In my experience, transport stream is more suitable for this. -- DanDennedy - 01 Sep 2009

The question is, where is the multiplexer.

We still do not have a separate muxer to do multiprogramm streams. Currently everythink (decoding, encoding, muxing) happens within FFMPEG. So we do not yet have a plan how to mux when using MLT. -- EdmundHumenberger - 02 Sep 2009

I think it mostly stays the same, but instead of receiving DV via pipe, you can receive something via UDP. That something can be DV or transport stream containing MPEG-2 Video or MPEG-4 (not AVC). -- DanDennedy - 04 Sep 2009

Discussion Archive:

For a start we might want to use only one melted server and output DV from the melted server over a pipe to our current FFMPEG pipeline. I dont think that we can run 4 melted servers on one machine and feed FFMPEG with 4 pipes to generate the TS at the same time. Depending on how much CPU power is needed by melted, I hope that we can run our current FFMPEG pipeline with 4 Grabber cards, 3 feeding FFMPEG direclty and one melted server grabbing and feeding FFMPEG. -- EdmundHumenberger - 01 Sep 2009

You can certainly choose to have 3 pipelines using your existing approach and 1 using melted. However, I think you will find melted not being much more overhead, and eventually want to consolidate to simplify the overall architecture and bring it under a common management UI. Currently, I consider making the Rugen-based client manage something besides Melted out of the scope of the project. -- DanDennedy - 01 Sep 2009

Is the IP transport UDP or TCP? -- DanDennedy - 01 Sep 2009

We use UDP to transport the finished Multiprogramm TS from the encoder box to the modulatorbox. Encoding and multiplexing is currently done one the same machine. -- EdmundHumenberger - 02 Sep 2009

If TCP, how is the connection established - which box is the server and which the client and what protocol? Will I use FFmpeg's network support or pipe MPEG-2 to your program that does the network handling? -- DanDennedy - 01 Sep 2009

Re: "(avoid random device numbers moving around at reboot)" This should not even be a problem in your current architecture because you should be using each Canopus converter's GUID with the dvgrab -guid option. There is no reasonable way to change the kernel drivers to try to do it any other way like by using PCI slot ids and physical Firewire connector number. You should lookup the GUID for each device and print adhesive labels containing the GUID for each converter. -- DanDennedy - 01 Sep 2009

(OK, Thank you) -- EdmundHumenberger - 02 Sep 2009

Hint: connect a single converter and run dvgrab. dvgrab should find it and print its GUID. The -guid option has a side effect currently - it attempts to establish a point-to-point connection with the device instead of the default broadcast connection. This is low level Firewire terminology - no need to be alarmed. However, sometime it can introduce an interoperability problem depending on the device. Please test this. I can leave out this "connection management" in the MLT integration to avoid this potential risk and to simplify first versions. Then, when I do introduce it, I can require an explicit option to use point-to-point. -- DanDennedy - 02 Sep 2009

There is no common uncompressed audio/video format suitable for pipes that maintains timestamps for A/V sync etc. Not to mention, shoving lots of uncompressed data through pipes is inefficient. MLT is a virtually a superset of FFmpeg. Maybe you mean you want to output compressed MPEG-2 Singel Programm Transport Stream from melted and pipe it to some other software you wrote, which does make a lot of sense. -- DanDennedy - 01 Sep 2009

WE currently pipe the output of multiple DVGRAB into FFMPEG. Therfore the Format is DV. Then FFMPEG pipes the TS ino our programm which sends the TS over UDP Packets to the sender. So what we miss in MLT is a Multiprogramm muxer to generate DVB-T compatible TS (or can you use FFMPEG within MLT to do that? and them MLT outputs the finished TS) -- EdmundHumenberger - 02 Sep 2009

OK. I interpreted your "programm stream" literally when you really meant single program transport stream. Now we understand each other. -- DanDennedy - 02 Sep 2009

Another question: Can we move "cut effects" to a phase 3? -- DanDennedy - 02 Sep 2009

Yes we can, no problem -- EdmundHumenberger - 04 Sep 2009

MLT will be able to output multiple single program transport streams over UDP. However, we will need a separate program to multiplex to multi-program TS; the architecture of MLT does not really make that possible. I believe ffmpeg can do that without transcoding, just remuxing. I will have to make some tests. Can we put SPTS-to-MPTS remuxer on a different box to reduce the burden on the Melted server? It does not need to be a totally separate box. Can it be the box doing the modulation? -- DanDennedy - 02 Sep 2009

Yes, it can be the box doing the modulation. There we have enough CPU power. Can FFMPEG receive multiple SPTS over UDP sent out by MLT? Is MLT really able to output multiple "Single program transport streams" over UDP? -- EdmundHumenberger - 04 Sep 2009

Yes, Melted manages multiple MLT consumers, each of which can ouput over UDP. I can let you specify a distinct address and port for each, or I can let you just specify a single address and base port and let it increment the port for each output. -- DanDennedy - 04 Sep 2009

The question is if we implement a kind of stylesheet/datasource concept with timeline schedule to define what gets rendered/overlaid when. Or we start out with a simple overlay definition and then go more powerfull later (using OpenShot as a starting point) -- EdmundHumenberger - 04 Sep 2009

OK, take my suggestion for the usage of OpenShot lightly for this phase. I am merely suggesting to reuse its code for the authoring of titles and graphic positioning: http://www.openshotvideo.com/2009/06/video-title-editor-for-linux.html. I suggest moving anything more sophisticated including timelines, templates, and datasources to Phase 3. -- DanDennedy - 04 Sep 2009

• General infos about Firewire and Firewire under Linux

http://www.cablelabs.com/news/newsletter/SPECS/spectechjuly/tech.pgs/leadstory.html http://ieee1394.wiki.kernel.org/index.php/Juju_Migration http://lwn.net/Articles/353043/ Video buffer management!