The TNmedia platform consists of three main parts:

    1. SceneSpotter.tv
    2. videoCMS
    3. Video Analysis, Transcoding and Asset Management platform

SceneSpotter.tv

Scenespotter.tv is a online service for content providers that enables them to make video browsable by associating pointers within the video resulting in sections that let users to watch or replay the sections they choose.  The pointers enable the content providers to link users directly to a specific relevant section.

Conceptually SceneSpotter.tv consists of two main components:

  1. Pointers
  2. Metadata

Pointers
The service enables creative producers to associate section pointers in the video without the assistance of a technical-savy person (e.g. video editor).  By adding section pointers the browseability of the video is increased. The pointers are not physically added to video but are managed external from the video source.

This increases flexibility and has following advantages:

  • Pointers can easily be adjusted, i.e. section starting point can be moved new sections can be added or removed.
  • A video may have multiple pointers sets. This allows content providers to create different end user experiences for the same video without having to store the video file again.
  • Video sections can be associated with other media files. For example, for a presentation recording it is possible to associate certain sections with the related slides from the powerpoint presentation.

Metadata
Moreover, it is possible to associate metadata to the section pointers allowing for the following:

  • Targetted contextual advertising specfic to the scene. It is possible to associate advertising keywors specific to sections. These keywords will be passed to the advertising intermediate when the section viewed.
  • Increased flexibility to describe sections in the video. The metadata can include section title as well as the transcript which can be displayed on the web site and used for browseability
  • Increased searchability. The addition of relevant terms to each scene, scenes will be included in the web site search results.
  • Increased visbility in web search engines such as Google (leading for more traffic). The metadata can include terms relevant to each scene. This information can be exposed to web crawlers and as result the link to the appropriate video section will appear in the web search results when users include the terms in their search query.

The metadata is stored and managed seperatly and can easily be changed, added or removed.


videoCMS

The videoCMS is an online service for content providers that enables them to create playlists of video and make them available to the user regardless of the media players available to the end-user.

Conceptually the videoCMS consists of three main components:

  1. Video player suitable multiple formats. Allows users to play the video - embedded in the web page- using the player of their choice (regardeless of the format that the content provider originally created). Supported formats are QuickTime, Windows Media and Flash Video.
  2. Dynamic web modules (video interactivity modules) which manage the interactivity with the video through the pointers and metadata added through the SceneSpotter. Enables content owners to use the pointers and metadata - managed through SceneSpotter – to create an interactive user experience. The system facilitates the construction of the web page. The content provider can easily add and modify information modules on the web page that interact with the video. The modules are listed below.
  3. Usage monitoring system. Provides insights into which videos as well as which scenes within the video are played most.

List of video interactivity modules:

  • Information module – information on the video and general information on the channel
  • Playlist module – containing the different videos available to user. Title is managed and stored through the ChannelPublisher
  • Scene module – listing the different sections of the video with their title as stored in the metadata from the SceneSpotter
  • Control module – displaying the standard control feature such as play, stop, pause etc.  in a consistent design across video players
  • Slides module – allowing for the association of the related slides from the powerpoint presentation to a presentation recording
  • Transcript module – displaying the relevant part of the video transcript in a auto scrolling box. The transcript lines are coupled to appropriate pointers in the video (stored in SceneSpotter)
  • Ad module – managing the interaction with the advertising intermediate using the keyword stored in the SceneSpotter
  • Search module – enables searching through the metadata

Video Analysis, Transcoding and Asset Management platform.

At TNlabs we develop applications that extract contextual information from video, so one of the main building blocks of our platform is the part that supports this analysis. Another main component is the transcoding of video and general asset management.

For example we extract timeline scene images from video in order to present viewers visual information of what’s happening in the videos. We also aim to analyze video to detect scene changes and extract text from spoken word.
Another type of module that we develop is the transcoding to several different video formats like Windows media, 3GP, Flash and Quicktime in configurable quality and bit rates.

Main goal is to automate the video analysis and transcoding process as much as possible. This will allow us to to manage in bulk thousands of videos and (re)analyse them when new modules are developed or updated.

In a very early stage of the system design process we recognized that the requirements for server performance and scalability are very high and we discovered that we need these high scalability figures intermittently. Another strong requirement is the need for reliable, secure and cost-effective storage for our video footage and metadata information.

Amazon Web Services

In our search we discovered Amazon Web Services as a possible cost-effective solution for our server computing and web storage needs. One of the services Amazon Web Services offers is the Amazon Elastic Compute Cloud (Amazon EC2). Currently still in Beta it provides resizable compute capacity and complete control of computing resources. On top of that it runs on Amazon’s proven computing environment.

Another very interesting service is the Amazon Simple Storage Service (Amazon S3). It provides a service to store and retrieve any amount of data, at any time, from anywhere on the web.
To “glue” our locally hosted VideoCMS application with Amazon EC2 and S3 services we used yet another Amazon Web Service called Amazon Simple Queue Service (Amazon SQS). SQS offers a reliable, highly scalable hosted messaging queue and by using it we can move data between our distributed application components, without losing messages or requiring each component to be always available.


RightScale: Cloud Computing Delivered

Next step after deploying S3 as our storage mechanism was to look into Elastic Computing with EC2. At that time the only problem was that EC2 was in limited beta and that there was a waiting list. Luckily we got in contact with the guys at RightScale who offer a free Developer account on their RighScale Dashboard platform. With the RightScale developer account we were able to experiment with EC2, basically it offered us all the EC2 functionalities to build, launch and manage our first custom build EC2 images.

After we got our own EC2 account but we continued to use the RightScale dashboard and other RightScale features. By using RightScale we are able to experimement, design, build, deploy and manage our EC2 based applications without having to handle the nitty-gritty EC2 details.

Besides the increased speed of deploying our EC2 applications by using RighScale we were also very happy with the friendly and good support that we got from them.

By now RightScale has introduced 3 premium editions of their RightScale plans: Website Edition, Grid Edition and Premium Edition. Website Edition has everything to run a full customer-facing website on Amazon EC2. Grid Edition controls and manages any background or batch processing worker tasks in a fault-tolerant environment. And Premium Edition combines both RightSite and RightGrid.

(no we dont have any RightScale shares. We just recognise a great service that helped us so much to utilise the allready great possibilities of the Amazon Web Services.)


TNmedia/TNlabs Video Analysis and Transcoding framework


As input we have the video catalog information that we import from our customers or from our own video hosting platform. Output consists of all extracted contextual video information, images and video copies in required video formats. Finally the output is imported by the TNmedia VideoCMS for direct publishing of the video items and/or manually editing by the producers.

Importer process
The Importer is a process that runs locally on the TNmedia/TNlabs servers in Amsterdam and it periodically interrogates the video archive of our customers or our own archive. It keeps an up to date database of available master videos and keeps track on the status of video analysis/transcoding. Once the Importer detects a new video or that a video needs to be reanalyzed it adds a message to the Analysis queue that is hosted on the Amazon SQS architecture in the US.

Analysis queue monitor
The monitor service watches the Analysis queue and based on the queue size it dynamically starts one or more TN-Analysis server instances at Amazon EC2.

TN-Analysis server instances
The TN-Analysis server instances are started from one of our own custom build EC2 server images which contain all the application logic to perform specific actions like scene detection or video transcoding.
When an instance is started it reads from the Analysis queue a message and based on the information provided in the messages it collects the specific video and requested tasks are performed against it. Once these tasks have been completed all generated contextual information is stored on Amazon S3 and a new message is added to the TN-Online queue.

All of this is done automatically, we can monitor detailed progress by using the RightScale dashboard. There is always up to date information about queue size, S3 usage, amount of active servers and duration of jobs. Also in case of emergency or for maintenance the Analysis flow can be manually controlled.

Publishing
The TN-Online queue is continuously monitored and once a new message is available the Publishing process reads its information. This information is updated in the Importer database and the associated video is marked as ready. Before the video item is added (or updated) in the TNmedia VideoCMS we verify the completeness, integrity and availability of the item. Also contextual data and video streams are checked by the Verification process.
Finally the TNmedia VideoCMS imports those video items which are marked as “ready” in the Importer database.

 

 

© 2010 TNmedia - contact