MediaTomb Scripting

This documentation is valid for MediaTomb version 0.11.0.

Copyright   2005 Gena Batsyan, Sergey Bostandzhyan

Copyright   2006-2007 Gena Batsyan, Sergey Bostandzhyan, Leonhard Wimmer

THIS SOFTWARE COMES WITH ABSOLUTELY NO WARRANTY! USE AT YOUR OWN RISK!

-------------------------------------------------------------------------------

Table of Contents

1. Introduction
2. How It Works

    2.1. Understanding Virtual Objects.
    2.2. Theory Of Operation

3. Global Variables And Constants

    3.1. The Media Object
    3.2. Constants

4. Functions

    4.1. Native Server Functions
    4.2. Helper Functions

5. Walkthrough

    5.1. Import Script
    5.2. Playlist Script

1. Introduction

MediaTomb allows you to customize the structure of how your media is being
presented to your renderer. One of the most important features introduced since
the version 0.8 are the virtual containers and virtual items. Let's think of
possible scenarios:

  * you may want to separate your content by music, photo, video, maybe create
    a special container with all non playable stuff

  * you may want your music to be sorted by genre, year, artist, album, or
    maybe by starting letters, so you can more easily find your favorite song
    when browsing the server

  * you want to have your photos that you took with your favorite digital
    camera to appear in a special folder, or maybe you even want to separate
    the photos that you took with flash-on from the ones that you made without
    flash

  * your media player does not support video, so you do not even want to see
    the Video container

  * it's up to your imagination :)

The scenarios described above and much more can be achieved with the help of an
import script.

Additionally, version 0.10.0 introduces a playlist parsing feature, which is
also handled by scripting.

2. How It Works

This section will give you some overview on how virtual objects work and on how
they are related to scripting.

2.1. Understanding Virtual Objects.

When you add a file or directory to the database via the web interface several
things happen.

 1. The object is inserted into the PC Directory. PC Directory is simply a
    special non-removable container. Any media file added will have an entry
    inside the PC Directory tree. PC Directory's hierarchy reflects the file
    system hierarchy, all objects inside the PC Directory including itself are
    NON-VIRTUAL objects. All virtual objects may have a different title,
    description, etc., but they are still references to objects in the
    PC-Directory. That's why it is not possible to change a location of a
    virtual object - the only exceptions are URL items and Active items.

 2. Once an item is added to the PC Directory it is forwarded to the virtual
    object engine. The virtual object engine's mission is to organize and
    present the media database in a logical hierarchy based on the available
    metadata of the items.

Each UPnP server implements this so called virtual object hierarchy in a
different way. Audio files are usually sorted by artist, album, some servers
may just present a view similar to the file system and so on. Most servers have
strong limitations on the structure of the virtual containers, they usually
offer a predefined layout of data and the user has to live with it. In
MediaTomb we try to address this shortcoming by introducing the scriptable
virtual object engine. It is designed to be:

  * maximally flexible

  * easily customizable and extendable

  * robust and efficient

We try to achieve these goals by embedding a scripting runtime environment that
allows the execution of ECMAScript-262 conform scripts better known as
JavaScript. We are using Mozilla's JavaScript implementation called
SpiderMonkey, it is a stand-alone easily embeddable javascript engine,
supporting JavaScript versions 1.0 through 1.4.

2.2. Theory Of Operation

After an item is added to the PC Directory it is automatically fed as input to
the import script. The script then creates one or more virtual items for the
given original item. Items created from scripts are always marked virtual.

When the virtual object engine gets notified of an added item, following
happens: a javascript object is created mirroring the properties of the item.
The object is introduced to the script environment and bound to the predefined
variable 'orig'. This way a variable orig is always defined for every script
invocation and represents the original data of the added item. Then the script
is invoked.

In the current implementation, if you modify the script then you will have to
restart the server for the new logic to take effect. Note, that the script is
only triggered when new objects are added to the database, also note that the
script does not modify any objects that already exist in the database - it only
processes new objects that are being added. When a playlist item is
encountered, it is automatically fed as input to the playlist script. The
playlist script attempts to parse the playlist and adds new item to the
database, the item is then processed by the import script.

3. Global Variables And Constants

In this section we will introduce the properties of the object that will be
processed by the script, as well as functions that are offered by the server.

3.1. The Media Object

As described in Section 2.2, each time an item is added to the database the
import script is invoked. So, one script invocation processes exactly one non
virtual item, and creates a number of virtual items and containers. The
original item is made available in the form of the global variable 'orig'.
Additionally, when the object being imported is a playlist, it is made
available to the playlist parser script in the form of the global variable
'playlist'. It is usually a good idea to only read from these variables and to
create and only modify local copies.

Note:

    modifying the properties of the orig object will not propagate the changes
    to the database, only a call to the addCdsObject() will permanently add the
    object.

3.1.1. General Properties

Here is a list of properties of an object, you can set them you create a new
object or when you modify a copy of the 'orig' object.

RW means read/write, i.e. - changes made to that property will be transferred
into the database.

RO means, that this is a read only property, any changes made to it will get
lost.

  * orig.objectType

    RW

    This defines the object type, following types are available:

      o OBJECT_TYPE_CONTAINER

        Object is a container.

      o OBJECT_TYPE_ITEM

        Object is an item.

      o OBJECT_TYPE_ACTIVE_ITEM

        Object is an active item.

      o OBJECT_TYPE_ITEM_EXTERNAL_URL

        Object is a link to a resource on the Internet.

      o OBJECT_TYPE_ITEM_INTERNAL_URL

        Object is an internal link.

  * orig.title

    RW

    This is the title of the original object, since the object represents an
    entry in the PC-Directory, the title will be set to it's file name. This
    field corresponds to dc:title in the DIDL-Lite XML.

  * orig.id

    RO

    The object ID, make sure to set all refID's (reference IDs) of your virtual
    objects to that ID.

  * orig.parentID

    RO

    The object ID of the parent container.

  * orig.upnpclass

    RW

    The UPnP class of the item, this corresponds to upnp:class in the DIDL-Lite
    XML.

  * orig.location

    RO

    Location on disk, given by the absolute path and file name.

  * orig.theora

    RO

    This property is a boolean value, it is non zero if the particular item is
    of type OGG Theora. This is useful to allow proper sorting of media and
    thus placing OGG Vorbis into the Audio container and OGG Theora into the
    Video container.

  * orig.onlineservice

    RO

    Identifies if the item belongs to an online service and thus has extended
    properties. Following types are available:

      o ONLINE_SERVICE_NONE

        The item does not belong to an online service and does not have
        extended properties.

  * orig.mimetype

    RW

    Mimetype of the object.

  * orig.meta

    RW

    Array holding the metadata that was extracted from the object (i.e. id3/
    exif/etc. information)

      o orig.meta[M_TITLE]

        RW

        Extracted title (for example the id3 title if the object is an mp3
        file), if you want that your new virtual object is displayed under this
        title you will have to set obj.title = orig.meta[M_TITLE]

      o orig.meta[M_ARTIST]

        RW

        Artist information, this corresponds to upnp:artist in the DIDL-Lite
        XML.

      o orig.meta[M_ALBUM]

        RW

        Album information, this corresponds to upnp:album in the DIDL-Lite XML.

      o orig.meta[M_DATE]

        RW

        Date, must be in the format of YYYY-MM-DD (required by the UPnP spec),
        this corresponds to dc:date in the DIDL-Lite XML.

      o orig.meta[M_GENRE]

        RW

        Genre of the item, this corresponds to upnp:genre in the DIDL-Lite XML.

      o orig.meta[M_DESCRIPTION]

        RW

        Description of the item, this corresponds to dc:description in the
        DIDL-Lite XML.

      o orig.meta[M_REGION]

        RW

        Region description of the item, this corresponds to upnp:region in the
        DIDL-Lite XML.

      o orig.meta[M_TRACKNUMBER]

        RW

        Track number of the item, this corresponds to upnp:originalTrackNumber
        in the DIDL-Lite XML.

      o orig.meta[M_AUTHOR]

        RW

        Author of the media, this corresponds to upnp:author in the DIDL-Lite
        XML.

      o orig.meta[M_DIRECTOR]

        RW

        Director of the media, this corresponds to upnp:director in the
        DIDL-Lite XML.

  * orig.aux

    RO

    Array holding the so called auxiliary data. Aux data is metadata that is
    not part of UPnP, for example - this can be a camera model that was used to
    make a photo, or the information if the photo was taken with or without
    flash. Currently aux data can be gathered from libexif and libextractor
    (see the Import section in the main documentation for more details). So,
    this array will hold the tags that you specified in your config.xml,
    allowing you to create your virtual structure according to your liking.

  * orig.playlistOrder

    RW

    This property is only available if the object is being created by the
    playlist script. It's similar to ID3 track number, but is used to set the
    position of the newly created object inside a parsed playlist container.
    Usually you will increment the number for each new object that you create
    while parsing the playlist, thus ensuring that the resulting order is the
    same as in the original playlist.

3.2. Constants

Actually there are no such things as constants in JS, so those are actually
predefined global variables that are set during JS engine initialization. Do
not assign any values to them, otherwise following script invocation will be
using wrong values.

  * UPNP_CLASS_CONTAINER

    Type: string

    Value: object.container

  * UPNP_CLASS_CONTAINER_MUSIC

    Type: string

    Value: object.container.musicContainer

  * UPNP_CLASS_CONTAINER_MUSIC_ARTIST

    Type: string

    Value: object.container.person.musicArtist

  * UPNP_CLASS_CONTAINER_MUSIC_GENRE

    Type: string

    Value: object.container.genre.musicGenre

  * UPNP_CLASS_CONTAINER_MUSIC_ALBUM

    Type: string

    Value: object.container.album.musicAlbum

    Note:

        this container class will be treated by the server in a special way,
        all music items in this container will be sorted by ID3 track number.

  * UPNP_CLASS_PLAYLIST_CONTAINER

    Type: string

    Value: object.container.playlistContainer

    Note:

        this container class will be treated by the server in a special way,
        all items in this container will be sorted by the number specified in
        the playlistOrder property (this is set when an object is created by
        the playlist script).

  * UPNP_CLASS_ITEM

    Type: string

    Value: object.item

  * UPNP_CLASS_ITEM_MUSIC_TRACK

    Type: string

    Value: object.item.audioItem.musicTrack

  * OBJECT_TYPE_CONTAINER

    Type: integer

    Value: 1

  * OBJECT_TYPE_ITEM

    Type: integer

    Value: 2

  * OBJECT_TYPE_ACTIVE_ITEM

    Type: integer

    Value: 4

  * OBJECT_TYPE_ITEM_EXTERNAL_URL

    Type: integer

    Value: 8

  * OBJECT_TYPE_ITEM_INTERNAL_URL

    Type: integer

    Value: 16

4. Functions

The server offers various native functions that can be called from the scripts,
additionally there are some js helper functions that can be used.

4.1. Native Server Functions

The so called native functions are implemented in C++ in the server and can be
called from the scripts.

4.1.1. Native Functions Available To All Scripts

The server offers three functions which can be called from within the import
and/or the playlist script:

  * addCdsObject(object, containerChain, lastContainerClass);

    This function adds a virtual object to the server database, the path in the
    database is defined by the containerChain parameter. The third argument is
    optional, it allows to set the upnp:class of the last container in the
    chain.

    Parameters:

      o object

        A virtual object that is either a copy of or a reference to 'orig', see
        Section 3.2 for a list of properties.

      o containerChain

        A string, defining where the object will be added in the database
        hierarchy. The containers in the chain are separated by a slash '/',
        for example, a value of '/Audio/All Music' will add the object to the
        Audio, All Music container in the server hierarchy. Make sure to
        properly escape the slash characters in container names. You will find
        more information on container chain escaping later in this chapter.

      o lastContainerClass

        A string, defining the upnp:class of the container that appears last in
        the chain. This parameter can be omitted, in this case the default
        value 'object.container' will be taken. Setting specific upnp container
        classes is useful to define the special meaning of a particular
        container; for example, the server will always sort songs by track
        number if upnp class of a container is set to
        'object.container.album.musicAlbum'.

  * copyObject(originalObject);

    This function returns a copy of the virtual object.

  * print(...);

    This function is useful for debugging scripts, it simply prints to the
    standard output.

  * f2i(string)

  * m2i(string)

  * p2i(string)

  * j2i(string)

    The above set of functions converts predefined characters sets to UTF-8.
    The 'from' charsets can be defined in the server configuration:

      o f2i: filesystem charset to internal

      o m2i: metadata charset to internal

      o j2i: js charset to internal

      o p2i: playlist charset to internal

4.1.2. Native Functions Available To The Playlist Script

The following function is only available to the playlist script.

  * readln();

    This function reads and returns exactly one line of text from the playlist
    that is currently being processed, end of line is identified by carriage
    return/line feed characters. Each subsequent call will return the next
    line, there is no way to go back.

    The idea is, that you can process your playlist line by line and gather the
    required information to create new objects which can be added to the
    database.

4.2. Helper Functions

There is a set of helper JavaScript functions which reside in the common.js
script. They can be used by the import and by the playlist script.

  * function escapeSlash(name);


    The first function escapes slash '/' characters in a string. This is
    necessary, because the container chain is defined by a slash separated
    string, where slash has a special meaning - it defines the container
    hierarchy. That means, that slashes that appear in the object's title need
    to be properly escaped.

  * 

    The following function makes it easier to work with container chains; it
    takes an array of container names as argument, makes sure that the names
    are properly escaped and adds the slash separators as necessary. It returns
    a string that is formatted to be used as a parameter for the addCdsObject
    function.

    function createContainerChain(arr)
    {
        var path = '';
        for (var i = 0; i < arr.length; i++)
        {
            path = path + '/' + escapeSlash(arr[i]);
        }
        return path;
    }

  * This function retrieves the year from a yyyy-mm-dd formatted string.

    function getYear(date)
    {
        var matches = date.match(/^([0-9]{4})-/);
        if (matches)
            return matches[1];
        else
            return date;
    }

  * This function identifies the type of the playlist by the mimetype, it is
    used in the playlist script to select an appropriate parser.

    function getPlaylistType(mimetype)
    {
        if (mimetype == 'audio/x-mpegurl')
            return 'm3u';
        if (mimetype == 'audio/x-scpls')
            return 'pls';
        return '';
    }

5. Walkthrough

Now it is time to take a closer look at the default scripts that are supplied
with MediaTomb. Usually it is installed in the /usr/share/mediatomb/js/
directory, but you will also find it in scripts/js/ in the MediaTomb source
tree.

Note:

    this is not a JavaScript tutorial, if you are new to JS you should probably
    make yourself familiar with the language.

5.1. Import Script

We start with a walkthrough of the default import script, it is called
import.js in the MediaTomb distribution.

Below are the import script functions that organize our content in the database
by creating the virtual structure. Each media type - audio, image and video is
handled by a separate function.

5.1.1. Audio Content Handler

The biggest one is the function that handles audio - the reason is simple: mp3
files offer a lot of metadata like album, artist, genre, etc. information, this
allows us to create a nice container layout.

function addAudio(obj)
{
    var desc = '';
    var artist_full;
    var album_full;

First we will gather all the metadata that is provided by our object, of course
it is possible that some fields are empty - we will have to check that to make
sure that we handle this case correctly.

    var title = obj.meta[M_TITLE];

Note the difference between obj.title and obj.meta[M_TITLE] - while
object.title will originally be set to the file name, obj.meta[M_TITLE] will
contain the parsed title - in this particular example the ID3 title of an MP3.


    if (!title) title = obj.title;
    var artist = obj.meta[M_ARTIST];
    if (!artist)
    {
        artist = 'Unknown';
        artist_full = null;
    }
    else
    {
        artist_full = artist;
        desc = artist;
    }

   var album = obj.meta[M_ALBUM];
   if (!album)
    {
        album = 'Unknown';
        album_full = null;
    }
    else
    {
        desc = desc + ', ' + album;
        album_full = album;
    }

    if (desc)
        desc = desc + ', ';

    desc = desc + title;

    var date = obj.meta[M_DATE];
    if (!date)
    {
        date = 'Unknown';
    }
    else
    {
        date = normalizeDate(date);
        desc = desc + ', ' + date;
    }

    var genre = obj.meta[M_GENRE];
    if (!genre)
    {
        genre = 'Unknown';
    }
    else
    {
        desc = desc + ', ' + genre;
    }

    var description = obj.meta[M_DESCRIPTION];
    if (!description)
    {

Note how we are setting properties of an object - in this case we put together
a description and we are setting for objects that did not already have one.

        obj.meta[M_DESCRIPTION] = desc;
    }

We finally gathered all data that we need, so let's create a nice layout for
our audio files. Note how we are constructing the chain, in the line below the
array 'chain' will be converted to 'Audio/All audio' by the
createContainerChain() function.

    var chain = new Array('Audio', 'All audio');
    obj.title = title;

The UPnP class argument to addCdsObject() is optional, if it is not supplied
the default UPnP class will be used. However, it is suggested to correctly set
UPnP classes of containers and objects - this information may be used by some
renderers to identify the type of the container and present the content in a
different manner .

    addCdsObject(obj, createContainerChain(chain), UPNP_CLASS_CONTAINER_MUSIC);

    chain = new Array('Audio', 'Artists', artist, 'All songs');
    addCdsObject(obj, createContainerChain(chain), UPNP_CLASS_CONTAINER_MUSIC);

    chain = new Array('Audio', 'All - full name');
    var temp = '';
    if (artist_full)
        temp = artist_full;

    if (album_full)
        temp = temp + ' - ' + album_full + ' - ';

    obj.title = temp + title;

    addCdsObject(obj, createContainerChain(chain), UPNP_CLASS_CONTAINER_MUSIC);

    chain = new Array('Audio', 'Artists', artist, 'All - full name');
    addCdsObject(obj, createContainerChain(chain), UPNP_CLASS_CONTAINER_MUSIC);

    chain = new Array('Audio', 'Artists', artist, album);
    obj.title = track + title;

Remember, the server will sort all items by ID3 track if the container class is
set to UPNP_CLASS_CONTAINER_MUSIC_ALBUM.

    addCdsObject(obj, createContainerChain(chain), UPNP_CLASS_CONTAINER_MUSIC_ALBUM);

    chain = new Array('Audio', 'Albums', album);
    obj.title = track + title;
    addCdsObject(obj, createContainerChain(chain), UPNP_CLASS_CONTAINER_MUSIC_ALBUM);

    chain = new Array('Audio', 'Genres', genre);
    addCdsObject(obj, createContainerChain(chain), UPNP_CLASS_CONTAINER_MUSIC_GENRE);

    chain = new Array('Audio', 'Year', date);
    addCdsObject(obj, createContainerChain(chain), UPNP_CLASS_CONTAINER_MUSIC);
}

5.1.2. Image Content Handler

This function takes care of images. Currently it does very little sorting, but
could easily be extended - photos made by digital cameras provide lots of
information in the Exif tag, so you could easily add code to sort your pictures
by camera model or anything Exif field you might be interested in.

Note:

    if you want to use those additional Exif fields you need to compile
    MediaTomb with libexif support and also specify the fields of interest in
    the import section of your configuration file (See documentation about
    library-options).

function addImage(obj)
{
    var chain = new Array('Photos', 'All Photos');
    addCdsObject(obj, createContainerChain(chain), UPNP_CLASS_CONTAINER);

    var date = obj.meta[M_DATE];
    if (date)
    {
        chain = new Array('Photos', 'Date', date);
        addCdsObject(obj, createContainerChain(chain), UPNP_CLASS_CONTAINER);
    }
}

Just like in the addAudio() function - we simply construct our container chain
and add the object.

5.1.3. Video Content Handler

Not much to say here... I think libextractor is capable of retrieving some
information from video files, however I seldom encountered any video files
populated with metadata. You could also try ffmpeg to get more information,
however by default we keep it very simple - we just put everything into the
'All Video' container.

function addVideo(obj)
{
    var chain = new Array('Video');
    addCdsObject(obj, createContainerChain(chain));
}

5.1.4. Putting it all together

This is the main part of the script, it looks at the mimetype of the original
object and feeds the object to the appropriate content handler.

if (getPlaylistType(orig.mimetype) == '')
{
    var arr = orig.mimetype.split('/');
    var mime = arr[0];

    var obj = orig;

All virtual objects are references to objects in the PC-Directory, so make sure
to correctly set the reference ID!

    obj.refID = orig.id;

    if ((mime == 'audio'))
    {
        addAudio(obj);
    }

    if (mime == 'video')
    {
        addVideo(obj);
    }

    if (mime == 'image')
    {
         addImage(obj);
    }

We now also have OGG Theora recognition, so we can ensure that Vorbis

    if (orig.mimetype == 'application/ogg')
    {
    if (obj.theora == 1)
            addVideo(obj);
        else
            addAudio(obj);
    }
}

5.2. Playlist Script

The default playlist parsing script is called playlists.js, similar to the
import script it works with a global object which is called 'playlist', the
fields are similar to the 'orig' that is used in the import script with the
exception of the playlistOrder field which is special to playlists.

Another big difference between playlist and import scripts is, that playlist
scripts can add new media to the database, while import scripts only process
already existing objects (the ones found in PC Directory) and just add
additional virtual items.

The default playlist script implementation supports parsing of m3u and pls
formats, but you can add support for parsing of any ASCII based playlist
format.

5.2.1. Adding Items

We will first look at a helper function:

addPlaylistItem(location, title, playlistChain);

It is defined in playlists.js, it receives the location (path on disk or HTTP
URL), the title and the desired position of the item in the database layout
(remember the container chains used in the import script).

The function first decides if we are dealing with an item that represents a
resource on the web, or if we are dealing with a local file. After that it
populates all item fields accordingly and calls the addCdsObject() that was
introduced earlier. Note, that if the object that is being added by the
playlist script is not yet in the database, the import script will be invoked.
Below is the complete function with some comments:

function addPlaylistItem(location, title, playlistChain)
{

Determine if the item that we got is an URL or a local file.

    if (location.match(/^.*:\/\//))
    {
        var exturl = new Object();

Setting the mimetype is crucial and tricky... if you get it wrong your renderer
may show the item as unsupported and refuse to play it. Unfortunately most
playlist formats do not provide any mimetype information.

        exturl.mimetype = 'audio/mpeg';

Make sure to correctly set the object type, then populate the remaining fields.

        exturl.objectType = OBJECT_TYPE_ITEM_EXTERNAL_URL;
        exturl.location = location;
        exturl.title = (title ? title : location);
        exturl.protocol = 'http-get';
        exturl.upnpclass = UPNP_CLASS_ITEM_MUSIC_TRACK;
        exturl.description = "Song from " + playlist.title;

This is a special field which ensures that your playlist files will be
displayed in the correct order inside a playlist container. It is similar to
the id3 track number that is used to sort the media in album containers.

        exturl.playlistOrder = playlistOrder++;

Your item will be added to the container named by the playlist that you are
currently parsing.

        addCdsObject(exturl, playlistChain,  UPNP_CLASS_PLAYLIST_CONTAINER);
    }

Here we are dealing with a local file.

    else
    {
        if (location.substr(0,1) != '/')
            location = playlistLocation + location;
        var item  = new Object();
        item.location = location;
        if (title)
            item.title = title;
        else
         {
            var locationParts = location.split('/');
            item.title = locationParts[locationParts.length - 1];
            if (! item.title)
                item.title = location;
        }
        item.objectType = OBJECT_TYPE_ITEM;
        item.playlistOrder = playlistOrder++;
        addCdsObject(item, playlistChain,  UPNP_CLASS_PLAYLIST_CONTAINER);
    }
}

5.2.2. Main Parsing

The actual parsing is done in the main part of the script. First, the type of
the playlist is determined (based on the playlist mimetype), then the correct
parser is chosen. The parsing itself is a loop, where each call to readln()
returns exactly one line of text from the playlist. There is no possibility to
go back, each readln() invocation will retrieve the next line until end of file
is reached.

To keep things easy we will only list the m3u parsing here. Again, if you are
not familiar with regular expressions, now is probably the time to take a
closer look.

...
else if (type == 'm3u')
{
    var line;
    var title = null;

Here is the do - while loop which will read the playlist line by line.

    do
    {

Read the line:

        line = readln();

Perform m3u specific parsing:

        if (line.match(/^#EXTINF:(\d+),(\S.+)$/i))
        {
            // duration = RegExp.$1; // currently unused
            title = RegExp.$2;
        }
        else if (! line.match(/^(#|\s*$)/))
        {

Call the helper function to add the item once you gathered the data:

            addPlaylistItem(line, title, playlistChain);
            title = null;
        }
    }

We will exit the loop when end of the playlist file is reached.

    while (line);
}
...

Happy scripting!

