tlRender, or timeline render, is an early stage project for rendering editorial timelines

Last update: Dec 30, 2022

Related tags

Overview

tlRender

tlRender, or timeline render, is an early stage project for rendering editorial timelines.

The project includes libraries for rendering timelines with OpenGL and Qt, and example applications showing the usage of the libraries.

The project is written in C++ and uses OpenTimelineIO for reading timelines. FFmpeg (https://ffmpeg.org/) and other open source libraries are used for movie and image I/O.

The tlRender code is provided under a BSD style open source license.

Supported:

Movie clips
Image sequences (Cineon, DPX, JPEG, OpenEXR, PNG, TIFF)
OpenColorIO

In progress:

Transitions
Rendering to files

To do:

Effects
Audio support
Hardware movie decoding
Nested timelines
Python bindings

Libraries

tlrCore

The core library providing timeline functionality, file I/O, and other utilities.

Dependencies:

Optional dependencies:

tlrGL

Timeline rendering using OpenGL.

Dependencies:

tlrQt

Qt widgets and objects for working with timelines.

Dependencies:

tlrApp

Utility library for the example applications.

Examples

tlrplay-glfw

The example "tlrplay-glfw" plays back timelines using a GLFW based application.

Timelines can be opened from the command line. A HUD (heads up display), keyboard shortcuts, and command line options provide controls for the application.

Dependencies:

GLFW

tlrplay-qwidget

The example "tlrplay-qwidget" plays back timelines using a QWidget based application.

Timelines can be opened from the "File/Open" menu, drag and dropped onto the window, or from the command line. Menus, playback buttons, a frame slider, and keyboard shortcuts provide controls for the application.

tlrbake-glfw

The example "tlrbake-glfw" is a GLFW based command-line application for rendering a timeline to a movie file or image file sequence.

Building

Dependencies

A CMake super build script is provided to build the dependencies from source.

Note however that Qt is not included in the super build, you must install it separately.

CMake Build Options

TLR_ENABLE_PYTHON - Enable Python support (for OTIO Python adapters)
TLR_BUILD_FFmpeg - Build FFmpeg support (Linux and macOS only)
TLR_BUILD_JPEG - Build JPEG support
TLR_BUILD_OpenEXR - Build OpenEXR support
TLR_BUILD_PNG - Build PNG support
TLR_BUILD_TIFF - Build TIFF support
TLR_BUILD_GL - Build OpenGL library (tlrGL)
TLR_BUILD_QT - Build QT library (tlrQt)
TLR_BUILD_EXAMPLES - Build examples

Building on Linux

Clone the repository:

git clone https://github.com/darbyjohnston/tlRender.git
cd tlRender

Create a build directory:

mkdir build
cd build

Run CMake with the super build script:

cmake ../etc/SuperBuild/ -DCMAKE_INSTALL_PREFIX=$PWD/install -DCMAKE_PREFIX_PATH=$PWD/install -DCMAKE_BUILD_TYPE=Debug

Start the build:

cmake --build . -j 4

Try running the "tlrplay-glfw" application:

export LD_LIBRARY_PATH=$PWD/install/lib:$LD_LIBRARY_PATH
./install/bin/tlrplay-glfw ../etc/SampleData/multiple_clips.otio

Building on Linux with Qt

Clone the repository:

git clone https://github.com/darbyjohnston/tlRender.git
cd tlRender

Create a build directory:

mkdir build
cd build

Run CMake with the super build script, adding the location of "Qt5Config.cmake" to "CMAKE_PREFIX_PATH" (make sure to use quotes), and enabling "TLR_BUILD_QT":

cmake ../etc/SuperBuild/ -DCMAKE_INSTALL_PREFIX=$PWD/install -DCMAKE_PREFIX_PATH="$PWD/install;$HOME/Qt/5.15.2/gcc_64/lib/cmake/Qt5" -DTLR_BUILD_QT=ON -DCMAKE_BUILD_TYPE=Debug

Start the build:

cmake --build . -j 4

Try running the "tlrplay-qwidget" application:

export LD_LIBRARY_PATH=$PWD/install/lib:$LD_LIBRARY_PATH
./install/bin/tlrplay-qwidget ../etc/SampleData/multiple_clips.otio

Notes for building on Linux

Running gcovr for viewing code coverage:

gcovr -r ../../../../tlRender/lib/tlrCore --html --object-directory $PWD --html-details --output gcov.html

Building on macOS

Clone the repository:

git clone https://github.com/darbyjohnston/tlRender.git
cd tlRender

Create a build directory:

mkdir build
cd build

Run CMake with the super build script:

cmake ../etc/SuperBuild/ -DCMAKE_INSTALL_PREFIX=$PWD/install -DCMAKE_PREFIX_PATH=$PWD/install -DCMAKE_BUILD_TYPE=Debug

Start the build:

cmake --build . -j 4

Try running the "tlrplay-glfw" application:

./install/bin/tlrplay-glfw ../etc/SampleData/multiple_clips.otio

Building on macOS with Qt

Clone the repository:

git clone https://github.com/darbyjohnston/tlRender.git
cd tlRender

Create a build directory:

mkdir build
cd build

Run CMake with the super build script, adding the location of "Qt5Config.cmake" to "CMAKE_PREFIX_PATH" (make sure to use quotes), and enabling "TLR_BUILD_QT":

cmake ../etc/SuperBuild/ -DCMAKE_INSTALL_PREFIX=$PWD/install -DCMAKE_PREFIX_PATH="$PWD/install;$HOME/Qt/5.15.2/clang_64/lib/cmake/Qt5" -DTLR_BUILD_QT=ON -DCMAKE_BUILD_TYPE=Debug

Start the build:

cmake --build . -j 4

Try running the "tlrplay-qwidget" application:

./tlRender/src/tlRender-build/bin/tlrplay-qwidget/tlrplay-qwidget ../etc/SampleData/multiple_clips.otio

Notes for building on macOS

The CMake variable "CMAKE_OSX_ARCHITECTURES" can be used to specify the build architecture:

-DCMAKE_OSX_ARCHITECTURES=x86_64

Building FFmpeg on Windows

Most of the third party software that tlRender depends upon is built as part of the CMake super build, except for FFmpeg on Windows. Instead the Windows Subsystem for Linux (WSL) is used to compile FFmpeg as a separate step before the CMake super build.

Enable the Windows Subsystem for Linux:

Open the Windows control panel and click on "Programs and Features"
Click on "Turn Windows features on or off" on the left side of the "Programs and Features" window
Check the "Windows Subsystem for Linux" item in the "Windows Features" window
Restart your computer

Install Ubuntu from the Windows app store, then open a shell and install necessary software:

sudo apt update
sudo apt install mingw-w64 yasm make unzip

Build FFmpeg, replacing $SOURCE_DIR and $BUILD_DIR with the same directories used in the "Building on Windows" section:

$SOURCE_DIR/etc/Windows/build_ffmpeg_wsl.sh $BUILD_DIR/install

Building on Windows

Clone the repository:

git clone https://github.com/darbyjohnston/tlRender.git
cd tlRender

Create a build directory:

mkdir build
cd build

Run CMake with the super build script:

cmake ../etc/SuperBuild/ -DCMAKE_INSTALL_PREFIX=%CD%/install -DCMAKE_PREFIX_PATH=%CD%/install -DCMAKE_BUILD_TYPE=Debug

Start the build:

cmake --build . -j 4 --config Debug

Try running the "tlrplay-glfw" application:

set PATH=%CD%\install\bin;%PATH%
.\install\bin\tlrplay-glfw ..\etc\SampleData\multiple_clips.otio

Building on Windows with Qt

Clone the repository:

git clone https://github.com/darbyjohnston/tlRender.git
cd tlRender

Create a build directory:

mkdir build
cd build

Run CMake with the super build script, adding the location of "Qt5Config.cmake" to "CMAKE_PREFIX_PATH" (make sure to use quotes), and enabling "TLR_BUILD_QT":

cmake ../etc/SuperBuild/ -DCMAKE_INSTALL_PREFIX=%CD%/install -DCMAKE_PREFIX_PATH="%CD%/install;C:\Qt\5.15.2\msvc2019_64\lib\cmake\Qt5" -DTLR_BUILD_QT=ON -DCMAKE_BUILD_TYPE=Debug

Start the build:

cmake --build . -j 4 --config Debug

Try running the "tlrplay-qwidget" application:

set PATH=%CD%\install\bin;%PATH%
.\install\bin\tlrplay-qwidget ..\etc\SampleData\multiple_clips.otio

Comments

Added beginRaster() and endRaster() to draw text in raster coordinates.

Here it is again the proposed API to render text in raster coordinates. The code is identical to that of Render::begin(), Render::end() but the canvas is not cleared and the shader is not created (it is assumed it was created before hand).

The Render::endRaster() function is a no-op. Only the Render::beginRaster(imaging::Size& viewportSize) changes the text shader matrix.

If the beginRaster() function is called before the Render::begin(), it will also not change the shader matrix.

opened by ggarra13 9
Fixes color pipeline to match exrdisplay math.

The gamma function needs to be applied as part of the exposure math. Note, however, that it makes the levels' checks useless. To fix them, they also would need to move to the exposureFunc() function.

opened by ggarra13 9
expat dependency in OCIO
There's an expat dependency in OCIO. I think the super-build should be responsible for fetching that bit... Building on macOS yields this (after I hack in TLR_PLATFORM_MACOS):

ld: library not found for -lOCIO_expat-NOTFOUND

I think the issue is perhaps with OCIO? expat in theory is packaged as a system library...
bug
opened by meshula 9
Added drawRectOutline function.

This function draws one pixel rectangles perfectly. It works much better for that than drawing a mesh as that outline can disappear when zoomed out on the display.

opened by ggarra13 6
Added matrix to drawMesh().

This is needed to support mvp projection for scaling and translation (zoom and panning) of the primitive together with the main viewport drawing of the image. Render::begin() just sets a raster projection without taking into account zooming and panning.

opened by ggarra13 6
Compiling libtiff with deflate support leads to undefined references

When libdeflate.a (or libdeflate.so) is found in one of the library dirs, compilation of the example programs fails with errors like:

tif_zip.c:(.text+0xfe): undefined reference to `libdeflate_free_decompressor'

The quick fix is to add:

-Dlibdeflate=OFF to the CMAKE_ARGS of BuildTIFF.cmake.

A better fix would be to include the deflate library in the linking phase, albeit I was unable to find where:

find_package(Deflate)

is found to see what library variable it sets.
bug

opened by ggarra13 4
Audio starts late

A commit made the audio start late. When the audio was added with the -a flag, all was still good. I didn't do a bisect, but audio was fine at least up to:

b476f32b5d121fec19615a125e8853512ac9c0de
bug

opened by ggarra13 4
Is a time comparison sufficient here?
_timelinePlayer->tick(); const auto& frame = _timelinePlayer->observeFrame()->get(); if (frame != _frame) { _frame = frame; _renderDirty = true; }

Efficiency question... It's sufficient just to compare frame.time, isn't it? the frame comparison also compares layers. Even if layers changed mid frame, we've already rendered for the time, in other words if time didn't change and layers did, I don't think it's necessary to trap it.
enhancement
opened by meshula 4
Small? movies seem to display black

BART_2021-02-07.m4v, PSR63_2012-06-02.mov and Dinky_2015-06-11.m4v display black on all players (linux tested only), at least on this commit:

d4f302f9666e4a5109701db16e2e3398d4239bbd
bug

opened by ggarra13 3
4K movies take too long to start

Using play-glfw on a 4K mp4 movie takes tlRender about 4 seconds to start. With tlplay creating the thumbnails it is much worse and playback is not smooth at least until the thumbnails are created.

In comparison, mrViewer opens the 4K movie in 1 second or less.
bug

opened by ggarra13 2
Fixes color pipeline to match exrdisplay math.

Returned gamma to its original place. Simplified shader.

Note, however, that colorsEnabled has to be checked off when messing with the gamma or else the gamma will be applied twice.
bug

opened by ggarra13 1
Add support for versioning (replacing) of clips

A feature of mrViewer is that it allows changing versions of a clip (movie or sequence). This would be an addition to the FilesModel to allow replacing a file with another one (and the likely recreation of the underlying timeline).

The versioning (using a regex or other convention to match versions) can be done by the viewing application.
enhancement

opened by ggarra13 7