VALEO Tutorial

VALEO lets you control external programs from Erlang.

1. Supervise an Erlang VM

In this first example, we’ll use Valeo to control an Erlang VM with some arguments.

%% copy the following into a file called: "vm.erl"
-module(vm).

-export([
         start/0, stop/1,
         %% behaviour callbacks
         valeo_init/1,
         valeo_handle_info/2,
         valeo_terminate/1
        ]).

-behavior(valeo).

start() -> %% run an Erlang VM under VALEO supervision
   ProgArgs =[
             {cmd_name, "erl"},
             {cmd_params, ["+Ktrue", "+B", "-noshell", {"-pa", "ebin"}]}
             ],
   PortArgs =[
              {port_id, ?MODULE},
              {port_pidfile, "vm.pid"}
             ],
   valeo:start_link(?MODULE, ProgArgs, PortArgs).

stop(Port) -> %% stop the Erlang VM
   valeo:stop(Port).

%% --- callback functions ---
valeo_init(_) ->
   {ok, []}.

valeo_handle_info({exit_status, 0}, State) ->
   {stop, normal}; %% normal exit of the VM, do not restart
valeo_handle_info({exit_status, Status}, State) ->
   io:format("VM exited with error code: ~p", [Status]),
   {stop, need_restart}. %% always restart-it

valeo_terminate(State) ->
   ok. %% cleanup

Compile:

c:\> erlc vm.erl

Then run:

c:\> werl
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:2:2] [async-threads:0] [kernel-poll:false]
Eshell V5.9.1  (abort with ^G)

1> {ok, Port} = vm:start().

2> valeo:proc_id(Port). %% returns the OS process ID
5056

3> valeo:stop(Port).

4> q().

We can check that the returned OS process ID is the same, 5056 in the example above.

The Windows Task Manager shows (use top on Unix):

Valeo as “heart” replacement?

Erlang already provides a heartbeat-based monitoring capability for

its runtime systems, with a restart ability if it fails

(http://erlang.org/doc/man/heart.html).

But heartbeat suffers from few drawbacks and can be annoying when

trying to shut down a misbehaving Erlang system. Try to kill the Erlang

process and heartbeat will restart it, and vice versa (you kill the heartbeat

process and the Erlang process will restart it).

As shown in the example, Valeo can be seen as a simple, viable and

portable alternative.

Recap

• Lines 2,10: declare a new module called vm which exports a public function called start/0 and three Valeo’s callbacks functions.
• Line 12: declare the custom behaviour Valeo (see OTP Design Principles for behaviour).
• The code starts a Valeo instance which runs an Erlang VM with a couple of arguments (“+Ktrue”, “+B”, etc.)
• Line 15: defines a proplists ProgArgs of the form [{cmd_name, “CMD”},{cmd_params,[“param1”,...]}] witch defines the external program to run and any optional parameters to pass as command line arguments.
  Valeo will always search the command specified by cmd_name in the local priv directory of the calling module, and then in the PATH environment variable.
  In this example, the user simply specified the erl command without using its full path (ex. /usr/bin/erl).
• Line 19: two fields are defined: port_id (an atom representing a locally registered name of the Valeo instance,

  used by a supervisor for example), and the mandatory port_pidfile (a temporary file containing the OS process ID).

• Line 29: the callback function valeo_init/1 is very similar to gen_server one. Use it to keep a state (ex. []) during the Valeo instance execution. Its parameter is the current process id (self()).

• Lines 32,36: the callback function valeo_handle_info/2 is called by Valeo when a timeout occurs or when it receives

  any message from the external port program. Here, the user only gets notified when the Erlang VM exits either normally or not (ex. crash).
• Line 38: the callback function valeo_terminate/1 is called by Valeo when it is about to terminate. It should be the opposite of valeo_init/1

  and do any necessary cleaning up.
• Line 25: stop the program under supervision.

2. Live Video Stream Archiving

In this more advanced example of Valeo usage. We will supervise an instance of the popular VLC media player to grab some RTSP stream

for archiving purposes.

Let’s see how to proceed!

%% copy the following into a file called: "vlc.erl"
-module(vlc).

-export([
         start/0, stop/1,
         %% behaviour callbacks
         valeo_init/1,
         valeo_handle_info/2,
         valeo_terminate/1
        ]).

-behavior(valeo).

-define(VLC_PARAMS(Dur,Stm,Out,Mux), "--stop-time " ++
  Dur ++ " --run-time=" ++ Dur ++ " " ++ Stm        ++
  "--sout='#std{access=file,mux=" ++ Mux ++ ",dst=" ++
  Out ++ "}' vlc://quit").

%%--------------------------------------------------------------------
%% @doc Drive VLC player from command line.
%% @end
%%--------------------------------------------------------------------
start() ->
   Stream   = "rtsp://media6.lsops.net:1935/live/bbcworld1_en_high.sdp",
   Output   = "bbc.mp4", %% output file
   Duration = 30, %% 30 sec.
   Muxer    = "ts",
   PidFile  = filename:basename(Output ++ ".pid"), %% must be unique name

   ProgArgs = [
               {cmd_name, "vlc"},
               {cmd_params,
                ["-I dummy", %% run without GUI (console mode)
                 "-q", %% quiet mode
                 ?VLC_PARAMS(integer_to_list(Duration), Stream, Output, Muxer)
                ]}
              ],
   PortArgs = [
               {port_id, ?MODULE},
               {port_mode, 'active'},
               {port_opts, [stderr_to_stdout]},
               {port_stop_after, 3 * 1000}, %% force stop after 3sec.
               {port_pidfile, PidFile}
              ],

   io:format("Capture live stream from: ~p~n", [Stream]),
   io:format("Stream duration ~psec.~n", [Duration]),
   io:format("Save the stream into: ~p~n", [Output]),

   %% run VLC player and capture live BBC stream
   {ok, Port} = valeo:start_link(?MODULE, ProgArgs, PortArgs),

   %% stop VLC after 30sec.
   {ok, _} = timer:apply_after(Duration * 1000, ?MODULE, stop, [Port]).


%%--------------------------------------------------------------------
%% @doc Stop VLC player
%% @end
%%--------------------------------------------------------------------
stop(Pid) ->
    valeo:stop(Pid).



%%--------------------------------------------------------------------
%% @doc Initiate the valeo server
%% @end
%%--------------------------------------------------------------------
valeo_init(Pid) ->
   io:format("VLC started: ~p~n", [Pid]),
   {ok, []}.


%%--------------------------------------------------------------------
%% @doc Callback to handle messages coming from the external program.
%% @end
%%--------------------------------------------------------------------
valeo_handle_info({exit_status, _}, _State) ->
   {stop, 'normal'};
valeo_handle_info({data, Data}, State) ->
   io:format("Got data from VLC: ~p~n", [Data]),
   {ok, State}.


%%--------------------------------------------------------------------
%% @doc Terminate the valeo server.
%% @end
%%--------------------------------------------------------------------
valeo_terminate(_State) ->
   io:format("VLC closed~n"),
   ok.

Compile:

$ erlc vlc.erl

Then run:

$ erl
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:2:2] [async-threads:0] [kernel-poll:false]
Eshell V5.9.1  (abort with ^G)

1> {ok, _} = vlc:start().

%% wait 30 seconds to let VLC automatically finish the task (no need to manually stop it).

2> q().

Handling restarts with VALEO

Valeo follows the OTP principles. If you want to restart

your external program after a crash, we strongly encourage you

to use it inside a supervision tree.

http://www.erlang.org/doc/design_principles/sup_princ.html

Recap

• Line 14: defines a macro helper which serialize capture parameters passed to VLC. This is in no way related to Valeo itself.
• Lines 24,28: define the BBC live video stream to archive, the output file and the stream duration (30 sec.).
• Lines 30-36: same as the previous example, we define our command (vlc) and its parameters.
• Lines 39-43: define a couple of advanced options to control the external program. The first being the port_mode set to ‘active’,

  meaning that we redirect the stdin and stdout of the external program. The other option port_opts is equivalent to the 2nd argument

  (PortSettings) of open_port/2 call.

  The stderr_to_stdout is pretty much the same as 2>&1 in a bash shell (redirecting stderr to stdout).
• Line 51: runs the VLC player in console mode.
• Line 54: automatically stops VLC after Duration x 1000 (30 sec.). Please note that we specified the same capture duration for VLC,

  this is to make sure we stop the program anyway.
• Lines 79-83: the interesting change from the previous tutorial is that we handle the program’s stdout/stderr. When the external

  program writes some data to it’s standard output, the function valeo_handle_info is called with {data, Data} as its first argument.

  In our example, we just send (print) everything to the screen.

Note

The complete production-ready VLC example is available at: vlc.erl

This simple design can scale linearly to archive thousand of live radios, video channels, do realtime transcoding, etc.

Exercise

To keep this gem easy to read, we simply saved the stream on disk (“bbc.mp4”).

Use WSDK to save it on a WARC file instead.