README.md

# About the RTMP IO plugin

The RTMP plugin provides a standard client implementation of the RTMP protocol.
It is used to provide RTMP input and output to videostitch softwares with a choice of 3 different encoders: libx264, QSV, NVENC.

The plugin implements a client with pull behavior when it is configured for input, and push behavior when configured for output.
In all cases, an RTMP server is needed.

It is tested with the following servers: NGINX, WOWZA, MIST, AKAMAI.

RTMP protocol specification:
https://www.adobe.com/content/dam/Adobe/en/devnet/rtmp/pdf/rtmp_specification_1.0.pdf

FLV/F4V format specification:
http://download.macromedia.com/f4v/video_file_format_spec_v10_1.pdf

The plugin provides support for pushing RTMP to a compatible server (Nginx, Wowza, Akamai, ...).
RTMP pull (when the server request the connection) is not supported


# RTMP Input configuration

```
"reader_config": {
    "name": "rtmp://127.0.0.1:1935/inputs/0_0",
    "type": "rtmp",
	"frame_rate": {
	    "den": 1,
	    "num": 30
	},
	"audio_samplerate" : 48000,
    "audio_channels" : 2,
    "audio_sample_depth" : "s16"
}
```

###### name
type : *string*
default : **required**
notes : rtmp://SERVER_IP:PORT/APP/STREAMKEY
RTMP authentication is not supported (untested).

###### frame_rate
notes : **required.** *den* & *num* properties are used to specify input framerate such as **framerate = num / den**
VFR (variable framerate) is not supported (untested).

###### frame_rate.den
type : int
default : ?

###### frame_rate.num
type : int
default : ?

###### audio_samplerate :
type : int
default : - optional
notes : 44100 or 48000

###### audio_channels
type : *int*
default : - optional
Notes :  1 (mono) or 2 (stereo)

###### audio_sample_depth
type : *string*
default : - optional
notes : "flt", "s16" or "s32". Specifies the expected audio sample format/depth from the RTMP input. Note that it is converted to "s16" for internal processing if not already in that format.


# RTMP Output configuration


```
{
 	// RTMP protocol settings
 	"type": "rtmp",
	"filename": "rtmp://127.0.0.1:1935/live/output",
    "downsampling_factor": 1,
    "pub_user" : "yourusername",
    "pub_enc_passwd" : "2Njux57MZPgCc6H7S1RcTg==\n",

	// video encoder settings depend on the selected encoder
	"video_codec" : "x264",
	...

	// audio encoder settings
	"audio_codec": "aac",
    ...

}
```


## RTMP output settings

##### filename
type : *string*
default : **required**
notes : (to be confirmed) the URL pattern for publishing rtmp://SERVER_IP:PORT/APP/STREAMKEY?username=%USER%:password=%PASSWORD%

###### pub_user
type : *string*
default : *optional*
notes : username for connecting to a secured RTMP server. Generated by VahanaVR when configuring RTMP with user/password.

###### pub_enc_passwd
type : *string*
default : *optional*
notes : encripted password to connect to a secured RTMP server. Generated & encripted by VahanaVR when configuring RTMP with user/password.

##### downsampling_factor
type : *int*
default : *1*
notes : downsamples the stitcher output by the given factor. eg.: pano size of 3840x1920 with downsampling_factor of 2 will encode a 1920x960 video.

###### bitrate_min
type : *int*
default : *-1*
notes : If output bitrate is higher than available network bandwidth and dynamic bitrate control supported by the video encoder the bitrate will be
automatically adjusted down to bitrate_min to avoid dropping packets (when output queue is full)



## AUDIO Encoding

Note that the f4v format specification which we use as reference for supported values: http://download.macromedia.com/f4v/video_file_format_spec_v10_1.pdf

In our implementation (using lame) please note that mp3 only supports 44100 Hz sampling rate: [lameEncode](src/lameEncoder.cpp#L38)

**AAC codec**
sampling rate : 48000 Hz
sample format : s16 | s32 | flt
channel layout : mono | stereo

**MP3 codec**
sampling rate : 44100 Hz
sample format : s16 | s32 | flt
channel layout : mono | stereo

###### Sample configuration
```
{
    "type": "rtmp"
...
    "audio_bitrate": 128,
    "audio_codec": "aac",
    "channel_layout": "stereo",
    "sample_format": "s16",
    "sampling_rate": 44100
...
}
```

###### audio_codec
type : string
default : ?
notes : "aac" or "mp3"

###### audio_bitrate
type : int
default : ?
notes :

###### channel_layout
type : string
default : ?
notes : "stereo" or "mono"

###### sampling_rate
type : int
default : ?
notes : 41000 / 48000

###### sample_format
type : string
default : ?
notes : "fltp", "s16" or "s32". According to the f4v format specifications, the



## VIDEO Encoding

The only supported video encoding is H264. A choice of 3 encoders is provided :


### libx264 encoder

This is the encoder for RTMP output used by VahanaVR by default.
It runs on CPU on all plateform and does not involve specific HW dependencies.

###### Sample configuration
```
{
	"type" : "rtmp",
	"video_codec" : "x264",
	"tune" : "",
	"profile" : "baseline",
	"level" : "4.2",
	"filename" : "rtmp://localhost:1935/live/stream",
	"bitrate_mode" : "VBR",
	"quality_balance" : 5,
	"b_frames" : 2,
	"bitrate" : 8001,
	"cbr_padding" : false,
	"target_usage" : 4,
	"gop" : -1,
}
```

###### bitrate
type : *int*
default : *2000*
notes : target bitrate, in *??*

###### bitrate_mode
type : **
default : **
notes :

###### buffer_size
type : **
default : **
notes :

###### cbr_padding
type : **
default : **
notes :

###### preset
type : **
default : **
notes :

###### tune
type : **
default : **
notes :

###### profile
type : **
default : **
notes :

###### level
type : **
default : **
notes :

###### quality_balance
type : **
default : **
notes :

###### gop
type : **
default : **
notes :

###### b_frames
type : **
default : **
notes :


### Intel Quicksync encoder

The QSV encoder works on Intel processors, provided that the Intel Quicksync video engine is available.
It leverages Intel GPU to provide HW accelerated H264 video encoding.

###### Sample configuration
```
{
    "type": "rtmp"
...
    "video_codec": "qsv",
    "bitrate": 4000,
    "target_usage": 5,
    "num_slice": 0,
    "gop": 30,
    "level": "5.1",
    "profile": "baseline",
    "bitrate_mode": "CBR",
...
}
```

###### bitrate
type : *int*
default : *4000*
notes : target bitrate, in *kbps*

###### bitrate_mode
type : *string*
default : *CBR*
notes : bit rate control mode (for compatibility with libx264) cqp, cbr, vbr (upper or lower case).

###### target_usage
type : *int*
default : 4
notes : trade-off between quality and speed, from 1 (quality) to 7 (fastest)

###### num_slice
type : *int*
default : *0*
notes : Number of slices in each video frame. If num_slice=0 , the encoder may choose any slice partitioning allowed by the codec standard.

###### profile
type : *string*
default : *baseline*
notes : H264 profile, baseline | main | extended | high | constrained_baseline | constrained_high | progressive_high
Specified values will be ignored if the resolution & fps do not fit in the requested profile.

###### level
type : *int*
default : *3.1*
notes : Any of the H264 standard levels.
Specified values will be ignored if the resolution & fps do not fit in the requested level.

###### gop
type : *int*
default : *250*
notes : The target GOP size. Unknown range (0~250 ?).
It is unknown wether scenecut detection can overide this or wether automatic GOP size is possible (eeg. w/ gop=0 as in libx264 encoder). All I-Frames are IDR-Frames.

###### memtype
type : *string*
default : *d3d11*
notes : This parameter is hardcoded. Any value here will be ignored.


### NVENC encoder

The NVENC encoder works on Nvidia GPU processors, provided that the Nvidia video engine is available.
It leverages Nvidia GPU to provide HW accelerated H264 video encoding.

###### Sample configuration
```
{
    "type": "rtmp"
...
    "video_codec": "h264_nvenc",
    "bitrate": 4000,
    "preset" : "slow",
    "profile": "baseline",
    "bitrate_mode": "CBR",
    "gop": 30,
    "vbvMaxBitrate" : 4000,
    "buffer_size" : 600,
...
}
```

###### bitrate
type : *int*
default : *5000*
notes : target bitrate, in *kbps*

###### preset
type : *string*
default : *default*
notes : trade-off between quality and speed : slow/hq, fast/hp, bd, ll, llhp, llhq, lossless, losslesshp, medium/default.

###### profile
type : *string*
default : *baseline*
notes : H264 profile, baseline | main | high | constrained_high | high444 | stereo
Specified values will be ignored if the resolution & fps do not fit in the requested profile.

###### bitrate_mode
type : *string*
default : *cqp*
notes : bit rate control mode (for compatibility with libx264) cqp, cbr, vbr, vbr_minqp (upper or lower case).
It is overriden by rcmode if used

###### rcmode
type : *int*
default : *0*
notes : bit rate control mode 0(CQP), 1(VBR), 2(CBR), 4(VBR_MINQP), 8(2_PASS_QUALITY), 16(2_PASS_FRAMESIZE), 32(2_PASS_VBR)

###### gop
type : *int*
default : *250*
notes : The target GOP size. Unknown range (0~250 ?).
It is unknown wether scenecut detection can overide this or wether automatic GOP size is possible (eeg. w/ gop=0 as in libx264 encoder). All I-Frames are IDR-Frames.

###### vbvMaxBitrate
type : *int*
default : *0*
notes : Video Buffering Verifier maximum bitrate, in *kbps*

###### buffer_size
type : *int*
default : *0*
notes : Video Buffering Verifier maximum buffer size, in *kb*

###### qp
type : *int*
default : *28*
notes : quantization parameter, QP, that ranges from 0 to 51

###### b_frames
type : *int*
default : *0*
notes : number of B frames between two P frames

###### video_codec
type : *string*
default : *0*
notes : video codec "h264_nvenc", "hevc_nvenc"