Usage

Learnable bilateral convolution layer is implemented as Permutohedral layer in Caffe neural network framework. The format of the filter follows closely the filter format of the standard spatial convolution. It has the shape (n, c, 1, f), with n = num_output, c = input_channels / groups, f = filter_size. Below is a sample prototxt for the layer along with explanations of different parameters of the layer.

layer {
  name: "permutohedral"

  type: "Permutohedral"

  bottom: "input"              # Input blob
  bottom: "in_features"        # Input features
  bottom: "out_features"       # Output features
  bottom: "in_lattice"         # (Optional) Use the lattice information from another permutohedral layer
                               # with the same features.

  top: "output"                # Output filtered blob
  top: "out_lattice"           # (Optional) Outputs the lattice information that can be used by another
                               # permutohedral layer with the same features.

  permutohedral_param {
    num_output: 1              # Number of filter banks == dimension of the output signal.

    group:      1              # Number of convolutional groups (default is 1). The input signal is cut into
                               # this many groups to compute the filtered result.
                               # Each one of the (input_channels / groups) are responsible for one element in
                               # the (num_output / groups) many output groups.

    neighborhood_size: 2       # Filter neighborhood size

    bias_term: true            # Whether to use bias term or not

    norm_type: SYMMETRIC       # SYMMENTRIC (default): Applies the signal normalization before and after the
                               #                       filtering;
                               # AFTER:                Applies the signal normalization after the filtering.

    offset_type: FULL          # FULL (default): Full Gaussian Offset;
                               # DIAG:           Diagonal Gaussian offset;
                               # NONE:           No offset.

    visualize_lattice: false   # false (default): nothing changes and works as usual
                               # true:            Will make barycentric coordinates uniform (useful for lattice visualisation)

    do_skip_blur: false        # false (default): nothing changes and works as usual
                               # true:            skips the blur step and only runs splat and slice

    repeated_init: false       # false (default): constructs a new lattice with in and out features
                               # true:            Makes use of in_lattice which is given as third bottom (required for this option)
 }
}

We also add PixelFeature layer in Caffe to extract features from the input image. Following is a sample prototxt to use that layer.

layer {
  name: "positional_rgb_features"
  type: "PixelFeature"
  bottom: "image_blob"            # Input data blob
  top: "positional_rgb_features"  # Output feature blob

  permutohedral_feature_param{
    type: POSITION_AND_RGB        # Feature type (others: RGB, POSITION, RGB_AND_POSITION)
    pos_scale: 0.1                # Position feature scale (default: 1.0)
    color_scale: 0.2              # color feature scale (default: 1.0)
  }
}