Using PanoTools projects within Natron
======================================

Those are preliminary notes on using PanoTools or Hugin projects from
within Natron.

Creating a PanoTools (pto) project
----------------------------------

Using the Hugin GUI
~~~~~~~~~~~~~~~~~~~

-  download `Hugin <http://hugin.sourceforge.net/>`__ - on macOS this
   can be done using `homebrew <https://brew.sh/>`__, by typing the
   command ``brew cask install hugin`` in a terminal.
-  read the `Hugin documentation <http://wiki.panotools.org/Hugin>`__ or
   start with a `Hugin
   tutorial <http://hugin.sourceforge.net/tutorials/overview/>`__
-  make your panorama

Using command-line tools
~~~~~~~~~~~~~~~~~~~~~~~~

Full details on using command-line tools are given in the `Panorama
scripting in a
nutshell <http://wiki.panotools.org/Panorama_scripting_in_a_nutshell>`__
documentation.

On macOS, if Hugin was installed using homebrew, the tools are available
in various directories, and they can be added to the PATH using:

::

    PATH="$PATH:/Applications/Hugin/tools_mac:/Applications/Hugin/Hugin.app/Contents/MacOS:/Applications/Hugin/HuginStitchProject.app/Contents/MacOS"

Here is an example of running the panorama tools from a set of JPEG
images (with suffix ``.jpg``) placed in the current directory to
generate e perspective panorama (other options are described in the
documentation):

::

    pto_gen -o project.pto *.JPG
    cpfind --multirow -o project.pto project.pto

    celeste_standalone -i project.pto -o project.pto
    cpclean -v --output project.pto project.pto

    autooptimiser -a -l -s -m -o project.pto project.pto

    nona -m TIFF_m -o project project.pto

    enblend --save-masks -o panorama.tif project*.tif

The result is:

-  A ``project.pto`` file, which is the Hugin project itself
-  The ``project*.tif`` images, which are images warped to the reference
   projection.
-  The ``mask-*.tif`` images, which are masks generated by ``enblend``.
-  The ``panorama.tif`` image, which is the final panorama.

Using Hugin/PanoTools parameters in Natron
------------------------------------------

The ``pto`` file syntax is described is the
`PTOptimizer <http://wiki.panotools.org/PTOptimizer>`__ and
`PTStitcher <http://wiki.panotools.org/PTStitcher>`__ docs.

This file can be viewed and edited in any text editor.

Project size
~~~~~~~~~~~~

The project size, or format, should be set to the panorama size, which
is given in pixels on the ``p`` line (at the start of the file).

Distortion correction
~~~~~~~~~~~~~~~~~~~~~

The distortion correction parameters for the input images are given in
the ``i`` lines (one for each image).

The values for the ``a``, ``b``, ``c``, ``d``, ``e``, ``g``, ``t``
parameters should be entered in a **LensDistortion** node, with
*Model*/``model``\ =PanoTools and *Direction*/``direction``\ =Undistort,
placed after the **Read** node for each input image or video. The script
name for these values are ``pt_a``, ``pt_b``, ``pt_c``, ``pt_d``,
``pt_e``, ``pt_g``, ``pt_t``.

Note that if all images share the same distortion parameters (this is
written as ``a=0 b=0 c=0 d=0 e=0 g=0 t=0`` on the ``i`` line), the
**LensDistortion** node can be cloned in Natron (right-click on node,
*Edit*, *Clone Nodes*).

Projection
~~~~~~~~~~

The projection (which can be performed by the
`Nona <http://wiki.panotools.org/Nona>`__ tool) can be done using
**Card3D** nodes placed after each **LensDistortion** node.

Each **Card3D** node must have its *Transform Order*/``cardXFormOrder``
set to STR, and the *Rotation Order*/``cardRotOrder`` should be left to
the default value (ZXY). Be careful, these are neither the Axis nor the
Cam Transform Order, which are in the two first groups of the
**Card3D**\ node.

Set the *Output Format* to Project.

The ``r`` ``p`` and ``y`` values from the ``i`` line are roll, yaw,
pitch angles. Their values should be put in the *Rotate* parameter of
the **Card3D** (script name is ``cardRotate``), using the following
convention: ``cardRotate.x = p``, ``cardRotate.y = -y``,
``cardRotate.z = -r``.

The ``v`` value from the ``i`` line corresponds to the horizontal field
of view, and has to be converted to an aperture value. The **Lens-In
H.Aperture**/``lensInHAperture`` param should be set to
``2*tan(v*pi/360)``. This expression can be directly typed in the value
field, with ``v`` replaced by the actual value from the corresponding
``i`` line of the ``pto`` file.

Setup all the Card3D nodes for each input image that way.

Camera
~~~~~~

In Natron 2, the camera used by the **Card3D** node is set in the *Cam*
group at the top of the parameters list.

In the **Card3D** node for the first image, unfold this group, and
unfold the *Cam Projection* group.

The ``v`` value on the ``p`` line (usually at the top of the ``pto``
file) gives the horizontal field of view of the output panorama.

Set the *Focal Length*/``camfocal`` to 1., and set the *Horiz.
Aperture*/``camhaperture`` to ``2*tan(v*pi/360)``, replacing ``v`` with
its actual value.

You can then copy these two parameters to all the **Card3D** nodes, or -
even better - link these parameters, so that the output camera for the
panorama can then be modified: Right-click on the parameter from the
first **Card3D**, Copy Link, then right-click on the same parameter of
every other **Card3D** node, Paste Link.

If you intend to modify the camera orientation later, you can also link
the same way the *Rotate*/``camRotate`` parameter (those for the camera,
not the card).

Note that when the Natron **Card3D** node is used in Nuke (where it
appears as **Card3DOFX**), the node has an external Cam input, to which
a Camera node may be connected. There may be a similar concept in future
versions of Natron.

Building the panorama in Natron
-------------------------------

First sketch: overlap the images
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Make sure that the *Output Components* parameter in all readers is set
to RGBA, so that images have a transparent value outside of their
domain.

Now, connect the outpput of the first *Card3D* to the B input of a
**Merge** node, connect the secont to the A input, the third to the A2
input, etc...

The output of the **Merge** node should show a first panorama, obtained
by overlapping all images.

Drawing the masks
~~~~~~~~~~~~~~~~~

Add a **Roto** node after each **LensDistortion**, before each
**Card3D**.

Only leave connected the A and B inputs to the **Merge** node to the two
first **Card3D** nodes.

Check *Premultiply* in each Roto node, check that only the A channel is
affected by **Roto** in its parameters, and start editing the roto mask
on the second image, for which the **LensDistortion** output is
connected to the A input of the **Merge**, while viewing the output of
the **Merge**.

Set the compositing operator of each roto shape to "min" instead of
"over", so that the original image alpha gets masked by the roto shape.

Do not forget to add feather, especially in the overlap area.

Then, reconnect the A2 input to the third **Card3D**, and start editing
its roto mask, always in "min" compositing mode.

Do the same with A3, A4, etc... and you should end up with a full
panorama.

Future work
-----------

Camera response
~~~~~~~~~~~~~~~

See `Camera response
curve <http://wiki.panotools.org/Camera_response_curve>`__ and
`Vig\_optimize <http://wiki.panotools.org/Vig_optimize>`__.

The values ``Ra`` ``Rb`` ``Rc`` ``Rd`` ``Re`` on the ``i`` line encode a
color response curve (EMoR). See ``EMoRParams``\ in the hugin source
code.

Exposure and color
~~~~~~~~~~~~~~~~~~

See `Vignetting <http://wiki.panotools.org/Vignetting>`__ and
`Vig\_optimize <http://wiki.panotools.org/Vig_optimize>`__.

``Eev`` encodes the exposure, see ``ExposureValue`` in the hugin source
code.

``Er`` and ``Eb`` encode the red and blue multipliers, see
``WhiteBalanceRed`` and ``WhiteBalanceBlue`` in the hugin source code.

Vignetting
~~~~~~~~~~

See `Vignetting <http://wiki.panotools.org/Vignetting>`__ and
`Vig\_optimize <http://wiki.panotools.org/Vig_optimize>`__.

``Va`` is always 1, see ``VigCorrMode`` in the hugin source code.

``Vb`` ``Vc`` ``Vd`` encode the degree 2, 4 and 6 coefficients for
vignette correction, see ``RadialVigCorrCoeff`` in the hugin source
code.

``Vx`` and ``Vy`` encode the vignetting center shift, see
``RadialVigCorrCenterShift`` in the hugin source code.

Blending
~~~~~~~~

-  importing enblend masks, using them as roto masks

-  executing enblend externally (using RunScript maybe?)