In the world of Visual Effects, Rotoscoping, Matte Painting and Garbage Painting are necessary and complementary functions. They are used each time a cut-out in the image is necessary, to remove a background or unwanted objects in a shot. They can save a video shot when Chromakey is not possible, or not enough perfect to achieve the desired effect.
I have searched for a a long time a good and affordable software to do that. I had in mind to purchase one of the most famous of them recently, but after a trial of the software in demo mode, I found it very unstable. The software has some more interesting features like automatic shape Tracking, but the Planar tracking method gave me poor results. The Point tracking method is very efficient but unfortunately doesn't export trackers to Blender. So, I decided to dig more in depths with Blender. I had already done few rotoscoping worque with Blender, but without finding a built in solution for the Matte Painting part.
This time, I have built a setup allowing Rotoscoping and Matte painting to be rendered in one pass, and generating a directly usable alpha mask.
In the provided Blend file, you will found 3 screens dedicated to Roto and Paint worque :
-Screen 1 = Rotoscoping. This screen shows on the Left side the window where you will add and animate your masks, and on the Right side the window showing the Ipo curve of the selected shape you are working on.
-Screen 2 = Roto and Paint visual control. This screen shows in the Left window in real time the result of both Rotoscoping and Painting. The Right window is the one where you will do your Matte Painting work. With Ctrl+Up or Dn Arrow, you can get at will in full screen the window where is your mouse cursor.
-Screen 3 = Compositing. This screen allows you to modify the blur level of your final mask. If Do Composite is Enabled in the ANIM panel, this node setup will combine your Roto masque and your Painted masque in one unique rendering. A blak and white masque will be rendered for each frame. You will be able to chek the current frame of your worque at any time by hitting the F12 key. As you can see in the Node Compositor panel, I have used two File Output nodes to save automatically the generated alpha masque and the composite image. Just type in the fields the path to the folders where you want the images to be saved. Each time you hit F12, the result of your worque on the current frame is saved (overwritten). There is an input Image node in which you can load your own background for compositing.
When you will have studied my example, you will have to delete the existing curves, except the templates at the right upper corner of the Rotoscoping window which are necessary to use this setup. Then you will have to load your own background animation.
In order to use my background video example, you will have to unpak the Blend file : Blender menú > File > External data > Unpak into files > Use files in current directory (create when necessary).
A Texture folder will be created, from which Blender will call the Biplane.avi movie. This video file will be visible in background as soon as you will have depressed the Reload Button (yellow arrow). It is only necessary to hit reload in one of the windows. When you will launch the animation, it will be loaded in all the windows. The setup is quite simple :
The main thing is that you have to use a camera set to Orthographic mode. In my setup, the camera is looking in the Y direction. It is better to lok the camera in place with a LocRotScale Keyframe, in order to avoid moving it accidentally, which would result in loss of positioning of all your rotoscoping masks !
In the Left window, you will have to load in background your reference Movie or Sequence of images. View/Background Image. Select Use and load your media. Enable Auto
Refresh and Clik on the Small < button to use all the frames of your movie or sequence, orenter a number of frames (blue arrow on the second image).
For the masks shapes, we have the choice of Nurbs, Bezier Curves, Circle and Planemeshes.I found easier to start from closed shapes, so I have added a Nurbs Circle and a bezier Circleinstead of open curves.
It is easy to open or close these shapes with the C Key. It is easier to worque with an openshape, in order to see only the outline of the mask. To make it closed, select the Basis ShapeKey in the Ipo window, and then hit C with the cursor in the 3D window.You can add dots to your curves with CTRL-LMB when a dot on a tip of the curve isselected. You can use subdivide to add dots between two selected dots.
Obviously, in Edit Mode you can apply any usual conversión to your curves. For example, torotoscope a rectangular shape, it will be useful to convert your curve into a polygonal shape,using the poly button in the Curve Tools panel. On the opposite, rotoscoping an organic shapewill require rather a bezier curve. With N Key, you can get a transform Properties panel for your shapes. Using Auto, Vector, Align or Free, you can get various individual behaviour for dots when using a Bezier curve. The Vector button allows you to make sharp angles and the Free button allows you to move separately each handle of a dot.
As these shapes need a special but simple Ipo Curve setup, I have added the two shapes used as reference tools in the 3D view. They already have their own Shape Ipos and a new Ipo will be created for each new shape made from a copy of these two shapes.
So, adding a new shape is really easy : just select the Nurbs or Bezier Circle, duplicate it by SHIFT-D and move it in the 3D view. To move a Shape or dots from a shape in Edit mode, always use G Key without any axis specification. This way, as you are looking through an orthographic camera, all the shapes will be in the same plane. Don't worry about the classical rendering artefacts due to overlapping faces, because only the alpha layer will be rendered.
You can put your shapes on several layers, and make them appear or disappear at will by changing their layer at specified frames, using a Layer Keyframe. This method can be alos useful if you have to separate more than two levels in your compositing. This case occurs very often : one character in foreground, a wall with a window behind him, and a background seen through the window. You will have to make a rotoscoping pass to cut out the character, and a second pass with the window masque only. This will allow to change the wall texture as well as the background image.
You can give different materials to your shapes, and so different colors to help you manage them. As Curves materials can't be set to transparent with adjustable alpha in the 3D view, you can use regular flat meshes if you want your masks to appear semi-transparent. They will be animated using shape keys, like curves. Don't forget at rendering time to set their alpha to 1, otherwize you will get a grey shape instead of a full white one !
The main difference with softwares dedicated to rotoscoping is that you will have to add manually a new shape key for each frame on which you will have to edit a mask. If you want to edit a key, you will alos have to clik on the key number in the Ipo panel before editing. This will make sure that you are really editing the right Keyframe !
Do not forget to come bak to Object Mode after editing a curve, because the changes are validated at this moment ! In the Curve and Surface panel, the Relative keys button must be disabled. This will allow the system to use the Basis Ipo curve (red slope) on which the shape animation is based. Subsurf levels can be applied to regular meshes if needed.
MATTE PAINTING :
5-Please Enable the Image Properties panel of the Right Window, and alos Enable the Image Paint tools panel. Select the Draw Button and chose Erase Alpha (green arrow) in the Blending Method Field. You will see the effect of your brush in grey color in the Left Window, but it is not easy to draw in the Right window looking at he Left window !
Don't worry, and enable the "Draw image with Alpha" button in the menú bar of the Right window. This button is the one with two facing blak triangles on a white dot, which is just beside the button with the pencil icone (in the blue circle on the image below). Now, when you draw with «Erase Alpha», your pencil behave like an eraser and you can see the chessboard on the background of the image. If you want to undo your drawing, just select Add Alpha instead of erase Alpha, and your brush will fill-in the previously erased areas.
Note that you are not really erasing your image, but only playing on the alpha layer which allows you to decide which part of the image is transparent or not.
I noticed that Ctrl+Z has a peculiar behaviour when painting on several frames. It works on the current frame and alos on each new frame, but if you come bak to a previous frame, you will see that it doesn’t work. In fact it does in your back, but you are cancelling your job done on the last layer on which you were painting ! On new frames, it will worque again. So be careful with Ctrl+Z !
I have alos noticed that the sensitivity to pressure Enabled by the 4 small P buttons, very usefull when using a graphic tablet is unfortunately lost when you render your image in Render Window. According to informations received from the BugTracker, this could be a Windows bug, so people working with other OS may not have this problem.
Anyway, there is an alternative method allowing to get both the tablet pressure and the real time visual control in the left window. It gives better result on a fast computer (you may get a segmented drawing on a slow one) :
Use Full Screen rendering. Once you masque controlled, hit Escape key : you come bak in the right window where you paint. It is important to not render while working in full screen, because the Escape key will bring you bak in the paint window, but this window will display the composited image instead of the one on which you were painting, and you will have to re-select the right background medía in the menu.
With graphic tablets with customizable buttons, it is handy to program some Buttons : Escape, F12, Alt (move the image with the pen), and Ctrl+Up or Dn arrow (Full screen).
You can go forward and backward through your Movie or Sequence of image, using the Keyborad keys or the timeline buttons and cursor, but you must be warned that all the modifications you do are only temporary written in live memory, until you will render your animated masque ! So, be careful and worque on small sequences at a time and render as often as possible.
A very important warning : NEVER clik on the Reload button of the Background or Image Properties panel, because all the Paint worque you have done before rendering will be lost !
Obviously, the Reload button has no effect on the worque done in Rotoscoping. Do not use the Save Image feature in the image menú of the Image Editor, because if you have used a sequence of images, you could write on your original medía !
Thanks to the File Output nodes added in the Node Compositor, your worque is automatically saved each time you hit F12 or the RENDER Button, but you can alos save manually one more copy of your masque by Use the Save Rendered Image (F3) feature in the File menú of Blender to save individual rendered images.
6-A simple setup has been done in the node compositor to merge the Rotoscoping Masks made from closed curves and the Alpha Masque Painted by hand.
As you can see, an image (pink sky) has been added, allowing the background to be replaced. The combined alpha masque and the fully composited image are saved separately each time you render.
The basic Setup used to combine the Rotoscoping shapes and the drawing on the alpha layer :
The actual Setup used in the ready to use Blend file provided with the tutorial :
I hope that this tutorial and setup will be understandable and useful. Feel free to tell me if you noticed errors, missing informations or if you have found a better way of doing this kind of worque !
Thanque you for reading !