Shadows from set geometry are accurately depicted on added objects
For more than twenty years, the software program Photoshop has been the industry standard for seamlessly manipulating images, especially the removal of unwanted items like blemishes, wires and telephone poles. When it comes to adding something to a photo, however, the process is still rather involved. Now, a team from the University of Illinois at Urbana-Champaign (UIUC), led by PhD candidate Kevin Karsch, is presenting a new algorithm at Siggraph Asia 2011 that promises to radically simplify the procedure of believably incorporating new or "synthetic" objects in still shots and the results are quite impressive.
To appreciate the elegance of the UIUC team's contribution, it's
helpful to understand just what an improvement it is over current
techniques. Look at any effects-heavy feature film and you'll see
instances where computer-generated objects, be they giant robots,
dinosaurs or futuristic vehicles, were added to previously-shot footage
(also called a background plate in the Biz). In order to sell
the illusion that the added objects were there in the first place,
technicians typically need to gather copious amounts of on-set data
(e.g. lens height and focal length, size of on-set objects and
direction/color temperature of visible light sources, etc.) from which
they recreate a virtual environment. When this is done poorly, it's a
dead giveaway that the added object is fake and was inserted after the
original image or footage was shot.
The genius of the Karsch team's approach lies in what the user doesn't
have to provide in order to create believable additions. (Currently,
the software only handles still images, but plans are in the works to
tackle film and video sequences before long). Starting with a still
photo about which nothing is known, novice users have been able to add
virtual subjects in just minutes with results good enough to fool
experts. That's great news for graphic artists, but maybe not so
thrilling for folks like crime scene investigators who'll soon have to
doubt the veracity of every object in frame!
To pull off this digital sleight-of-hand, the process is broken up
into intuitive stages, the first being the creation of 3D geometry from
2D data. "The main idea is we want to take our picture and we want to
get a 3D model of the scene from it," Karsch explained. "We first apply
an automatic estimation procedure which tries to find the boundaries of
the scene - it tries to find the walls, the ceiling, the floor." Since
that's occasionally inaccurate, the software also includes an interface
that allows the user to refine the estimate.
"With two vanishing points, we can recover the focal length of the camera, and then using the rest of the input we can also recover a rough representation of the 3D geometry of the scene: where the floors are at, where the walls are at and where the ceiling is at," Karsch added.
Next, the Lighting Stage requires the user to identify light sources
in the image, whether in frame or not. In the traditional computer
graphics approach, lighting sources and characteristics are identified
by placing a chrome reference ball in front of the camera. The UIUC
team's approach eliminates the need for that and a host of other steps
as it calculates a decent representation of the actual lighting
conditions.
If the object to be added will land behind another opaque item, the Occluding Object Stage (in which the user basically scribbles on the objects that will block part of the object to be added) comes next. Also called a spectral matting procedure, this generates an object mask which, when applied, creates the illusion that the added object is behind the masked one.
Enabling beginners to generate professional results with minimal time and effort is sure to make the Karsch team's algorithm a big hit in the graphics world. Judging from the sample images provided from the team's paper, the approach adeptly handles just about every lighting condition conceivable, even for transparent and shadowed subjects. For those of you itching to get your hands on it, plans are in the works to create an online version that anyone can use. In the meantime, it seems we'll just have to make do with run-of-the-mill reality!
Source: Kevin Karsch
Check out the video below for a look at the software in action:
"With two vanishing points, we can recover the focal length of the camera, and then using the rest of the input we can also recover a rough representation of the 3D geometry of the scene: where the floors are at, where the walls are at and where the ceiling is at," Karsch added.
If the object to be added will land behind another opaque item, the Occluding Object Stage (in which the user basically scribbles on the objects that will block part of the object to be added) comes next. Also called a spectral matting procedure, this generates an object mask which, when applied, creates the illusion that the added object is behind the masked one.
Enabling beginners to generate professional results with minimal time and effort is sure to make the Karsch team's algorithm a big hit in the graphics world. Judging from the sample images provided from the team's paper, the approach adeptly handles just about every lighting condition conceivable, even for transparent and shadowed subjects. For those of you itching to get your hands on it, plans are in the works to create an online version that anyone can use. In the meantime, it seems we'll just have to make do with run-of-the-mill reality!
Source: Kevin Karsch
Check out the video below for a look at the software in action:
0 comments:
Post a Comment