Difference between revisions of "Symmetry Analogies"

From CSWiki
Jump to: navigation, search
Line 16: Line 16:
 
** '''g3''' None. E.g. Target is defined via symmetry maximization/minimization.
 
** '''g3''' None. E.g. Target is defined via symmetry maximization/minimization.
 
'''D Types of deformations''' <br>
 
'''D Types of deformations''' <br>
* '''D1''' Surface control points. E.g. piecewise linear coords, human pose angles, mesh cage coordinates...
+
* '''D1''' Surface control points. E.g. piecewise linear coords, human pose angles, patch coordinates (cut patches from some image f, blend them in using Efros & Freeman ''quilting'' technique.)
* '''D2''' Image Warp. E.g. move control points in overlay lattice and interpolate values in-between.
+
* '''D2''' Warp. E.g. move control points in overlay lattice and interpolate values in-between, caged mesh
* '''D3''' Splat patches. E.g. cut patches from some image f, blend them in using Efros & Freeman ''quilting'' technique.
+
* '''D3''' Semantic Deforms. E.g. move face elements only to create valid human face, restrict human pose angles to have valid human, etc...
* '''D4''' Semantic Deforms. E.g. move face elements only to create valid human face, restrict human pose angles to have valid human, etc...
 
 
'''E Types of energy''' <br>
 
'''E Types of energy''' <br>
 
* '''E1''' ||S(g) - S(f)|| Difference in symmetry space
 
* '''E1''' ||S(g) - S(f)|| Difference in symmetry space
Line 30: Line 29:
  
 
== Valid combination table ==
 
== Valid combination table ==
 +
Note that:
 +
* Input image ('''f''') + Deformation ('''D''') => Space of valid solutions
 +
* Energy ('''E''') + target image ('''g''') => Minimal energy solution (i.e. output).
 +
So, space of valid solutions can be described as:
 +
 +
{| class="wikitable" border="1"
 +
|-
 +
!
 +
! D1: Surface Ctrl Pnts
 +
! D2: Warp
 +
! D3: Semantic-aware
 +
|-
 +
! f1: Single Shape
 +
| Human in all possible poses (vary angles of skeleton)
 +
| Image/Mesh under non-rigid transformations
 +
| Human Faces with some elements moved
 +
|-
 +
! f2: Many Shapes
 +
| Texture consisting of quilts (vary location of quilts)
 +
| Cars/Windows/regular objects moved around (some parts shrunk/expanded)
 +
| Leafs on tree moved along branch
 +
|}
 +
 +
<br>
 
{| class="wikitable" border="1"
 
{| class="wikitable" border="1"
 
|-
 
|-
Line 51: Line 74:
 
== Example Details ==
 
== Example Details ==
 
* '''Inpainting''' f=image with hole, copy patches from f on itself to maximize symmetry. '''E2''', '''D3''', any '''S_norm'''.
 
* '''Inpainting''' f=image with hole, copy patches from f on itself to maximize symmetry. '''E2''', '''D3''', any '''S_norm'''.
* '''Texture Synthesis''' Copy quilts s.t. either target symmetries of g are achieved or symmetry is maximized. Any '''E''', '''D3''', any '''S'''. Note: varying '''S''' should result in different images, especially if '''g''' is not given.
+
* '''Texture Synthesis''' Copy quilts s.t. either target symmetries of g are achieved or symmetry is maximized. Any '''E''', '''D1''', any '''S'''. Note: varying '''S''' should result in different images, especially if '''g''' is not given.
 
* '''Symmetry Transfer''' Copy symmetry of one object (possibly synthetic) to another object.  
 
* '''Symmetry Transfer''' Copy symmetry of one object (possibly synthetic) to another object.  
 
** Copy symmetry of faces - might require semantic details, and putting control points
 
** Copy symmetry of faces - might require semantic details, and putting control points

Revision as of 19:18, 21 July 2009

This is a discussion page for symmetry-space editing framework. People currently involved in this project are: Vladimir Kim, Yaron Lipman and Thomas Funkhouser. Feel free to contact us if you would like to participate. Some unorganized examples can be found here.

Examples Taxonomy

Classification Legend

Types of input/target

  • Input f Deformable image
    • f1 One smooth image s.t. all of its parts are smoothly related via some transformation. E.g. face, human pose,
    • f2 Disconnected components related by some transformation. E.g. windows, leafs, cars in the traffic
  • Target g Target symmetry image
    • g1 Real image. E.g. same as f1, but with different subject.
    • g2 Synthetic (perfect) shape. E.g. Perfect oval, plane of symmetry
    • g3 None. E.g. Target is defined via symmetry maximization/minimization.

D Types of deformations

  • D1 Surface control points. E.g. piecewise linear coords, human pose angles, patch coordinates (cut patches from some image f, blend them in using Efros & Freeman quilting technique.)
  • D2 Warp. E.g. move control points in overlay lattice and interpolate values in-between, caged mesh
  • D3 Semantic Deforms. E.g. move face elements only to create valid human face, restrict human pose angles to have valid human, etc...

E Types of energy

  • E1 ||S(g) - S(f)|| Difference in symmetry space
  • E2 ||S(f)||, if g is not provided, and where ||Sf|| is assumed to specify how symmetric is object with respect to symmetry operator S.

S Types of Symmetry operators. S(f) maps smooth function to its symmetry space.

  • S={pointwise, translation, PRST}, and in addition:
    • S_masked masked based on area of influence (e.g. local vs global)
    • S_thresh masked based on strongest response (e.g. plane of symmetry)
    • S_norm normalized (e.g. in the beginning or at each step)

Valid combination table

Note that:

  • Input image (f) + Deformation (D) => Space of valid solutions
  • Energy (E) + target image (g) => Minimal energy solution (i.e. output).

So, space of valid solutions can be described as:

D1: Surface Ctrl Pnts D2: Warp D3: Semantic-aware
f1: Single Shape Human in all possible poses (vary angles of skeleton) Image/Mesh under non-rigid transformations Human Faces with some elements moved
f2: Many Shapes Texture consisting of quilts (vary location of quilts) Cars/Windows/regular objects moved around (some parts shrunk/expanded) Leafs on tree moved along branch


f1: Single shape f2: Many shapes
g1: Real Symmetry Transfer
Assymetrization
Texture Synthesis
Assymetrization
g2: Synth Symmetry Transfer
Assymetrization
Texture Synthesis
Assymetrization
g3: None Inpainting
Symmetrization
Inpainting
Texture Synthesis

Example Details

  • Inpainting f=image with hole, copy patches from f on itself to maximize symmetry. E2, D3, any S_norm.
  • Texture Synthesis Copy quilts s.t. either target symmetries of g are achieved or symmetry is maximized. Any E, D1, any S. Note: varying S should result in different images, especially if g is not given.
  • Symmetry Transfer Copy symmetry of one object (possibly synthetic) to another object.
    • Copy symmetry of faces - might require semantic details, and putting control points
    • Copy symmetry of humans to align them into same 'canonical' pose. One approach: extract skeleton from target pose (mesh g) and source pose (mesh f). Find Sf and Sg, and optimize with respect to E1.
    • Align shape with respect to plane of symmetry - e.g. user draws line/plane of symmetry and shape becomes more symmetric with respect to this line
  • Symmetrization Make shape more symmetric. Note that this can be also achieved via symmetry transfer, where target is some symmetric synthetic image/shape. For any deformation model optimize with respect to E2.
    • Mesh symmetrization
  • Assymetrization make some shape less symmetric. f=regular image, g=asymmetric image.
    • Pseudo-random points distribution f=regular points, S(g) = S(f) + gaussian blur.

To do

  • Inpainting
  • Texture Synthesis
  • Canonical Pose in 3D
    • Finding skeleton
    • PRST in 3D