# Difference between revisions of "Symmetry Analogies"

Jump to: navigation, search

This is a discussion page for symmetry-space editing framework. People currently involved in this project are: Vladimir Kim, Yaron Lipman and Thomas Funkhouser. Feel free to contact us if you would like to participate. Some unorganized examples can be found here.

# Examples Taxonomy

## Classification Legend

Types of input/target

• Input f Deformable image
• f1 One smooth image s.t. all of its parts are smoothly related via some transformation. E.g. face, human pose,
• f2 Disconnected components related by some transformation. E.g. windows, leafs, cars in the traffic
• Target g Target symmetry image
• g1 Real image. E.g. same as f1, but with different subject.
• g2 Synthetic (perfect) shape. E.g. Perfect oval, plane of symmetry
• g3 None. E.g. Target is defined via symmetry maximization/minimization.

D Types of deformations

• D1 Surface control points. E.g. piecewise linear coords, human pose angles, patch coordinates (cut patches from some image f, blend them in using Efros & Freeman quilting technique.)
• D2 Warp. E.g. move control points in overlay lattice and interpolate values in-between, caged mesh
• D3 Semantic Deforms. E.g. move face elements only to create valid human face, restrict human pose angles to have valid human, etc...

E Types of energy

• E1 ||S(g) - S(f)|| Difference in symmetry space
• E2 ||S(f)||, if g is not provided, and where ||Sf|| is assumed to specify how symmetric is object with respect to symmetry operator S.

S Types of Symmetry operators. S(f) maps smooth function to its symmetry space.

• S={pointwise, translation, PRST}, and in addition:
• S_masked masked based on area of influence (e.g. local vs global)
• S_thresh masked based on strongest response (e.g. plane of symmetry)
• S_norm normalized (e.g. in the beginning or at each step)

## Valid combination table

Note that:

• Input image (f) + Deformation (D) => Space of valid solutions
• Energy (E) + target image (g) => Minimal energy solution (i.e. output).

So, space of valid solutions can be described as:

D1: Surface Ctrl Pnts D2: Warp D3: Semantic-aware
f1: Single Shape 1. Human in all possible poses (vary angles of skeleton)
2. All surfaces/curves with given topology (vary coordinates of vertices)
Object Image/Mesh under non-rigid transformations (vary cage control points) Human Faces with some elements moved (vary location and dimensions of face elements)
f2: Many Shapes Texture consisting of quilts (vary location of quilts) Objects Image: Cars/Windows/regular deformed (some parts shrunk/expanded) Leafs on tree moved along branch

So, for each input the desired output depends on Energy + target symmetry S(g) combination.

E=L2[S(g_real) - S(f)] E=L2[S(g_synth) - S(f)] E=Energy[S(f)]
f1D1 1.Align human to other human's pose.
2.Transfer symmetry of surface
1.Align human to sketch pose (or perfect pose)
2. Transfer symmetry of user-specified sketch
1.Make human pose perfectly symmetric (with respect to plane, translations, etc...)
2. Make shape perfectly symmetric (keeping topology)
f1D2 Transfer symmetry of another image/surface: symmetrize/assymetrize. Transfer symmetry of user sketch. Transfer symmetry of (im)perfect shape: symmetrize/assymetrize. Make surface/image perfectly symmetric.
f1D3 Copy face symmetries of one human to another. Note: only source needs user input control points. Make face symmetric/assymetric as in given synthetic example. Change plane of symmetry of a given face. Make face more symmetric. Note: changing S should produce interesting results, should face have more planar symmetry? more translation symmetry?
f2D1
f2D2
f2D3

f1: Single shape f2: Many shapes
g1: Real Symmetry Transfer
Assymetrization
Texture Synthesis
Assymetrization
g2: Synth Symmetry Transfer
Assymetrization
Texture Synthesis
Assymetrization
g3: None Inpainting
Symmetrization
Inpainting
Texture Synthesis

## Example Details

• Inpainting f=image with hole, copy patches from f on itself to maximize symmetry. E2, D3, any S_norm.
• Texture Synthesis Copy quilts s.t. either target symmetries of g are achieved or symmetry is maximized. Any E, D1, any S. Note: varying S should result in different images, especially if g is not given.
• Symmetry Transfer Copy symmetry of one object (possibly synthetic) to another object.
• Copy symmetry of faces - might require semantic details, and putting control points
• Copy symmetry of humans to align them into same 'canonical' pose. One approach: extract skeleton from target pose (mesh g) and source pose (mesh f). Find Sf and Sg, and optimize with respect to E1.
• Align shape with respect to plane of symmetry - e.g. user draws line/plane of symmetry and shape becomes more symmetric with respect to this line
• Symmetrization Make shape more symmetric. Note that this can be also achieved via symmetry transfer, where target is some symmetric synthetic image/shape. For any deformation model optimize with respect to E2.
• Mesh symmetrization
• Assymetrization make some shape less symmetric. f=regular image, g=asymmetric image.
• Pseudo-random points distribution f=regular points, S(g) = S(f) + gaussian blur.

# To do

• Inpainting
• Texture Synthesis
• Canonical Pose in 3D
• Finding skeleton
• PRST in 3D