4.18.24 Movement Legacies, Reflection, and Review.pptx
Morphing Image
1. T H E U N I V E R S I T Y of T E X A S
HEALTH SCIENCE CENTER AT HOUSTON
S C H O O L of H E A L T H I N F O R M A T I O N S C I E N C E S
Interpolation and Morphing
For students of HI 5323
“Image Processing”
Willy Wriggers, Ph.D.
School of Health Information Sciences
http://biomachina.org/courses/processing/05.html
70. Inverting Matrix Transformations
If
v′ = M v
then
v = M-1v′
Thus, to invert the transformation, invert the matrix
Useful for computing the backward mapping given the forward
transform
For more info see e.g.
http://home.earthlink.net/~jimlux/radio/math/matinv.htm
82. Image Blending
• Goal is smooth transformation between image of one object and
another
• The idea is to get a sequence of intermediate images which when
put together with the original images would represent the change
from one image to the other
• Realized by
– Image warping
– Color blending
• Image blending has been widely used in creating movies, music
videos and television commercials
– Terminator 2
83. Cross-Dissolve (Cross-Fading)
• Simplest approach is cross-dissolve:
– linear interpolation to fade from one image (or volume) to another
• No geometrical alignment between images (or volumes)
• Pixel-by-pixel (or voxel by voxel) interpolation
• No smooth transitions, intermediate states not realistic
from G. Wolberg, CGI ‘96
84. Problems
• Problem with cross-dissolve is that if
features don’t line up exactly, we get a
double image
• Can try shifting/scaling/etc. one entire
image to get better alignment, but this
doesn’t always fix problem
• Can handle more situations by applying
different warps to different pieces of image
– Manually chosen
– Takes care of feature correspondences
Image IS with mesh Image IT, mesh MT
MS defining pieces
from G. Wolberg, CGI ‘96
89. Mesh Warping
• Source and target images are meshed
• The meshes for both images are interpolated
• The intermediate images are cross-dissolved
• Here, we look at 2D example
90. Mesh Warping Algorithm
• Algorithm
for each frame f do
– interpolate mesh M, between Ms and Mt
– warp Image Is to I1, using meshes Ms and M
– warp Image It to I2, using meshes Mt and M
– interpolate image I1 and I2
end
– Is : source image, It : target image
– source image has mesh Ms, target image has mesh Mt
113. Control Point Warping
Instead of a warping mesh, use arbitrary correspondence
points:
Tip of one person’s nose to the tip of another, eyes to eyes,
etc.
Interpolate between correspondence points to determine how
points move
Apply standard warping (forward or backward):
In-between image is a weighted average of the source and
destination corresponding pixels
Here we look at 3D example…
114. Finding Control Points in 3D Structures
Alternative
Actin filament: Reconstruction from EM data at 20Å resolution rmsd: 1.1Å
Wriggers et al., J. Molecular Biology, 1998, 284: 1247-1254
115. Control Point Displacements
Have 2 conformations,
both source and target
characterized by
control points
RNA Polymerase, Wriggers, Structure, 2004, Vol. 12, pp. 1-2.
116. Piecewise-Linear Inter- / Extrapolation
For each probe position find 4 closest control points. w 3 f3
Ansatz: Fx ( x, y, z ) = ax + by + cz + d w 4 f4
F = ( Fx , Fy , Fz )
Fx (w1 ) = f1, x ,
Fx (w 2 ) = f 2, x , w1 f1 w 2 f2
Fx (w 3 ) = f3, x ,
Fx (w 4 ) = f 4, x (similar for Fy , Fz ).
Cramer’s rule:
f1, x w1, y w1, z 1 w1, x f1, y w1, z 1
f 2, x w2, y w2, z 1 w2, x f 2, y w2, z 1 w1, x w1, y w1, z 1
f 3, x w3, y w3, z 1 w3, x f3, y w3, z 1 w2, x w2, y w2, z 1
D=
f 4, x w4, y w4, z 1 w4, x f 4, y w4, z 1 w3, x w3, y w3, z 1
a= , b= , , w w4, y w4, z 1
D D 4, x
See e.g. http://mathworld.wolfram.com/CramersRule.html
117. Non-Linear Kernel Interpolation
Consider all k control points and interpolation kernel function U(r).
Ansatz: k
Fx ( x, y, z ) = a1 + ax x + a y y + az z + ∑ bi ⋅ U ( w i − ( x, y, z ) )
k =1
Fx (w i ) = fi , x , ∀i (similar for Fy , Fz ).
Solve :
L−1 ( f1, x , , f k , x , 0, 0, 0, 0) = (b1 , , bk , a1 , ax , a y , az )T ,
⎛1 w1, x w1, y w1, z ⎞
⎛ P Q⎞ ⎜ ⎟
where L = ⎜ T
⎜Q ⎟, Q=⎜ ⎟ , k × 4,
⎝ 0⎟
⎠ ⎜1 wk , x wk , y wk , z ⎟
⎝ ⎠
⎛ 0 U ( w12 ) U ( w1k ) ⎞
⎜ ⎟
P= ⎜ U ( w21 ) 0 U ( w2 k ) ⎟
, k × k.
⎜ ⎟
⎜
⎜U (w ) U (w ) ⎟
⎝ k1 k2 0 ⎟ ⎠
118. Bookstein “Thin-Plate” Splines
• kernel function U(r) is principal solution of biharmonic equation that arises in
elasticity theory of thin plates:
∆ U (r ) = ∇ U (r ) = δ(r ).
2 4
• variational principle: U(r) minimizes the bending energy (not shown).
• 1D: U(r) = |r3| (cubic spline)
• 2D: U(r) = r2 log r2
• 3D: U(r) = |r|
2D: U(r) F ( x, y )
See Bookstein, Morphometric Tools for Landmark Data, Cambridge U Press, 1997
123. Resources
Textbooks:
Kenneth R. Castleman, Digital Image Processing, Chapter 8
John C. Russ, The Image Processing Handbook, Chapter 3
Online Graphics Animations:
http://nis-lab.is.s.u-tokyo.ac.jp/~nis/animation.html