LiveChromaKey and JSARToolKit presentation

Spark project 勉強会 #09

JSARToolKit / LiveChromaKey

2009.05.27
Yusuke Kawasaki
川﨑有亮 (kawanet)
http://www.kawa.net/

http://www.kawa.net/ 1

Yusuke Kawasaki（川﨑有亮）

• Perl monger
– XML::TreePP, XML::FeedPP, ...
• JavaScript addict
– Wiimote over HTTP, Gainer, ...
• ActionScript 3.0 newbie
– Today!
• Media Technology Labs
– Recruit Co., Ltd. in Japan

http://twitter.com/kawa0117

§1 - JSARToolKit

JavaScript Augmented Reality

JSARToolKit

• JSAR（ジャバスクリプト拡張現実）
– JavaScript Augmented Reality

• JSARToolKit
– My first work using ActionScript 3.0
– Powered by SWFObject!
– A library to run AR by JavaScript
– A proxy wrapper for a Flash app using FLARToolKit
– Debuted at OSDC.TW 2009 in Taipei

• Genealogy
– 2008.03 – NyARToolKit for Java - by A⻁@nyatla.jp
– 2008.05 – FLARToolKit for ActionScript3 by Saqoosha
– 2009.04 – JSARToolKit for JavaScript by Kawanet

• Saqoosha さん、本当にありがとうございます！！_o_


JSARToolKit Synopsis

It needs just several lines of JavaScript code to run AR.
When a marker found, it calls onDetect callback function.

< s c r i p t t y p e = quot; t e x t / j a v a s c r i p t quot; s r c = quot; s w f o b j e c t . j s quot; > < / s c r i p t >
< s c r i p t t y p e = quot; t e x t / j a v a s c r i p t quot; s r c = quot; j s a r . j s quot; > < / s c r i p t >
< s c r i p t t y p e = quot; t e x t / j a v a s c r i p t quot; > < ! - -
v a r p a t s = [ ' j s a r l o g o . p a t ' ] ;
v a r j s a r = n e w J S A R ( ' j s a r _ h e r e ' ) ;
j s a r . d r a w M a r k e r R e c t = t r u e ;
j s a r . o n D e t e c t e d = f u n c t i o n ( r e s u l t ) { . . . } ;
j s a r . i n i t ( ) ;
j s a r . s e t M a r k e r ( p a t s ) ;
- - > < / s c r i p t >
< d i v i d = quot; j s a r _ h e r e quot; > < / d i v >


JSAR returns a JavaScript Object

{
quot; d e t e c t e d quot; : t r u e ,
quot; m a r k e r s quot; : [
{
quot; c o d e i d quot; : 0 ,
quot; d i r e c t i o n quot; : 2 ,
quot; c o n f i d e n c e quot; : 0 . 5 1 2 6 3 7 ,
quot; v e r t e x quot; : [
{ quot; x quot; : 3 4 6 , quot; y quot; : 2 0 1 } ,
{ quot; x quot; : 3 9 5 , quot; y quot; : 1 5 4 }
]
} ,
] ,
}


JSARToolKit Layers

Flash Layer jsar.swf

HTML Canvas Layer

HTML Div Layer


AiR Xiaolongbao（エア⼩籠包）

2009.04.19 @ Taipei (OSDC.TW)


§2 - LiveChromaKey

Bluescreen-less augmented IN reality

LiveChromaKey
• A new image synthesizing engine for AR
– 2009.05 released by Kawanet (me!)
• Pure ActionScripe 3.0
– No library dependencies
– No JavaScript
• Only webcam needed
– No blue back screen
– No need to print
AR marker PDF


Step 1/3 - Stationary background

At first, connect you webcam and run LiveChromaKey.
Never move until it recognizes stationary background.


Step 2/3 – Bluescreen on the fly

It generates bluescreen automatically.
Live Video – Background = Bluescreen

－ ⇒


Step 3/3 - Transparent foreground

Get the foreground image as a translarent Sprite.
Live Video – Bluescreen = Foreground

－ ⇒


LiveChromaKey Synopsis

LiveChromaKey provides four kinds of Sprites.

v a r c h r o m a k e y : L C K _ C o r e = n e w L C K _ C o r e ( ) ;
c h r o m a k e y . i n i t ( ) ;

v a r s p L i v e : S p r i t e = c h r o m a k e y . g e t L i v e ( ) ;
v a r s p B a c k : S p r i t e = c h r o m a k e y . g e t B a c k g r o u n d ( ) ;
v a r s p M a s k : S p r i t e = c h r o m a k e y . g e t M a s k ( ) ;
v a r s p F o r e : S p r i t e = c h r o m a k e y . g e t F o r e g r o u n d ( ) ;

t h i s . a d d C h i l d ( s p L i v e ) ;
t h i s . a d d C h i l d ( s p B a c k ) ;
t h i s . a d d C h i l d ( s p M a s k ) ;
t h i s . a d d C h i l d ( s p F o r e ) ;


Example: Travelling in Egypt

egypt.swf

Sprites augmented IN reality

The key feature of LiveChromaKey is to insert
Sprites augmented between background and
foreground images.

Sprite


Example: Slideshow in reality

minority.swf

Multiarmed deity (or 美川健⼀)

exile.swf

ひとりおそ松くん

exile.swf

ひとり Exile

exile.swf

§3 - LivePointers

Live Color Pointer Detection Library

LivePointers

• Live Color Pointer Detection Library
– will be released soon!

• Pure ActionScripe 3.0
– No library dependencies
– No JavaScript

• Webcam as a 3D human interface
– No need to print AR marker PDF


Pointer = Pixel Cluster

• Ex.
– Cluster #0
– 250 pixels
– RGB: 25D985
– H: 152
– S: 0.83
– V: 0.85


Labeled Pointer Detection

• Labeled pointer needs a guide color.

• ex. fingercap
– 0x2197A9 (uint)
– ￥105（キャン・ドゥ）

• Fingercap is definitely
important user interface
device of the future.


LivePointers（⽇本語のみスマソ）

• 最短距離法 (Nearest Neighbor Clustering) をベース
にして、ActionScript 3.0 でリアルタイムに表⽰でき
る程度に単純化したクラスタ分析処理を⾏う。

• 画⾯内の単⾊のカラーマーカーを検出します。
– ⽩・⿊・グレー・淡⾊は検出しません。
– （S・V値が⾼い）くっきりした⾊が得意です。
– 模様は検出しません。（⾊のみを認識する）
– 通常は、画⾯内の全ての検出⽤ガイドとしてRGBで
⾊指定した「ラベル付マーカー」を利⽤する。

• 3次元⼊⼒のポインティングデバイスとして利⽤可能。


Now on the Spark!

JSARToolKit
http://www.libspark.org/svn/js/JSARToolKit/

LiveChromaKey
http://www.libspark.org/svn/as3/LiveChromaKey/

LivePointers
... coming soon!


Thank you!

Yusuke Kawasaki
川﨑有亮 (kawanet)


LiveChromaKey and JSARToolKit presentation

Recomendados

Recomendados

Más contenido relacionado

Similar a LiveChromaKey and JSARToolKit presentation

Similar a LiveChromaKey and JSARToolKit presentation (20)

Más de Yusuke Kawasaki

Más de Yusuke Kawasaki (20)

Último

Último (20)

LiveChromaKey and JSARToolKit presentation