The presentation summarizes the process and results of a marker-based Augmented Reality pilot project implemented using open source software, e.g. FLARToolKit.
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Augmented Reality with Open Source Software
1. Open Source AR Software
Business Case DirectFactory
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 1
2. BeCompany – About Us
Head office in Zurich
Independent integrator
Technology evaluation, focus on new technologies
Consulting, implementation, support
Specialized in open source technologies
Swiss Post solution provider:
DirectFactory and other web applications
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 2
3. DirectFactory & AR: Requirements
DirectFactory as tool for AR campaigns
Allow users to create/import 3D scenes
Use postcards as advertisement and
carrier/interaction medium
Personalization (text/other possibilities)
AR platforms: Desktop, iOS, Android
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 3
4. Marker-based AR
Video
Tracking
3D Model
3D Processing
Marker position
in 3D space
Camera position
for rendering Textures
Composition
3D Rendering
& Animation Animation
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 4
5. Business Case DirectFactory
Step 1: Technology Evaluation
Step 2: Prototype
Step 3: Pilot
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 5
6. Step 1: Technology Evaluation
Technologies (open & closed source)
Technical requirements & conditions
Legal conditions (licensing)
Options & capabilities (supported content &
markers, quality, reliability, personalization, …)
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 6
7. Step 1: Technology Evaluation
Closed Source Vendors
High quality showcases
Cross-platform solutions
No out-of-the-box personalization capability
Core customization requires cooperation with
vendor
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 7
8. Step 1: Technology Evaluation
Open Source Vendors
Lack of high-quality showcases
Projects focusing on single platforms, driven by
individual use cases, not market penetration
Small core development communities
Large supply of knowledgeable developers for
individual platforms, but not across platforms
Low initial costs
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 8
9. Open Source Marker-Based AR Platforms
Runtime
Hardware Operating System Video Tracking 3D Rendering
Environment
Away3D
Flash FLARToolKit
Papervision3D
Windows,
Desktop
Mac OS
Silverlight 5 3D
Silverlight SLARToolKit
Balder
Native OpenSceneGraph
iOS ARToolKit for iOS
(Objective C) (OpenGL)
Android Native (Java) ARToolKit/AndAR OpenGL
Mobile
Native (C#) /
Windows Phone Mango / SLARToolKit Native (C#)
Silverlight
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 9
10. Flash vs. Silverlight
Flash
Nov '10 - Apr '11
Source: statowl.com
Silverlight
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 10
11. Decision: Flash + FLARToolKit
Based on proven ARToolKit algorithms
Large supply of knowledgeable developers
Dual license (GPL for non-commercial use)
Supports multiple 3D engines:
Papervision3D, Away3D, Sandy, Alternativa3D
Marker: simple geometric forms, asymmetric
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 11
12. Step 2: Prototype
Goals
Proof of concept: Integration DF & AR
Basic technical capabilities & limitations
Basic project workflow
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 12
13. Step 2: Desktop Prototype
Lessons learned during Implementation
Easy integration of DF personalization service
(objects rendered independent from 3D scene)
Very fast implementation
Postcard feasible as marker carrier
No no-go criteria found
But: no anmiation yet – critical!
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 13
14. Step 3: Pilot
Goals
Reality check, verify assumptions
Actual implementation cost & complexity
3D model & anmiation workflow
Acceptance, understanding by recipients
Real-world technical limitations (webcam, …)
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 14
15. Limitations
& Conclusion
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 15
16. Technical Limitations
AR solutions still in early development stage:
„Peak of Inflated Expectations“ (Gartner)
No cross-platform runtime environment
(except proprietary plug-ins)
Flash usability problems:
Browser window size bug on Firefox Mac,
Keyboard focus
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 16
17. Handling Limitations
Users not familiar with AR applications
Webcam not available in older computers
Rigid lenses (notebooks) can impede handling
for some applications
Consideration: Alternative content for users
without webcam
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 17
18. 3D Scene Limitations
Existing standards (OpenGL) not consistently
available across platforms
No universal standard for 3D scenes
(3D modelling -> 3D rendering):
Built in Cinema4D, problems importing in Away3D
Generic application + arbitrary pluggable 3D scenes
very hard to achieve („AR scene library“)
Close cooperation between designer and AR/Flash
developer necessary
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 18
19. Rendering Performance Limitations
3D Rendering Flash Stage3D
Flash („Molehill“)
DirectX 9,
zzzz…
OpenGL 1.3,
OpenGL ES 2.0
„Future version of
Flash Player“
CPU GPU
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 19
20. Marker Limitations
Black & white
Simple, asymmetric shape (orientation
detection)
Size approx. 10 x 10 cm
Surface not shiny (postcard varnishing!)
Rigid carrier material to ensure that the
marker stays flat
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 20
21. Personalization Limitations
Personalization of 3D scene problematic
(personalized objects separate from 3D scene)
Marker-based personalization not possible with
ARToolKit
Alphanumeric code: cumbersome, risk of typing
errors
Visual code (QR): additional visual marker could
be confusing
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 21
22. Conclusion – as things are now :)
Many technical problems not related to AR itself
Multiple components with individual limitations
involved, incompatibilities
Technical limitations dictate conceptual & functional
possibilities – „function follows form“
Easy implementation, low initial costs
Realization in short timeframe possible
Be aware of non-technical limitations
(appeal, handling, usability, guidance)
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 22
23. When to Use Closed Source
High quality requirements
Complex 3D scenes (many polygons)
Image (e.g. photo) as marker
Support for multiple platforms (desktop, mobile)
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 23
24. When to Use Open Source
Low initial costs
Low quality/complexity requirements
(3D scene, marker)
Short time-to-market
Single target platform
(desktop or particular mobile platform)
Customizability (personalization, integration in
other services, …)
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 24
25. The End
Thank you for your attention!
Andreas Hartmann: Open Source AR Software – Business Case DirectFactory 2011-05-31 25