1. PISA
Production, Indexing and Search
of Audio-visual Material
Computer Assisted Manufacturing
Dieter Van Rijsselbergen, IBBT – MMLab – UGent

2. PISA - Overview
Computer Assisted Design Search Engine
Concept
Indexing
Script Editing
Retrieval
• Parse scenario Script Editing • Timecode based indexing
• Shooting script editor
• Geo-temporal reference
• Storyboard
• Taxonomy based indexing and search
Abstract • Facetted search
Information
Virtual Intelligent Analysis and
Model Driven Development: Modelling Analysis Quantization
• Setting (Stage properties, light) Interpretation
• Character • Character identification
• Synthetic Speech • Background categorisation
• Sound effects Virtual and identification
• Character animation Model • Topic and eventdetection
• Virtual camera
Quantization
Computer Assisted Manufacturing
Automated Reverse Engineering
Realisation Production • Shot segmentation
• Ingest Footage • Video footprint and reuse detection
• Editing • Biometric face detection
• Mastering • Background analysis
• Reproduction to alternative distribution channels • Speech-to-text
35

3. Computer Assisted Manufacturing
! Computer Assisted Manufacturing is about automation
of media production processes
! Based on extensive production metadata
! high level Product Engineering for narratives
! low level Manufacturing Engineering for actual
manufacturing specifications
! Using clever algorithms to fill in the gaps
! supported by Trouvaille
36

4. Drama Manufacturing Processes
Synopsis Dialogue Product Engineering
Writing Writing
Manufacturing Engineering
Shooting 3-D Quality
Editing
Scripting Previsualization Assurance
Manufacturing
? Acquisition ? ?
37

5. Objectives
! Extend manufacturing workflow with 3-D previsualization
! Build evolving products
! Re-use previsualization parameters
during manufacturing
! Increase efficiency and automation in production
processes
! 1) Rough Cut Editing based on 3-D previsualization
! 2) Repurposing for multiple output channels
38

6. Computer Assisted
Manufacturing
Rough Cut Editing

7. Rough Cut Editing
! Inclusion of 3-D previsualization into the
drama manufacturing process
! Use previsualization as shooting script
! Integration into the MAMS and Craft Editing systems
! Building evolving products
! mixture of previsualized and shot footage
! evolve from all-previsualized to all-shot composition
40

8. Rough Cut Editing Processes
Synopsis Dialogue Product Engineering
Writing Writing
Manufacturing Engineering
3-D Quality
Editing
Previsualization Assurance
Manufacturing
Previz.
Acquisition
Rendering
41

9. Computer Assisted
Manufacturing
Repurposing

10. Computer Aided Repurposing (CAR)
! Automated Repurposing of Drama
! Different output channels
! HDTV
! SDTV
! iPod/Cellular
! True Universal Media Experience
! requires semantically sensible scaling/cropping
43

11. Repurposing Concept
! Automated Repurposing of Drama
! Spatial adaptation using semantic Objects of Interest
! as in “focus on Capt. Kirk”
! decided by creative staff in function of the narrative
! is a form of Manufacturing Engineering
! can be defined in early 3-D Previsualization
! preparation not only of shooting script
but also of adaptations
! parameters refined in Quality Assurance
44

12. Repurposing: How?
! 1) Determine where the semantic elements are
! Semantic Objects of Interest
are linked with regions of interest
! manual annotation
! automated
algorithms
! aided by
Product
Engineering
metadata!
Jenny Waldek
45

13. Repurposing: How?
! 2) Provide reframing instructions, e.g.:
! Object Orientation and Range
! “The Bad must be shown on the right…”
0.75
! Object Size
! “… and I want it to be a full shot of The Bad.”
Frame Size
Object of Interest
Size: (30%, 90%)
46

14. Repurposing: Channels
! HDTV
! Originally acquired
high definition material
! SDTV ! Mobile (iPod)
! Legacy applications ! Semantically
! Common pan/scan repurposed
! Editing
(AVID)
47

15. Repurposing Processes
Synopsis Dialogue Product Engineering
Writing Writing
Manufacturing Engineering
3-D Repurposing Quality
Scripting Editing
Previsualization Assurance
Manufacturing
Previz. EDL
Acquisition Repurposing
Rendering Rendering
48

16. Workflow (1): Synopsis
49

17. Workflow (2): Management
50

18. Workflow (3): 3-D Previz. – Scenario
51

19. Workflow (4): 3-D Previz. – Shooting Script
52

20. Workflow (5): 3-D Previz. Editing
53

21. Workflow (6): Acquisition
54

22. Workflow (7): Editing
55

23. Workflow (8): Quality Assurance & Analysis
Plane
56

24. Workflow (9): Repurposing
57

25. Workflow (10): Conform & Publish
58

26. Computer Aided Manufacturing: Workflow
59