#3dconfsc15

1. 1
A Copyright Protection Technology
for
3D Printing Model(STL)
Jong (John) CHOI and SK Park(Markany)
Oct 22, 2015

2. 2
1. Clearing Center
2. Feature Extraction
3. Shape Estimation
4. THZ Detector

3. 3
1. Copyright in 3D Printing
Virtuous Cycling for national
Industry Eco-system and enhancing
National Competiveness
Capability Building for
Traditional Industry
Clusters (Automobile,
Construction, Manufacturing),
which have been competitive
3D Printing as a means of
Conversion Industry to create
new market and foundation of
national competiveness3D Innovation in Manufacturing
for converting into service
industry and for Creating value
added competiveness
Automobile
Construction
Space, aircraft
3D 프린터산업
의류
medical
3D Printing Industry

4. 4
2. Copyright in 3D Eco System

5. 5
2. Eco Systems for 3D Industry Promotion
standardization
certificate
copyright
protection
system
Fee
MGT
system계
quality Certificate
copy prevention
system
product
Stability
certificate
Industry Control System
Supportive Technology
Development
Industrial
convergence
Outputs
3D scan
system
Markany
copyright
protection
systems
Fee MGT
systems
authoring
tools
copy
Prevention
systems
인텔코리아 MarkanyETRI
Distribution
Tool
System
construction
medical
artsautomobile fashionmaterial
3D
printing characters
I/O
parts
SW
3D printer
Inputs
(design)
medical
info systems
distribution
platform systems
Slicing Technology

6. 6
3. Copyright Clearing Center
Feature Management (FMS)
Meta/feature
database
Main function
Meta/feature scann Fature extraction
Meta/fature
registration
Meta/fature
deletion
License issue proxy
Proxy/right policy
DB
Right
Policy info
Proxy server
Proxy info
Management Platform
Content MGT Member MGT
Statistic Member
Bulletin MGT
Service platform
Content category
Search/recom
mend service
SNS
User review Interesting
content
Minimum
price srvice
Streaming/archiving
storage
Thumb nail
Content/backup
storage
Registration/extraction
Database
Encrypt
content
History of
Illegal
Distribution
Collect
content
ViolationMonitoring CIMS)
service/met/
violation
Monitoring server
functions
Menu Design
Request
Feature
Recognition
Check
log data
Features
Recognitionadmin
Admin
System MGT
Monitoring
Violation judgement
Illegal distribution of 3D printing filesWeb Hard, TorrentMonitor illegal distribution
Illegal
distributor
3Dprinter
Purchaser
Print 3D
Distribution
Service
Delivery
content
Purchase
content
Content distribution server
(CDS)
Content distribution
Digital Right MGT (DRM)
Packing/license/device
Packaging server
Content
encryption
Key MGT
License server
License issue Log MGT
Integrated with
purchase
history
Device
authentication
Forensic marking
Embed Forensic
Mark
Extract forensic
mark
Content Management(CMS)
copyright/co
ntract MGT
Content/me
ta data
MGT
Integrate with CP/SP transfer
fee/payment clearing system
Request of content package
Register
Content
Creator
Database Database

7. 7
4. 3D Printing Process
Forward Engineering
Reverse Engineering
PC
3D Scanner
3D Modeling
CADIAN
Sketchup
Blender
Nurbs
Surface
solid
STL SLICE
Clean Mesh
Fix Mesh
Check volume
Scale
G CODE 3D Print
Controller
Product

8. 8
5. Issues in Copyright Protection (3D Printing)
(1) Illegal Copy of Model File (STL)
www.shapeway.com (www.****.com)
(2) 3D Scanning and Production
(www.****.com)

9. 9
6. Solutions for Illegal Copy and Legal Dispute
(1) Illegal Copy of 3D Model (STL)
(2) 3D Scanning and Production
(3)Modification of Model File
Encryption (DRM)
Feature(DNA)
Extraction
Shape Estimation
and Matching
THZ Detection
ChungAng Univ. Markany
KAIST
SangMyung Univ.
KEIT

10. 10
7. Encryption of License Storage and STL file
Distribution
legacy
3D Printing Clearance Center
Payment system
(PMS)
Content Management system
(CMS)
License server
License proxy server CAD Application
Clearing system
DRM server
DRM decryption
Agent
DRM Storage
(DRM NAS)
DRM DB
DRM
encryption
Storage
(NAS)
DB
License file
server
purchaser
Copyright management
Systems
Admin
Web
DRM
Policy
MGT

11. 11
Seller
UCI
Reg. Date
Distributor
Metadata
(Dublin Core)
Data Size
Padding Size
Signature Base Header Encrypted STL Data Padding
PurchasingSTLfile
Download
Signature Header Size DRM Header Encrypted Data Padding
Creator
Contributor
Identifier
Publisher
Rights
Title
Subject
Type
Format
Description
.
Signature Header Size DRM Header Encrypted Data Padding
 Algorithms: Seed, AES, MACrypto(KCMVP Module), etc Encryption of STL File
8. Encryption of STL Model

12. 12
 User Authentication
 Content Encryption
 License Management
 Software Authentication
 Hardware Authentication
Intellectual Property Protection
3D model (STL)
user 3D Printer
Purchase of 3D
model (STL)
Application
- Encrypted STL file
- Check whether License
conditions are satisfied
or not
Hardware
authentication
Software
authentication
Encryption
Issue License
- Check whether SW is legal
- Embed IP information
- Authenticate hardware
- Display/Print IP
information
-micro licensing
- Content encryption (Seed, AES,
KCMVP)
Use 3D Model(STL) Print 3D (STL)
9. STL File Protection

13. 13
1. Clearing Center
2. Feature Extraction
3. Shape Estimation
4. THZ Detector

14. 14
Copyright Protection of
3D STL Model
Certification of
User’s Purchase
History
Monitoring and
Tracing 3D STL
Model
Response to
Collusion Attack
Non-Blind Digital
Forensic for 3D
STL Model
Skeleton Based
Original Model
Estimation
Digital Forensic based on 3D STL
Skeleton with Original Estimation
 Digital Forensic Resistant to Collusion Attack
 Certification of Purchase History for 3D STL model
 Monitoring and Tracing 3D STL Distribution
 Extraction of 3D STL model skeleton
 Estimation of Original 3D STL Model based on Skeleton
1. Shape Estimation for STL model

15. 15
Cluster Maximization
Feature-Extraction (3D Gradient, Distance,
Surface). Similarity Analysis Algorithm
3D Object’s Feature
Extraction/analysis
ODiSC
Algorithm
Based
Clustering
Genetic
Algorithm
Based
Clustering
Topology
Based
Feature
Extraction
Histogram
based
Feature
Extraction
2D View
Based
Feature
Extraction
1. Feature Extraction and Similarity Measurement

16. 16
2. Skeleton Extraction for STL structure Analysis
• 3D medial axis extraction
original 3D Model (ex) Medial Axis Extraction

17. 17
3D STL model
Defined
skeleton
Extracted
skeleton
Structure
Analysis and
Similarity
Measurement
Extraction Module of
3D STL skeleton
Analysis Module of 3D
STL model skeleton
Module for Rapid Extraction and Analysis of 3D STL Model Skeleton
 Non-Blind Digital Forensic Algorithm for 3D STL Model
 Skeleton Extraction Technology for Topological Structure of
3D STL Model
2. Feature Extraction

18. 18
Estimated
Original
STL Model
of 3D STL
Model
3D STL
Model
Topological
Modification using
Skeleton Structure
Recovery of Surface
based on Skeleton
3 D STL Model Estimation of 3 D STL model Based on Skeleton Structure
Skeleton Structure
DB
Topologically
Distorted/transf
ormed 3D STL
Model
Topologically
corrected 3D
STL Model
3D STL
Model of
Surface
Recovered
 Digital Forensic Extraction and Tracing Technology for 3D STL
Model
 Estimation of Original Model based on 3D STL Skeleton Structure
3. Feature Extraction and Structure Analysis

19. 19
 Feature Extraction/Analysis of STL
Model
Address Length Type Descriptions
0
80
...
...
84
88
92
...
96
100
104
...
...
108
...
...
...
112
80
4
...
...
4
4
4
...
4
4
4
...
...
2
...
...
...
4
ASCII
integer
...
...
real
real
real
...
real
real
real
...
...
integer
...
...
...
real
Header information(not used)
Number of facet(triangles)
First triangle defination
normal vector
x coordinates of normal vector
y coordinates of normal vector
z coordinates of normal vector
first vector
x coordinate
y coordinate
z coordinate
Attribute information
Number of attributes(set to zero)
Second triangle defination
normal vector
x coordinates of normal vector
 2D_view based feature extraction
2D view based methods
Histogram based methods
Topology based methods
STL File Format in Binary data
4. Feature Extraction of STL Model

20. 20
 Optimization of Features
 Feature Extraction/Analysis of 3D Object
 Developing Method of Maximizing Homogeneity of Single Unit within Group using ODiSC(Optimized
Discriminative Subspace Clustering) algorithm and Genetic Algorithm
 Developing Technology of Extracting Features through Clustering for Pattern Recognition
5. Feature Extraction of STL Model

21. 21
 Reverse Analysis of 3D
Model
 Feature Extraction DB and optimization
< 2D View DB > < Topology DB> < Histogram DB>
< clustering>
< optimized DB>
6. Feature Description of STL Model

22. 22
3D content
features
features DB
Clustering
Detection result
2D based Feature Extraction
After extraction
of each features .
Features are
combined
Classification of
features based on
clustering algorithms
Extraction of topological features
features
3D feature
comparsion
/analysis
Extraction of Histogram Features
 Technology of Similarity based on Features
Extraction of topological features
Extraction of Histogram Features
2D based Feature Extraction
7. Feature Description of STL Model

23. 23
1. Clearing Center
2. Feature Extraction
3. Shape Estimation
4. THZ Detector

24. 24
1. Feature Based Similarity Recognition
Feature Based
Similarity Recognition
Clustering Maximization
3D Object Feature Extraction
and Analysis
ODiSC
Based
Clustering
algorithm
Genetic
Based
Clustering
algorithm
Topology
based
Extraction
Histogram
based
Extraction
2D View
based
Extraction

25. 25
Combined Depth Image based Deformed Image Recognition Algorithm
2. Feature Extractions from 3D Model
3D Model
(DB Build)
Generate
Depth Image
Combine
Depth Image
Feature
Extractions
3D Model
Normalization
Deformed
Image(Query)
Depth Image
Generation
Store
Features
Feature
Extraction
Feature
Comparison
Output

26. 26
• Normalize using Principal Components Analysis (PCA) , Implicit Shape
Representation (ISR), and weighted implicit Shape Representation(WISR)
technique.
• WISR algorithm can normalize 3D Model so that view point is pointed to sphere
containing the most features
• The depth in 2D images can be invariant, comparing before transformation and after
transformation, when WISR algorithm is applied to.
3. Shape Normalization based on Weighted Implicit Shape Representation

27. 27
• Extract features & Matching: Features of each
image can be extracted, and by measuring
Euclidean distance between features similarity is
evaluated. Then, features of high values are
clustered to generate homography.
• Preprocessing : generate Converter Module
which will be used in RANSAC sampling process
and decide size of grids which 2D image will be
segmented into
• Calculate Homography: Homography will be
created between Pairs of features generated in
“feature extract & match stage” and the image
generated in preprocessing stage
• RANSAC algorithm is implemented to create
Homography between pairs of features generated
in ‘Feature Extraction & Matching’ stage and 2
images generated in Converter Module
(above) matching features,
(below) second time feature matching
after complementation
4. Image Preprocessing Based on Homography

28. 28
- Left) Character of Dinosaur model. 3D model with cropped Tail part .
- Center) Original 3D Model
- Right) Two models are recognized the same model and merged
 Case) Deformed image recognition based on combined depth image
5. 3D Model Recognition Algorithm based on Combined Depth Image

29. 29
1. Clearing Center
2. Feature Extraction
3. Shape Estimation
4. THZ Detector

30. 30
3D Pattern
(license)
Inside 3D printed
product
Product based license
(3D Pattern layer)
• 3D Printer Resolution : 0.2mm(ave)
• THZ x,y resolution : 0.5mm
• Expected 3D Pattern size : 5 x 5 x 2
• 10 x 10 x 10 bit  125byte
2mm
 Scanning: Production Data Embedded
within 3D Printed Products
1. THZ 3D Watermarking: THZ Recognition Technology

31. 31
THz Antenna
3D printed
product
Laser
Degree of Signal transmission
Is different depending on Output Material
 THz Imaging Principle
2. THZ 3D Watermarking

32. 32
1. Original Copyright
Holder of 3D Model
2. Date of Printing
3. Printer
(material, type of
printer, IP)
4. Host PC Information
5. Original Right Holder
of Contents
6. Use of Permission
…
1. ㈜samsung P
2. 20150317/11:25
3. ㈜Stratasys, FDP130s
(material, type of
printer, IP)
4. John CHOI
5. ㈜iconix
6. Toys
1. 20byte
2. 10byte
3. 40byte
4. 20byte
5. 10byte
…
Min : 125 byte
3D Pattern Layer
2mm
…
 3D Pattern Recording
3. THZ 3D Watermarking

33. 33
……
THZ data
signal/image
processing
3D pattern
integration
3D Pattern
analysis
…
1. Copyright information of 3D
Model
2. Information of Printing Time
3. Type of Used Printer
(material, type of printer)
4. Data of Host PC
5. Use permission
Tomography
(single layer image
capturing )
 THZ Detector Imaging Process
4. THZ 3D Watermarking

34. 34
TDS Module TDS transponder
equipme
nt
1. Femtosecond laser
2. DAQ equipment (Lab view)
3. Delay & optics
1. TDS antenna module
2. Thz transmitter module
3. Object Stage & optics
function THZ Generation & Image processing Thz transmitter & Object reflector part
photo
TDS module case Optical Table
5. THZ Detection Device

35. 35