Overview of the paper "What's in Unison? A Formal Specication and Reference Implementation of a File Synchronizer" by Benjamin C. Pierce Jerome Vouillon presented at Oregon State University for "Domain Specific Languages" class on May 20th 2014. Presentation time: 20 min

1. WHAT’S IN UNISON?
A FORMAL SPECIFICATION AND
REFERENCE IMPLEMENTATION
OF A FILE SYNCHRONIZER
Presentation type: paper presentation
Class: CS 589 – Domain Specific Languages
Presenter: Sergii Shmarkatiuk
Date: 5/20/2014

2. TARGET DOMAIN: FILE REPLICATION AND
SYNCHRONIZATION
2
File
synchronization
Replication
MirroringDeployment
Cloud storage

3. COMMON USE CASE: CLOUD STORAGE AND
SYNCHRONIZATION BETWEEN DEVICES
3

4. COMMON USE CASE: DEPLOYMENT OF
WEB-APPLICATION
4
development instance production instance
http://localhost/MyApp http://myapp.com
3. upload
2. test
ftp://@myapp.com/var/www/html/
1. edit files
4. test
/var/www/html/MyApp

5. COMMON USE CASE: DEPLOYMENT OF
WEB-APPLICATION
5
development instance production instance
http://localhost/MyApp http://myapp.com
4. test
/var/www/html/MyApp ftp://@myapp.com/var/www/html/
1. edit files
5. test
2. edit files
3. sync
+

6. FILE SYNCHRONIZATION SOFTWARE
6
Rsync
Beyond Compare
Synchronize It!
Dropbox
Google Drive
Box.net
OneDrive
Manual synchronization
Cloud storage

7. SOFTWARE DEPLOYMENT TOOLS
7
Puppet
Chef
Capistrano
CFEngine
Ansible

8. UNISON
File synchronization tool
Command line interface
Implemented in OCaml (DSL) and C
(tool)
Available for all major platforms (UNIX,
Win, Mac)
Unlike rsync, Unison is not included into
basic UNIX distributions
8

9. PAPER CONTRIBUTIONS
Paper presents mathematical model,
DSL, mechanics and challenges of file
synchronization
Authors proved some properties of file
synchronization operations using Coq
Authors described the gap between
idealistic representation and actual tool
implementation
9

10. QUESTIONS
Keying Xu, Chao Peng:
What is the semantic domain of Unison? Is it a deep or
shallow embedded DSL?
10
Unison is a deep embedded DSL

11. UNISON: SYNTAX AND SEMANTIC DOMAIN
11
 Current states of the
replicas
 2 file trees
 Archive (last
synchronized state)
 Synchronized
replicas
 2 file trees
 Archive (last
synchronized state)
Syntax Semantic domain
sem
A B
~
A B
=

12. QUESTIONS
Chao Peng:
What is the basic object of Unison?
12

13. UNISON: BASIC DATA STRUCTURES
FILESYSTEM
13
type name = string
type contents = string
type properties = string
type fs = Dir of properties * dContents
| File of properties * contents
| Symlink of contents
| Bot
and dContents = (name * fs) list
OCaml

14. UNISON: BASIC DATA STRUCTURES
UPDATE DETECTION
14
type prevState = DIR
| FILE
| SYMLINK
| ABSENT
type ’a leafUpdate = LeafSame
| LeafUpdated of ’a * ’a option
OCaml

15. UNISON: BASIC DATA STRUCTURES
UPDATE DETECTION
15
type updateItem = Same
| Updated of updateContent * prevState
| Error
and updateContent =
UCDir of properties leafUpdate * updateChildren
| UCFile of properties leafUpdate * contents leafUpdate
| UCSymlink of contents leafUpdate
| UCAbsent
and updateChildren = (name * updateItem) list
OCaml

16. UNISON: BASIC DATA STRUCTURES
RECONCILIATION
16
type direction = Conflict
| LeftToRight
| RightToLeft
| Equal
type transportInstr =
Instr of updateItem * updateItem * direction
| NoInstr
| Problem
type transportInstrTree = Node of transportInstr * transp
ortInstrList
and transportInstrList = (name * transportInstrTree) list
OCaml

17. UNISON: BASIC OPERATIONS
17
• Comparison of two file trees
• Description of detected difference
Update detection (buildUpdates)
• Building set of transport instructions
Reconciliation (reconcile)
• Performing transport instructions
• Giving user the opportunity to verify changes
Propagation (propagate)

18. UNISON: SEMANTIC FUNCTIONS
FILESYSTEM OPERATIONS
18
• assoc
• lookupPath
• reverse
• mem
• fsKind
• remove
• deletions
• buildUpdates
• buildUpdateChildren

19. UNISON: SEMANTIC FUNCTIONS
UPDATE DETECTION
19
• hasErrors
• hasErrorsChildren
• propagateErrors
• noConflictInstr
• reconcileNoConflict
• reconcileNoConflictChildren
• leafDirection
• combineDirections
• uassoc
• reconcile
• reconcileChildren
• reconcileLeft

20. UNISON: SEMANTIC FUNCTIONS
RECONCILIATION
20
• leafApply
• updateArchive
• updateArchiveChildren
• emptySource
• copyRec
• copyChildren
• copy
• checkNoUpdates
• replaceRec
• replace

21. UNISON: SEMANTIC FUNCTIONS
PROPAGATION
21
• performInstrLeaf
• newProps
• unchangedProps
• performInstrDir
• performInstr
• propagateLocally
• propagate
• propagateInChildren

22. QUESTIONS
Keying Xu:
How do authors deal with modeling gap between the
reference implementation and the specification?
22
Authors describe „modeling gap‟ limitations in
their paper

23. UNISON: THE “MODELING GAP”
23
 Functional program (Ocaml)
 Returns new replicas without
changing content
 Written as if it “owns”
filesystems
 Regards filesystems as simple,
mathematical tree structures
 Assumes that all operations
can be implemented
111111111111
 Treats archive as full-blown
filesystem
 Imperative program (C)
 Modifies real filesystems in-
place
 Runs with live filesystems
123123
 Operates on real
implementations of filesystems
(POSIX, NTFS, …)
 Deals with operations that might
be impossible to implement
 Stores just a fingerprint of each
file‟s contents
Reference implementation
(DSL)
Real implementation (software
tool)

24. QUESTIONS
Brent Carmer:
What is the connection between Unison and your DSL?
Panini Patapanchala:
Relation with the version control and what aspects you
can take from this paper.
24
• SCMF-DSL also uses concept of replication
• SCMF-DSL also operates with file trees
• SCMF-DSL also detects file changes to perform such
automatic actions as version numbering

25. UNISON VS SCMF-DSL
25
Unison
SCMF-
DSL
Deployment
Replication Version control
Version numberingConflicts

26. UNISON VS SCMF-DSL
26
 Takes into account only latest
synchronized state
 Allows synchronization only
between latest states
 Treats both file trees as equal
sources of changes (everything
is writable)11111111111
 There is no defined direction for
replication 1111111111111111111
 Operates with file trees
 Replicates contents of
filesystem
 Might generate incompatible
changes (conflicts)
 Saves information about all
synchronized states
 Allows to roll back to previous
states
 Treats file trees as primary and
secondary (can be writable or
read-only)
 Operates with certain direction
for replication: from primary
replica to secondary replica
 Operates with version trees
 Replicates contents of version
control system
 Does not generate incompatible
changes
Unison SCMF-DSL

27. UNISON VS SCMF-DSL
27
A B
~
A B
=
sem
Unison: file trees
SCMF-DSL: version trees
sem
1
x
2
3
57
8
9
11
x
4
6
x
10
1
x
2
3
57
8
9
11
x
4
6
x
10
12

28. UNISON VS SCMF-DSL
28
A B
~
A B
=
sem
Unison: file trees, 2 platforms (A, B)
SCMF-DSL: version trees, N platforms (P1, P2, … PN)
sem
1
x
2
3
57
8
9
11
x
4
6
x
10
1
x
2
3
57
8
9
11
x
4
6
x
10
12

29. QUESTIONS
Brent Carmer:
Does the user ever construct things using the types
listed in the reference implementation?
29
NO
User uses real implementation (tool) instead of
reference implementation (DSL)

30. QUESTIONS
Amin Alipour:
How can they make sure that function synch is run
atomically?
Brent Carmer:
How do they use Coq to verify their reference
implementation?
Panini Patapanchala:
Maximal runs are unique can you justify the theorem with an
example.
30

31. QUESTIONS
31
Authors use Coq to prove following properties of their DSL:
• Laziness is safe (replication with itself is safe )
• Mirroring is a special case (replication with previously
synchronized state o and replica a gives replica a)
• Maximal runs are unique (it is impossible to generate
two different synchronizations on the same two replicas
a and b)
• Success in the absence of conflicts (if replication does
not generate conflicts first time, it won‟t generate
conflicts next time as well)

32. QUESTIONS
Amin Alipour:
The paper assumes that there are only two replica of filesystem's.
Is that right? If so, how it can scale synchronization to more than
two replicas?
Rui Qin:
How about more than two replicas, does it also work?
Panini Patapanchala:
I feel the paper explained the base cases for reconciliation and
presently the more important problems are the one presented in
future scope like multi-replica synchronization for more number of
replicas.
32
NO
Unison works only with pairs of replicas

33. QUESTIONS
Amin Alipour:
What is the relation of conflict as described in the paper and
merge in git?
Chao Peng:
Is Unison easily extensible? can you conclude some low-
level or high-level aspects of Unison?
33

34. QUESTIONS
Panini Patapanchala:
The buildupdate of the implementation is a bit like
imperative implementation than functional.
34
This is partly because of mixed nature of OCaml language -
it incorporates functional, imperative and
object-oriented paradigms