One of the presentations used in a discussion meeting about GlusterFS held on Sep. 14, 2011 in Japan. Ust: http://www.ustream.tv/channel/glusterfs Togetter: http://togetter.com/li/188183

1. GlusterFS
@keithseahus
September 14, 2011

2. Agenda
!
!
!
!
!
2
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

3. Agenda
!
!
!
!
!
3
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

4. ! @keithseahus
–
! @jincl16
–
!
– NTTPC
–
– Gluster 4
4
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

5. from NTTPC
! @moromizu
–
! @tk_hamaguchi
–
5
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

6. Agenda
!
!
!
!
!
6
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

7. ! C
! 231,277 , 200 (*.c)
– 8,693
7
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

8. !
!
shared object
!
! volum
e
8
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

9. ! (glusterfsd)
– storage/posix
– protocol/server
– performance/io-threads
– features/locks
! (glusterfs)
– protocol/client
– cluster/dht
– cluster/afr
– cluster/storipe
– performance/read-ahead
– performance/io-cache
– performance/quick-read
– performance/stat-prefetch
– performance/write-behind
9
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

10. storage/posix
!
POSIX I/O
!
– o-direct GF_OPTION_TYPE_BOOL
– directory GF_OPTION_TYPE_PATH
– export-statfs-size GF_OPTION_TYPE_BOOL
– mandate-attribute GF_OPTION_TYPE_BOOL
10
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

11. protocol/server
! glusterfsd
glusterfs
!
– transport-type GF_OPTION_TYPE_STR
• tcp|socket|ib-verbs|unix|ib-sdp|tcp/client|ib-verbs/client
– volume-filename.* GF_OPTION_TYPE_PATH
– inode-lru-limit GF_OPTION_TYPE_INT 0-(1 * GF_UNIT_MB
– client-volume-filename GF_OPTION_TYPE_PATH
11
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

12. protocol/client
! glusterfs
glusterfsd
!
– username GF_OPTION_TYPE_ANY
– password GF_OPTION_TYPE_ANY
– transport-type GF_OPTION_TYPE_STR
• tcp|socket|ib-verbs|unix|ib-sdp|tcp/client|ib-verbs/client
– remote-host GF_OPTION_TYPE_ANY
– remote-subvolume GF_OPTION_TYPE_ANY
– transport-timeout GF_OPTION_TYPE_TIME
• 5-1013
12
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

13. cluster/dht
! Elastic Hashing Algorithm
• brick 32bit
• 12 bricks –
• brick1 = 1- 357913941
• brick2 = 357913942 - 715827883
• brick3 = 715827884 – 1073741823
• …brick12 = 3937053354 – 4294967295
•
• /share/somefiles/data/glusterrules.txt = 815827884
• brick I/O
•
13
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

14. cluster/afr
! Automatic File Replication
! self-heal
!
– read-subvolume GF_OPTION_TYPE_XLATOR
– favorite-child GF_OPTION_TYPE_XLATOR
– data-self-heal GF_OPTION_TYPE_BOOL
– metadata-self-heal GF_OPTION_TYPE_BOOL
– entry-self-heal GF_OPTION_TYPE_BOOL
– data-change-log GF_OPTION_TYPE_BOOL
– metadata-change-log GF_OPTION_TYPE_BOOL
– entry-change-log GF_OPTION_TYPE_BOOL
– data-lock-server-count GF_OPTION_TYPE_INT 0
– metadata-lock-server-count GF_OPTION_TYPE_INT 0
– entry-lock-server-count GF_OPTION_TYPE_INT 0
14
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

15. afr - Gluster Native Clinet (FUSE)
15
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

16. afr - gNFS
glusterfs =
gNFS (gluste
r
NFS )
16
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

17. cluster/stripe
!
!
– block-size GF_OPTION_TYPE_ANY
– use-xattr GF_OPTION_TYPE_BOOL
17
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

18. performance/read-ahead
!
!
– force-atime-update GF_OPTION_TYPE_BOOL
– page-size GF_OPTION_TYPE_SIZET
• (64 * GF_UNIT_KB)-(2 * GF_UNIT_MB)
– page-count GF_OPTION_TYPE_INT
• 1-16
18
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

19. performance/io-cache
!
!
– priority GF_OPTION_TYPE_ANY
– cache-timeout (force-revalidate-timeout) GF_OPTION_TYPE_INT
• 0-60
– page-size GF_OPTION_TYPE_SIZET
• (16 * GF_UNIT_KB)-(4 * GF_UNIT_MB)
– cache-size GF_OPTION_TYPE_SIZET
• (4 * GF_UNIT_MB)-(6 * GF_UNIT_GB)
19
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

20. performance/quick-read
!
! open(), read
(), close() 1
!
– cache-timeout GF_OPTION_TYPE_INT
• 1-60
– max-file-size GF_OPTION_TYPE_SIZET
• 0-(1000 * GF_UNIT_KB)
20
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

21. performance/stat-prefetch
! stat() dentry
21
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

22. performance/write-behind
! write()
!
write()
!
– flush-behind GF_OPTION_TYPE_BOOL
– aggregate-size GF_OPTION_TYPE_SIZET
• (128 * GF_UNIT_KB)-(4 * GF_UNIT_MB)
– window-size GF_OPTION_TYPE_SIZET
• (512 * GF_UNIT_KB)-(1 * GF_UNIT_GB)
– enable-O_SYNC GF_OPTION_TYPE_BOOL
– disable-for-first-nbytes GF_OPTION_TYPE_SIZET
• 1 - (1 * GF_UNIT_MB)
22
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

23. performance/io-threads
! glusterfsd I/O
! glusterfs I/
O
!
– thread-count GF_OPTION_TYPE_INT
• 1-32
23
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

24. features/locks
! glusterfsd fcntl() flock()
!
– mandatory-locks (mandatory) GF_OPTION_TYPE_BOOL
24
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

25. glusterd-volgen
! volume
gluster
25
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

26. CLI
!
– gluster peer {probe|detach|status|help}
!
– gluster volume {create|delete|start|stop|set|
info|add-brick|remove-brick|rebalance|
replace-brick|log|sync|reset|profile|quota|top|
geo-replication}
26
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

27. Agenda
!
!
!
!
!
27
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

28. ! CIFS
– Samba + Native Client
–
– CTDB
! WebDAV
– Apache httpd + Native Client
! iSCSI
– iSCSI Enterprise Target + Native Client
gNFS
28
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

29. 29
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

30. Agenda
!
!
!
!
!
30
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

31. !
– 1
!
– glusterfs-logrotate volume
nfs.log logrotate.d orz
! self-heal
– find hoge | xargs stat ls –lRf
– entire-self-heal
!
– GlusterFS QoS
– tc
31
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

32. Agenda
!
!
!
!
!
32
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

33. ! glusterfs-1.3.x
– symlink self-heal …
! glusterfs-2.0.x
– write-behind …
! glusterfs-3.0.x
– thin-provisioning
33
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

34. GlusterFS
! on Google Groups
– http://groups.google.com/group/gluster-ja
– @keithseahus + @thatsdone
34
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.

35. !
!
!
!
!
35
Copyright © 2011 NTTPC Communications, Inc. And Gluster, Inc. All Rights Reserved.