why we need ext4

1. Why we need ext4? Robin Dong <sanbai@taobao.com>

2. ext2 global layout Image from: http://learn.akae.cn/media/ch29s02.html

3. ext2 global layout Super Block (1 block) GDT (multi blocks) Block Bitmap (1 block) Inode Bitmap (1 block) Inode table (multi blocks)

4. Super-block and GDT are vital, therefore other groups will store their copies. If mkfs with “sparse_super”(default) not all groups have the copy of super block and GDT, only Group 0,1,3,5,7,3 2 ,5 2 ,7 2 ,3 3 ,5 3 ,7 3 ....have it. ext2 global layout

5. There is a structure called “Reserved GDT” which is putted after GDT and before Block-bitmap, it is also a large file. It is used for “resize” feature which could expand the size of whole filesystem.

6. ext2 file layout Image from: http://e2fsprogs.sourceforge.net/ext2intro.html

9. ext2 directory remove Image from: http://blog.csdn.net/anghlq/archive/2011/05/17/6427052.aspx

11. Optimize directories in filesystem. This option causes e2fsck to try to optimize all directories, either by reindexing them if the filesystem supports directory indexing, or by sorting and compressing directories for smaller directories, or for filesystems using traditional linear directories.

13. Fast Symlink: link path is stored in inode (if link path is smaller than 56 bytes) ext2 symlink

14. ext2 symlink Image from: http://www.pluto.it/files/journal/pj9811/e2fs.html

15. ext2 hard link

16. ext2 xattr

18. * | header |

19. * | entry 1 | |

20. * | entry 2 | | growing downwards

21. * | entry 3 | v

22. * | four null bytes |

23. * | . . . |

24. * | value 1 | ^

25. * | value 3 | | growing upwards

26. * | value 2 | |

27. * +--------------------+

29. If meta-data is in bad blocks,e2fsck will try to allocate new block for it.

31. ext3 could be looked like an ext2 filesystem with a journal file

32. dir_index

33. more efficent directory-searching

35. Journal will ensure filesystem consistent or recovery filesystem on system boot.

37. Ordered

38. Journal

40. Find dx_entry against hash value in root block by binary-search

41. Find ext3_dir_entry in leaf block one by one

43. Imaging an ext3 filesystem with 4K block size, a directory could contain about 5 million files (file name is 100 bytes)

44. Disadvantage: when add files to a directory, the b-tree will split, but after deleting files, the b-tree will not merge.

45. A directory with a few files will occupy many blocks.

47. Less IO

48. mkfs.ext3 -I 256 /dev/sda

51. Larger filesystem

52. Max file size: 16TB

53. Max filesystem size: 1EB(1048576TB)

54. ext4: meta_bg Image from: http://www.ibm.com/developerworks/cn/linux/l-cn-filesrc5/

56. Imaging an ext4 filesystem with block_size = 1K

57. 1K/64 = 16

58. a meta group will contain 16 groups.

59. The meta-GDT(1 block) will be put in Group 0, Group1, Group15

60. Group 16, Group17, Group31

61. Group 32, Group33, Group63

62. …...

63. ext4: flex_bg

65. The position of Super-block and GDT follow the rule of “sparse”

66. Advantage: save the space of Group 1,Group 2,Group 3 (especially for the extent of ext4)

68. Create a filesystem without initializing all of the block groups. This feature also enables checksums and highest-inode-used statistics in each blockgroup. This feature can speed up filesystem creation time noticeably (if lazy_itable_init is enabled), and can also reduce e2fsck time dramatically.

70. ext4_new_inode -> ext4_read_block_bitmap

71. ext4: extent Image from: http://www.ibm.com/developerworks/cn/linux/l-cn-filesrc5/

73. Example: a 300G file in ext3 will occupied 300MB meta-data-blocks, but in ext4 it only occupuied 36KB

75. This improves performance and reduces fragmentation by improving block allocation decisions based on the actual file size

76. Q & A Thanks!