This is a self made slide covering topics related to storage systems available in the market with certain information of file systems to understand the fundamentals.
There are also some information is available related to how the whole stuff works.
1. Seminar Outline
Introduction to STORAGE.
Various File Systems.
Why Storage Systems?
Types of STORAGE SYSTEMS.
Differences between SAN & NAS.
Conclusion
Glossary Of Terms
2. Introduction to Storage
In today’s era computers are the most useful devices because of its
usability has reached to a new extent.
As we already know for making computer fully functional, we need
lots of small to large scale chips need to implement various
functionalities which we day to day use or computer serves to us
through.
Let’s consider few of which is very important part of computer in
order to maintain or make available functionality to the user.
Motherboard
Microprocessor OR Microcontroller(depends on particular system)
Input & Output console and devices.
Memory (the most essential part of the system and critical one)
This discussion essentially focuses on the matters related to a
memory at a new level.
3. Introduction( Continues… )
As we discussed in the previous slide, memory is the
essential element or component of the computer system
which enable us to perform several or may be all kind of the
operations.
Let’s take a real world example of a memory is none other
than our BRAIN which maintains tons of information for
making decision and doing processing in real time or in
future, it also has future assumptions and past which
happened with us.
In similar scenario storage systems or devices help us to
accomplish a certain task, help us to remember some data or
information based on its capacity.
Capacity varies according to its requirement such as if we
want to store student information then we would not be
needing a TERA BYTE storage it can be fulfilled within
few MEGAs or GIGA bytes.
4. Various File Systems
file system is a data structure that organizes files and directories on
a disk partition so that they can be easily retrieved. Only one file
system can reside on a disk partition.
A file is a one-dimensional array of bytes with no other structure
implied. Information about each file is stored in structures called
inodes.
A directory is a container that stores files and other directories. It is
merely another type of file that the user is permitted to use, but not
allowed to write; the operating system itself retains the
responsibility for writing directories. Directories cannot span file
systems. The combination of directories and files make up a file
system.
At systems level a file systems works like a register which
maintains information regarding system and memory resources like
FILE NODE, FILE LOCATION, FILE META INFORMATION ,
POINTERS ETC.
5. File Systems(continue)
FAT(File Allocation Table):
The FAT file system is simple and reliable. It does not lose data because
the computer crashed in the middle of an update. It does not use a lot of
memory. It does, however, do a lot of extra administrative I/O to
different areas of the partition. The directory is allocated at the start of
the partition and it contains the table of free space. To write a new
dataset, or to add data to an old one, the disk arm must be constantly
moved between the location of the directory and the place where the data
is being written. Without optimization, a file can end up fragmented into
many small pieces.
By design FAT supports a maximum of 64K allocation units. When the
disk partition is 32 megabytes or less, then an allocation unit is a 512
byte sector. However, as the disk gets larger, the units get larger. A 64
megabyte disk partition has 1K allocation units. A 128 meg partition has
2K units. A 256 meg partition has 4K allocation units. Each file occupies
one or more allocation unit. As the allocation units get large, any large
number of small files wastes a lot of disk space.
6. File Systems(continue)
NTFS(New Technology File System): its a new file system for
Windows NT. NTFS is designed to be all things to all people and to
include all the features of every other file system in common use.
NTFS support long UNICODE file names. In theory, an NTFS file
can have its name in Chinese or Hebrew characters. At the same
time, NTFS maintains an 8.3 name for the file so that it can be used
by a DOS program. NTFS also supports case-sensitive file access
(for Unix programs) and case- insensitive file access (for DOS, OS/2,
and Windows programs).NTFS writes updates to a log area of each
volume. After a system crash, this log area can be used to cleanup
problems almost instantaneously.
NTFS supports "volume sets" where a single disk letter is associated
with a "volume" created from a number of separate free space areas
scattered across several disks. If an NTFS volume fills up, it can be
dynamically expanded by adding an extra chunk of free disk space
from the same or from another hard disk.
7. File Systems (Linux Case Study)
Linux File System structure: In Linux, everything is configured as a
file. This includes not only text files, images and compiled programs
(also referred to as executables), but also directories, partitions and
hardware device drivers.
One important term about file systems is NATIVE FILE SYSTEMS
means Native means that the file systems were either developed
originally for Linux or were first developed for other operating systems
and then rewritten so that they would have functions and performance
on Linux comparable or superior to those of file systems originally
developed for Linux.
Linux has following file systems: ext2,ext3,ext4.
Here, EXT2 stands for SECOND EXTENDED FILE
SYSTEM,THIRD EXTENDED FILE SYSTEMS & FOURTH
EXTENDED FILESYSTEM.
8. Linux File System
EXTFS(Extended File System):The ext file-system uses a system called
inodes to track information about the files stored in the virtual
directory. The inode system creates a separate table on each physical
device, called the inode table , to store the file information. Each stored
file in the virtual directory has an entry in the inode table. The
extended part of the name comes from the additional data that it tracks
on each file, which consists of:
The file name
The file size
The owner of the file
The group the file belongs to
Access permissions for the file
Pointers to each disk block that contains data from the file
Linux references each inode in the inode table using a unique number
(called the inode number ), assigned by the file system as data files are
created. The file system uses the inode number to identify the file
rather than having to use the full file name and path.
There are several versions also avelible like ext2,ext3,ext4.
9. Why Storage systems?
This is era of computing, world needs to use lots of resources
from all over the world and data requirement is very much
for certain things to know end users usually use search
engines on internet those search engines are not simple one
to implement.
First question arises where they keep such huge amount of
the data for producing or presenting in front of users?
They certainly need a very vast storage to handle frequent
and speedy coming data request.
So, every data giant company needs a very powerful
STORAGE SYSTEM.
10. Types of Storage Systems
Embedded Storage
Storage Area Network(SAN)
Network Attached Storage(NAS)
11. Types of Storage Systems(continue)
Embedded Storage(Flash storage):This is most smallest storage medium
that stores data internally in devices like mobile phones storage, any
embedded devices , SCADA systems that means from small to large
scale all kind of applications can be applied to use for Embedded
Storage or say memory.
Specially used when we wanted to create a small computing application
which is used into small computing devices sometimes to boot OS
stored under any flash devices or small memory.
This kind of memory is also useful for small machine dependent
application such as small robots or any hardware specific code written
on microcontroller in those situation a flash storage is very much useful
for us to make an portable machine / hardware application.
12. Storage Networking
Storage networking is a most evolving area of
information technology which consists of
following:
DAS(Direct Attached Storage)
NAS(Network Attached Storage)
SAN(Storage Area Network)
13. Types of Storage Systems(continue)
Storage Area Network(SAN): The Storage Networking Industry
Association (SNIA) defines the storage area network (SAN) as a
network whose primary purpose is the transfer of data between
computer systems and storage elements. A SAN consists of a
communication infrastructure, which provides physical connections. It
also includes a management layer, which organizes the connections,
storage elements, and computer systems so that data transfer is secure
and robust. The term SAN is usually (but not necessarily) identified
with block I/O services rather than file access services.
SAN introduces the flexibility of networking to enable one server or
many heterogeneous servers to share a common storage utility. A
network might include many storage devices, including disk, tape, and
optical storage. Additionally, the storage utility might be located far
from the servers that it uses.
15. Storage Area Network continue…
In a diagram this shows that how a storage system uses what kind of
general model.
Within a diagram you can see that terminals arranged having different
platforms like NT, UNIX, LINUX etc.
These things actually connected by any network devices such as
network hubs, routers, switches etc. which enable us to connect them
internally with each and every other component within a diagram.
All connected terminal use network components to connect to RAID,
Optical Storage, Disk or tape so that its really useful to make an contact
with external storage devices through a communication channel.
16. Storage Area Network
Protocols(Continue…)
Network protocols used in SAN is listed as under:
iSCSI FCIP iFCP
As with usual networking environment having some protocols to
communicate within various devices over the networking environment
same way storage systems has their own networking infrastructure to
maintain memory access requests.
iSCSI: The most common type of IP SAN uses iSCSI to
encapsulate SCSI commands and assemble data into packets for transfer
between the host servers and storage devices. IP SAN protocols
typically run over a standard Ethernet network and use the Transmission
Control Protocol/Internet Protocol (TCP/IP) for communication.
An IP SAN for block-based data is often referred to as an iSCSI SAN.
17. Storage Area Network
Protocols(Continue…)
Fiber Channel over IP (FCIP or FC/IP): known as Fiber Channel
tunneling or storage tunneling) is an Internet Protocol (IP)-based
storage networking technology developed by the Internet Engineering
Task Force (IETF). FCIP mechanisms enable the transmission of Fibre
Channel (FC) information by tunneling data between storage area
network (SAN) facilities over IP networks; this capacity facilitates data
sharing over a geographically distributed enterprise. One of two main
approaches to storage data transmission over IP networks, FCIP is
among the key technologies expected to help bring about rapid
development of the storage area network market by increasing the
capabilities and performance of storage data transmission.
18. Storage Area Network Protocols(Continue…)
iFCP(internet Fiber Channel Protocol):it is a gateway-to-gateway
protocol It provides Fiber Channel fabric services to Fiber Channel
devices over an IP network. iFCP uses TCP to provide congestion
control, error detection, and recovery. The primary purpose of iFCP
is to allow interconnection and networking of existing Fiber
Channel devices at wire speeds over an IP network.
Under iFCP, IP components and technology replace the Fiber
Channel switching and routing infrastructure. iFCP was originally
developed by Nishan Systems, acquired by McDATA in September
2003. McDATA was then acquired by Brocade.
19. Network Attached Storage(NAS)
Network-attached storage (NAS) is basically a LAN-attached file server
that serves files by using a network protocol such as Network File System
(NFS). NAS is a term that is used to refer to storage elements that
connect to a network and provide file access services to computer
systems. A NAS storage element consists of an engine that implements
the file services (by using access protocols such as NFS or Common
Internet File System (CIFS)) and one or more devices, on which data is
stored. NAS elements might be attached to any type of network.
NAS storage provides acceptable performance and security, and it is often
less expensive for servers to implement (for example, Ethernet adapters are
less expensive than Fiber Channel adapters).
Protocols by NAS: NFS(Network File System), CIFS(Common Internet File
System)
21. Network Attached Storage Architecture
NAS Architecture: This is essentially important to know what’s
inside actual NAS?
It consists of various fixed disks and RAIDs( Redundant Arrays of
Independent Disks) which used to maintain a large amount of
storage of data even may be in PETA BYTES.
In previous image of NAS it dictates how any NAS is useful and
how its connected to various devices such as wireless devices,
modems,digi cams, TV and many more flavor of devices.
It doesn’t end here NAS can also be connected to internet to be
treated as an cloud storage as well as private company storage
devices or even in a company’s intranet we can connect it.
22. Key Differences between SAN & NAS
SAN:
Only server class devices with SCSI Fiber Channel can connect to the
SAN.
The Fiber Channel of the SAN has a limit of around 10km at best.
A SAN addresses data by disk block number and transfers raw disk blocks.
File Sharing is operating system dependent and does not exist in many
operating systems also it can be managed by File System managed by
servers.
NAS:
Almost any machine that can connect to the LAN (or is interconnected to the
LAN through a WAN) can use NFS, CIFS or HTTP protocol to connect to a
NAS and share files.
A NAS identifies data by file name and byte offsets, transfers file data or file
meta-data (file's owner, permissions, creation data, etc.), and handles security,
user authentication, file locking.
A NAS allows greater sharing of information especially between disparate
operating systems such as Unix and NT it can also be managed by Head unit
of NAS.
23. Conclusion
As we seen in our whole presentation it’s really important to look after
a matter of storage in a huge environments.
Starting from smallest memory unit we have tried to explore how it can
grow up to very large scale and can be useful to implement bigger
network application which can access a very large data store let’s name
such an application such as any MNC’s Data warehouse, Google
Search Engine, Data mining applications generally these technologies
are implemented to manage vary enormous amount of data storage.
24. Glossary of terms
FAT: File Allocation Table
NTFS: New Technology File System
EXT2,EXT3,EXT4: Second Extended, Third Extended, Four Extended
File Systems.
SAN: Storage Area Network
NAS: Network Attached Storage
SCSI: Small Computer Systems Interface
iSCSI: internet Small Computer Systems Interface
FC: Fiber Channel
iFCP: internet Fiber Channel Protocol
FCIP: Fiber Channel over IP
NFS: Network File system
CIFS: Common Internet File System