Operating
Systems:
Internals
and
Design
Principles
Chapter 12
File Management
Ninth Edition
By William Stallings
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Files
◼
Data collections created by users
◼
The File System is one of the most important parts of the OS to a user
◼
Desirable properties of files:
Long-term existence
• Files are stored on disk or other secondary storage and do not disappear when a user logs off
Sharable between processes
• Files have names and can have associated access permissions that permit controlled sharing
Structure
• Files can be organized into hierarchical or more complex structure to reflect the relationships
among files
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
File Systems
◼
Provide a means to store data organized as files as well as a
collection of functions that can be performed on files
◼
Maintain a set of attributes associated with the file
◼
Typical operations include:
◼
Create
◼
Delete
◼
Open
◼
Close
◼
Read
◼
Write
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
File Structure
Four terms are
commonly used when
discussing files:
Field
Record
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
File
Database
Structure Terms
File
Field
◼
Basic element of data
◼
Collection of similar records
◼
Contains a single value
◼
Treated as a single entity
◼
Fixed or variable length
◼
May be referenced by name
Database
◼
Access control restrictions
usually apply at the file level
◼
Collection of related data
◼
Relationships among elements
of data are explicit
◼
Designed for use by a number
of different applications
◼
Consists of one or more types
of files
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Record
◼
Collection of related fields that
can be treated as a unit by some
application program
◼
Fixed or variable length
File Management System
◼
Meet the data management needs of the user
◼
Guarantee that the data in the file are valid
◼
Optimize performance
◼
Provide I/O support for a variety of storage device types
◼
Minimize the potential for lost or destroyed data
◼
Provide a standardized set of I/O interface routines to user
processes
◼
Provide I/O support for multiple users in the case of multipleuser systems
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Minimal User Requirements
▪ Each user:
1
2
3
4
5
6
• Should be able to create, delete, read, write and modify files
• May have controlled access to other users’ files
• May control what type of accesses are allowed to the users’ files
• Should be able to move data between files
• Should be able to back up and recover files in case of damage
• Should be able to access his or her files by name rather than by numeric identifier
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
User Program
Pile
Indexed
Sequential
Sequential
Indexed
Hashed
Logical I/O
Basic I/O Supervisor
Basic File System
Disk Device Driver
Tape Device Driver
Figure 12.1 File System Software Architecture
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Device Drivers
◼
Lowest level
◼
Communicates directly with peripheral devices
◼
Responsible for starting I/O operations on a device
◼
Processes the completion of an I/O request
◼
Considered to be part of the operating system
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Basic File System
◼
Also referred to as the physical I/O level
◼
Primary interface with the environment outside the computer system
◼
Deals with blocks of data that are exchanged with disk or tape systems
◼
Concerned with the placement of blocks on the secondary storage
device
◼
Concerned with buffering blocks in main memory
◼
Does not understand the content of the data or the
structure of the files involved
◼
Considered part of the operating system
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Basic I/O Supervisor
◼
Responsible for all file I/O initiation and termination
◼
At this level, control structures are maintained that deal with device
I/O, scheduling, and file status
◼
Selects the device on which I/O is to be performed
◼
Concerned with scheduling disk and tape accesses to optimize
performance
◼
I/O buffers are assigned and secondary memory is allocated at this
level
◼
Part of the operating system
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Logical I/O
Enables users
and
applications to
access records
Provides
generalpurpose
record I/O
capability
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Maintains
basic data
about file
Access Method
◼
Level of the file system closest to the user
◼
Provides a standard interface between applications and the file
systems and devices that hold the data
◼
Different access methods reflect different file structures and
different ways of accessing and processing the data
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Records
Directory
management
User & program
comands
Operation,
File name
File
Structure
Access
method
Physical blocks
in main memory
buffers
Blocking
Physical blocks
in secondary
storage (disk)
Disk
scheduling
File
manipulation
functions
I/O
File
allocation
User access
control
File management concerns
Operating system concerns
Figure 12.2 Elements of File Management
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Free storage
management
File Organization and Access
◼
File organization is the logical structuring of the records as
determined by the way in which they are accessed
◼
◼
In choosing a file organization, several criteria are important:
◼
Short access time
◼
Ease of update
◼
Economy of storage
◼
Simple maintenance
◼
Reliability
Priority of criteria depends on the application that will use the file
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
File Organization Types
The pile
The
sequential
file
The direct, or
hashed, file
Five of the common
file organizations are:
The
indexed file
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
The indexed
sequential file
Variable-length records
Variable set of fields
Chronological order
Fixed-length records
Fixed set of fields in fixed order
Sequential order based on key field
(a) Pile File
(b) Sequential File
Exhaustive
index
Exhaustive
index
Partial
index
n
Index
levels
2
1
Main File
Index
Overflow
File
(c) Indexed Sequential File
Primary File
(variable-length records)
(d) Indexed File
Figure 12.3 Common File Organizations
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
The Pile
◼
Least complicated form of
file organization
◼
Data are collected in the
order they arrive
◼
Each record consists of one
burst of data
◼
Purpose is simply to
accumulate the mass of
data and save it
◼
Record access is by
exhaustive search
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Variable-length records
Variable set of fields
Chronological order
(a) Pile File
E
The Sequential
File
◼
Most common form of file
structure
◼
A fixed format is used for
records
◼
Key field uniquely identifies
the record
◼ Typically
ble-length
records used in batch
ble set of applications
fields
nological order
Only organization that is
easily stored on tape as well
(a)
asPile
diskFile
◼
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Fixed-length records
Fixed set of fields in fixed order
Sequential order based on key field
(b) Sequential File
Exhaustive
Exhaustive
Partial
(a) Pile File
Indexed
Sequential File
◼
Adds an index to the file
to support random access
◼
Adds an overflow file
◼
Greatly reduces the time
required to access a single
record
◼
Multiple levels of
indexing can be used to
provide greater efficiency
in access
n
Index
levels
2
1
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Main File
Index
Overflow
File
(c) Indexed Sequential File
(a) Pile File
Indexed File
(b) Sequential File
Exhaustive
index
Exhaustive
index
Partial
index
n
Records
are accessed only through
Main File
Indextheir indexes
levels
Index
◼ 2Variable-length records can be
1 employed
◼
◼
◼
◼
◼
Exhaustive index contains one
entry for every record in the main
file
Overflow
Partial index contains
Fileentries to
records where the field of interest
exists
(c) Indexed
File
Used
mostly inSequential
applications
where
timeliness of information is
critical
Examples would be airline
reservation systems and inventory
control systems
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Primary File
(variable-length records)
(d) Indexed File
Direct or Hashed File
◼
Access directly any block of a known
address
◼
Makes use of hashing on the key value
◼
Often used where:
◼ Very rapid access is required
◼ Fixed-length records are used
◼ Records are always accessed
one at a time
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Examples are:
•
•
•
•
Directories
Pricing tables
Schedules
Name lists
B-Trees
◼
A balanced tree structure with all branches of equal length
◼
Standard method of organizing indexes for databases
◼
Commonly used in OS file systems
◼
Provides for efficient searching, adding, and deleting of items
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Key1
Subtree1
Subtree2
Key2
Keyk–1
Subtree3
Subtreek–1
Figure 12.4 A B-tree Node with k Children
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Subtreek
B-Tree
◼
Every node has at most 2d – 1
keys and 2d children or,
equivalently, 2d pointers
◼
Every node, except for the root,
has at least d – 1 keys and d
pointers, as a result, each internal
node, except the root, is at least
half full and has at least d
children
◼
The root has at least 1 key and 2
children
◼
All leaves appear on the same
level and contain no information.
This is a logical construct to
terminate the tree; the actual
implementation may differ
◼
A nonleaf node with k pointers
contains k – 1 keys
Characteristics
A B-tree is characterized by its
minimum degree d and satisfies
the following properties:
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
23 51 61 71
2 10
30 32 39 43 44
52 59 60
67 68
73 85 88 96
67 68
73 85 88 90 96
(a) B-tree of minimum degree d = 3.
23 51 61 71
2 10
30 32 39 43 44
52 59 60
(b) Key = 90 inserted. This is a simple insertion into a node.
23 39 51 61 71
2 10
30 32
43 44 45
52 59 60
67 68
73 85 88 90 96
(c) Key = 45 inserted. This requires splitting a node into two parts and promoting one key to the root node.
51
23 39
2 10
30 32
61 71 88
43 44 45
52 59 60
67 68
73 84 85
90 96
(d) Key = 84 inserted. This requires splitting a node into two parts and promoting one key to the root node
This then requires the root node to be split and a new root created.
Figure 12.5 Inserting Nodes into a B-tree
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Table 12.1
Basic Information
File Name
Name as chosen by creator (user or program). Must be unique within a specific
directory.
File Type
For example: text, binary, load module, etc.
File Organization
For systems that support different organizations
Address Information
Information
Elements of a
File Directory
Volume
Indicates device on which file is stored
Starting Address
Starting physical address on secondary storage (e.g., cylinder, track, and block
number on disk)
Size Used
Current size of the file in bytes, words, or blocks
Size Allocated
The maximum size of the file
Access Control Information
Owner
User who is assigned control of this file. The owner may be able to grant/deny
access to other users and to change these privileges.
Access Information
A simple version of this element would include the user's name and password for
each authorized user.
Permitted Actions
Controls reading, writing, executing, transmitting over a network
Usage Information
(Table can be found on page 537 in textbook)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Date Created
When file was first placed in directory
Identity of Creator
Usually but not necessarily the current owner
Date Last Read Access
Date of the last time a record was read
Identity of Last Reader
User who did the reading
Date Last Modified
Date of the last update, insertion, or deletion
Identity of Last Modifier
User who did the modifying
Date of Last Backup
Date of the last time the file was backed up on another storage medium
Current Usage
Information about current activity on the file, such as process or processes that
have the file open, whether it is locked by a process, and whether the file has been
updated in main memory but not yet on disk
Operations Performed
on a Directory
◼
To understand the requirements for a file structure, it is helpful to
consider the types of operations that may be performed on the
directory:
Search
Create
files
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Delete
files
List
directory
Update
directory
Two-Level Scheme
There is one
directory for each
user and a master
directory
Master directory has
an entry for each user
directory providing
address and access
control information
Names must be
unique only within the
collection of files of a
single user
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Each user directory
is a simple list of
the files of that user
File system can easily
enforce access
restriction on
directories
Master Directory
Subirectory
Subirectory
Subirectory
Subirectory
Subirectory
File
File
File
File
Figure 12.6 Tree-Structured Directory
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Master Directory
System
User_A
User_B
User_C
Directory
"User_C"
Directory
"User_A"
Directory "User_B"
Draw
Word
Directory "Word"
Directory "Draw"
Unit_A
ABC
Directory "Unit_A"
ABC
File
"ABC"
Pathname: /User _B/Draw/ABC
File
"ABC"
Pathname: /User _B/Word/Unit_A/ABC
Figure 12.7 Example of Tree-Structured Directory
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
File Sharing
Two issues arise
when allowing files
to be shared among
a number of users:
Access rights
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Management of
simultaneous
access
Access Rights
◼
None
◼
◼
◼
The user would not be allowed
to read the user directory that
includes the file
Knowledge
◼ The user can determine that the
◼
◼
file exists and who its owner is
and can then petition the owner
for additional access rights
◼
Execution
◼
◼
◼
◼
The user can read the file for any
purpose, including copying and
execution
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
The user can modify, delete, and
add to the file’s data
Changing protection
◼
◼
The user can add data to the file
but cannot modify or delete any
of the file’s contents
Updating
◼
The user can load and execute a
program but cannot copy it
Reading
Appending
The user can change the access
rights granted to other users
Deletion
◼
The user can delete the file from
the file system
User Access Rights
Owner
Specific
Users
User
Groups
Usually the
initial creator
of the file
Has full rights
Individual
users who are
designated by
user ID
May grant
rights to
others
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
A set of users
who are not
individually
defined
All
All users who
have access to
this system
These are
public files
Record Blocking
◼
Blocks are the unit of I/O
with secondary storage
◼
▪
For I/O to be
performed records
must be organized as
blocks
Given the size of a block,
three methods of blocking
can be used:
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
1)
Fixed-Length Blocking – fixedlength records are used, and an
integral number of records are
stored in a block
Internal fragmentation –
2)
3)
unused space at the end of each
block
Variable-Length Spanned Blocking
– variable-length records are used
and are packed into blocks with no
unused space
Variable-Length Unspanned
Blocking – variable-length records
are used, but spanning is not
employed
Record 1
Record 2
Record 3
Record 4
Track 1
Record 5
Record 6
Record 7
Record 8
Track 2
(a) Fixed Blocking
Record 1
Record 4 (rest)
Record 2
Record 5
Record 3
Record 4
Ptr
Record 1
Ptr Track 2
Record 6
Track 1
(b) Variable Blocking: Spanned
Record 1
Record 2
Record 4
Record 5
Record 3
Record 6
(c) Variable Blocking: Unspanned
Figure 12.8 Record Blocking Methods
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Track 1
Track 2
File Allocation
▪ On secondary storage, a file consists of a collection of blocks
▪ The operating system or file management system is responsible for
allocating blocks to files
▪ The approach taken for file allocation may influence the approach
taken for free space management
▪ Space is allocated to a file as one or more portions (contiguous set
of allocated blocks)
▪ File allocation table (FAT)
▪
Data structure used to keep track of the portions assigned to a file
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Preallocation vs
Dynamic Allocation
◼
A preallocation policy requires that the maximum size of a file be
declared at the time of the file creation request
◼
For many applications it is difficult to estimate reliably the maximum
potential size of the file
◼
◼
Tends to be wasteful because users and application programmers tend
to overestimate size
Dynamic allocation allocates space to a file in portions as needed
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Portion Size
◼
In choosing a portion size there is a trade-off between efficiency from
the point of view of a single file versus overall system efficiency
◼
Items to be considered:
1) Contiguity of space increases performance, especially for
Retrieve_Next operations, and greatly for transactions
running in a transaction-oriented operating system
2) Having a large number of small portions increases the size of
tables needed to manage the allocation information
3) Having fixed-size portions simplifies the reallocation of space
4) Having variable-size or small fixed-size portions minimizes
waste of unused storage due to overallocation
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Alternatives
Two major alternatives:
Variable, large
contiguous portions
• Provides better performance
• The variable size avoids waste
• The file allocation tables are
small
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Blocks
• Small fixed portions provide
greater flexibility
• They may require large tables
or complex structures for their
allocation
• Contiguity has been
abandoned as a primary goal
• Blocks are allocated as needed
Table 12.2
File Allocation Methods
Contiguous
Chained
Indexed
Preallocation?
Necessary
Possible
Fixed or variable
size portions?
Variable
Fixed blocks
Fixed blocks
Variable
Portion size
Large
Small
Small
Medium
Allocation
frequency
Once
Low to high
High
Low
Time to allocate
Medium
Long
Short
Medium
File allocation
table size
One entry
One entry
Large
Medium
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Possible
File Allocation Table
File Name Start Block
File A
0
1
5
6
10
11
15
16
20
21
25
26
File D
31
30
2
3
4
7
8
File B
12
13
9
14
17
18
File C
22
23
File E
27
28
19
32
34
33
File A
File B
File C
File D
File E
2
9
18
30
26
Length
3
5
8
2
3
24
29
Figure 12.9 Contiguous File Allocation
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
File Allocation Table
File Name Start Block
File A
0
15
1
2
File B
6
7
File C
11
12
File E
16
17
20
21
25
30
5
10
3
4
8
9
13
18
14
File D
19
22
23
24
26
27
28
29
31
32
33
34
File A
File B
File C
File D
File E
0
3
8
19
16
Length
3
5
8
2
3
Figure 12.10 Contiguous File Allocation (After Compaction)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
File Allocation Table
File Name Start Block
0
File B
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
File B
1
Figure 12.11 Chained Allocation
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Length
5
File Allocation Table
File Name Start Block
0
File B
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
File B
0
Length
5
Figure 12.12 Chained Allocation (After Consolidation)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
File Allocation Table
0
File B
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
File Name
Index Block
File B
24
1
8
3
14
28
Figure 12.13 Indexed Allocation with Block Portions
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
File Allocation Table
0
File B
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
File Name
Index Block
File B
24
Start Block
Length
1
28
14
3
4
1
Figure 12.14 Indexed Allocation with Variable-Length Portions
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Free Space Management
◼
Just as allocated space must be managed, so must the unallocated
space
◼
To perform file allocation, it is necessary to know which blocks are
available
◼
A disk allocation table is needed in addition to a file allocation table
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Bit Tables
◼
This method uses a vector containing one bit for each block on the
disk
◼
Each entry of a 0 corresponds to a free block, and each 1
corresponds to a block in use
Advantages:
• Works well with any file
allocation method
• It is as small as possible
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Chained Free Portions
◼
The free portions may be chained together by using a pointer and
length value in each free portion
◼
Negligible space overhead because there is no need for a disk
allocation table
◼
Suited to all file allocation methods
Disadvantages:
• Leads to fragmentation
• Every time you allocate a block you need to read
the block first to recover the pointer to the new
first free block before writing data to that block
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Indexing
◼
Treats free space as a file and uses an index table as it would for file
allocation
◼
For efficiency, the index should be on the basis of variable-size
portions rather than blocks
◼
This approach provides efficient support for all of the file allocation
methods
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Free Block List
Each block is assigned a
number sequentially
The list of the
numbers of all free
blocks is maintained
in a reserved portion
of the disk
Depending on the size of
the disk, either 24 or 32
bits will be needed to store
a single block number
The size of the free
block list is 24 or 32
times the size of the
corresponding bit
table and must be
stored on disk
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
There are two effective
techniques for storing a
small part of the free
block list in main
memory:
The list can be treated
as a push-down stack
with the first few
thousand elements of
the stack kept in main
memory
The list can be treated
as a FIFO queue, with a
few thousand entries
from both the head and
the tail of the queue in
main memory
Volumes
◼
A collection of addressable sectors in secondary
memory that an OS or application can use for
data storage
◼
The sectors in a volume need not be consecutive
on a physical storage device
◼
◼
They need only appear that way to the OS or application
A volume may be the result of assembling and
merging smaller volumes
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
UNIX File
Management
◼
In the UNIX file system, six
types of files are distinguished:
Regular, or ordinary
• Contains arbitrary data in zero or more data blocks
Directory
• Contains a list of file names plus pointers to associated inodes (index nodes)
Special
• Contains no data but provides a mechanism to map physical devices to file names
Named pipes
• An interprocess communications facility
Links
• An alternative file name for an existing file
Symbolic links
• A data file that contains the name of the file to which it is linked
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Inodes
◼
All types of UNIX files are administered by the OS by means of
inodes
◼
An inode (index node) is a control structure that contains the key
information needed by the operating system for a particular file
◼
Several file names may be associated with a single inode
◼
An active inode is associated with exactly one file
◼
Each file is controlled by exactly one inode
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
mode
Data
Data
Data
Data
Data
Data
Data
Data
Data
Data
owners (2)
Data
timestamps (4)
size
Data
direct(0)
direct (1)
Pointers
Pointers
direct(12)
single indirect
Pointers
Data
Pointers
double indirect
triple indirect
Pointers
Data
block count
reference count
Pointers
flags (2)
generation number
Pointers
Data
Pointers
Pointers
blocksize
Pointers
Data
extended attr size
extended
attribute
blocks
Pointers
Data
Inode
Figure 12.15 Structure of FreeBSD inode and File
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
File Allocation
◼
File allocation is done on a block basis
◼
Allocation is dynamic, as needed, rather than using preallocation
◼
An indexed method is used to keep track of each file, with part of
the index stored in the inode for the file
◼
In all UNIX implementations the inode includes a number of direct
pointers and three indirect pointers (single, double, triple)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Table 12.3
Capacity of a FreeBSD File with 4-Kbyte Block Size
Level
Number of Blocks
Direct
12
48K
512
2M
512 ´ 512 = 256K
1G
Single Indirect
Double Indirect
Triple Indirect
Number of Bytes
512 ´ 256K = 128M
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
512G
Inode table
Directory
i1
Name1
i2
Name2
i3
Name3
i4
Name4
Figure 12.16 UNIX Directories and Inodes
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Volume Structure
◼
A UNIX file
system resides
on a single
logical disk or
disk partition
and is laid out
with the
following
elements:
Boot block
Superblock
Inode table
Data
blocks
Contains
code
required to
boot the
operating
system
Contains
attributes and
information
about the file
system
Collection
of inodes
for each file
Storage space
available for
data files and
subdirectories
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
User applications
User
space
GNU C library
System call interface
Inode
cache
File
system
Virtual File
System (VFS)
Directory
cache
Individual File
Systems
Buffer cache
Device drivers
Figure 12.17 Linux Virtual File System Context
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Kernel
space
System calls
using VFS
user interface
Linux
Virtual
File
System
VFS
system
calls
Mapping
function
to file
system X
System calls
using file
system X
interface
File
System X
Disk I/O
calls
User
Process
Files on secondary
storage maintained
by file system X
Figure 12.18 Linux Virtual File System Concept
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Primary Object Types in VFS
Superblock
Object
• Represents a specific
mounted file system
Dentry Object
• Represents a specific
directory entry
Inode Object
File Object
• Represents a specific
file
• Represents an open
file associated with a
process
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
The Superblock Object
◼
Stores information describing a specific file system
◼
Typically, the superblock corresponds to the file system superblock or file system control
block, which is stored in a special sector on disk
◼
The superblock object consists of a number of data items
◼ Examples include:
◼ The device this file system is mounted on
◼ The basic block size of the file system
◼ Dirty flag, to indicate that the superblock has been changed but not written back
to disk
◼ File system type
◼ Flags, such as a read-only flag
◼ Pointer to the root of the file system directory
◼ List of open files
◼ Semaphore for controlling access to the file system
◼ List of superblock operations
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
The Inode Object
◼
An inode is associated with each file
◼
The inode object holds all the information about a named file except its
name and the actual data contents of the file
◼
Items contained in an inode object include owner, group, permissions,
access times for a file, size of data it holds, and number of links
◼
The inode object also includes an inode operations object that describes the
file system’s implemented functions that the VFS can invoke on an inode
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
The Dentry Object
◼
A dentry (directory entry) is a specific component in a path
◼
The component may be either a directory name or a file name
◼
Dentry objects facilitate quick lookups to files and directories, and are used
in a dentry cache for that purpose
◼
The dentry object includes a pointer to the inode and superblock
◼
It also includes a pointer to the parent dentry and pointers to any
subordinate dentrys
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
The File Object
◼
The file object is used to represent a file opened by a process
◼
The object is created in response to the open() system call, and destroyed in response to
the close () system call
◼
The file object also includes an inode operations object that describes the file system’s
implemented functions that the VFS can invoke on a file object
◼
The methods defined for the file object include read, write, open, release, and lock
◼
The file object consists of a number of items, including:
◼ Dentry object associated with the file
◼ File system containing the file
◼ File objects usage counter
◼ User’s user ID
◼ User’s group ID
◼ File pointer, which is the current position in the file from which the next
operation will take place
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Caches
◼
The VFS employs three caches to improve performance:
◼ Inode cache
◼
◼
◼
Because every file and directory is represented by a VFS inode, a directory listing
command or a file access command causes a number of inodes to be accessed
◼
The inode cache stores recently visited inodes to make access quicker
Directory cache
◼
The directory cache stores the mapping between the full directory names and their inode
numbers
◼
This speeds up the process of listing a directory
Buffer cache
◼
The buffer cache is independent of the file systems and is integrated into the mechanisms
that the Linux kernel uses to allocate and read and write data buffers
◼
As the real file systems read data from the underlying physical disks, this results in
requests to the block device drivers to read physical blocks from the device that they
control
◼
If the same data is needed often, it will be retrieved from the buffer cache rather than
read from the disk
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Windows File System
◼
The developers of Windows NT designed a new file system, the New
Technology File System (NTFS) which is intended to meet high-end
requirements for workstations and servers
◼
Key features of NTFS:
◼
Recoverability
◼
Security
◼
Large disks and large files
◼
Multiple data streams
◼
Journaling
◼
Compression and encryption
◼
Hard and symbolic links
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
NTFS Volume
and File Structure
◼
NTFS makes use of the following disk storage concepts:
Sector
• The smallest physical storage unit on the disk
• The data size in bytes is a power of 2 and is almost always
512 bytes
Cluster
• One or more contiguous sectors
• The cluster size in sectors is a power of 2
Volume
• A logical partition on a disk, consisting of one or more
clusters and used by a file system to allocate space
• Can be all or a portion of a single disk or it can extend
across multiple disks
• The maximum volume size for NTFS is 264 clusters
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Table 12.4
Windows NTFS Partition and Cluster Sizes
Volume Size
Sectors per Cluster
Cluster Size
£ 512 Mbyte
1
512 bytes
512 Mbyte - 1 Gbyte
2
1K
1 Gbyte - 2 Gbyte
4
2K
2 Gbyte - 4 Gbyte
8
4K
4 Gbyte - 8 Gbyte
16
8K
8 Gbyte - 16 Gbyte
32
16K
16 Gbyte - 32 Gbyte
64
32K
> 32 Gbyte
128
64K
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
partition
boot
sector
Master File Table
System
Files
Figure 12.19 NTFS Volume Layout
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
File Area
Master File Table (MFT)
◼
The heart of the Windows file system is the MFT
◼
The MFT is organized as a table of 1,024-byte rows, called records
◼
Each row describes a file on this volume, including the MFT itself,
which is treated as a file
◼
Each record in the MFT consists of a set of attributes that serve to
define the file (or folder) characteristics and the file contents
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Table 12.5
Windows NTFS File and Directory Attribute Types
Attribute Type
Description
Standard information
Includes access attributes (read-only, read/write, etc.); time
stamps, including when the file was created or last modified;
and how many directories point to the file (link count).
Attribute list
A list of attributes that make up the file and the file reference
of the MFT file record in which each attribute is located. Used
when all attributes do not fit into a single MFT file record.
File name
A file or directory must have one or more names.
Security descriptor
Specifies who owns the file and who can access it.
Data
The contents of the file. A file has one default unnamed data
attribute and may have one or more named data attributes.
Index root
Used to implement folders.
Index allocation
Used to implement folders.
Volume information
Includes volume-related information, such as the version and
name of the volume.
Bitmap
Provides a map representing records in use on the MFT or
folder.
Note: Colored rows refer to required file attributes; the other attributes are optional.
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
I/O Manager
Log File
Service
Log the transaction
Read/write
the file
Flush the Write the
log file
cache
Cache
Manager
NTFS Driver
Fault Tolerant
Driver
Read/write a
mirrored or
striped volume
Disk Driver
Load data from
disk into
memory
Access the mapped
file or flush the cache
Virtual Memory
Manager
Figure 12.20 Windows NTFS Components
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Read/write
the disk
/(root)
/system
(ro)
bin
etc
lib
/data
(rw)
usr
/cache
/mnt/sdcard
(rw)
removable storage (rw)
ro: mounted as read only
rw: mounted as read and write
Figure 12.21 Typical Directory Tree of Android
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
SQLite
◼
Most widely deployed SQL database engine in the world
◼
Based on the Structured Query Language (SQL)
◼
Designed to provide a streamlined SQL-based database management
system suitable for embedded systems and other limited memory systems
◼
The full SQLite library can be implemented in under 400 KB
◼
In contrast to other database management systems, SQLite is not a separate
process that is accessed from the client application
◼
The SQLite library is linked in, and thus becomes an integral part of
the application program
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Summary
◼
◼
◼
◼
◼
◼
◼
◼
File structure
File management systems
File organization and access
◼ The pile
◼ The sequential file
◼ The indexed sequential file
◼ The indexed file
◼ The direct or hashed file
B-Trees
File directories
◼ Contents
◼ Structure
◼ Naming
File sharing
◼ Access rights
◼ Simultaneous access
Record blocking
Android file management
◼ File system
◼ SQLite
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
◼
◼
◼
Secondary storage management
◼ File allocation
◼ Free space management
◼ Volumes
◼ Reliability
UNIX file management
◼ Inodes
◼ File allocation
◼ Directories
◼ Volume structure
Linux virtual file system
◼ Superblock object
◼ Inode object
◼ Dentry object
◼ File object
◼ Caches
Windows file system
◼ Key features of NTFS
◼ NTFS volume and file structure
◼ Recoverability
Operating
Systems:
Internals
and
Design
Principles
Chapter 11
I/O Management
and Disk Scheduling
Ninth Edition
By William Stallings
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
External devices that engage in I/O with computer
systems can be grouped into three categories:
Human readable
• Suitable for communicating with the computer user
• Printers, terminals, video display, keyboard, mouse
Machine readable
• Suitable for communicating with electronic equipment
• Disk drives, USB keys, sensors, controllers
Communication
• Suitable for communicating with remote devices
• Modems, digital line drivers
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Devices differ in a number of areas:
Data Rate
• There may be differences of magnitude between the data transfer rates
Application
• The use to which a device is put has an influence on the software
Complexity of Control
• The effect on the operating system is filtered by the complexity of the I/O module that controls the
device
Unit of Transfer
• Data may be transferred as a stream of bytes or characters or in larger blocks
Data Representation
• Different data encoding schemes are used by different devices
Error Conditions
• The nature of errors, the way in which they are reported, their consequences, and
the available range of responses differs from one device to another
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Gigabit Ethernet
Graphics display
Hard disk
Ethernet
Optical disk
Scanner
Laser printer
Floppy disk
Modem
Mouse
Keyboard
101
102
103
104
105
106
107
Data Rate (bps)
Figure 11.1 Typical I/O Device Data Rates
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
108
109
◼
Three techniques for performing I/O are:
◼
Programmed I/O
◼
◼
Interrupt-driven I/O
◼
◼
The processor issues an I/O command on behalf of a process to an I/O module;
that process then busy waits for the operation to be completed before proceeding
The processor issues an I/O command on behalf of a process
◼ If non-blocking – processor continues to execute instructions from the process
that issued the I/O command
◼ If blocking – the next instruction the processor executes is from the OS,
which will put the current process in a blocked state and schedule another
process
Direct Memory Access (DMA)
◼
A DMA module controls the exchange of data between main memory and an
I/O module
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
I/O-to-memory transfer
through processor
No Interrupts
Use of Interrupts
Programmed I/O
Interrupt-driven I/O
Direct I/O-to-memory
transfer
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Direct memory access (DMA)
1
2
3
4
• Processor directly controls a peripheral device
• A controller or I/O module is added
• Same configuration as step 2, but now interrupts are employed
• The I/O module is given direct control of memory via DMA
5
• The I/O module is enhanced to become a separate processor, with
a specialized instruction set tailored for I/O
6
• The I/O module has a local memory of its own and is, in fact, a
computer in its own right
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Data
Count
Data Lines
Data
Register
Address Lines
Address
Register
Request to DMA
Acknowledge from DMA
Interrupt
Read
Write
Control
Logic
Figure 11.2 Typical DMA Block Diagram
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Processor
DMA
I/O
Memory
I/O
(a) Single-bus, detached DMA
Processor
DMA
Memory
DMA
I/O
I/O
I/O
(b) Single-bus, Integrated DMA-I/O
System bus
Processor
Memory
DMA
I/O bus
I/O
I/O
I/O
(c) I/O bus
Figure 11.3 Alternative DMA Configurations
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Efficiency
Generality
◼
Major effort in I/O design
◼
◼
Important because I/O
operations often form a
bottleneck
Desirable to handle all devices in
a uniform manner
◼
Most I/O devices are extremely
slow compared with main
memory and the processor
Applies to the way processes view
I/O devices and the way the
operating system manages I/O
devices and operations
◼
The area that has received the
most attention is disk I/O
Diversity of devices makes it
difficult to achieve true generality
◼
Use a hierarchical, modular
approach to the design of the I/O
function
◼
◼
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
User
Processes
User
Processes
User
Processes
Directory
Management
Logical
I/O
Communication
Architecture
File
System
Physical
Organization
Device
I/O
Device
I/O
Device
I/O
Scheduling
& Control
Scheduling
& Control
Scheduling
& Control
Hardware
Hardware
Hardware
(a) Local peripheral device
(b) Communications port
(c) File system
Figure 11.4 A Model of I/O Organization
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
To avoid overheads and inefficiencies, it is sometimes convenient to perform
input transfers in advance of requests being made, and to perform output
transfers some time after the request is made
Block-oriented device
Stream-oriented device
• Stores information in blocks
that are usually of fixed size
• Transfers are made one block
at a time
• Possible to reference data by
its block number
• Disks and USB keys are
examples
• Transfers data in and out as a
stream of bytes
• No block structure
• Terminals, printers,
communications ports,
mouse and other pointing
devices, and most other
devices that are not
secondary storage are
examples
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
No Buffer
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Without a buffer, the OS
directly accesses the device
when it needs
Single Buffer
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
The simplest type of support
that the operating system can
provide
◼
When a user process issues
an I/O request, the OS
assigns a buffer in the system
portion of main memory to
the operation
◼
Input transfers are made to the system buffer
◼
Reading ahead/anticipated input
◼
◼
◼
Approach generally provides a speedup compared to the lack of system buffering
◼
◼
◼
Is done in the expectation that the block will eventually be needed
When the transfer is complete, the process moves the block into user space and
immediately requests another block
The user process can be processing one block of data while the next block is being
read in
The OS is able to swap the process out because the input operation is taking place in
system memory rather than user process memory
Disadvantages:
◼
◼
Complicates the logic in the operating system
Swapping logic is also affected
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Can be used in a line-at-a-time
fashion or a byte-at-a-time
fashion
◼
Line-at-a-time operation is
appropriate for scroll-mode
terminals (dumb terminals)
◼
With this form of terminal,
user input is one line at a
time, with a carriage return
signaling the end of a line
◼
Output to the terminal is
similarly one line at a time
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Byte-at-a-time operation is
used on forms-mode
terminals, when each
keystroke is significant and
for many other peripherals,
such as sensors and
controllers
Double Buffer
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Assigning two system buffers to
the operation
◼
A process now transfers data to
or from one buffer while the
operating system empties or fills
the other buffer
◼
Also known as buffer swapping
Circular Buffer
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
When more than two buffers
are used, the collection of
buffers is itself referred to as
a circular buffer
◼
Each individual buffer is one
unit in the circular buffer
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Disk
Performance
Parameters
Wait for
Device
Wait for
Channel
Seek
The actual details of disk I/O
operation depend on the:
◼ Computer system
◼ Operating system
◼ Nature of the I/O
channel and disk
controller hardware
Rotational
Delay
Device Busy
Figure 11.6 Timing of a Disk I/O Transfer
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Data
Transfer
◼
When the disk drive is operating, the disk is rotating at constant speed
◼
To read or write the head must be positioned at the desired track and
at the beginning of the desired sector on that track
◼
Track selection involves moving the head in a movable-head system or
electronically selecting one head on a fixed-head system
◼
On a movable-head system the time it takes to position the head at the
track is known as seek time
◼
The time it takes for the beginning of the sector to reach the head is
known as rotational delay
◼
The sum of the seek time and the rotational delay equals the access
time
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Seek Time
◼
The time required to move the disk arm to the required track
◼
Consists of two key components:
◼
◼
◼
The initial startup time
The time taken to traverse the tracks that have to be crossed once the
access arm is up to speed
Settling time
◼
Time after positioning the head over the target track until track
identification is confirmed
◼
Much improvement comes from smaller and lighter disk
components
◼
A typical average seek time on contemporary hard disks is under
10ms
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Disk Performance
◼
Rotational delay
◼
The time required for the addressed area of the disk to rotate into a
position where it is accessible by the read/write head
◼
Disks rotate at speeds ranging from 3,6000 rpm (for handheld devices
such as digital cameras) up to 15,000 rpm
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
track number
track number
0
25
50
75
100
125
150
175
199
track number
track number
0
25
50
75
100
125
150
175
199
0
25
50
75
100
125
150
175
199
0
25
50
75
100
125
150
175
199
(a) FIFO
Time
(b) SSTF
Time
(c) SCAN
Time
(d) C-SCAN
Figure 11.7 Comparison of Disk Scheduling Algorithms (see Table 11.3)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Time
(a) FIFO
(b) SSTF
(starting at track 100)
(starting at track 100)
Next
track
accessed
55
58
39
18
90
160
150
38
184
Average
seek
length
Number Next
of tracks track
traversed accessed
45
3
19
21
72
70
10
112
146
55.3
90
58
55
39
38
18
150
160
184
Average
seek
length
(c) SCAN
(d) C-SCAN
(starting at track 100, (starting at track 100,
in the direction of
in the direction of
increasing track
increasing track
number)
number)
Number Next
Number Next
Number
of tracks track
of tracks track
of tracks
traversed accessed traversed accessed traversed
10
32
3
16
1
20
132
10
24
27.5
150
160
184
90
58
55
39
38
18
Average
seek
length
50
10
24
94
32
3
16
1
20
27.8
150
160
184
18
38
39
55
58
90
Average
seek
length
Table 11.2 Comparison of Disk Scheduling Algorithms
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
50
10
24
166
20
1
16
3
32
35.8
track number
First-In, First-Out (FIFO)
0
25
50
75
100
125
150
175
199
◼
Processes in sequential order
◼
Fair to all processes
◼
Approximates random scheduling in performance
if there are many processes competing for the disk
(a) FIFO
0
© 2017 Pearson
25 Education, Inc., Hoboken, NJ. All rights reserved.
Time
Name
Description
Remarks
Selection according to requestor
Random
Random scheduling
For analysis and simulation
FIFO
First in first out
Fairest of them all
PRI
Priority by process
Control outside of disk queue
management
LIFO
Last in first out
Maximize locality and
resource utilization
Selection according to requested item
SSTF
Shortest service time first
High utilization, small queues
SCAN
Back and forth over disk
Better service distribution
C-SCAN
One way with fast return
Lower service variability
N-step-SCAN
SCAN of N records at a time
Service guarantee
FSCAN
N-step-SCAN with N = queue Load sensitive
size at beginning of SCAN
cycle
Table 11.3 Disk Scheduling Algorithms
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Control of the scheduling is outside the control of disk management
software
◼
Goal is not to optimize disk utilization but to meet other objectives
◼
Short batch jobs and interactive jobs are given higher priority
◼
Provides good interactive response time
◼
Longer jobs may have to wait an excessively long time
◼
A poor policy for database systems
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
track number
track number
Shortest Service
Time First
(SSTF)
0
25
50
75
100
125
150
175
199
0
25
50
75
100
125
150
175
199
0
◼
Select the disk I/O request
that requires the least
movement of the disk arm
from its current position
◼
Always choose the
minimum seek time
(a) FIFO
Time
(b) SSTF
Time
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
track
track number
track number
100
125
150
175
199
0
25
50
75
100
125
150
175
199
SCAN
0
25
50
75
100
125
150
175
199
0
25
◼
Also known as the elevator algorithm
(a) FIFO
◼
Arm moves in one direction only
◼
◼
Time
Satisfies all outstanding requests until it
reaches the last track in that direction
then the direction is reversed
Favors jobs whose requests are for tracks
nearest to both innermost and outermost
tracks and favors the latest-arriving jobs
(b) SSTF
Time
(c) SCAN
Time
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
trac
125
150
175
199
track number
C-SCAN
0
25
50
75
100
125
150
175
199
track number
(Circular SCAN)
0
25
50
75
100
125
150
175
199
◼
Restricts scanning to one
direction only
◼
When the last track has been
visited in one direction, the arm
is returned to the opposite end of
the disk and the scan begins
again
(b) SSTF
Time
(c) SCAN
Time
(d) C-SCAN
© 2017 Pearson
Education,
Inc., Hoboken,
NJ. All rightsof
reserved.
Figure
11.7
Comparison
Disk Scheduling
Time
Algorithms (see Table 11.3)
◼
Segments the disk request queue into subqueues of length N
◼
Subqueues are processed one at a time, using SCAN
◼
While a queue is being processed new requests must be added to
some other queue
◼
If fewer than N requests are available at the end of a scan, all of
them are processed with the next scan
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Uses two subqueues
◼
When a scan begins, all of the requests are in one of the queues,
with the other empty
◼
During scan, all new requests are put into the other queue
◼
Service of new requests is deferred until all of the old requests have
been processed
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Redundant Array
of Independent
Disks
◼
Consists of seven
levels, zero through
six
RAID is a set of
physical disk drives
viewed by the operating
system as a single logical
drive
Redundant disk capacity is
used to store parity
information, which
guarantees data
recoverability in case of a
disk failure
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Design
architectures
share three
characteristics:
Data are distributed
across the physical
drives of an array in
a scheme known as
striping
◼
The term was originally coined in a paper by a group of researchers at the
University of California at Berkeley
◼
◼
The paper outlined various configurations and applications and
introduced the definitions of the RAID levels
Strategy employs multiple disk drives and distributes data in such a way as
to enable simultaneous access to data from multiple drives
◼
Improves I/O performance and allows easier incremental increases in
capacity
◼
The unique contribution is to address effectively the need for redundancy
◼
Makes use of stored parity information that enables the recovery of data
lost due to a disk failure
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Category
Striping
Mirroring
Level Description
0
Nonredundant
Disks
Data availability
required
Lower than single
disk
Very high
2N
Higher than RAID
2, 3, 4, or 5; lower
than RAID 6
Higher than single
disk for read;
similar to single
disk for write
Very high for both read
and write
Up to twice that of a
single disk for read;
similar to single disk
for write
Highest of all
listed alternatives
Approximately twice
that of a single disk
Highest of all
listed alternatives
Approximately twice
that of a single disk
Mirrored
2
Redundant via
Hamming code
N+m
3
Bit-interleaved parity
N+1
Parallel
access
Independent
access
Much higher than
single disk;
comparable to
RAID 3, 4, or 5
Much higher than
single disk;
comparable to
RAID 2, 4, or 5
N+1
Much higher than
single disk;
comparable to
RAID 2, 3, or 5
5
Block-interleaved
distributed parity
N+1
Much higher than
single disk;
comparable to
RAID 2, 3, or 4
6
Block-interleaved dual
distributed parity
N+2
Highest of all
listed alternatives
Block-interleaved
parity
N = number of data disks;
Small I/O request rate
N
1
4
Large I/O data
transfer capacity
m proportional to log N
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Similar to RAID 0
for read;
significantly lower
than single disk
for write
Similar to RAID 0
for read; lower
than single disk
for write
Similar to RAID 0
for read; lower
than RAID 5 for
write
Similar to RAID 0 for
read; significantly
lower than single disk
for write
Similar to RAID 0 for
read; generally lower
than single disk for
write
Similar to RAID 0 for
read; significantly
lower than RAID 5 for
write
Table 11.4 RAID Levels
(Page 498 in textbook)
strip 0
strip 1
strip 2
strip 3
strip 4
strip 5
strip 6
strip 7
strip 8
strip 9
strip 10
strip 11
strip 12
strip 13
strip 14
strip 15
(a) RAID 0 (non-redundant)
strip 0
strip 1
strip 2
strip 3
strip 0
strip 1
strip 2
strip 3
strip 4
strip 5
strip 6
strip 7
strip 4
strip 5
strip 6
strip 7
strip 8
strip 9
strip 10
strip 11
strip 8
strip 9
strip 10
strip 11
strip 12
strip 13
strip 14
strip 15
strip 12
strip 13
strip 14
strip 15
b2
b3
f0(b)
f1(b)
f2(b)
(b) RAID 1 (mirrored)
b0
b1
(c) RAID 2 (redundancy through Hamming code)
Figure 11.8 RAID Levels (page 1 of 2)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
b0
b1
b2
b3
P(b)
(d) RAID 3 (bit-interleaved parity)
block 0
block 1
block 2
block 3
P(0-3)
block 4
block 5
block 6
block 7
P(4-7)
block 8
block 9
block 10
block 11
P(8-11)
block 12
block 13
block 14
block 15
P(12-15)
(e) RAID 4 (block-level parity)
block 0
block 1
block 2
block 3
P(0-3)
block 4
block 5
block 6
P(4-7)
block 7
block 8
block 9
P(8-11)
block 10
block 11
block 12
P(12-15)
block 13
block 14
block 15
P(16-19)
block 16
block 17
block 18
block 19
(f) RAID 5 (block-level distributed parity)
block 0
block 1
block 2
block 3
P(0-3)
Q(0-3)
block 4
block 5
block 6
P(4-7)
Q(4-7)
block 7
block 8
block 9
P(8-11)
Q(8-11)
block 10
block 11
block 12
P(12-15)
Q(12-15)
block 13
block 14
block 15
(g) RAID 6 (dual redundancy)
Figure 11.8 RAID Levels (page 2 of 2)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
RAID
Level 0
◼
Not a true RAID because it does not
include redundancy to improve
performance or provide data protection
◼
User and system data are distributed
across all of the disks in the array
◼
Logical disk is divided into strips
strip 0
strip 1
strip 2
strip 3
strip 4
strip 5
strip 6
strip 7
strip 8
strip 9
strip 10
strip 11
strip 12
strip 13
strip 14
strip 15
(a) RAID 0 (non-redundant)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
RAID
Level 1
◼
Redundancy is achieved by the simple
expedient of duplicating all the data
◼
There is no “write penalty”
strip 0
strip 1
strip 2
strip 3
strip 4
strip 5
strip 6
strip 8
strip 9
strip 10
When a drive fails the data may still be
from the second drive
strip accessed
11
strip 12
strip 13
strip 14
strip 15
◼
strip 7
◼
Principal disadvantage is the cost
(a) RAID 0 (non-redundant)
strip 0
strip 1
strip 2
strip 3
strip 0
strip 1
strip 2
strip 3
strip 4
strip 5
strip 6
strip 7
strip 4
strip 5
strip 6
strip 7
strip 8
strip 9
strip 10
strip 11
strip 8
strip 9
strip 10
strip 11
strip 12
strip 13
strip 14
strip 15
strip 12
strip 13
strip 14
strip 15
(b) RAID 1 (mirrored)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
strip 4
strip 5
strip 6
strip 7
strip 8
strip 9
strip 10
strip 11
strip 12
strip 13
strip 14
strip 15
(a) RAID 0 (non-redundant)
RAID
Level 2
◼
Makes use of a parallel access
technique
◼
Data
is 0used
strip 3 striping
strip
◼
strip 7
strip 4
strip 5
Typically
a Hamming
code
is usedstrip 6
strip 0
strip 1
strip 2
strip 4
strip 5
strip 6
strip 8
strip 9
strip 10
strip 12
strip 13
strip 14
◼ Effective
strip 15
b2
b3
strip 11
strip 8
strip 1
strip 9
strip 2
strip 10
choice
in 14
strip 12in an environment
strip 13
strip
which many disk errors occur
(b) RAID 1 (mirrored)
b0
b1
f0(b)
f1(b)
(c) RAID 2 (redundancy through Hamming code)
Figure 11.8 RAID Levels (page 1 of 2)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
f2(b)
RAID
Level 3
b0
◼
Requires only a single redundant disk,
no matter how large the disk array
◼
Employs parallel access, with data
distributed in small strips
◼
Can achieve very high data transfer
rates
b1
(d) RAID 3 (bit-interleaved parity)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
b2
b3
P(b)
RAID
Level 4
b0
◼
Makes use of an independent access
technique
◼
A bit-by-bit parity strip is calculated across
corresponding strips on each data disk,
b2
b3
P(b)
and the
parity bits are
stored in the
corresponding strip on the parity disk
◼
Involves a write penalty when an I/O write
request of small size is performed
b1
(d) RAID 3 (bit-interleaved parity)
block 0
block 1
block 2
block 3
P(0-3)
block 4
block 5
block 6
block 7
P(4-7)
block 8
block 9
block 10
block 11
P(8-11)
block 12
block 13
block 14
block 15
P(12-15)
(e) RAID 4 (block-level parity)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
block 0
block 1
block 4
block 5
RAID
Level 5
block 8
block 9
block 12
block 13
block 2
P(0-3)
block 11
P(8-11)
Similar to RAID-4
but distributes
the
block 7
P(4-7)
parity bits across all disks
◼
block 6
block 10
◼
block 3
Typical allocation is a round-robin
block 15
P(12-15)
scheme
block 14
Has the characteristic that the loss of
any one disk does not result in data loss
◼
(e) RAID 4 (block-level parity)
block 0
block 1
block 2
block 3
P(0-3)
block 4
block 5
block 6
P(4-7)
block 7
block 8
block 9
P(8-11)
block 10
block 11
block 12
P(12-15)
block 13
block 14
block 15
P(16-19)
block 16
block 17
block 18
block 19
(f) RAID 5 (block-level distributed parity)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
(e) RAID 4 (block-level parity)
Two different parity calculations are
block 2 carriedblock
P(0-3)
out 3and stored
in separate blocks
block 6 on different
P(4-7) disks block 7
◼
block 0
block 1
RAID
Level 6
block 4
block 5
block 8
block 9
block 12
P(12-15)
P(16-19)
block 16
P(8-11)
blockextremely
10
block
◼ Provides
high11data availability
block 13
block 14
block 15
◼ Incurs a substantial write penalty
block 17
block 18
block 19
because each write affects two parity
blocks
(f) RAID 5 (block-level distributed parity)
block 0
block 1
block 2
block 3
P(0-3)
Q(0-3)
block 4
block 5
block 6
P(4-7)
Q(4-7)
block 7
block 8
block 9
P(8-11)
Q(8-11)
block 10
block 11
block 12
P(12-15)
Q(12-15)
block 13
block 14
block 15
(g) RAID 6 (dual redundancy)
Figure 11.8 RAID Levels (page 2 of 2)
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Cache memory is used to apply to a memory that is smaller and faster than
main memory and that is interposed between main memory and the
processor
◼
Reduces average memory access time by exploiting the principle of locality
◼
Disk cache is a buffer in main memory for disk sectors
◼
Contains a copy of some of the sectors on the disk
If YES
The request is satisfied
via the cache
If NO
The requested sector is
read into the disk
cache from the disk
When an I/O request is made
for a particular sector, a check
is made to determine if the
sector is in the disk cache
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Most commonly used algorithm that deals with the design issue of
replacement strategy
◼
The block that has been in the cache the longest with no reference
to it is replaced
◼
A stack of pointers reference the cache
◼
Most recently referenced block is on the top of the stack
◼
When a block is referenced or brought into the cache, it is placed on the
top of the stack
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
The block that has experienced the fewest references is replaced
◼
A counter is associated with each block
◼
Counter is incremented each time block is accessed
◼
When replacement is required, the block with the smallest count is
selected
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
New Section
Old Section
MRU
LRU
Re-reference;
count unchanged
Re-reference;
count := count + 1
Miss (new block brought in)
count := 1
(a) FIFO
New Section
Middle Section
Old Section
MRU
LRU
(b) Use of three sections
Figure 11.9 Frequency-Based Replacement
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
60
Disk cache miss rate (%)
50
VAX UNIX
40
30
20
IBM MVS
10
IBM SVS
0
0
5
10
15
20
25
30
Cache size (megabytes)
Figure 11.10 Some Disk Cache Performance Results Using LRU
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
70
60
Disk cache miss rate (%)
50
IBM VM
40
30
IBM MVS
20
VAX UNIX
10
0
0
5
10
15
20
25
30
Cache size (megabytes)
Figure 11.11 Disk Cache Performance Using Frequency-Based Replacement [ROBI90]
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
File Subsystem
Buffer Cache
Character
Block
Device Drivers
Figure 11.12 UNIX I/O Structure
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
UNIX Buffer Cache
◼
Is essentially a disk cache
◼
◼
The data transfer between the buffer cache and the user process space
always occurs using DMA
◼
◼
◼
I/O operations with disk are handled through the buffer cache
Does not use up any processor cycles
Does consume bus cycles
Three lists are maintained:
◼ Free list
◼
◼
Device list
◼
◼
List of all slots in the cache that are available for allocation
List of all buffers currently associated with each disk
Driver I/O queue
◼
List of buffers that are actually undergoing or waiting for I/O on a particular
device
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Hash Pointers
Buffer Cache
Free List Pointers
Hash Table
Device#, Block#
Free List
Pointer
Figure 11.13 UNIX Buffer Cache Organization
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Used by character oriented devices
Terminals and printers
Either written by the I/O device and read by the process or
vice versa
Producer/consumer model is used
Character queues may only be read once
As each character is read, it is effectively destroyed
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Is simply DMA between device and process space
◼
Is always the fastest method for a process to perform I/O
◼
Process is locked in main memory and cannot be swapped out
◼
I/O device is tied up with the process for the
duration of the transfer making it unavailable
for other processes
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Unbuffered I/O
Buffer Cache
Disk drive
X
X
Tape drive
X
X
Character Queue
Terminals
X
Communication lines
X
Printers
X
Table 11.5 Device I/O in UNIX
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
X
◼
Very similar to other UNIX implementation
◼
Associates a special file with each I/O device driver
◼
Block, character, and network devices are recognized
◼
Default disk scheduler in Linux 2.4 is the Linux Elevator
For Linux 2.6 the Elevator algorithm has been
augmented by two additional algorithms:
• The deadline I/O scheduler
• The anticipatory I/O scheduler
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Maintains a single queue for disk read and write requests and
performs both sorting and merging functions on the queue
◼
When a new request is added to the queue, four operations are
considered in order:
◼
If the request is to the same on-disk sector or an immediately adjacent
sector to a pending request in the queue, then the existing request and
the new request are merged into one request
◼
If a request in the queue is sufficiently old, the new request is inserted at
the tail of the queue
◼
If there is a suitable location, the new request is inserted in sorted order
◼
If there is no suitable location, the new request is placed at the tail of
the queue
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Two problems manifest themselves with the elevator scheme:
◼
◼
◼
A distant block request can be delayed for a substantial time because the queue is
dynamically updated
A stream of write requests can block a read request for a considerable time, and thus
block a process
To overcome these problems, a new deadline I/O scheduler was developed in 2002
◼
◼
◼
This scheduler makes use of two pairs of queues
In addition to each incoming request being placed in a sorted elevator queue as before,
the same request is placed at the tail of a read FIFO queue for a read request or a
write FIFO queue for a write request
When a request is satisfied, it is removed from the head of the sorted queue and also
from the appropriate FIFO queue
◼
However, when the item at the head of one of the FIFO queues becomes older than its expiration
time, then the scheduler next dispatches from that FIFO queue, taking the expired request, plus the
next few requests from the queue
◼
As each request is dispatched, it is also removed from the sorted queue
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Elevator and deadline scheduling can be counterproductive if there
are numerous synchronous read requests
◼
In Linux, the anticipatory scheduler is superimposed on the deadline
scheduler
◼
When a read request is dispatched, the anticipatory scheduler causes
the scheduling system to delay
◼
There is a good chance that the application that issued the last read
request will issue another read request to the same region of the disk
◼
That request will be serviced immediately
◼
Otherwise the scheduler resumes using the deadline scheduling algorithm
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
The NOOP Scheduler
This is the simplest among Linux I/O schedulers
It is a minimal scheduler that inserts I/O requests into a FIFO
queue and uses merging
Its main uses include nondisk-based block devices such as
memory devices, and specialized software or hardware
environments that do their own scheduling and need only minimal
support in the kernel
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Completely Fair Queuing I/O
Scheduler (CFQ)
◼
Was developed in 2003
◼
Is the default I/O scheduler in Linux
◼
The CFQ scheduler guarantees a fair allocation of the disk I/O bandwidth
among all processes
◼
It maintains per process I/O queues
◼
◼
◼
◼
Each process is assigned a single queue
Each queue has an allocated timeslice
Requests are submitted into these queues and are processed in round robin
When the scheduler services a specific queue, and there are no more
requests in that queue, it waits in idle mode for a predefined time interval for
new requests, and if there are no requests, it continues to the next queue
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
For Linux 2.4 and later there is a single unified
page cache for all traffic between disk and main
memory
◼
Benefits:
◼
When it is time to write back dirty pages to disk, a
collection of them can be ordered properly and
written out efficiently
◼
Because of the principle of temporal locality, pages in
the page cache are likely to be referenced again before
they are flushed from the cache, thus saving a disk
I/O operation
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
I/O Manager
Cache
Manager
File System
Drivers
Network
Drivers
Hardware
Device Drivers
Figure 11.15 Windows I/O Manager
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Cache Manager
◼
◼
Maps regions of files into
kernel virtual memory and
then relies on the virtual
memory manager to copy
pages to and from the files
on disk
File System Drivers
◼
◼
Sends I/O requests to the
software drivers that
manage the hardware
device adapter
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Network Drivers
◼ Windows includes
integrated networking
capabilities and support
for remote file systems
◼ The facilities are
implemented as software
drivers
Hardware Device Drivers
◼ The source code of
Windows device drivers
is portable across
different processor types
Windows offers two
modes of I/O
operation
Asynchronous
Is used whenever
possible to optimize
application
performance
An application initiates an
I/O operation and then
can continue processing
while the I/O request is
fulfilled
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
Synchronous
The application is
blocked until the I/O
operation completes
▪
Windows provides five different techniques
for signaling I/O completion:
1
• Signaling the file object
2
• Signaling an event object
3
• Asynchronous procedure call
4
• I/O completion ports
5
• Polling
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Windows supports two sorts of RAID configurations:
Hardware
RAID
Software RAID
Separate physical
disks combined into
one or more logical
disks by the disk
controller or disk
storage cabinet
hardware
Noncontiguous disk
space combined into
one or more logical
partitions by the
fault-tolerant
software disk driver,
FTDISK
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Volume Shadow
Copies
◼
Efficient way of making
consistent snapshots of
volumes so they can be
backed up
◼
Also useful for archiving
files on a per-volume basis
◼
Implemented by a software
driver that makes copies of
data on the volume before
it is overwritten
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Volume
Encryption
◼
Windows uses
BitLocker to encrypt
entire volumes
◼
More secure than
encrypting individual
files
◼
Allows multiple
interlocking layers of
security
Summary
◼
◼
I/O devices
Organization of the I/O function
◼ The evolution of the I/O function
◼ Direct memory access
◼
Operating system design issues
◼ Design objectives
◼ Logical structure of the I/O function
◼
I/O Buffering
◼ Single/double/circular buffer
◼ The utility of buffering
◼
Disk scheduling
◼ Disk performance parameters
◼ Disk scheduling policies
◼
Raid
© 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved.
◼
Raid levels 0 – 6
◼
Disk cache
◼
Design and performance considerations
◼
UNIX SVR4 I/O
◼ Buffer cache
◼ Character queue
◼ Unbuffered I/O
◼ UNIX devices
◼
Linux I/O
◼ Disk scheduling
◼ Linux page cache
◼
Windows I/O
◼ Basic I/O facilities
◼ Asynchronous and Synchronous I/O
◼ Software RAID
◼ Volume shadow copies/encryption
CHAPTER 7
1. Introduction to Information Security
2. Unintentional Threats to Information
Systems
3. Deliberate Threats to Information
Systems
4. What Organizations Are Doing to
Protect Information Resources
5. Information Security Controls
1. Identify the five factors that contribute
to the increasing vulnerability of
information resources and specific
examples of each factor.
2. Compare and contrast human
mistakes and social engineering, along
with specific examples of each one.
3. Discuss the 10 types of deliberate
attacks.
4. Describe the three risk mitigation
strategies and examples of each one
in the context of owning a home.
5. Identify the three major types of
controls that organizations can use to
protect their information resources
along with an example of each one.
Opening Case
• The St. Louis Cardinals
Investigated for Hacking the
Houston Astros
1. Describe how the Cardinals apparently were
able to gain access to the Astros’ computer
system.
2. What lessons should the Astros learn from
this security breach?
7.1
Introduction to
Information Security
Information Security
Threat
Exposure
Vulnerability
Five Key Factors Increasing
Vulnerability
• Cybercrime
•
•
•
•
•
Five Key Factors Increasing
Vulnerability
1. Today’s interconnected, interdependent,
wirelessly networked business
environment
2. Smaller, faster, cheaper computers and
storage devices
3. Decreasing skills necessary to be a
computer hacker
4. International organized crime taking over
cybercrime
5. Lack of management support
7.2
Unintentional Threats to
Information Systems
• Human Errors
• Social Engineering
Figure 7.1
Security
Threats
Human Errors
• Higher employee levels = higher
levels of security risk
• Most Dangerous Employees
• Human Mistakes
Dangerous Employees
• Two organizational areas pose the
greatest risk
– Human Resources
– Information Systems
• Janitors and Guards Frequently
Overlooked
Human Mistakes
• Carelessness with laptops
• Carelessness with computing
devices
• Opening questionable e-mails
• Careless Internet surfing
• Poor password selection and use
• Carelessness with one’s office
Human Mistakes
(continued)
• Carelessness using unmanaged
devices
• Carelessness with discarded
equipment
• Careless monitoring of
environmental hazards
Table 7.1: Human Mistakes
Social
Engineering
• Social Engineering:
– an attack in which the perpetrator uses
social skills to trick or manipulate
legitimate employees into providing
confidential company information such
as passwords.
7.3
Deliberate Threats to
Information Systems
1. Espionage or Trespass
2. Information Extortion
3. Sabotage or Vandalism
4. Theft of Equipment or Information
5. Identity Theft
6. Compromises to Intellectual
Property
7.3
Deliberate Threats to
Information Systems
(continued)
7. Software Attacks
8. Alien Software
9. Supervisory Control and Data
Acquisition Attacks
10. Cyberterrorism and Cyberwarfare
6. Compromises to
Intellectual Property
•
•
•
•
Intellectual Property
Trade Secret
Patent
Copyright
7. Software Attacks: Three
Categories
1. Remote Attacks Requiring User
Action
– Virus
– Worm
– Phishing Attack
– Spear Phishing
7. Software Attacks: Three
Categories (continued)
2. Remote Attacks Needing No User
Action
– Denial-of-Service Attack
– Distributed Denial-of-Service Attack
7. Software Attacks: Three
Categories (continued)
3. Attacks by a Programmer
Developing a System
– Trojan Horse
– Back Door
– Logic bomb
ABOUT BUSINESS 7.1
• Ransomeware
1. Why is ransomware more than a nuisance?
2. Are your digital files adequately backed up?
Why or why not?
8. Alien Software
•
•
•
•
•
Adware
Spyware
Spamware
Spam
Cookies
7.4
•
•
•
•
What Organizations Are
Doing to Protect
Information Resources
Risk
Risk Management
Risk Analysis
Risk Mitigation
Table 7.3: The Difficulties in
Protecting Information Resources
ABOUT BUSINESS 7.2
• Catching a Hacker
1. Why did the FBI need to “argue with law
enforcement officials in various countries”?
2. Describe the diff iculties that investigators
encounter in bringing cybercriminals to
justice. Can you propose any additional
strategies they should consider?
Risk Management
Three Processes of Risk Management:
1. risk analysis
2. risk mitigation
3. controls evaluation
Risk Analysis
Three Steps of Risk Analysis
1. assessing the value of each asset
being protected
2. estimating the probability that each
asset will be compromised
3. comparing the probable costs of the
asset’s being compromised with the
costs of protecting that asset
Risk Mitigation
• Risk Acceptance
• Rick Limitation
• Risk Transference
7.5
•
•
•
•
•
Information Security
Controls
Physical Controls
Access Controls
Communications Controls
Business Continuity Planning
Information Systems Auditing
Figure 7.2: Where Defense
Mechanisms are Located.
Physical Controls
•
•
•
•
Walls
Doors
Fencing
Gates
•
•
•
•
Locks
Badges
Guards
Alarm Systems
Access Controls
• Authentication
• Authorization
– Something
– Something
– Something
– Something
the
the
the
the
user
user
user
user
is (Biometrics)
has
does
knows
ABOUT BUSINESS 7.3
• Trustev: Helping to Prevent Credit
Card Fraud
1. Describe how Trustev’s authentication
method differs from other authentication
methods.
2. What are potential disadvantages with
Trustev’s authentication method?
Communications Controls
• Firewall
– Demilitarized Zone (DMZ)
•
•
•
•
•
Anti-malware Systems
Whitelisting
Blacklisting
Encryption
Virtual Private Network (VPN)
Communications Controls
(Continued)
• Transport Layer Security (formerly
called Secure Socket Layer)
• Employee Monitoring Systems
Figure 7.3: (a) Basic Firewall for Home
Computer. (b) Organization with Two
Firewalls and Demilitarized Zone
Figure 7.4: How Public-key
Encryption Works
Figure 7.5: How Digital
Certificates Work.
Figure 7.6: Virtual Private
Network (VPN) and Tunneling
Business Continuity
Planning
• Business Continuity
• Business Continuity Plan
Information Systems
Auditing
• Internal Audits
• External Audits
• Three Categories of IS auditing
procedures
Three Categories of IS
auditing procedures:
• Auditing Around the Computer
• Auditing Through the Computer
• Auditing With the Computer
ONLINE ACCESS
Thank you for purchasing a new copy of Operating Systems: Internals and Design
Principles, Eighth Edition. Your textbook includes six months of prepaid access to the
book’s Premium Content. This prepaid subscription provides you with full access to the
following student support areas:
r Animations and Videonotes
r Supplemental Chapters
r Supplemental Appendices
r Supplemental homework problems with solutions
Use a coin to scratch off the coating and reveal your student access code.
Do not use a knife or other sharp object as it may damage the code.
To access the Operating Systems: Internals and Design Principles, Eighth Edition,
Premium Content for the first time, you will need to register online using a computer with
an Internet connection and a web browser. The process takes just a couple of minutes and
only needs to be completed once.
1. Go to http://www.pearsonhighered.com/stallings/
2. Click on the Premium Content.
3. Click on the Register button.
4. On the registration page, enter your student access code* found beneath the scratchoff panel. Do not type the dashes. You can use lower- or uppercase.
5. Follow the on-screen instructions. If you need help at any time during the online
registration process, simply click the Need Help? icon.
6. Once your personal Login Name and Password are confirmed, you can begin using
the Operating Systems: Internals and Design Principles Premium Content Website!
To log in after you have registered:
You only need to register for this Premium Content once. After that, you can log in any
time at http://www.pearsonhighered.com/stallings/ by providing your Login Name and
Password when prompted.
*Important: The access code can only be used once. This subscription is valid for six
months upon activation and is not transferable. If this access code has already been revealed, it may no longer be valid.
If this is the case, you can purchase a subscription by going to
http://www.pearsonhighered.com/stallings/ and following the on-screen instructions.
OPERATING SYSTEMS
INTERNALS AND DESIGN
PRINCIPLES
This page intentionally left blank
OPERATING SYSTEMS
INTERNALS AND DESIGN
PRINCIPLES
EIGHTH EDITION
William Stallings
Boston Columbus Indianapolis New York San Francisco Upper Saddle River
Amsterdam Cape Town Dubai London Madrid Milan Munich Paris Montréal Toronto
Delhi Mexico City São Paulo Sydney Hong Kong Seoul Singapore Taipei Tokyo
Editorial Director, ECS: Marcia Horton
Executive Editor: Tracy Johnson (Dunkelberger)
Editorial Assistant: Jenah Blitz-Stoehr
Director of Marketing: Christy Lesko
Marketing Manager: Yez Alayan
Marketing Assistant: Jon Bryant
Director of Program Management: Erin Gregg
Program Management-Team Lead: Scott Disanno
Program Manager: Carole Snyder
Project Management-Team Lead: Laura Burgess
Project Manager: Robert Engelhardt
Procurement Specialist: Linda Sager
Cover Designer: Bruce Kenselaar
Permissions Supervisor: Michael Joyce
Permissions Administrator: William Opaluch
Director, Image Asset Services: Annie Atherton
Manager,Visual Research: Karen Sanatar
Cover Art: © Fixer00/Shutterstock
Media Project Manager: Renata Butera
Full-Service Project Management: Integra Software Services
Pvt. Ltd. /Allan Rayer
Credits and acknowledgments borrowed from other sources and reproduced, with permission, in this textbook appear on the
appropriate page within text.
Microsoft® and Windows® are registered trademarks of the Microsoft Corporation in the United States of America
and other countries. Screenshots and icons reprinted with permission from the Microsoft Corporation. This book is not
sponsored or endorsed by or affiliated with the Microsoft Corporation.
Copyright © 2015, 2012, 2009 Pearson Education, Inc., publishing as Prentice Hall, one Lake Street, Upper Saddle River,
New Jersey, 07458. All rights reserved. Printed in the United States of America. This publication is protected by Copyright,
and permission should be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system,
or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise. To obtain
permission(s) to use material from this work, please submit a written request to Pearson Education, Inc., Permissions
Department, One Lake Street, Upper Saddle River, New Jersey 07458, or you may fax your request to 201-236-3290.
Many of the designations by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those
designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed in
initial caps or all caps.
Library of Congress Cataloging-in-Publication Data on File
10 9 8 7 6 5 4 3 2 1
ISBN 10:
0-13-380591-3
ISBN 13: 978-0-13-380591-8
For Tricia
This page intentionally left blank
CONTENTS
Online Resources xiii
VideoNotes xv
Preface xvii
About the Author xxvii
Chapter 0 Guide for Readers and Instructors 1
0.1
0.2
0.3
0.4
Outline of this Book 2
Example Systems 2
A Roadmap for Readers and Instructors 3
Internet and Web Resources 4
PART 1 BACKGROUND 7
Chapter 1 Computer System Overview 7
1.1
Basic Elements 8
1.2
Evolution of the Microprocessor 10
1.3
Instruction Execution 10
1.4
Interrupts 13
1.5
The Memory Hierarchy 24
1.6
Cache Memory 27
1.7
Direct Memory Access 31
1.8
Multiprocessor and Multicore Organization 32
1.9
Recommended Reading 36
1.10
Key Terms, Review Questions, and Problems 36
1A
Performance Characteristics of Two-Level Memories 39
Chapter 2 Operating System Overview 46
2.1
Operating System Objectives and Functions 47
2.2
The Evolution of Operating Systems 52
2.3
Major Achievements 61
2.4
Developments Leading to Modern Operating Systems 70
2.5
Fault Tolerance 73
2.6
OS Design Considerations for Multiprocessor and Multicore 76
2.7
Microsoft Windows Overview 79
2.8
Traditional UNIX Systems 86
2.9
Modern UNIX Systems 89
2.10
Linux 91
2.11
Android 96
2.12
Recommended Reading and Animations 102
2.13
Key Terms, Review Questions, and Problems 103
vii
viii
CONTENTS
PART 2 PROCESSES 105
Chapter 3 Process Description and Control 105
3.1
What Is a Process? 107
3.2
Process States 109
3.3
Process Description 124
3.4
Process Control 133
3.5
Execution of the Operating System 139
3.6
UNIX SVR4 Process Management 142
3.7
Summary 147
3.8
Recommended Reading and Animations 147
3.9
Key Terms, Review Questions, and Problems 148
Chapter 4 Threads 152
4.1
Processes and Threads 153
4.2
Types of Threads 159
4.3
Multicore and Multithreading 166
4.4
Windows 8 Process and Thread Management 171
4.5
Solaris Thread and SMP Management 178
4.6
Linux Process and Thread Management 182
4.7
Android Process and Thread Management 186
4.8
Mac OS X Grand Central Dispatch 190
4.9
Summary 193
4.10
Recommended Reading 193
4.11
Key Terms, Review Questions, and Problems 194
Chapter 5 Concurrency: Mutual Exclusion
and Synchronization 199
5.1
Principles of Concurrency 202
5.2
Mutual Exclusion: Hardware Support 210
5.3
Semaphores 214
5.4
Monitors 227
5.5
Message Passing 233
5.6
Readers/Writers Problem 240
5.7
Summary 244
5.8
Recommended Reading and Animations 245
5.9
Key Terms, Review Questions, and Problems 246
Chapter 6 Concurrency: Deadlock and Starvation 259
6.1
Principles of Deadlock 260
6.2
Deadlock Prevention 269
6.3
Deadlock Avoidance 271
6.4
Deadlock Detection 277
6.5
An Integrated Deadlock Strategy 279
6.6
Dining Philosophers Problem 280
6.7
UNIX Concurrency Mechanisms 282
6.8
Linux Kernel Concurrency Mechanisms 286
CONTENTS
6.9
6.10
6.11
6.12
6.13
6.14
Solaris Thread Synchronization Primitives 293
Windows 7 Concurrency Mechanisms 295
Android Interprocess Communication 299
Summary 300
Recommended Reading and Animations 301
Key Terms, Review Questions, and Problems 302
PART 3 MEMORY 309
Chapter 7 Memory Management 309
7.1
7.2
7.3
7.4
7.5
7.6
7.7
7A
Memory Management Requirements 310
Memory Partitioning 314
Paging 325
Segmentation 328
Summary 330
Recommended Reading and Animations 330
Key Terms, Review Questions, and Problems 330
Loading and Linking 333
Chapter 8 Virtual Memory 340
8.1
Hardware and Control Structures 341
8.2
Operating System Software 358
8.3
UNIX and Solaris Memory Management 377
8.4
Linux Memory Management 383
8.5
Windows Memory Management 386
8.6
Android Memory Management 389
8.7
Summary 389
8.8
Recommended Reading and Animations 390
8.9
Key Terms, Review Questions, and Problems 391
PART 4 SCHEDULING 397
Chapter 9 Uniprocessor Scheduling 397
9.1
Types of Processor Scheduling 398
9.2
Scheduling Algorithms 402
9.3
Traditional UNIX Scheduling 424
9.4
Summary 426
9.5
Recommended Reading and Animations 427
9.6
Key Terms, Review Questions, and Problems 428
Chapter 10 Multiprocessor, Multicore, and Real-Time
Scheduling 432
10.1
Multiprocessor and Multicore Scheduling 433
10.2
Real-Time Scheduling 446
10.3
Linux Scheduling 461
10.4
UNIX SVR4 Scheduling 465
10.5
UNIX FreeBSD Scheduling 466
ix
x
CONTENTS
10.6
10.7
10.8
10.9
Windows Scheduling 470
Summary 472
Recommended Reading 473
Key Terms, Review Questions, and Problems 473
PART 5 INPUT/OUTPUT AND FILES 477
Chapter 11 I/O Management and Disk Scheduling 477
11.1
I/O Devices 478
11.2
Organization of the I/O Function 480
11.3
Operating System Design Issues 483
11.4
I/O Buffering 486
11.5
Disk Scheduling 489
11.6
RAID 496
11.7
Disk Cache 505
11.8
UNIX SVR4 I/O 509
11.9
Linux I/O 512
11.10 Windows I/O 515
11.11 Summary 517
11.12 Recommended Reading and Animations 518
11.13 Key Terms, Review Questions, and Problems 519
Chapter 12
12.1
12.2
12.3
12.4
12.5
12.6
12.7
12.8
12.9
12.10
12.11
12.12
12.13
12.14
File Management 522
Overview 523
File Organization and Access 529
B-Trees 533
File Directories 536
File Sharing 541
Record Blocking 542
Secondary Storage Management 544
UNIX File Management 552
Linux Virtual File System 557
Windows File System 561
Android File Management 566
Summary 567
Recommended Reading 568
Key Terms, Review Questions, and Problems 568
PART 6 EMBEDDED SYSTEMS 571
Chapter 13 Embedded Operating Systems 571
13.1
Embedded Systems 572
13.2
Characteristics of Embedded Operating Systems 573
13.3
Embedded Linux 576
13.4
TinyOS 577
13.5
Recommended Reading 587
13.6
Key Terms, Review Questions, and Problems 587
CONTENTS
Chapter 14
14.1
14.2
14.3
14.4
14.5
14.6
14.7
14.8
14.9
14.10
14.11
14.12
Virtual Machines 590
Approaches to Virtualization 593
Processor Issues 596
Memory Management 598
I/O Management 600
VMware ESXi 602
Microsoft Hyper-V and Xen Variants 604
Java VM 606
Linux VServer Virtual Machine Architecture 606
Android Virtual Machine 609
Summary 611
Recommended Reading 612
Key Terms, Review Questions, and Problems 613
Chapter 15
15.1
15.2
15.3
15.4
15.5
15.6
15.7
15.8
15.9
15.10
Operating System Security 614
Intruders and Malicious Software 615
Buffer Overflow 619
Access Control 627
UNIX Access Control 635
Operating Systems Hardening 638
Security Maintenance 642
Windows Security 643
Summary 648
Recommended Reading 649
Key Terms, Review Questions, and Problems 649
Chapter 16 Distributed Processing, Client/Server, and Clusters 652
16.1
Client/Server Computing 653
16.2
Distributed Message Passing 664
16.3
Remote Procedure Calls 667
16.4
Clusters 671
16.5
Windows Cluster Server 676
16.6
Beowulf and Linux Clusters 678
16.7
Summary 680
16.8
Recommended Reading 681
16.9
Key Terms, Review Questions, and Problems 682
APPENDICES
Appendix A Topics in Concurrency A-1
Appendix B
Programming and Operating System Projects B-1
References 685
Credits 699
Index 703
xi
This page intentionally left blank
ONLINE CHAPTERS AND APPENDICES1
Chapter 17 Network Protocols
17.1
The Need for a Protocol Architecture 17-4
17.2
The TCP/IP Protocol Architecture 17-9
17.3
Sockets 17-19
17.4
Linux Networking 17-26
17.5
Summary 17-29
17.6
Recommended Reading and Web Sites 17-30
17.7
Key Terms, Review Questions, and Problems 17-31
17A
The Trivial File Transfer Protocol 17-42
Chapter 18 Distributed Process Management
18.1
Process Migration 18-2
18.2
Distributed Global States 18-14
18.3
Distributed Mutual Exclusion 18-22
18.4
Distributed Deadlock 18-41
18.5
Summary 18-59
18.6
Recommended Reading 18-60
18.7
Key Terms, Review Questions, and Problems 18-61
Chapter 19 Overview of Probability and Stochastic Processes
19.1
Probability 19-2
19.2
Random Variables 19-10
19.3
Elementary Concepts of Stochastic Processes 19-19
19.4
Recommended Reading and Web Sites 19-34
19.5
Key Terms, Review Questions, and Problems 19-35
Chapter 20 Queueing Analysis
20.1
How Queues Behave—A Simple Example 20-3
20.2
Why Queuing Analysis? 20-8
20.3
Queueing Models 20-12
20.4
Single-Server Queues 20-24
20.5
Multiserver Queues 20-28
20.6
Examples 20-30
20.7
Queues with Priorities 20-38
20.8
Networks of Queues 20-40
20.9
Other Queueing Models 20-46
20.10 Estimating Model Parameters 20-48
20.11 Recommended Reading and Web Sites 20-53
20.12 Key Terms, Review Questions, and Problems 20-54
Programming Project One Developing a Shell
Programming Project Two The HOST Dispatcher Shell
1
Online chapters, appendices, and other documents are Premium Content, available via the access card
at the front of this book.
xiii
xiv
ONLINE CHAPTERS AND APPENDICES
Appendix C Topics in Computer Organization C-1
Appendix D Object-Oriented Design D-1
Appendix E Amdahl’s Law E-1
Appendix F
Hash Tables F-1
Appendix G Response Time G-1
Appendix H Queueing System Concepts H-1
Appendix I
The Complexity of Algorithms I-1
Appendix J
Disk Storage Devices J-1
Appendix K Cryptographic Algorithms K-1
Appendix L
Standards Organizations L-1
Appendix M Sockets: A Programmer’s Introduction M-1
Appendix N The International Reference Alphabet N-1
Appendix O BACI: The Ben-Ari Concurrent Programming System O-1
Appendix P Procedure Control P-1
Appendix Q Ecos Q-1
Glossary
VideoNote
VIDEONOTES
Locations of VideoNotes
http://www.pearsonhighered.com/stallings
Chapter 5 Concurrency: Mutual Exclusion and Synchronization 199
5.1
Illustration of Mutual Exclusion 208
5.2
Hardware Support for Mutual Exclusion 212
5.3
A Definition of Semaphore Primitives 216
5.4
A Definition of Binary Semaphore Primitives 217
5.6
Mutual Exclusion Using Semaphores 219
5.9
An Incorrect Solution to the Infinite-Buffer Producer/Consumer Problem
Using Binary Semaphores 222
5.10
A Correct Solution to the Infinite-Buffer Producer/Consumer Problem
Using Binary Semaphores 224
5.11
A Solution to the Infinite-Buffer Producer/Consumer Problem
Using Semaphores 225
5.13
A Solution to the Bounded-Buffer Producer/Consumer Problem
Using Semaphores 226
5.14
Two Possible Implementations of Semaphores 227
5.16
A Solution to the Bounded-Buffer Producer/Consumer Problem
Using a Monitor 230
5.17
Bounded-Buffer Monitor Code for Mesa Monitor 232
5.20
Mutual Exclusion Using Messages 238
5.21
A Solution to the Bounded-Buffer Producer/Consumer Problem
Using Messages 239
5.22
A Solution to the Readers/Writers Problem Using Semaphores:
Readers Have Priority 241
5.23
A Solution to the Readers/Writers Problem Using Semaphores:
Writers Have Priority 243
5.24
A Solution to the Readers/Writers Problem Using Message Passing 244
5.25
An Application of Coroutines 248
Chapter 6 Concurrency: Deadlock and Starvation 259
6.9
Deadlock Avoidance Logic 276
6.12
A First Solution to the Dining Philosophers Problem 281
6.13
A Second Solution to the Dining Philosophers Problem 282
6.14
A Solution to the Dining Philosophers Problem Using a Monitor 283
6.18
Another Solution to the Dining Philosophers Problem Using a Monitor 307
Chapter 13 Embedded Operating Systems 571
13.6
Condition Variable Example Code 589
xv
xvi
VIDEONOTES
Chapter 18 Distributed Process Management
18.11 Token.Passing Algorithm (for Process Pi) 40
18.14 A Distributed Deadlock Detection Algorithm 50
Appednix A Topics in Concurrency A-1
A.1
Mutual Exclusion Attempts A-3
A.2
Dekker’s Algorithm A-6
A.3
Peterson’s Algorithm for Two Processes A-7
A.5
An Unfair Barbershop A-17
A.6
A Fair Barbershop A-20
PREFACE
WHAT’S NEW IN THE EIGHTH EDITION
Since the seventh edition of this book was published, the field has seen continued
innovations and improvements. In this new edition, I try to capture these changes
while maintaining a broad and comprehensive coverage of the entire field. To begin
the process of revision, the seventh edition of this book was extensively reviewed by
a number of professors who teach the subject and by professionals working in the
field. The result is that, in many places, the narrative has been clarified and tightened,
and illustrations have been improved.
Beyond these refinements to improve pedagogy and user friendliness, the technical content of the book has been updated throughout, to reflect the ongoing changes
in this exciting field, and the instructor and student support has been expanded. The
most noteworthy changes are as follows:
r Windows 8: Windows 8 is Microsoft’s latest OS offering for PCs, workstations,
and servers, which includes a number of changes to the internal architecture.
The new edition provides details on Windows 8 internals in all of the key
technology areas covered in this book, including process/thread management,
scheduling, memory management, security, file systems, and I/O.
r Android operating system: Android is the fastest growing mobile platform.
The real-world constraints and operating environment of mobile devices are
quite different from traditional desktop or server computers. It is important
for students to learn this new environment.
r Embedded Linux: The use of a minimal version of Linux for embedded systems has grown in popularity. This new edition provides an overview of the key
elements of the embedded Linux approach.
r Virtual machines: Server virtualization and other forms of virtual machines
are becoming increasingly widespread. A new chapter deals with the operating
system design issues for virtual machines.
r Multicore design issues: The dominant computer architecture is now multicore. This raises new OS design issues that are addressed in this new edition.
r I/O standards: The book has been updated to reflect the latest developments,
including Thunderbolt.
r Storage hardware: The discussion of storage hardware has been updated and
now includes discussion of solid-state drives.
r Fault tolerance: The ACM/IEEE Computer Science Curricula 2013 lists fault
tolerance as one of the core topics for an OS course. A new section provides an
overview of fault tolerance.
xvii
xviii
PREFACE
OBJECTIVES
This book is about the concepts, structure, and mechanisms o...
Purchase answer to see full
attachment