Operating Systems: Internals and Design Principles Chapter 12 File Management Ninth Edition By William Stallings © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Files ◼ Data collections created by users ◼ The File System is one of the most important parts of the OS to a user ◼ Desirable properties of files: Long-term existence • Files are stored on disk or other secondary storage and do not disappear when a user logs off Sharable between processes • Files have names and can have associated access permissions that permit controlled sharing Structure • Files can be organized into hierarchical or more complex structure to reflect the relationships among files © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. File Systems ◼ Provide a means to store data organized as files as well as a collection of functions that can be performed on files ◼ Maintain a set of attributes associated with the file ◼ Typical operations include: ◼ Create ◼ Delete ◼ Open ◼ Close ◼ Read ◼ Write © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. File Structure Four terms are commonly used when discussing files: Field Record © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. File Database Structure Terms File Field ◼ Basic element of data ◼ Collection of similar records ◼ Contains a single value ◼ Treated as a single entity ◼ Fixed or variable length ◼ May be referenced by name Database ◼ Access control restrictions usually apply at the file level ◼ Collection of related data ◼ Relationships among elements of data are explicit ◼ Designed for use by a number of different applications ◼ Consists of one or more types of files © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Record ◼ Collection of related fields that can be treated as a unit by some application program ◼ Fixed or variable length File Management System ◼ Meet the data management needs of the user ◼ Guarantee that the data in the file are valid ◼ Optimize performance ◼ Provide I/O support for a variety of storage device types ◼ Minimize the potential for lost or destroyed data ◼ Provide a standardized set of I/O interface routines to user processes ◼ Provide I/O support for multiple users in the case of multipleuser systems © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Minimal User Requirements ▪ Each user: 1 2 3 4 5 6 • Should be able to create, delete, read, write and modify files • May have controlled access to other users’ files • May control what type of accesses are allowed to the users’ files • Should be able to move data between files • Should be able to back up and recover files in case of damage • Should be able to access his or her files by name rather than by numeric identifier © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. User Program Pile Indexed Sequential Sequential Indexed Hashed Logical I/O Basic I/O Supervisor Basic File System Disk Device Driver Tape Device Driver Figure 12.1 File System Software Architecture © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Device Drivers ◼ Lowest level ◼ Communicates directly with peripheral devices ◼ Responsible for starting I/O operations on a device ◼ Processes the completion of an I/O request ◼ Considered to be part of the operating system © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Basic File System ◼ Also referred to as the physical I/O level ◼ Primary interface with the environment outside the computer system ◼ Deals with blocks of data that are exchanged with disk or tape systems ◼ Concerned with the placement of blocks on the secondary storage device ◼ Concerned with buffering blocks in main memory ◼ Does not understand the content of the data or the structure of the files involved ◼ Considered part of the operating system © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Basic I/O Supervisor ◼ Responsible for all file I/O initiation and termination ◼ At this level, control structures are maintained that deal with device I/O, scheduling, and file status ◼ Selects the device on which I/O is to be performed ◼ Concerned with scheduling disk and tape accesses to optimize performance ◼ I/O buffers are assigned and secondary memory is allocated at this level ◼ Part of the operating system © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Logical I/O Enables users and applications to access records Provides generalpurpose record I/O capability © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Maintains basic data about file Access Method ◼ Level of the file system closest to the user ◼ Provides a standard interface between applications and the file systems and devices that hold the data ◼ Different access methods reflect different file structures and different ways of accessing and processing the data © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Records Directory management User & program comands Operation, File name File Structure Access method Physical blocks in main memory buffers Blocking Physical blocks in secondary storage (disk) Disk scheduling File manipulation functions I/O File allocation User access control File management concerns Operating system concerns Figure 12.2 Elements of File Management © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Free storage management File Organization and Access ◼ File organization is the logical structuring of the records as determined by the way in which they are accessed ◼ ◼ In choosing a file organization, several criteria are important: ◼ Short access time ◼ Ease of update ◼ Economy of storage ◼ Simple maintenance ◼ Reliability Priority of criteria depends on the application that will use the file © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. File Organization Types The pile The sequential file The direct, or hashed, file Five of the common file organizations are: The indexed file © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. The indexed sequential file Variable-length records Variable set of fields Chronological order Fixed-length records Fixed set of fields in fixed order Sequential order based on key field (a) Pile File (b) Sequential File Exhaustive index Exhaustive index Partial index n Index levels 2 1 Main File Index Overflow File (c) Indexed Sequential File Primary File (variable-length records) (d) Indexed File Figure 12.3 Common File Organizations © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. The Pile ◼ Least complicated form of file organization ◼ Data are collected in the order they arrive ◼ Each record consists of one burst of data ◼ Purpose is simply to accumulate the mass of data and save it ◼ Record access is by exhaustive search © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Variable-length records Variable set of fields Chronological order (a) Pile File E The Sequential File ◼ Most common form of file structure ◼ A fixed format is used for records ◼ Key field uniquely identifies the record ◼ Typically ble-length records used in batch ble set of applications fields nological order Only organization that is easily stored on tape as well (a) asPile diskFile ◼ © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Fixed-length records Fixed set of fields in fixed order Sequential order based on key field (b) Sequential File Exhaustive Exhaustive Partial (a) Pile File Indexed Sequential File ◼ Adds an index to the file to support random access ◼ Adds an overflow file ◼ Greatly reduces the time required to access a single record ◼ Multiple levels of indexing can be used to provide greater efficiency in access n Index levels 2 1 © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Main File Index Overflow File (c) Indexed Sequential File (a) Pile File Indexed File (b) Sequential File Exhaustive index Exhaustive index Partial index n Records are accessed only through Main File Indextheir indexes levels Index ◼ 2Variable-length records can be 1 employed ◼ ◼ ◼ ◼ ◼ Exhaustive index contains one entry for every record in the main file Overflow Partial index contains Fileentries to records where the field of interest exists (c) Indexed File Used mostly inSequential applications where timeliness of information is critical Examples would be airline reservation systems and inventory control systems © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Primary File (variable-length records) (d) Indexed File Direct or Hashed File ◼ Access directly any block of a known address ◼ Makes use of hashing on the key value ◼ Often used where: ◼ Very rapid access is required ◼ Fixed-length records are used ◼ Records are always accessed one at a time © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Examples are: • • • • Directories Pricing tables Schedules Name lists B-Trees ◼ A balanced tree structure with all branches of equal length ◼ Standard method of organizing indexes for databases ◼ Commonly used in OS file systems ◼ Provides for efficient searching, adding, and deleting of items © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Key1 Subtree1 Subtree2 Key2 Keyk–1 Subtree3 Subtreek–1 Figure 12.4 A B-tree Node with k Children © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Subtreek B-Tree ◼ Every node has at most 2d – 1 keys and 2d children or, equivalently, 2d pointers ◼ Every node, except for the root, has at least d – 1 keys and d pointers, as a result, each internal node, except the root, is at least half full and has at least d children ◼ The root has at least 1 key and 2 children ◼ All leaves appear on the same level and contain no information. This is a logical construct to terminate the tree; the actual implementation may differ ◼ A nonleaf node with k pointers contains k – 1 keys Characteristics A B-tree is characterized by its minimum degree d and satisfies the following properties: © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. 23 51 61 71 2 10 30 32 39 43 44 52 59 60 67 68 73 85 88 96 67 68 73 85 88 90 96 (a) B-tree of minimum degree d = 3. 23 51 61 71 2 10 30 32 39 43 44 52 59 60 (b) Key = 90 inserted. This is a simple insertion into a node. 23 39 51 61 71 2 10 30 32 43 44 45 52 59 60 67 68 73 85 88 90 96 (c) Key = 45 inserted. This requires splitting a node into two parts and promoting one key to the root node. 51 23 39 2 10 30 32 61 71 88 43 44 45 52 59 60 67 68 73 84 85 90 96 (d) Key = 84 inserted. This requires splitting a node into two parts and promoting one key to the root node This then requires the root node to be split and a new root created. Figure 12.5 Inserting Nodes into a B-tree © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Table 12.1 Basic Information File Name Name as chosen by creator (user or program). Must be unique within a specific directory. File Type For example: text, binary, load module, etc. File Organization For systems that support different organizations Address Information Information Elements of a File Directory Volume Indicates device on which file is stored Starting Address Starting physical address on secondary storage (e.g., cylinder, track, and block number on disk) Size Used Current size of the file in bytes, words, or blocks Size Allocated The maximum size of the file Access Control Information Owner User who is assigned control of this file. The owner may be able to grant/deny access to other users and to change these privileges. Access Information A simple version of this element would include the user's name and password for each authorized user. Permitted Actions Controls reading, writing, executing, transmitting over a network Usage Information (Table can be found on page 537 in textbook) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Date Created When file was first placed in directory Identity of Creator Usually but not necessarily the current owner Date Last Read Access Date of the last time a record was read Identity of Last Reader User who did the reading Date Last Modified Date of the last update, insertion, or deletion Identity of Last Modifier User who did the modifying Date of Last Backup Date of the last time the file was backed up on another storage medium Current Usage Information about current activity on the file, such as process or processes that have the file open, whether it is locked by a process, and whether the file has been updated in main memory but not yet on disk Operations Performed on a Directory ◼ To understand the requirements for a file structure, it is helpful to consider the types of operations that may be performed on the directory: Search Create files © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Delete files List directory Update directory Two-Level Scheme There is one directory for each user and a master directory Master directory has an entry for each user directory providing address and access control information Names must be unique only within the collection of files of a single user © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Each user directory is a simple list of the files of that user File system can easily enforce access restriction on directories Master Directory Subirectory Subirectory Subirectory Subirectory Subirectory File File File File Figure 12.6 Tree-Structured Directory © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Master Directory System User_A User_B User_C Directory "User_C" Directory "User_A" Directory "User_B" Draw Word Directory "Word" Directory "Draw" Unit_A ABC Directory "Unit_A" ABC File "ABC" Pathname: /User _B/Draw/ABC File "ABC" Pathname: /User _B/Word/Unit_A/ABC Figure 12.7 Example of Tree-Structured Directory © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. File Sharing Two issues arise when allowing files to be shared among a number of users: Access rights © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Management of simultaneous access Access Rights ◼ None ◼ ◼ ◼ The user would not be allowed to read the user directory that includes the file Knowledge ◼ The user can determine that the ◼ ◼ file exists and who its owner is and can then petition the owner for additional access rights ◼ Execution ◼ ◼ ◼ ◼ The user can read the file for any purpose, including copying and execution © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. The user can modify, delete, and add to the file’s data Changing protection ◼ ◼ The user can add data to the file but cannot modify or delete any of the file’s contents Updating ◼ The user can load and execute a program but cannot copy it Reading Appending The user can change the access rights granted to other users Deletion ◼ The user can delete the file from the file system User Access Rights Owner Specific Users User Groups Usually the initial creator of the file Has full rights Individual users who are designated by user ID May grant rights to others © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. A set of users who are not individually defined All All users who have access to this system These are public files Record Blocking ◼ Blocks are the unit of I/O with secondary storage ◼ ▪ For I/O to be performed records must be organized as blocks Given the size of a block, three methods of blocking can be used: © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. 1) Fixed-Length Blocking – fixedlength records are used, and an integral number of records are stored in a block Internal fragmentation – 2) 3) unused space at the end of each block Variable-Length Spanned Blocking – variable-length records are used and are packed into blocks with no unused space Variable-Length Unspanned Blocking – variable-length records are used, but spanning is not employed Record 1 Record 2 Record 3 Record 4 Track 1 Record 5 Record 6 Record 7 Record 8 Track 2 (a) Fixed Blocking Record 1 Record 4 (rest) Record 2 Record 5 Record 3 Record 4 Ptr Record 1 Ptr Track 2 Record 6 Track 1 (b) Variable Blocking: Spanned Record 1 Record 2 Record 4 Record 5 Record 3 Record 6 (c) Variable Blocking: Unspanned Figure 12.8 Record Blocking Methods © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Track 1 Track 2 File Allocation ▪ On secondary storage, a file consists of a collection of blocks ▪ The operating system or file management system is responsible for allocating blocks to files ▪ The approach taken for file allocation may influence the approach taken for free space management ▪ Space is allocated to a file as one or more portions (contiguous set of allocated blocks) ▪ File allocation table (FAT) ▪ Data structure used to keep track of the portions assigned to a file © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Preallocation vs Dynamic Allocation ◼ A preallocation policy requires that the maximum size of a file be declared at the time of the file creation request ◼ For many applications it is difficult to estimate reliably the maximum potential size of the file ◼ ◼ Tends to be wasteful because users and application programmers tend to overestimate size Dynamic allocation allocates space to a file in portions as needed © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Portion Size ◼ In choosing a portion size there is a trade-off between efficiency from the point of view of a single file versus overall system efficiency ◼ Items to be considered: 1) Contiguity of space increases performance, especially for Retrieve_Next operations, and greatly for transactions running in a transaction-oriented operating system 2) Having a large number of small portions increases the size of tables needed to manage the allocation information 3) Having fixed-size portions simplifies the reallocation of space 4) Having variable-size or small fixed-size portions minimizes waste of unused storage due to overallocation © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Alternatives Two major alternatives: Variable, large contiguous portions • Provides better performance • The variable size avoids waste • The file allocation tables are small © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Blocks • Small fixed portions provide greater flexibility • They may require large tables or complex structures for their allocation • Contiguity has been abandoned as a primary goal • Blocks are allocated as needed Table 12.2 File Allocation Methods Contiguous Chained Indexed Preallocation? Necessary Possible Fixed or variable size portions? Variable Fixed blocks Fixed blocks Variable Portion size Large Small Small Medium Allocation frequency Once Low to high High Low Time to allocate Medium Long Short Medium File allocation table size One entry One entry Large Medium © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Possible File Allocation Table File Name Start Block File A 0 1 5 6 10 11 15 16 20 21 25 26 File D 31 30 2 3 4 7 8 File B 12 13 9 14 17 18 File C 22 23 File E 27 28 19 32 34 33 File A File B File C File D File E 2 9 18 30 26 Length 3 5 8 2 3 24 29 Figure 12.9 Contiguous File Allocation © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. File Allocation Table File Name Start Block File A 0 15 1 2 File B 6 7 File C 11 12 File E 16 17 20 21 25 30 5 10 3 4 8 9 13 18 14 File D 19 22 23 24 26 27 28 29 31 32 33 34 File A File B File C File D File E 0 3 8 19 16 Length 3 5 8 2 3 Figure 12.10 Contiguous File Allocation (After Compaction) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. File Allocation Table File Name Start Block 0 File B 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 File B 1 Figure 12.11 Chained Allocation © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Length 5 File Allocation Table File Name Start Block 0 File B 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 File B 0 Length 5 Figure 12.12 Chained Allocation (After Consolidation) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. File Allocation Table 0 File B 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 File Name Index Block File B 24 1 8 3 14 28 Figure 12.13 Indexed Allocation with Block Portions © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. File Allocation Table 0 File B 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 File Name Index Block File B 24 Start Block Length 1 28 14 3 4 1 Figure 12.14 Indexed Allocation with Variable-Length Portions © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Free Space Management ◼ Just as allocated space must be managed, so must the unallocated space ◼ To perform file allocation, it is necessary to know which blocks are available ◼ A disk allocation table is needed in addition to a file allocation table © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Bit Tables ◼ This method uses a vector containing one bit for each block on the disk ◼ Each entry of a 0 corresponds to a free block, and each 1 corresponds to a block in use Advantages: • Works well with any file allocation method • It is as small as possible © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Chained Free Portions ◼ The free portions may be chained together by using a pointer and length value in each free portion ◼ Negligible space overhead because there is no need for a disk allocation table ◼ Suited to all file allocation methods Disadvantages: • Leads to fragmentation • Every time you allocate a block you need to read the block first to recover the pointer to the new first free block before writing data to that block © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Indexing ◼ Treats free space as a file and uses an index table as it would for file allocation ◼ For efficiency, the index should be on the basis of variable-size portions rather than blocks ◼ This approach provides efficient support for all of the file allocation methods © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Free Block List Each block is assigned a number sequentially The list of the numbers of all free blocks is maintained in a reserved portion of the disk Depending on the size of the disk, either 24 or 32 bits will be needed to store a single block number The size of the free block list is 24 or 32 times the size of the corresponding bit table and must be stored on disk © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. There are two effective techniques for storing a small part of the free block list in main memory: The list can be treated as a push-down stack with the first few thousand elements of the stack kept in main memory The list can be treated as a FIFO queue, with a few thousand entries from both the head and the tail of the queue in main memory Volumes ◼ A collection of addressable sectors in secondary memory that an OS or application can use for data storage ◼ The sectors in a volume need not be consecutive on a physical storage device ◼ ◼ They need only appear that way to the OS or application A volume may be the result of assembling and merging smaller volumes © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. UNIX File Management ◼ In the UNIX file system, six types of files are distinguished: Regular, or ordinary • Contains arbitrary data in zero or more data blocks Directory • Contains a list of file names plus pointers to associated inodes (index nodes) Special • Contains no data but provides a mechanism to map physical devices to file names Named pipes • An interprocess communications facility Links • An alternative file name for an existing file Symbolic links • A data file that contains the name of the file to which it is linked © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Inodes ◼ All types of UNIX files are administered by the OS by means of inodes ◼ An inode (index node) is a control structure that contains the key information needed by the operating system for a particular file ◼ Several file names may be associated with a single inode ◼ An active inode is associated with exactly one file ◼ Each file is controlled by exactly one inode © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. mode Data Data Data Data Data Data Data Data Data Data owners (2) Data timestamps (4) size Data direct(0) direct (1) Pointers Pointers direct(12) single indirect Pointers Data Pointers double indirect triple indirect Pointers Data block count reference count Pointers flags (2) generation number Pointers Data Pointers Pointers blocksize Pointers Data extended attr size extended attribute blocks Pointers Data Inode Figure 12.15 Structure of FreeBSD inode and File © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. File Allocation ◼ File allocation is done on a block basis ◼ Allocation is dynamic, as needed, rather than using preallocation ◼ An indexed method is used to keep track of each file, with part of the index stored in the inode for the file ◼ In all UNIX implementations the inode includes a number of direct pointers and three indirect pointers (single, double, triple) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Table 12.3 Capacity of a FreeBSD File with 4-Kbyte Block Size Level Number of Blocks Direct 12 48K 512 2M 512 ´ 512 = 256K 1G Single Indirect Double Indirect Triple Indirect Number of Bytes 512 ´ 256K = 128M © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. 512G Inode table Directory i1 Name1 i2 Name2 i3 Name3 i4 Name4 Figure 12.16 UNIX Directories and Inodes © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Volume Structure ◼ A UNIX file system resides on a single logical disk or disk partition and is laid out with the following elements: Boot block Superblock Inode table Data blocks Contains code required to boot the operating system Contains attributes and information about the file system Collection of inodes for each file Storage space available for data files and subdirectories © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. User applications User space GNU C library System call interface Inode cache File system Virtual File System (VFS) Directory cache Individual File Systems Buffer cache Device drivers Figure 12.17 Linux Virtual File System Context © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Kernel space System calls using VFS user interface Linux Virtual File System VFS system calls Mapping function to file system X System calls using file system X interface File System X Disk I/O calls User Process Files on secondary storage maintained by file system X Figure 12.18 Linux Virtual File System Concept © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Primary Object Types in VFS Superblock Object • Represents a specific mounted file system Dentry Object • Represents a specific directory entry Inode Object File Object • Represents a specific file • Represents an open file associated with a process © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. The Superblock Object ◼ Stores information describing a specific file system ◼ Typically, the superblock corresponds to the file system superblock or file system control block, which is stored in a special sector on disk ◼ The superblock object consists of a number of data items ◼ Examples include: ◼ The device this file system is mounted on ◼ The basic block size of the file system ◼ Dirty flag, to indicate that the superblock has been changed but not written back to disk ◼ File system type ◼ Flags, such as a read-only flag ◼ Pointer to the root of the file system directory ◼ List of open files ◼ Semaphore for controlling access to the file system ◼ List of superblock operations © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. The Inode Object ◼ An inode is associated with each file ◼ The inode object holds all the information about a named file except its name and the actual data contents of the file ◼ Items contained in an inode object include owner, group, permissions, access times for a file, size of data it holds, and number of links ◼ The inode object also includes an inode operations object that describes the file system’s implemented functions that the VFS can invoke on an inode © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. The Dentry Object ◼ A dentry (directory entry) is a specific component in a path ◼ The component may be either a directory name or a file name ◼ Dentry objects facilitate quick lookups to files and directories, and are used in a dentry cache for that purpose ◼ The dentry object includes a pointer to the inode and superblock ◼ It also includes a pointer to the parent dentry and pointers to any subordinate dentrys © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. The File Object ◼ The file object is used to represent a file opened by a process ◼ The object is created in response to the open() system call, and destroyed in response to the close () system call ◼ The file object also includes an inode operations object that describes the file system’s implemented functions that the VFS can invoke on a file object ◼ The methods defined for the file object include read, write, open, release, and lock ◼ The file object consists of a number of items, including: ◼ Dentry object associated with the file ◼ File system containing the file ◼ File objects usage counter ◼ User’s user ID ◼ User’s group ID ◼ File pointer, which is the current position in the file from which the next operation will take place © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Caches ◼ The VFS employs three caches to improve performance: ◼ Inode cache ◼ ◼ ◼ Because every file and directory is represented by a VFS inode, a directory listing command or a file access command causes a number of inodes to be accessed ◼ The inode cache stores recently visited inodes to make access quicker Directory cache ◼ The directory cache stores the mapping between the full directory names and their inode numbers ◼ This speeds up the process of listing a directory Buffer cache ◼ The buffer cache is independent of the file systems and is integrated into the mechanisms that the Linux kernel uses to allocate and read and write data buffers ◼ As the real file systems read data from the underlying physical disks, this results in requests to the block device drivers to read physical blocks from the device that they control ◼ If the same data is needed often, it will be retrieved from the buffer cache rather than read from the disk © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Windows File System ◼ The developers of Windows NT designed a new file system, the New Technology File System (NTFS) which is intended to meet high-end requirements for workstations and servers ◼ Key features of NTFS: ◼ Recoverability ◼ Security ◼ Large disks and large files ◼ Multiple data streams ◼ Journaling ◼ Compression and encryption ◼ Hard and symbolic links © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. NTFS Volume and File Structure ◼ NTFS makes use of the following disk storage concepts: Sector • The smallest physical storage unit on the disk • The data size in bytes is a power of 2 and is almost always 512 bytes Cluster • One or more contiguous sectors • The cluster size in sectors is a power of 2 Volume • A logical partition on a disk, consisting of one or more clusters and used by a file system to allocate space • Can be all or a portion of a single disk or it can extend across multiple disks • The maximum volume size for NTFS is 264 clusters © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Table 12.4 Windows NTFS Partition and Cluster Sizes Volume Size Sectors per Cluster Cluster Size £ 512 Mbyte 1 512 bytes 512 Mbyte - 1 Gbyte 2 1K 1 Gbyte - 2 Gbyte 4 2K 2 Gbyte - 4 Gbyte 8 4K 4 Gbyte - 8 Gbyte 16 8K 8 Gbyte - 16 Gbyte 32 16K 16 Gbyte - 32 Gbyte 64 32K > 32 Gbyte 128 64K © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. partition boot sector Master File Table System Files Figure 12.19 NTFS Volume Layout © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. File Area Master File Table (MFT) ◼ The heart of the Windows file system is the MFT ◼ The MFT is organized as a table of 1,024-byte rows, called records ◼ Each row describes a file on this volume, including the MFT itself, which is treated as a file ◼ Each record in the MFT consists of a set of attributes that serve to define the file (or folder) characteristics and the file contents © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Table 12.5 Windows NTFS File and Directory Attribute Types Attribute Type Description Standard information Includes access attributes (read-only, read/write, etc.); time stamps, including when the file was created or last modified; and how many directories point to the file (link count). Attribute list A list of attributes that make up the file and the file reference of the MFT file record in which each attribute is located. Used when all attributes do not fit into a single MFT file record. File name A file or directory must have one or more names. Security descriptor Specifies who owns the file and who can access it. Data The contents of the file. A file has one default unnamed data attribute and may have one or more named data attributes. Index root Used to implement folders. Index allocation Used to implement folders. Volume information Includes volume-related information, such as the version and name of the volume. Bitmap Provides a map representing records in use on the MFT or folder. Note: Colored rows refer to required file attributes; the other attributes are optional. © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. I/O Manager Log File Service Log the transaction Read/write the file Flush the Write the log file cache Cache Manager NTFS Driver Fault Tolerant Driver Read/write a mirrored or striped volume Disk Driver Load data from disk into memory Access the mapped file or flush the cache Virtual Memory Manager Figure 12.20 Windows NTFS Components © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Read/write the disk /(root) /system (ro) bin etc lib /data (rw) usr /cache /mnt/sdcard (rw) removable storage (rw) ro: mounted as read only rw: mounted as read and write Figure 12.21 Typical Directory Tree of Android © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. SQLite ◼ Most widely deployed SQL database engine in the world ◼ Based on the Structured Query Language (SQL) ◼ Designed to provide a streamlined SQL-based database management system suitable for embedded systems and other limited memory systems ◼ The full SQLite library can be implemented in under 400 KB ◼ In contrast to other database management systems, SQLite is not a separate process that is accessed from the client application ◼ The SQLite library is linked in, and thus becomes an integral part of the application program © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Summary ◼ ◼ ◼ ◼ ◼ ◼ ◼ ◼ File structure File management systems File organization and access ◼ The pile ◼ The sequential file ◼ The indexed sequential file ◼ The indexed file ◼ The direct or hashed file B-Trees File directories ◼ Contents ◼ Structure ◼ Naming File sharing ◼ Access rights ◼ Simultaneous access Record blocking Android file management ◼ File system ◼ SQLite © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ ◼ ◼ ◼ Secondary storage management ◼ File allocation ◼ Free space management ◼ Volumes ◼ Reliability UNIX file management ◼ Inodes ◼ File allocation ◼ Directories ◼ Volume structure Linux virtual file system ◼ Superblock object ◼ Inode object ◼ Dentry object ◼ File object ◼ Caches Windows file system ◼ Key features of NTFS ◼ NTFS volume and file structure ◼ Recoverability Operating Systems: Internals and Design Principles Chapter 11 I/O Management and Disk Scheduling Ninth Edition By William Stallings © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. External devices that engage in I/O with computer systems can be grouped into three categories: Human readable • Suitable for communicating with the computer user • Printers, terminals, video display, keyboard, mouse Machine readable • Suitable for communicating with electronic equipment • Disk drives, USB keys, sensors, controllers Communication • Suitable for communicating with remote devices • Modems, digital line drivers © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Devices differ in a number of areas: Data Rate • There may be differences of magnitude between the data transfer rates Application • The use to which a device is put has an influence on the software Complexity of Control • The effect on the operating system is filtered by the complexity of the I/O module that controls the device Unit of Transfer • Data may be transferred as a stream of bytes or characters or in larger blocks Data Representation • Different data encoding schemes are used by different devices Error Conditions • The nature of errors, the way in which they are reported, their consequences, and the available range of responses differs from one device to another © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Gigabit Ethernet Graphics display Hard disk Ethernet Optical disk Scanner Laser printer Floppy disk Modem Mouse Keyboard 101 102 103 104 105 106 107 Data Rate (bps) Figure 11.1 Typical I/O Device Data Rates © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. 108 109 ◼ Three techniques for performing I/O are: ◼ Programmed I/O ◼ ◼ Interrupt-driven I/O ◼ ◼ The processor issues an I/O command on behalf of a process to an I/O module; that process then busy waits for the operation to be completed before proceeding The processor issues an I/O command on behalf of a process ◼ If non-blocking – processor continues to execute instructions from the process that issued the I/O command ◼ If blocking – the next instruction the processor executes is from the OS, which will put the current process in a blocked state and schedule another process Direct Memory Access (DMA) ◼ A DMA module controls the exchange of data between main memory and an I/O module © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. I/O-to-memory transfer through processor No Interrupts Use of Interrupts Programmed I/O Interrupt-driven I/O Direct I/O-to-memory transfer © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Direct memory access (DMA) 1 2 3 4 • Processor directly controls a peripheral device • A controller or I/O module is added • Same configuration as step 2, but now interrupts are employed • The I/O module is given direct control of memory via DMA 5 • The I/O module is enhanced to become a separate processor, with a specialized instruction set tailored for I/O 6 • The I/O module has a local memory of its own and is, in fact, a computer in its own right © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Data Count Data Lines Data Register Address Lines Address Register Request to DMA Acknowledge from DMA Interrupt Read Write Control Logic Figure 11.2 Typical DMA Block Diagram © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Processor DMA I/O Memory I/O (a) Single-bus, detached DMA Processor DMA Memory DMA I/O I/O I/O (b) Single-bus, Integrated DMA-I/O System bus Processor Memory DMA I/O bus I/O I/O I/O (c) I/O bus Figure 11.3 Alternative DMA Configurations © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Efficiency Generality ◼ Major effort in I/O design ◼ ◼ Important because I/O operations often form a bottleneck Desirable to handle all devices in a uniform manner ◼ Most I/O devices are extremely slow compared with main memory and the processor Applies to the way processes view I/O devices and the way the operating system manages I/O devices and operations ◼ The area that has received the most attention is disk I/O Diversity of devices makes it difficult to achieve true generality ◼ Use a hierarchical, modular approach to the design of the I/O function ◼ ◼ © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. User Processes User Processes User Processes Directory Management Logical I/O Communication Architecture File System Physical Organization Device I/O Device I/O Device I/O Scheduling & Control Scheduling & Control Scheduling & Control Hardware Hardware Hardware (a) Local peripheral device (b) Communications port (c) File system Figure 11.4 A Model of I/O Organization © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ To avoid overheads and inefficiencies, it is sometimes convenient to perform input transfers in advance of requests being made, and to perform output transfers some time after the request is made Block-oriented device Stream-oriented device • Stores information in blocks that are usually of fixed size • Transfers are made one block at a time • Possible to reference data by its block number • Disks and USB keys are examples • Transfers data in and out as a stream of bytes • No block structure • Terminals, printers, communications ports, mouse and other pointing devices, and most other devices that are not secondary storage are examples © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ No Buffer © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Without a buffer, the OS directly accesses the device when it needs Single Buffer © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ The simplest type of support that the operating system can provide ◼ When a user process issues an I/O request, the OS assigns a buffer in the system portion of main memory to the operation ◼ Input transfers are made to the system buffer ◼ Reading ahead/anticipated input ◼ ◼ ◼ Approach generally provides a speedup compared to the lack of system buffering ◼ ◼ ◼ Is done in the expectation that the block will eventually be needed When the transfer is complete, the process moves the block into user space and immediately requests another block The user process can be processing one block of data while the next block is being read in The OS is able to swap the process out because the input operation is taking place in system memory rather than user process memory Disadvantages: ◼ ◼ Complicates the logic in the operating system Swapping logic is also affected © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Can be used in a line-at-a-time fashion or a byte-at-a-time fashion ◼ Line-at-a-time operation is appropriate for scroll-mode terminals (dumb terminals) ◼ With this form of terminal, user input is one line at a time, with a carriage return signaling the end of a line ◼ Output to the terminal is similarly one line at a time © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Byte-at-a-time operation is used on forms-mode terminals, when each keystroke is significant and for many other peripherals, such as sensors and controllers Double Buffer © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Assigning two system buffers to the operation ◼ A process now transfers data to or from one buffer while the operating system empties or fills the other buffer ◼ Also known as buffer swapping Circular Buffer © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ When more than two buffers are used, the collection of buffers is itself referred to as a circular buffer ◼ Each individual buffer is one unit in the circular buffer © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Disk Performance Parameters Wait for Device Wait for Channel Seek The actual details of disk I/O operation depend on the: ◼ Computer system ◼ Operating system ◼ Nature of the I/O channel and disk controller hardware Rotational Delay Device Busy Figure 11.6 Timing of a Disk I/O Transfer © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Data Transfer ◼ When the disk drive is operating, the disk is rotating at constant speed ◼ To read or write the head must be positioned at the desired track and at the beginning of the desired sector on that track ◼ Track selection involves moving the head in a movable-head system or electronically selecting one head on a fixed-head system ◼ On a movable-head system the time it takes to position the head at the track is known as seek time ◼ The time it takes for the beginning of the sector to reach the head is known as rotational delay ◼ The sum of the seek time and the rotational delay equals the access time © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Seek Time ◼ The time required to move the disk arm to the required track ◼ Consists of two key components: ◼ ◼ ◼ The initial startup time The time taken to traverse the tracks that have to be crossed once the access arm is up to speed Settling time ◼ Time after positioning the head over the target track until track identification is confirmed ◼ Much improvement comes from smaller and lighter disk components ◼ A typical average seek time on contemporary hard disks is under 10ms © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Disk Performance ◼ Rotational delay ◼ The time required for the addressed area of the disk to rotate into a position where it is accessible by the read/write head ◼ Disks rotate at speeds ranging from 3,6000 rpm (for handheld devices such as digital cameras) up to 15,000 rpm © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. track number track number 0 25 50 75 100 125 150 175 199 track number track number 0 25 50 75 100 125 150 175 199 0 25 50 75 100 125 150 175 199 0 25 50 75 100 125 150 175 199 (a) FIFO Time (b) SSTF Time (c) SCAN Time (d) C-SCAN Figure 11.7 Comparison of Disk Scheduling Algorithms (see Table 11.3) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Time (a) FIFO (b) SSTF (starting at track 100) (starting at track 100) Next track accessed 55 58 39 18 90 160 150 38 184 Average seek length Number Next of tracks track traversed accessed 45 3 19 21 72 70 10 112 146 55.3 90 58 55 39 38 18 150 160 184 Average seek length (c) SCAN (d) C-SCAN (starting at track 100, (starting at track 100, in the direction of in the direction of increasing track increasing track number) number) Number Next Number Next Number of tracks track of tracks track of tracks traversed accessed traversed accessed traversed 10 32 3 16 1 20 132 10 24 27.5 150 160 184 90 58 55 39 38 18 Average seek length 50 10 24 94 32 3 16 1 20 27.8 150 160 184 18 38 39 55 58 90 Average seek length Table 11.2 Comparison of Disk Scheduling Algorithms © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. 50 10 24 166 20 1 16 3 32 35.8 track number First-In, First-Out (FIFO) 0 25 50 75 100 125 150 175 199 ◼ Processes in sequential order ◼ Fair to all processes ◼ Approximates random scheduling in performance if there are many processes competing for the disk (a) FIFO 0 © 2017 Pearson 25 Education, Inc., Hoboken, NJ. All rights reserved. Time Name Description Remarks Selection according to requestor Random Random scheduling For analysis and simulation FIFO First in first out Fairest of them all PRI Priority by process Control outside of disk queue management LIFO Last in first out Maximize locality and resource utilization Selection according to requested item SSTF Shortest service time first High utilization, small queues SCAN Back and forth over disk Better service distribution C-SCAN One way with fast return Lower service variability N-step-SCAN SCAN of N records at a time Service guarantee FSCAN N-step-SCAN with N = queue Load sensitive size at beginning of SCAN cycle Table 11.3 Disk Scheduling Algorithms © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Control of the scheduling is outside the control of disk management software ◼ Goal is not to optimize disk utilization but to meet other objectives ◼ Short batch jobs and interactive jobs are given higher priority ◼ Provides good interactive response time ◼ Longer jobs may have to wait an excessively long time ◼ A poor policy for database systems © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. track number track number Shortest Service Time First (SSTF) 0 25 50 75 100 125 150 175 199 0 25 50 75 100 125 150 175 199 0 ◼ Select the disk I/O request that requires the least movement of the disk arm from its current position ◼ Always choose the minimum seek time (a) FIFO Time (b) SSTF Time © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. track track number track number 100 125 150 175 199 0 25 50 75 100 125 150 175 199 SCAN 0 25 50 75 100 125 150 175 199 0 25 ◼ Also known as the elevator algorithm (a) FIFO ◼ Arm moves in one direction only ◼ ◼ Time Satisfies all outstanding requests until it reaches the last track in that direction then the direction is reversed Favors jobs whose requests are for tracks nearest to both innermost and outermost tracks and favors the latest-arriving jobs (b) SSTF Time (c) SCAN Time © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. trac 125 150 175 199 track number C-SCAN 0 25 50 75 100 125 150 175 199 track number (Circular SCAN) 0 25 50 75 100 125 150 175 199 ◼ Restricts scanning to one direction only ◼ When the last track has been visited in one direction, the arm is returned to the opposite end of the disk and the scan begins again (b) SSTF Time (c) SCAN Time (d) C-SCAN © 2017 Pearson Education, Inc., Hoboken, NJ. All rightsof reserved. Figure 11.7 Comparison Disk Scheduling Time Algorithms (see Table 11.3) ◼ Segments the disk request queue into subqueues of length N ◼ Subqueues are processed one at a time, using SCAN ◼ While a queue is being processed new requests must be added to some other queue ◼ If fewer than N requests are available at the end of a scan, all of them are processed with the next scan © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Uses two subqueues ◼ When a scan begins, all of the requests are in one of the queues, with the other empty ◼ During scan, all new requests are put into the other queue ◼ Service of new requests is deferred until all of the old requests have been processed © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Redundant Array of Independent Disks ◼ Consists of seven levels, zero through six RAID is a set of physical disk drives viewed by the operating system as a single logical drive Redundant disk capacity is used to store parity information, which guarantees data recoverability in case of a disk failure © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Design architectures share three characteristics: Data are distributed across the physical drives of an array in a scheme known as striping ◼ The term was originally coined in a paper by a group of researchers at the University of California at Berkeley ◼ ◼ The paper outlined various configurations and applications and introduced the definitions of the RAID levels Strategy employs multiple disk drives and distributes data in such a way as to enable simultaneous access to data from multiple drives ◼ Improves I/O performance and allows easier incremental increases in capacity ◼ The unique contribution is to address effectively the need for redundancy ◼ Makes use of stored parity information that enables the recovery of data lost due to a disk failure © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Category Striping Mirroring Level Description 0 Nonredundant Disks Data availability required Lower than single disk Very high 2N Higher than RAID 2, 3, 4, or 5; lower than RAID 6 Higher than single disk for read; similar to single disk for write Very high for both read and write Up to twice that of a single disk for read; similar to single disk for write Highest of all listed alternatives Approximately twice that of a single disk Highest of all listed alternatives Approximately twice that of a single disk Mirrored 2 Redundant via Hamming code N+m 3 Bit-interleaved parity N+1 Parallel access Independent access Much higher than single disk; comparable to RAID 3, 4, or 5 Much higher than single disk; comparable to RAID 2, 4, or 5 N+1 Much higher than single disk; comparable to RAID 2, 3, or 5 5 Block-interleaved distributed parity N+1 Much higher than single disk; comparable to RAID 2, 3, or 4 6 Block-interleaved dual distributed parity N+2 Highest of all listed alternatives Block-interleaved parity N = number of data disks; Small I/O request rate N 1 4 Large I/O data transfer capacity m proportional to log N © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Similar to RAID 0 for read; significantly lower than single disk for write Similar to RAID 0 for read; lower than single disk for write Similar to RAID 0 for read; lower than RAID 5 for write Similar to RAID 0 for read; significantly lower than single disk for write Similar to RAID 0 for read; generally lower than single disk for write Similar to RAID 0 for read; significantly lower than RAID 5 for write Table 11.4 RAID Levels (Page 498 in textbook) strip 0 strip 1 strip 2 strip 3 strip 4 strip 5 strip 6 strip 7 strip 8 strip 9 strip 10 strip 11 strip 12 strip 13 strip 14 strip 15 (a) RAID 0 (non-redundant) strip 0 strip 1 strip 2 strip 3 strip 0 strip 1 strip 2 strip 3 strip 4 strip 5 strip 6 strip 7 strip 4 strip 5 strip 6 strip 7 strip 8 strip 9 strip 10 strip 11 strip 8 strip 9 strip 10 strip 11 strip 12 strip 13 strip 14 strip 15 strip 12 strip 13 strip 14 strip 15 b2 b3 f0(b) f1(b) f2(b) (b) RAID 1 (mirrored) b0 b1 (c) RAID 2 (redundancy through Hamming code) Figure 11.8 RAID Levels (page 1 of 2) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. b0 b1 b2 b3 P(b) (d) RAID 3 (bit-interleaved parity) block 0 block 1 block 2 block 3 P(0-3) block 4 block 5 block 6 block 7 P(4-7) block 8 block 9 block 10 block 11 P(8-11) block 12 block 13 block 14 block 15 P(12-15) (e) RAID 4 (block-level parity) block 0 block 1 block 2 block 3 P(0-3) block 4 block 5 block 6 P(4-7) block 7 block 8 block 9 P(8-11) block 10 block 11 block 12 P(12-15) block 13 block 14 block 15 P(16-19) block 16 block 17 block 18 block 19 (f) RAID 5 (block-level distributed parity) block 0 block 1 block 2 block 3 P(0-3) Q(0-3) block 4 block 5 block 6 P(4-7) Q(4-7) block 7 block 8 block 9 P(8-11) Q(8-11) block 10 block 11 block 12 P(12-15) Q(12-15) block 13 block 14 block 15 (g) RAID 6 (dual redundancy) Figure 11.8 RAID Levels (page 2 of 2) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. RAID Level 0 ◼ Not a true RAID because it does not include redundancy to improve performance or provide data protection ◼ User and system data are distributed across all of the disks in the array ◼ Logical disk is divided into strips strip 0 strip 1 strip 2 strip 3 strip 4 strip 5 strip 6 strip 7 strip 8 strip 9 strip 10 strip 11 strip 12 strip 13 strip 14 strip 15 (a) RAID 0 (non-redundant) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. RAID Level 1 ◼ Redundancy is achieved by the simple expedient of duplicating all the data ◼ There is no “write penalty” strip 0 strip 1 strip 2 strip 3 strip 4 strip 5 strip 6 strip 8 strip 9 strip 10 When a drive fails the data may still be from the second drive strip accessed 11 strip 12 strip 13 strip 14 strip 15 ◼ strip 7 ◼ Principal disadvantage is the cost (a) RAID 0 (non-redundant) strip 0 strip 1 strip 2 strip 3 strip 0 strip 1 strip 2 strip 3 strip 4 strip 5 strip 6 strip 7 strip 4 strip 5 strip 6 strip 7 strip 8 strip 9 strip 10 strip 11 strip 8 strip 9 strip 10 strip 11 strip 12 strip 13 strip 14 strip 15 strip 12 strip 13 strip 14 strip 15 (b) RAID 1 (mirrored) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. strip 4 strip 5 strip 6 strip 7 strip 8 strip 9 strip 10 strip 11 strip 12 strip 13 strip 14 strip 15 (a) RAID 0 (non-redundant) RAID Level 2 ◼ Makes use of a parallel access technique ◼ Data is 0used strip 3 striping strip ◼ strip 7 strip 4 strip 5 Typically a Hamming code is usedstrip 6 strip 0 strip 1 strip 2 strip 4 strip 5 strip 6 strip 8 strip 9 strip 10 strip 12 strip 13 strip 14 ◼ Effective strip 15 b2 b3 strip 11 strip 8 strip 1 strip 9 strip 2 strip 10 choice in 14 strip 12in an environment strip 13 strip which many disk errors occur (b) RAID 1 (mirrored) b0 b1 f0(b) f1(b) (c) RAID 2 (redundancy through Hamming code) Figure 11.8 RAID Levels (page 1 of 2) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. f2(b) RAID Level 3 b0 ◼ Requires only a single redundant disk, no matter how large the disk array ◼ Employs parallel access, with data distributed in small strips ◼ Can achieve very high data transfer rates b1 (d) RAID 3 (bit-interleaved parity) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. b2 b3 P(b) RAID Level 4 b0 ◼ Makes use of an independent access technique ◼ A bit-by-bit parity strip is calculated across corresponding strips on each data disk, b2 b3 P(b) and the parity bits are stored in the corresponding strip on the parity disk ◼ Involves a write penalty when an I/O write request of small size is performed b1 (d) RAID 3 (bit-interleaved parity) block 0 block 1 block 2 block 3 P(0-3) block 4 block 5 block 6 block 7 P(4-7) block 8 block 9 block 10 block 11 P(8-11) block 12 block 13 block 14 block 15 P(12-15) (e) RAID 4 (block-level parity) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. block 0 block 1 block 4 block 5 RAID Level 5 block 8 block 9 block 12 block 13 block 2 P(0-3) block 11 P(8-11) Similar to RAID-4 but distributes the block 7 P(4-7) parity bits across all disks ◼ block 6 block 10 ◼ block 3 Typical allocation is a round-robin block 15 P(12-15) scheme block 14 Has the characteristic that the loss of any one disk does not result in data loss ◼ (e) RAID 4 (block-level parity) block 0 block 1 block 2 block 3 P(0-3) block 4 block 5 block 6 P(4-7) block 7 block 8 block 9 P(8-11) block 10 block 11 block 12 P(12-15) block 13 block 14 block 15 P(16-19) block 16 block 17 block 18 block 19 (f) RAID 5 (block-level distributed parity) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. (e) RAID 4 (block-level parity) Two different parity calculations are block 2 carriedblock P(0-3) out 3and stored in separate blocks block 6 on different P(4-7) disks block 7 ◼ block 0 block 1 RAID Level 6 block 4 block 5 block 8 block 9 block 12 P(12-15) P(16-19) block 16 P(8-11) blockextremely 10 block ◼ Provides high11data availability block 13 block 14 block 15 ◼ Incurs a substantial write penalty block 17 block 18 block 19 because each write affects two parity blocks (f) RAID 5 (block-level distributed parity) block 0 block 1 block 2 block 3 P(0-3) Q(0-3) block 4 block 5 block 6 P(4-7) Q(4-7) block 7 block 8 block 9 P(8-11) Q(8-11) block 10 block 11 block 12 P(12-15) Q(12-15) block 13 block 14 block 15 (g) RAID 6 (dual redundancy) Figure 11.8 RAID Levels (page 2 of 2) © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Cache memory is used to apply to a memory that is smaller and faster than main memory and that is interposed between main memory and the processor ◼ Reduces average memory access time by exploiting the principle of locality ◼ Disk cache is a buffer in main memory for disk sectors ◼ Contains a copy of some of the sectors on the disk If YES The request is satisfied via the cache If NO The requested sector is read into the disk cache from the disk When an I/O request is made for a particular sector, a check is made to determine if the sector is in the disk cache © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Most commonly used algorithm that deals with the design issue of replacement strategy ◼ The block that has been in the cache the longest with no reference to it is replaced ◼ A stack of pointers reference the cache ◼ Most recently referenced block is on the top of the stack ◼ When a block is referenced or brought into the cache, it is placed on the top of the stack © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ The block that has experienced the fewest references is replaced ◼ A counter is associated with each block ◼ Counter is incremented each time block is accessed ◼ When replacement is required, the block with the smallest count is selected © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. New Section Old Section MRU LRU Re-reference; count unchanged Re-reference; count := count + 1 Miss (new block brought in) count := 1 (a) FIFO New Section Middle Section Old Section MRU LRU (b) Use of three sections Figure 11.9 Frequency-Based Replacement © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. 60 Disk cache miss rate (%) 50 VAX UNIX 40 30 20 IBM MVS 10 IBM SVS 0 0 5 10 15 20 25 30 Cache size (megabytes) Figure 11.10 Some Disk Cache Performance Results Using LRU © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. 70 60 Disk cache miss rate (%) 50 IBM VM 40 30 IBM MVS 20 VAX UNIX 10 0 0 5 10 15 20 25 30 Cache size (megabytes) Figure 11.11 Disk Cache Performance Using Frequency-Based Replacement [ROBI90] © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. File Subsystem Buffer Cache Character Block Device Drivers Figure 11.12 UNIX I/O Structure © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. UNIX Buffer Cache ◼ Is essentially a disk cache ◼ ◼ The data transfer between the buffer cache and the user process space always occurs using DMA ◼ ◼ ◼ I/O operations with disk are handled through the buffer cache Does not use up any processor cycles Does consume bus cycles Three lists are maintained: ◼ Free list ◼ ◼ Device list ◼ ◼ List of all slots in the cache that are available for allocation List of all buffers currently associated with each disk Driver I/O queue ◼ List of buffers that are actually undergoing or waiting for I/O on a particular device © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Hash Pointers Buffer Cache Free List Pointers Hash Table Device#, Block# Free List Pointer Figure 11.13 UNIX Buffer Cache Organization © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Used by character oriented devices Terminals and printers Either written by the I/O device and read by the process or vice versa Producer/consumer model is used Character queues may only be read once As each character is read, it is effectively destroyed © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Is simply DMA between device and process space ◼ Is always the fastest method for a process to perform I/O ◼ Process is locked in main memory and cannot be swapped out ◼ I/O device is tied up with the process for the duration of the transfer making it unavailable for other processes © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Unbuffered I/O Buffer Cache Disk drive X X Tape drive X X Character Queue Terminals X Communication lines X Printers X Table 11.5 Device I/O in UNIX © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. X ◼ Very similar to other UNIX implementation ◼ Associates a special file with each I/O device driver ◼ Block, character, and network devices are recognized ◼ Default disk scheduler in Linux 2.4 is the Linux Elevator For Linux 2.6 the Elevator algorithm has been augmented by two additional algorithms: • The deadline I/O scheduler • The anticipatory I/O scheduler © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Maintains a single queue for disk read and write requests and performs both sorting and merging functions on the queue ◼ When a new request is added to the queue, four operations are considered in order: ◼ If the request is to the same on-disk sector or an immediately adjacent sector to a pending request in the queue, then the existing request and the new request are merged into one request ◼ If a request in the queue is sufficiently old, the new request is inserted at the tail of the queue ◼ If there is a suitable location, the new request is inserted in sorted order ◼ If there is no suitable location, the new request is placed at the tail of the queue © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Two problems manifest themselves with the elevator scheme: ◼ ◼ ◼ A distant block request can be delayed for a substantial time because the queue is dynamically updated A stream of write requests can block a read request for a considerable time, and thus block a process To overcome these problems, a new deadline I/O scheduler was developed in 2002 ◼ ◼ ◼ This scheduler makes use of two pairs of queues In addition to each incoming request being placed in a sorted elevator queue as before, the same request is placed at the tail of a read FIFO queue for a read request or a write FIFO queue for a write request When a request is satisfied, it is removed from the head of the sorted queue and also from the appropriate FIFO queue ◼ However, when the item at the head of one of the FIFO queues becomes older than its expiration time, then the scheduler next dispatches from that FIFO queue, taking the expired request, plus the next few requests from the queue ◼ As each request is dispatched, it is also removed from the sorted queue © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Elevator and deadline scheduling can be counterproductive if there are numerous synchronous read requests ◼ In Linux, the anticipatory scheduler is superimposed on the deadline scheduler ◼ When a read request is dispatched, the anticipatory scheduler causes the scheduling system to delay ◼ There is a good chance that the application that issued the last read request will issue another read request to the same region of the disk ◼ That request will be serviced immediately ◼ Otherwise the scheduler resumes using the deadline scheduling algorithm © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. The NOOP Scheduler This is the simplest among Linux I/O schedulers It is a minimal scheduler that inserts I/O requests into a FIFO queue and uses merging Its main uses include nondisk-based block devices such as memory devices, and specialized software or hardware environments that do their own scheduling and need only minimal support in the kernel © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Completely Fair Queuing I/O Scheduler (CFQ) ◼ Was developed in 2003 ◼ Is the default I/O scheduler in Linux ◼ The CFQ scheduler guarantees a fair allocation of the disk I/O bandwidth among all processes ◼ It maintains per process I/O queues ◼ ◼ ◼ ◼ Each process is assigned a single queue Each queue has an allocated timeslice Requests are submitted into these queues and are processed in round robin When the scheduler services a specific queue, and there are no more requests in that queue, it waits in idle mode for a predefined time interval for new requests, and if there are no requests, it continues to the next queue © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ For Linux 2.4 and later there is a single unified page cache for all traffic between disk and main memory ◼ Benefits: ◼ When it is time to write back dirty pages to disk, a collection of them can be ordered properly and written out efficiently ◼ Because of the principle of temporal locality, pages in the page cache are likely to be referenced again before they are flushed from the cache, thus saving a disk I/O operation © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. I/O Manager Cache Manager File System Drivers Network Drivers Hardware Device Drivers Figure 11.15 Windows I/O Manager © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Cache Manager ◼ ◼ Maps regions of files into kernel virtual memory and then relies on the virtual memory manager to copy pages to and from the files on disk File System Drivers ◼ ◼ Sends I/O requests to the software drivers that manage the hardware device adapter © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Network Drivers ◼ Windows includes integrated networking capabilities and support for remote file systems ◼ The facilities are implemented as software drivers Hardware Device Drivers ◼ The source code of Windows device drivers is portable across different processor types Windows offers two modes of I/O operation Asynchronous Is used whenever possible to optimize application performance An application initiates an I/O operation and then can continue processing while the I/O request is fulfilled © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. Synchronous The application is blocked until the I/O operation completes ▪ Windows provides five different techniques for signaling I/O completion: 1 • Signaling the file object 2 • Signaling an event object 3 • Asynchronous procedure call 4 • I/O completion ports 5 • Polling © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Windows supports two sorts of RAID configurations: Hardware RAID Software RAID Separate physical disks combined into one or more logical disks by the disk controller or disk storage cabinet hardware Noncontiguous disk space combined into one or more logical partitions by the fault-tolerant software disk driver, FTDISK © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Volume Shadow Copies ◼ Efficient way of making consistent snapshots of volumes so they can be backed up ◼ Also useful for archiving files on a per-volume basis ◼ Implemented by a software driver that makes copies of data on the volume before it is overwritten © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Volume Encryption ◼ Windows uses BitLocker to encrypt entire volumes ◼ More secure than encrypting individual files ◼ Allows multiple interlocking layers of security Summary ◼ ◼ I/O devices Organization of the I/O function ◼ The evolution of the I/O function ◼ Direct memory access ◼ Operating system design issues ◼ Design objectives ◼ Logical structure of the I/O function ◼ I/O Buffering ◼ Single/double/circular buffer ◼ The utility of buffering ◼ Disk scheduling ◼ Disk performance parameters ◼ Disk scheduling policies ◼ Raid © 2017 Pearson Education, Inc., Hoboken, NJ. All rights reserved. ◼ Raid levels 0 – 6 ◼ Disk cache ◼ Design and performance considerations ◼ UNIX SVR4 I/O ◼ Buffer cache ◼ Character queue ◼ Unbuffered I/O ◼ UNIX devices ◼ Linux I/O ◼ Disk scheduling ◼ Linux page cache ◼ Windows I/O ◼ Basic I/O facilities ◼ Asynchronous and Synchronous I/O ◼ Software RAID ◼ Volume shadow copies/encryption CHAPTER 7 1. Introduction to Information Security 2. Unintentional Threats to Information Systems 3. Deliberate Threats to Information Systems 4. What Organizations Are Doing to Protect Information Resources 5. Information Security Controls 1. Identify the five factors that contribute to the increasing vulnerability of information resources and specific examples of each factor. 2. Compare and contrast human mistakes and social engineering, along with specific examples of each one. 3. Discuss the 10 types of deliberate attacks. 4. Describe the three risk mitigation strategies and examples of each one in the context of owning a home. 5. Identify the three major types of controls that organizations can use to protect their information resources along with an example of each one. Opening Case • The St. Louis Cardinals Investigated for Hacking the Houston Astros 1. Describe how the Cardinals apparently were able to gain access to the Astros’ computer system. 2. What lessons should the Astros learn from this security breach? 7.1 Introduction to Information Security Information Security Threat Exposure Vulnerability Five Key Factors Increasing Vulnerability • Cybercrime • • • • • Five Key Factors Increasing Vulnerability 1. Today’s interconnected, interdependent, wirelessly networked business environment 2. Smaller, faster, cheaper computers and storage devices 3. Decreasing skills necessary to be a computer hacker 4. International organized crime taking over cybercrime 5. Lack of management support 7.2 Unintentional Threats to Information Systems • Human Errors • Social Engineering Figure 7.1 Security Threats Human Errors • Higher employee levels = higher levels of security risk • Most Dangerous Employees • Human Mistakes Dangerous Employees • Two organizational areas pose the greatest risk – Human Resources – Information Systems • Janitors and Guards Frequently Overlooked Human Mistakes • Carelessness with laptops • Carelessness with computing devices • Opening questionable e-mails • Careless Internet surfing • Poor password selection and use • Carelessness with one’s office Human Mistakes (continued) • Carelessness using unmanaged devices • Carelessness with discarded equipment • Careless monitoring of environmental hazards Table 7.1: Human Mistakes Social Engineering • Social Engineering: – an attack in which the perpetrator uses social skills to trick or manipulate legitimate employees into providing confidential company information such as passwords. 7.3 Deliberate Threats to Information Systems 1. Espionage or Trespass 2. Information Extortion 3. Sabotage or Vandalism 4. Theft of Equipment or Information 5. Identity Theft 6. Compromises to Intellectual Property 7.3 Deliberate Threats to Information Systems (continued) 7. Software Attacks 8. Alien Software 9. Supervisory Control and Data Acquisition Attacks 10. Cyberterrorism and Cyberwarfare 6. Compromises to Intellectual Property • • • • Intellectual Property Trade Secret Patent Copyright 7. Software Attacks: Three Categories 1. Remote Attacks Requiring User Action – Virus – Worm – Phishing Attack – Spear Phishing 7. Software Attacks: Three Categories (continued) 2. Remote Attacks Needing No User Action – Denial-of-Service Attack – Distributed Denial-of-Service Attack 7. Software Attacks: Three Categories (continued) 3. Attacks by a Programmer Developing a System – Trojan Horse – Back Door – Logic bomb ABOUT BUSINESS 7.1 • Ransomeware 1. Why is ransomware more than a nuisance? 2. Are your digital files adequately backed up? Why or why not? 8. Alien Software • • • • • Adware Spyware Spamware Spam Cookies 7.4 • • • • What Organizations Are Doing to Protect Information Resources Risk Risk Management Risk Analysis Risk Mitigation Table 7.3: The Difficulties in Protecting Information Resources ABOUT BUSINESS 7.2 • Catching a Hacker 1. Why did the FBI need to “argue with law enforcement officials in various countries”? 2. Describe the diff iculties that investigators encounter in bringing cybercriminals to justice. Can you propose any additional strategies they should consider? Risk Management Three Processes of Risk Management: 1. risk analysis 2. risk mitigation 3. controls evaluation Risk Analysis Three Steps of Risk Analysis 1. assessing the value of each asset being protected 2. estimating the probability that each asset will be compromised 3. comparing the probable costs of the asset’s being compromised with the costs of protecting that asset Risk Mitigation • Risk Acceptance • Rick Limitation • Risk Transference 7.5 • • • • • Information Security Controls Physical Controls Access Controls Communications Controls Business Continuity Planning Information Systems Auditing Figure 7.2: Where Defense Mechanisms are Located. Physical Controls • • • • Walls Doors Fencing Gates • • • • Locks Badges Guards Alarm Systems Access Controls • Authentication • Authorization – Something – Something – Something – Something the the the the user user user user is (Biometrics) has does knows ABOUT BUSINESS 7.3 • Trustev: Helping to Prevent Credit Card Fraud 1. Describe how Trustev’s authentication method differs from other authentication methods. 2. What are potential disadvantages with Trustev’s authentication method? Communications Controls • Firewall – Demilitarized Zone (DMZ) • • • • • Anti-malware Systems Whitelisting Blacklisting Encryption Virtual Private Network (VPN) Communications Controls (Continued) • Transport Layer Security (formerly called Secure Socket Layer) • Employee Monitoring Systems Figure 7.3: (a) Basic Firewall for Home Computer. (b) Organization with Two Firewalls and Demilitarized Zone Figure 7.4: How Public-key Encryption Works Figure 7.5: How Digital Certificates Work. Figure 7.6: Virtual Private Network (VPN) and Tunneling Business Continuity Planning • Business Continuity • Business Continuity Plan Information Systems Auditing • Internal Audits • External Audits • Three Categories of IS auditing procedures Three Categories of IS auditing procedures: • Auditing Around the Computer • Auditing Through the Computer • Auditing With the Computer ONLINE ACCESS Thank you for purchasing a new copy of Operating Systems: Internals and Design Principles, Eighth Edition. Your textbook includes six months of prepaid access to the book’s Premium Content. This prepaid subscription provides you with full access to the following student support areas: r
Animations and Videonotes r
Supplemental Chapters r
Supplemental Appendices r
Supplemental homework problems with solutions Use a coin to scratch off the coating and reveal your student access code. Do not use a knife or other sharp object as it may damage the code. To access the Operating Systems: Internals and Design Principles, Eighth Edition, Premium Content for the first time, you will need to register online using a computer with an Internet connection and a web browser. The process takes just a couple of minutes and only needs to be completed once. 1. Go to http://www.pearsonhighered.com/stallings/ 2. Click on the Premium Content. 3. Click on the Register button. 4. On the registration page, enter your student access code* found beneath the scratchoff panel. Do not type the dashes. You can use lower- or uppercase. 5. Follow the on-screen instructions. If you need help at any time during the online registration process, simply click the Need Help? icon. 6. Once your personal Login Name and Password are confirmed, you can begin using the Operating Systems: Internals and Design Principles Premium Content Website! To log in after you have registered: You only need to register for this Premium Content once. After that, you can log in any time at http://www.pearsonhighered.com/stallings/ by providing your Login Name and Password when prompted. *Important: The access code can only be used once. This subscription is valid for six months upon activation and is not transferable. If this access code has already been revealed, it may no longer be valid. If this is the case, you can purchase a subscription by going to http://www.pearsonhighered.com/stallings/ and following the on-screen instructions. OPERATING SYSTEMS INTERNALS AND DESIGN PRINCIPLES This page intentionally left blank OPERATING SYSTEMS INTERNALS AND DESIGN PRINCIPLES EIGHTH EDITION William Stallings Boston Columbus Indianapolis New York San Francisco Upper Saddle River Amsterdam Cape Town Dubai London Madrid Milan Munich Paris Montréal Toronto Delhi Mexico City São Paulo Sydney Hong Kong Seoul Singapore Taipei Tokyo Editorial Director, ECS: Marcia Horton Executive Editor: Tracy Johnson (Dunkelberger) Editorial Assistant: Jenah Blitz-Stoehr Director of Marketing: Christy Lesko Marketing Manager: Yez Alayan Marketing Assistant: Jon Bryant Director of Program Management: Erin Gregg Program Management-Team Lead: Scott Disanno Program Manager: Carole Snyder Project Management-Team Lead: Laura Burgess Project Manager: Robert Engelhardt Procurement Specialist: Linda Sager Cover Designer: Bruce Kenselaar Permissions Supervisor: Michael Joyce Permissions Administrator: William Opaluch Director, Image Asset Services: Annie Atherton Manager,Visual Research: Karen Sanatar Cover Art: © Fixer00/Shutterstock Media Project Manager: Renata Butera Full-Service Project Management: Integra Software Services Pvt. Ltd. /Allan Rayer Credits and acknowledgments borrowed from other sources and reproduced, with permission, in this textbook appear on the appropriate page within text. Microsoft® and Windows® are registered trademarks of the Microsoft Corporation in the United States of America and other countries. Screenshots and icons reprinted with permission from the Microsoft Corporation. This book is not sponsored or endorsed by or affiliated with the Microsoft Corporation. Copyright © 2015, 2012, 2009 Pearson Education, Inc., publishing as Prentice Hall, one Lake Street, Upper Saddle River, New Jersey, 07458. All rights reserved. Printed in the United States of America. This publication is protected by Copyright, and permission should be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise. To obtain permission(s) to use material from this work, please submit a written request to Pearson Education, Inc., Permissions Department, One Lake Street, Upper Saddle River, New Jersey 07458, or you may fax your request to 201-236-3290. Many of the designations by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed in initial caps or all caps. Library of Congress Cataloging-in-Publication Data on File 10 9 8 7 6 5 4 3 2 1 ISBN 10: 0-13-380591-3 ISBN 13: 978-0-13-380591-8 For Tricia This page intentionally left blank CONTENTS Online Resources xiii VideoNotes xv Preface xvii About the Author xxvii Chapter 0 Guide for Readers and Instructors 1 0.1 0.2 0.3 0.4 Outline of this Book 2 Example Systems 2 A Roadmap for Readers and Instructors 3 Internet and Web Resources 4 PART 1 BACKGROUND 7 Chapter 1 Computer System Overview 7 1.1 Basic Elements 8 1.2 Evolution of the Microprocessor 10 1.3 Instruction Execution 10 1.4 Interrupts 13 1.5 The Memory Hierarchy 24 1.6 Cache Memory 27 1.7 Direct Memory Access 31 1.8 Multiprocessor and Multicore Organization 32 1.9 Recommended Reading 36 1.10 Key Terms, Review Questions, and Problems 36 1A Performance Characteristics of Two-Level Memories 39 Chapter 2 Operating System Overview 46 2.1 Operating System Objectives and Functions 47 2.2 The Evolution of Operating Systems 52 2.3 Major Achievements 61 2.4 Developments Leading to Modern Operating Systems 70 2.5 Fault Tolerance 73 2.6 OS Design Considerations for Multiprocessor and Multicore 76 2.7 Microsoft Windows Overview 79 2.8 Traditional UNIX Systems 86 2.9 Modern UNIX Systems 89 2.10 Linux 91 2.11 Android 96 2.12 Recommended Reading and Animations 102 2.13 Key Terms, Review Questions, and Problems 103 vii viii CONTENTS PART 2 PROCESSES 105 Chapter 3 Process Description and Control 105 3.1 What Is a Process? 107 3.2 Process States 109 3.3 Process Description 124 3.4 Process Control 133 3.5 Execution of the Operating System 139 3.6 UNIX SVR4 Process Management 142 3.7 Summary 147 3.8 Recommended Reading and Animations 147 3.9 Key Terms, Review Questions, and Problems 148 Chapter 4 Threads 152 4.1 Processes and Threads 153 4.2 Types of Threads 159 4.3 Multicore and Multithreading 166 4.4 Windows 8 Process and Thread Management 171 4.5 Solaris Thread and SMP Management 178 4.6 Linux Process and Thread Management 182 4.7 Android Process and Thread Management 186 4.8 Mac OS X Grand Central Dispatch 190 4.9 Summary 193 4.10 Recommended Reading 193 4.11 Key Terms, Review Questions, and Problems 194 Chapter 5 Concurrency: Mutual Exclusion and Synchronization 199 5.1 Principles of Concurrency 202 5.2 Mutual Exclusion: Hardware Support 210 5.3 Semaphores 214 5.4 Monitors 227 5.5 Message Passing 233 5.6 Readers/Writers Problem 240 5.7 Summary 244 5.8 Recommended Reading and Animations 245 5.9 Key Terms, Review Questions, and Problems 246 Chapter 6 Concurrency: Deadlock and Starvation 259 6.1 Principles of Deadlock 260 6.2 Deadlock Prevention 269 6.3 Deadlock Avoidance 271 6.4 Deadlock Detection 277 6.5 An Integrated Deadlock Strategy 279 6.6 Dining Philosophers Problem 280 6.7 UNIX Concurrency Mechanisms 282 6.8 Linux Kernel Concurrency Mechanisms 286 CONTENTS 6.9 6.10 6.11 6.12 6.13 6.14 Solaris Thread Synchronization Primitives 293 Windows 7 Concurrency Mechanisms 295 Android Interprocess Communication 299 Summary 300 Recommended Reading and Animations 301 Key Terms, Review Questions, and Problems 302 PART 3 MEMORY 309 Chapter 7 Memory Management 309 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7A Memory Management Requirements 310 Memory Partitioning 314 Paging 325 Segmentation 328 Summary 330 Recommended Reading and Animations 330 Key Terms, Review Questions, and Problems 330 Loading and Linking 333 Chapter 8 Virtual Memory 340 8.1 Hardware and Control Structures 341 8.2 Operating System Software 358 8.3 UNIX and Solaris Memory Management 377 8.4 Linux Memory Management 383 8.5 Windows Memory Management 386 8.6 Android Memory Management 389 8.7 Summary 389 8.8 Recommended Reading and Animations 390 8.9 Key Terms, Review Questions, and Problems 391 PART 4 SCHEDULING 397 Chapter 9 Uniprocessor Scheduling 397 9.1 Types of Processor Scheduling 398 9.2 Scheduling Algorithms 402 9.3 Traditional UNIX Scheduling 424 9.4 Summary 426 9.5 Recommended Reading and Animations 427 9.6 Key Terms, Review Questions, and Problems 428 Chapter 10 Multiprocessor, Multicore, and Real-Time Scheduling 432 10.1 Multiprocessor and Multicore Scheduling 433 10.2 Real-Time Scheduling 446 10.3 Linux Scheduling 461 10.4 UNIX SVR4 Scheduling 465 10.5 UNIX FreeBSD Scheduling 466 ix x CONTENTS 10.6 10.7 10.8 10.9 Windows Scheduling 470 Summary 472 Recommended Reading 473 Key Terms, Review Questions, and Problems 473 PART 5 INPUT/OUTPUT AND FILES 477 Chapter 11 I/O Management and Disk Scheduling 477 11.1 I/O Devices 478 11.2 Organization of the I/O Function 480 11.3 Operating System Design Issues 483 11.4 I/O Buffering 486 11.5 Disk Scheduling 489 11.6 RAID 496 11.7 Disk Cache 505 11.8 UNIX SVR4 I/O 509 11.9 Linux I/O 512 11.10 Windows I/O 515 11.11 Summary 517 11.12 Recommended Reading and Animations 518 11.13 Key Terms, Review Questions, and Problems 519 Chapter 12 12.1 12.2 12.3 12.4 12.5 12.6 12.7 12.8 12.9 12.10 12.11 12.12 12.13 12.14 File Management 522 Overview 523 File Organization and Access 529 B-Trees 533 File Directories 536 File Sharing 541 Record Blocking 542 Secondary Storage Management 544 UNIX File Management 552 Linux Virtual File System 557 Windows File System 561 Android File Management 566 Summary 567 Recommended Reading 568 Key Terms, Review Questions, and Problems 568 PART 6 EMBEDDED SYSTEMS 571 Chapter 13 Embedded Operating Systems 571 13.1 Embedded Systems 572 13.2 Characteristics of Embedded Operating Systems 573 13.3 Embedded Linux 576 13.4 TinyOS 577 13.5 Recommended Reading 587 13.6 Key Terms, Review Questions, and Problems 587 CONTENTS Chapter 14 14.1 14.2 14.3 14.4 14.5 14.6 14.7 14.8 14.9 14.10 14.11 14.12 Virtual Machines 590 Approaches to Virtualization 593 Processor Issues 596 Memory Management 598 I/O Management 600 VMware ESXi 602 Microsoft Hyper-V and Xen Variants 604 Java VM 606 Linux VServer Virtual Machine Architecture 606 Android Virtual Machine 609 Summary 611 Recommended Reading 612 Key Terms, Review Questions, and Problems 613 Chapter 15 15.1 15.2 15.3 15.4 15.5 15.6 15.7 15.8 15.9 15.10 Operating System Security 614 Intruders and Malicious Software 615 Buffer Overflow 619 Access Control 627 UNIX Access Control 635 Operating Systems Hardening 638 Security Maintenance 642 Windows Security 643 Summary 648 Recommended Reading 649 Key Terms, Review Questions, and Problems 649 Chapter 16 Distributed Processing, Client/Server, and Clusters 652 16.1 Client/Server Computing 653 16.2 Distributed Message Passing 664 16.3 Remote Procedure Calls 667 16.4 Clusters 671 16.5 Windows Cluster Server 676 16.6 Beowulf and Linux Clusters 678 16.7 Summary 680 16.8 Recommended Reading 681 16.9 Key Terms, Review Questions, and Problems 682 APPENDICES Appendix A Topics in Concurrency A-1 Appendix B Programming and Operating System Projects B-1 References 685 Credits 699 Index 703 xi This page intentionally left blank ONLINE CHAPTERS AND APPENDICES1 Chapter 17 Network Protocols 17.1 The Need for a Protocol Architecture 17-4 17.2 The TCP/IP Protocol Architecture 17-9 17.3 Sockets 17-19 17.4 Linux Networking 17-26 17.5 Summary 17-29 17.6 Recommended Reading and Web Sites 17-30 17.7 Key Terms, Review Questions, and Problems 17-31 17A The Trivial File Transfer Protocol 17-42 Chapter 18 Distributed Process Management 18.1 Process Migration 18-2 18.2 Distributed Global States 18-14 18.3 Distributed Mutual Exclusion 18-22 18.4 Distributed Deadlock 18-41 18.5 Summary 18-59 18.6 Recommended Reading 18-60 18.7 Key Terms, Review Questions, and Problems 18-61 Chapter 19 Overview of Probability and Stochastic Processes 19.1 Probability 19-2 19.2 Random Variables 19-10 19.3 Elementary Concepts of Stochastic Processes 19-19 19.4 Recommended Reading and Web Sites 19-34 19.5 Key Terms, Review Questions, and Problems 19-35 Chapter 20 Queueing Analysis 20.1 How Queues Behave—A Simple Example 20-3 20.2 Why Queuing Analysis? 20-8 20.3 Queueing Models 20-12 20.4 Single-Server Queues 20-24 20.5 Multiserver Queues 20-28 20.6 Examples 20-30 20.7 Queues with Priorities 20-38 20.8 Networks of Queues 20-40 20.9 Other Queueing Models 20-46 20.10 Estimating Model Parameters 20-48 20.11 Recommended Reading and Web Sites 20-53 20.12 Key Terms, Review Questions, and Problems 20-54 Programming Project One Developing a Shell Programming Project Two The HOST Dispatcher Shell 1 Online chapters, appendices, and other documents are Premium Content, available via the access card at the front of this book. xiii xiv ONLINE CHAPTERS AND APPENDICES Appendix C Topics in Computer Organization C-1 Appendix D Object-Oriented Design D-1 Appendix E Amdahl’s Law E-1 Appendix F Hash Tables F-1 Appendix G Response Time G-1 Appendix H Queueing System Concepts H-1 Appendix I The Complexity of Algorithms I-1 Appendix J Disk Storage Devices J-1 Appendix K Cryptographic Algorithms K-1 Appendix L Standards Organizations L-1 Appendix M Sockets: A Programmer’s Introduction M-1 Appendix N The International Reference Alphabet N-1 Appendix O BACI: The Ben-Ari Concurrent Programming System O-1 Appendix P Procedure Control P-1 Appendix Q Ecos Q-1 Glossary VideoNote VIDEONOTES Locations of VideoNotes http://www.pearsonhighered.com/stallings Chapter 5 Concurrency: Mutual Exclusion and Synchronization 199 5.1 Illustration of Mutual Exclusion 208 5.2 Hardware Support for Mutual Exclusion 212 5.3 A Definition of Semaphore Primitives 216 5.4 A Definition of Binary Semaphore Primitives 217 5.6 Mutual Exclusion Using Semaphores 219 5.9 An Incorrect Solution to the Infinite-Buffer Producer/Consumer Problem Using Binary Semaphores 222 5.10 A Correct Solution to the Infinite-Buffer Producer/Consumer Problem Using Binary Semaphores 224 5.11 A Solution to the Infinite-Buffer Producer/Consumer Problem Using Semaphores 225 5.13 A Solution to the Bounded-Buffer Producer/Consumer Problem Using Semaphores 226 5.14 Two Possible Implementations of Semaphores 227 5.16 A Solution to the Bounded-Buffer Producer/Consumer Problem Using a Monitor 230 5.17 Bounded-Buffer Monitor Code for Mesa Monitor 232 5.20 Mutual Exclusion Using Messages 238 5.21 A Solution to the Bounded-Buffer Producer/Consumer Problem Using Messages 239 5.22 A Solution to the Readers/Writers Problem Using Semaphores: Readers Have Priority 241 5.23 A Solution to the Readers/Writers Problem Using Semaphores: Writers Have Priority 243 5.24 A Solution to the Readers/Writers Problem Using Message Passing 244 5.25 An Application of Coroutines 248 Chapter 6 Concurrency: Deadlock and Starvation 259 6.9 Deadlock Avoidance Logic 276 6.12 A First Solution to the Dining Philosophers Problem 281 6.13 A Second Solution to the Dining Philosophers Problem 282 6.14 A Solution to the Dining Philosophers Problem Using a Monitor 283 6.18 Another Solution to the Dining Philosophers Problem Using a Monitor 307 Chapter 13 Embedded Operating Systems 571 13.6 Condition Variable Example Code 589 xv xvi VIDEONOTES Chapter 18 Distributed Process Management 18.11 Token.Passing Algorithm (for Process Pi) 40 18.14 A Distributed Deadlock Detection Algorithm 50 Appednix A Topics in Concurrency A-1 A.1 Mutual Exclusion Attempts A-3 A.2 Dekker’s Algorithm A-6 A.3 Peterson’s Algorithm for Two Processes A-7 A.5 An Unfair Barbershop A-17 A.6 A Fair Barbershop A-20 PREFACE WHAT’S NEW IN THE EIGHTH EDITION Since the seventh edition of this book was published, the field has seen continued innovations and improvements. In this new edition, I try to capture these changes while maintaining a broad and comprehensive coverage of the entire field. To begin the process of revision, the seventh edition of this book was extensively reviewed by a number of professors who teach the subject and by professionals working in the field. The result is that, in many places, the narrative has been clarified and tightened, and illustrations have been improved. Beyond these refinements to improve pedagogy and user friendliness, the technical content of the book has been updated throughout, to reflect the ongoing changes in this exciting field, and the instructor and student support has been expanded. The most noteworthy changes are as follows:







r
Windows 8: Windows 8 is Microsoft’s latest OS offering for PCs, workstations, and servers, which includes a number of changes to the internal architecture. The new edition provides details on Windows 8 internals in all of the key technology areas covered in this book, including process/thread management, scheduling, memory management, security, file systems, and I/O. r
Android operating system: Android is the fastest growing mobile platform. The real-world constraints and operating environment of mobile devices are quite different from traditional desktop or server computers. It is important for students to learn this new environment. r
Embedded Linux: The use of a minimal version of Linux for embedded systems has grown in popularity. This new edition provides an overview of the key elements of the embedded Linux approach. r
Virtual machines: Server virtualization and other forms of virtual machines are becoming increasingly widespread. A new chapter deals with the operating system design issues for virtual machines. r
Multicore design issues: The dominant computer architecture is now multicore. This raises new OS design issues that are addressed in this new edition. r
I/O standards: The book has been updated to reflect the latest developments, including Thunderbolt. r
Storage hardware: The discussion of storage hardware has been updated and now includes discussion of solid-state drives. r
Fault tolerance: The ACM/IEEE Computer Science Curricula 2013 lists fault tolerance as one of the core topics for an OS course. A new section provides an overview of fault tolerance. xvii xviii PREFACE OBJECTIVES This book is about the concepts, structure, and mechanisms o...
Purchase answer to see full attachment