UNIX Filesystem Administration

This chapter is from the book

UNIX Unleashed, 4th Edition

Learn More Buy

This chapter is from the book

This chapter is from the book 

UNIX Unleashed, 4th Edition

Learn More Buy

Dividing Disk Space Wisely

So, now you have a chain of devices on your machine, all properly attached, named, and ready to go. Many of these devices are likely to be hard disks, whether they are internal or external. The next step to make these disks active and accessible from the system is partitioning. Because partitioning more or less permanently divides disk space, you will need to know a little bit about the final layer between the hardware and the user—the filesystem—so that you can make wise decisions.

Virtual Devices: Partitions

Partitions are the lowest level of disk space organization over which admins have any control. As mentioned before, sectors, tracks, and cylinders all have a preset, fixed size on a given disk. Admins have no way—or real need—to alter these parameters.

The key thing about partitions is that they are independent of any operating system. Every major OS recognizes these low-level disk divisions and addresses each partition as a separate logical device. Information about partitions is kept in a partition table stored in the disk label or volume header of the physical hard disk. This is how disks maintain information about themselves and why they can be transferred from one machine to another without losing the information.

Partitions allow admins to multi-boot systems with various Operating Systems stored on the same hard disk. Only with the release of Red Hat 7.1 could you run an OS out of an existing partition.

Originally, there was a strict one-to-one mapping of partitions to filesystems.¹ Partitions could be considered containers for the filesystems.

Now, you can span one filesystem over multiple partitions with logical volumes (see the later section, "Logical Volumes"). Partitions can now be thought of more as building blocks for system storage rather than simple containers.

What you still can't do is assign multiple filesystems to one partition (without redividing the space). Here, partitions are acting as boundaries; regular filesystems cannot extend past the edges of the partition, but neither can the partition be subdivided.

Because partitions are hardware-level divisions that predated filesystem quotas, they can provide low-level overflow protection. With the right kind of schema, certain partitions can fill up without affecting overall system performance. At the very least, one partition filling up will not directly affect space on another partition.

All flavors of UNIX require a minimum of two partitions: / and swap. In this sort of system, / holds the entire operating system, local software, configurations, local accounts, and logs.

Swap Space

Swap space, usually known simply as swap, allows the Operating System to access raw disk space directly without the intervention of a filesystem. Swap was designed to act as an extension of the system's main memory (RAM), and it allows the system to surmount that memory's size limitations.

The main OS mechanisms that access this space are swapping and paging. Swapping and paging are distinct functions, although the two are often confused—even by more experienced sysadmins.

A computer has one physical linear memory space, some of which is reserved for objects that must remain resident (such as the Operating System, paging tables, and some other critical data structures). Resident elements must always remain in memory so that they are quickly available for use.

Of course, many other processes and objects have no residency requirements. When the Operating System detects that a certain threshold of resource usage has been reached, it looks for ways to switch allocations to processes that need it most.

Swapping is one of the mechanisms used by the OS to manage memory allocations. When a process is swapped, it is written to the swap space, along with all its currently associated data and other state information. The process's slot in main memory is now free for use by another process.

When a process goes into a wait state for a relatively long time, it doesn't actively need CPU time, so it is swapped, along with all its concomitant state information, out of main memory and onto a disk. Conversely, if a process has been resource-starved, the process will be swapped out of disk storage and back into main memory, allowing it access to the CPU. Intensive swapping, called thrashing, indicates that the entire system is severely starved for resources—usually RAM.

Paging also makes use of the swap disk area. Unlike swapping, which stores an entire process along with all its current state information, paging stores only parts of a process's executable code and discards state information. This is a normal memory- management procedure that takes place even on systems with sufficient resources. The Operating System predetermines the size of segments that can be retrieved and kept in main memory; these segments are called pages. So, when a process is paged into memory from disk or swap, or is paged out of memory into swap space, the transfer is done in page-sized increments.

In short, paging is a way to make more memory available to processes than physically exists on the system. Without paging, all parts of a program that might require real memory would have to be accommodated at process initiation. Programs would need to be very tiny (hard to do with complex programs) or would have to run sequentially (inefficient for all concerned).

Aside from process manipulation, swap space is also used to boot into miniroot and to hold memory images if the system crashes.² Because a memory image is literally a dump of the main memory contents, you will need at least as much swap space as RAM (plus a small margin) if you want to be able to examine the file later. In fact, there should be more swap space than RAM. The rule of thumb for sizing swap will be discussed later.

Referring to Partitions—Device Files

One of the two great truisms of system administration is that everything that exists in UNIX is a file.³ This means that a UNIX system "sees" its hardware devices as files. The advantage of this management method is that it allows the Operating System to treat physical disk access like flat⁴ file I/O.

At the most fundamental level, files are nothing more than strings of ones and zeros. So, file I/O is simply moving bits in and out of specific file locations. It doesn't matter to UNIX whether the file goes on to pass the bits to a disk, into memory, or off to Never Never Land; the kernel just shifts bits in and out of the file handle.

Device files are the lowest-level construct that allows the OS to interact with physical devices. Device files are required to create filesystems (see the later section "Local Filesystem Creation").

Devices can be divided into two categories: raw/character devices and block devices.

Raw/character device files are so named because no buffering is done between whatever function is generating (or retrieving) the bytes and the device that is storing them. These device files are most often used for bytewise copy procedures (like those done with dd). Most tape devices have only raw device files.

Conversely, block device files are buffered, meaning that data is transferred in chunks rather than 1 byte at a time. This tends to be a more efficient way to transfer data for most applications.

Note that, under Solaris, hard disks have both a character device file and a block device file. Each is needed at different times. Most daily disk operations will go through the block device, but filesystem creation is done via the raw/character device name.

Remember that one function of partitions is to make a disk accessible to the system as one or more devices. As you might have guessed, the naming schemes outlined in Chapter 2, "Managing Disk Hardware," identify the literal, on-disk partition and device names, not the system path to their interface files.

All standard UNIX device files were originally tidily kept in /dev. Some Red Hat implementations still maintain this system.

Under Solaris and some newer implementations of Red Hat, the contents of /dev tend to be a combination of device files and symbolic links to actual device files stored in different directories. In this sort of system, /dev acts as a sort of organizational redirector. Under Red Hat, look for a directory called /devfs; under Solaris, look for a directory called /devices.

The directories and device files in /dev vary immensely, even between versions of the same Operating System. Table 3.1 shows some of the most common and useful denizens of /dev. Although many items are in slightly different locations or have slightly different names, overall, both Operating Systems have a lot in common. We encourage you to use Table 3.1 as a starting point for your own explorations of your local system's device files.

Table 3.1 Items of Interest under /dev

Lest you think we're done with partitions, remember that we still need to discuss sizing and actual creation mechanisms. But before we do that, we need to consider the next layer up and what goes into it.

Logical Constructs: Filesystems

As mentioned before, you can't use your system's attached disk space (except for swap) until the space is divided and filesystems⁵ are created. But before you go carving up your disks, you should know what components you are arranging space for.

Note that we are not going into the kind of depth that an OS developer or low-level coder might need. This section is designed to give admins the foundational understanding they need to initially get their systems up and reliably running.

Definitions

A filesystem is a logical construct that provides a framework for storing and retrieving information from a random-access storage device.⁶ In the UNIX world, all information is considered to be a file,⁷ so essentially everything enduring on the system is processed, stored, and retrieved through a filesystem.

The OS is directly responsible for maintaining filesystems because they are complex, logical overlays and not simple, low-level space divisions. This also means that filesystems can be far more flexible and broad with regard to the kind of information that they manage internally. Filesystems store quite a bit of meta-information (literally, information about information) about not only their own constructs, but also the files they manage.

UNIX filesystems are generally hierarchical—that is, they consist of multiple directories organized in an inverted tree structure. Inverted tree diagrams, are depicted with the core node—or root—at the top. In Figure 3.1, directories are referred to as "nodes." Nodes that can contain leaf nodes or other child nodes are referred to as "child nodes"; nodes that contain no further subnodes are referred to as "leaf nodes."

Figure 3.1 Simple hierarchical inverted tree diagram.

Every directory contains a reference to itself (".") and to its parent (".."). Even the initial node—or root of the directory structure—has the parent reference; in this special case, the parent reference simply points to itself. In Figure 3.2, you can find a node's parent by going up to the previous tier.

Pathnames specifying the location of a file from the root directory down are called "absolute pathnames." Pathnames that begin with "../", "./", or anything other than "/" are called relative pathnames because they are specified relative to the current directory.

What Lives Where in UNIX Filesystems

Figure 3.3 presents a high-level view of a "standard" UNIX filesystem. Remember that we are presenting only a generalization of what you are likely to find on your system.

Figure 3.2 Hierarchical inverted tree diagram with relation labels.

Figure 3.3 Hierarchical inverted tree diagram—highlights of standard UNIX filesystems.

The root filesystem (/) contains a number of interesting directories:

• /bin—General binaries

• /dev—Device files

• /etc—System configuration files and bootup scripts

• /sbin—System-related binaries

/home can either be its own filesystem or just a subdirectory of /. On Solaris NFS servers, it will often be a subdirectory of /export. /home generally holds the local system's user home directories.

/opt can either be its own filesystem or just a subdirectory of /. It is intended to hold optional system software (for instance, compilers on Solaris systems).

Temporary file storage usually is located in the /tmp directory. This directory may be accessed on most systems as /tmp, /usr/tmp, and /var/tmp. Two of the three references will be actual directories, the third will be a symlink to one of the others. The variety of UNIX determines which reference is the symlink. We recommend that you reconfigure these directories so that only one is an actual directory and the other two are symlinks to it. We further recommend that the actual directory be a distinct, mounted filesystem. Note that many applications use /tmp as temporary "scratch" space. Because this means that users can inadvertently (or deliberately) write large files to /tmp, the separate mount will protect / and /usr from filling up with temporary user files.

/usr is usually in its own filesystem, although it also can be a subdirectory of /. Subdirectories of interest include these:

• bin—Binaries.

• include—Include files for programming.

• lib—Programming libraries.

• local—Locally built software packages; libraries, include files, and binaries. (/usr/local is often a mount point for a separate filesystem, to prevent software installation from filling up the /usr filesystem.)

• sbin—System binaries.

• share—Shared information, including man pages.

/var is occasionally in its own filesystem (which we recommend), although it can also be a subdirectory of / or /usr. Subdirectories of interest include these:

• adm—Log file storage (Solaris)

• crash—Crash diagnostics and memory image storage

• log—Log file storage (Red Hat and Solaris)

• mail—Mail delivery area

• spool—Spooling area for printing, cron, and other managed services

• yp—NIS configuration file storage

The /boot directory generally is found only on Red Hat machines, although there is no particular reason why it can't be used with Solaris as well. This directory, located in its own, small filesystem, has two main functions: to hold the basic files needed at boot time and to allow large disks to boot properly. Due to PC BIOS limitations, only boot sectors within the first 1024 cylinders of a disk are honored. So, a small boot partition in the very first cylinders of the disk is a wise idea.

The /proc pseudofilesystem contains system statistics and process information. /proc is considered a pseudofilesystem because the files there are not actual files on disk. Instead, each file is an interface to data about a particular running process or other system status.

Philosophy of Division

There are a number of competing philosophies about disk space divisions—the best space allocations, how many partitions to have, and so on. Like most longstanding philosophical differences, this question has blossomed into a kind of religious war. Remember that although we are going to give background information and make suggestions, we're not taking sides or saying that we have an insight on The Best!" way of doing things.

There are good reasons for each viewpoint, and the way that you finally deal with partitions must be very situation-specific. For instance, a server might not require separate home directory space, but it might need a lot of log space instead. In such a case, it might also be important to keep the logs in a separate partition from the OS so that the admins can access the machine even when the logs are brimming.

First we'll present some points to keep in mind for any kind of install and then we'll talk about specific requirements for different classes of install (server, workstation, and so on).

General Points

The most basic requirement, no matter what class of install you are doing, is that there be a minimum of two partitions: one for / and one for swap. In this most simple of systems, / holds the entire OS, logs, user home directories, and so on.

So, then, how much swap should you allocate? (Warning: entering religious debate zone.)

Our most basic rule of thumb is to set aside somewhere between two and two and a half times as much swap space as you have system RAM. This means that when RAM increases, swap should, too (though this is only true up to a point). Remember, though, you might have applications running that require more swap space—check your documentation.

Red Hat has a good discussion of swap-related issues at http://www.redhat.com/support/docs/gotchas/7.1/gotchas-71.html. As they say, "It's better to have it and not need it than to need it and not have it."

The next consideration will be the number of partitions that you want on your system. Without delving into the idea of installation classes, first consider a few points:

• Are there any data areas that need to have guaranteed space allocations?

The answer to this question is always "yes." If nothing else, the / and /usr file- systems must be protected. Some utilities won't operate correctly if they can't make system log entries or write scratch files to /tmp, so it's a good idea to protect the /var and /tmp filesystems, too.

• Should the machine continue running even if its logging area fills up?

If so, then /var must have its own filesystem.

• Are there any system or data areas that will see increasing storage use?

Again, /var is the default location for system logs and can be expected to fill up unless logs are rotated out regularly. Other examples could include a directory to which developed software is delivered for testing on a large project.

• Are there any system or data areas that will see frequent access?

/var/mail, for instance, can be a very busy directory. Directories containing important databases also might see high access rates and should be located in their own filesystems.

• Will any directories be exported through NFS?

Such directories should certainly have their own filesystems. Never export more of your system than absolutely necessary.

• Will users need especially large workspaces for large files?

Creating data filesystems is a good way to discourage users from cluttering their home directories too much.

Data directories for a specific project accessed only by certain users might well be grouped into a single filesystem. This means that they will also require their own partition—as we mentioned earlier, there cannot be more than one filesystem on a partition.

The related question, though, is why not just keep these data areas in their own directories rather than allocating a whole filesystem? The answer is two-fold. First, you cannot guarantee space for a data area that is not in its own partition—there are currently no mechanisms to achieve this. Second, you cannot limit space consumption by a directory within a filesystem. (For those of you thinking about filesystem quotas, remember that they can only be assigned to a user, not to a directory.)

As we pointed out earlier, partitions are an ironclad way of guaranteeing space and enforcing boundaries.

So, as we go on to talk about install classes, consider which data and system areas might need room to expand and enforced limits to that expansion. Also consider whether there might be critical system areas that must not get crowded out (for instance, if there is no room in /tmp, users can't log into the system).

Install Classes

When considering partitioning, machines are generally classified into two types: servers and workstations.⁸ There are always systems that fall into the gray areas and oddities that challenge this division, but we find that this is a good starting point.

The first piece of information that you need to judge partition sizes is the projected amount of space that various items will need. The best way to get a feel for this sort of thing is to get a relatively large disk and simply install the OS of your choice in various ways, noting the differences in size and usability.

Most sysadmins, unfortunately, are not allowed time for this sort of constructive play, so here is a reasonable working estimate:

By the numbers just given, you can tell a few things about Red Hat and Sun. Although their full installs are the same size, Red Hat tends to install less by default than does Sun. Also notice that a Red Hat server install is the smallest of all the types. This is because a server is presumed to be a dedicated machine that doesn't require things like X Windows in the base install. Sun's philosophy is the opposite: Servers get all the components that a workstation gets, plus more. We tend to advocate a minimalist approach: Only install what you actually need. If it's not installed, it doesn't have to be maintained and it won't hurt you.

More About Red Hat

Red Hat officially recommends that you divide up your space into three partitions: swap, /boot, and / (root).⁹ The official site recommends the following space division guidelines:

If you choose the defaults for a server-class install, your partition layout will look something like the following:

A few cautions:

• If your server does not serve home directories, then the space set aside for /home should probably be reapportioned elsewhere.

• If your server's particular application generates a lot of logs or is left unattended for extended periods of time, /var should probably be larger. If you intend to keep up to n multiple crash images, /var should be at least n times the size of real memory.

• If your server is dedicated to NFS service, /usr might need to be larger (unless the applications/files being served are located elsewhere).

• If your server has large amounts of RAM or runs swap-intensive software, swap should be larger.

• All "tmp"-type directories should point into /var/tmp, to avoid filling up other key partitions, or have their own partition.

Red Hat Linux 7.1 workstation-class defaults are as follows:

Our concerns about these defaults:

• This is probably not enough swap space—most new machines come with more than 32Mb of RAM. Swap should be increased accordingly.

• There is no separate partition for logs. This means that if the machine is left unattended for an extended period of time, or if the logs grow very large for some other reason, the machine might not be capable of functioning properly.

• "tmp"-type directories cannot be pointed to a noncritical partition. If users or applications make heavy use of any tmp directory, it could affect system performance.

More About Solaris

Sun has a mechanism called autolayout that proposes sizes based on the amount of disk space that it detects on your system during installation.¹⁰ Our experience suggests that a 4GB disk is a "comfortable minimum" requirement for a Solaris workstation installation.

For servers, Sun recommends the following partitions:

swap

/

/usr

/opt

/export

/export/home

/export/swap

Because a server should have space allocated for logging and for the installation of its services, it is our practice to specify an 18GB disk for Solaris server installations.

Sun's intention is that everything to be served out should be placed somewhere under /export. Note the absence of both a /boot and a /var partition. A few cautions:

• If your server does not serve home directories, then the space set aside for /export/home should probably be reapportioned elsewhere.

• If your server does not serve diskless clients, then the space set aside for /export/swap should probably be reapportioned elsewhere.

• If your server's particular application generates a lot of logs or is left unattended for extended periods of time, /var should probably be a separate partition.

• If your server is dedicated to NFS service, /opt might need to be larger (unless the applications/files being served are located elsewhere).

• If your server has large amounts of RAM or runs swap-intensive software, swap should be larger.

• All "tmp"-type directories should point into /var/tmp, to avoid filling up other key partitions, or move their own partition.

For workstations (what Sun calls "standalone" machines), the following partitions are proposed by default:

/

swap

/opt

/usr

/home

Our concerns about these defaults:

• There is no separate partition for logs. This means that if the machine is left unattended for an extended period of time, or if the logs grow very large for some other reason, the machine might not be capable of functioning properly.

• "tmp"-type directories cannot be pointed to a noncritical partition. If users or applications make heavy use of any tmp directory, it could affect system performance.

• If the workstation does not have local home directories, then the space set aside for /home should probably be reapportioned elsewhere.

• If the workstation does not have local optional software installed, then the space set aside for /opt should probably be reapportioned elsewhere.

Red Hat's Partitionless Install

There is a different kind of install supported by Red Hat that is quite outside our model. In a sort of try-before-you-buy approach, you now can install Red Hat Linux in a partitionless mode.¹¹

The trick to this kind of install is to piggyback Linux on an existing DOS/FAT file- system. Of course, this means that you can't boot into Linux from the local partition—you have to have a boot floppy.

See http://www.redhat.com/support/manuals/RHL-7.1-Manual/install-guide/ch-part-less.html for more information on partitionless installs.

Technical Details of Partitioning

Enough theory—it's finally time to make the partitions actually appear on the disk. As you have probably already guessed, Red Hat and Solaris use different means to achieve the same ends. Let's look at these tools.

Red Hat

At install time, you are presented with two front ends for disk partitioning: fdisk and diskdruid. The familiar trade-off exists between the two: diskdruid is more straightforward to use, while fdisk allows you finer control over all parameters.

If you want to partition a new disk added to an extant system, you will need to use a variant fdisk to do the job. Although plain old fdisk is the standard, we suggest that you use the curses-based front end called cfdisk. It comes standard on Red Hat 7.1 and is quite intuitive.

Invoking cfdisk with no arguments will bring up a listing of /dev/hda along with command options:

                       cfdisk 2.10s

                     Disk Drive: /dev/hda
                    Size: 40000000000 bytes
             Heads: 255  Sectors per Track: 63  Cylinders: 4863

   Name      Flags     Part Type  FS Type        [Label]       Size (MB)
 -------------------------------------------------------------------------------
   hda1                Primary   Linux ext2      [/boot]         32.91  
   hda2      Boot      Primary   NTFS            [^C]         26222.20
   hda3                Primary   Linux swap                    2113.90
   hda4                Primary   Linux ext2      [/]          11630.55

   [Bootable] [ Delete ] [ Help ] [Maximize] [ Print ] [ Quit ] [ Type ]
   [ Units ] [ Write ]

               Toggle bootable flag of the current partition

This works like most curses-based¹² front ends; the keyboard arrow keys move you up and down along the list of devices, and the Tab key moves between the command options along the bottom. The currently selected device and command are highlighted in gray. Also notice that there is a description of the command's purpose at the very bottom of the output. Pressing the Return or Enter key will run the highlighted command and refresh the display to reflect the update.

Assuming that you have a fresh disk, you will want to add a partition to it.

With the -l option, fdisk can also be used to report on current partition information:

[linux:6 ~]fdisk -l

Disk /dev/hda: 255 heads, 63 sectors, 4863 cylinders
Units = cylinders of 16065 * 512 bytes

  Device  Boot  Start    End  Blocks  Id System
/dev/hda1           1      4   32098+ 83 Linux
/dev/hda2  *        5   3192 25607610  7 HPFS/NTFS
/dev/hda3        3193   3449 2064352+ 82 Linux swap
/dev/hda4        3450   4863 11357955 83 Linux

Solaris

At both install-time and from within the OS, Solaris offers the format command to partition disks. When invoked, format presents you with a list of all currently accessible drives:

[sun:18: ~]format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
    0. c0t0d0 <ST315320A cyl 29649 alt 2 hd 16 sec 63>
     /pci@1f,0/ide@d/dad@0,0
    1. c0t1d0 <SUN2.1G cyl 2733 alt 2 hd 19 sec 80>
     /sbus@1f,0/SUNW,fas@e,8800000/sd@1,0
    2. c1t0d0 <IBM-DNES-318350-SA30 cyl 11199 alt 2 hd 10 sec 320>
     /sbus@1f,0/QLGC,isp@0,10000/sd@0,0
    3. c1t2d0 <SEAGATE-ST118273W-6244 cyl 7499 alt 2 hd 20 sec 237>
     /sbus@1f,0/QLGC,isp@0,10000/sd@2,0
Specify disk (enter its number):

Simply type in the number of the disk that you are interested in. Let's select disk 0:

selecting c0t0d0
[disk formatted, no defect list found]
Warning: Current Disk has mounted partitions.

FORMAT MENU:
    disk      - select a disk
    type      - select (define) a disk type
    partition - select (define) a partition table
    current   - describe the current disk
    format    - format and analyze the disk
    repair    - repair a defective sector
    show      - translate a disk address
    label     - write label to the disk
    analyze   - surface analysis
    defect    - defect list management
    backup    - search for backup labels
    verify    - read and display labels
    save      - save new disk/partition definitions
    volname   - set 8-character volume name
    !<cmd>    - execute <cmd>, then return
    quit
format>

To view the current partition table on the disk, enter "verify":

format>verify
Primary label contents:

Volume name = <    >
ascii name = <ST315320A cyl 29649 alt 2 hd 16 sec 63>
pcyl       = 29651
ncyl       = 29649
acyl       =  2
nhead      =  16
nsect      =  63
Part   Tag    Flag    Cylinders      Size      Blocks
 0    root     wm   1041 - 13524    6.00GB   (12484/0/0) 12583872
 1    swap     wu       0 - 1040   512.37MB  (1041/0/0)  1049328
 2  backup     wm      0 - 29648    14.25GB  (29649/0/0) 29886192
 3    home     wm  13525 - 29646     7.75GB  (16122/0/0) 16250976
 4 unassigned  wm      0             0       (0/0/0)      0
 5 unassigned  wm      0             0       (0/0/0)      0
 6 unassigned  wm      0             0       (0/0/0)      0
 7 unassigned  wm  29647 - 29648     0.98MB  (2/0/0)     2016

format>

Slice 2, called backup, represents the entire disk. As a result, it is not ever a useable partition and should not be altered in any way.

Now we're done discussing partitions and can move into entirely logical constructs, such as filesystems.