General overview of the Linux file system

"On a UNIX system, everything is a file; if something is not a file, it is a process."

A Linux system, just like UNIX, makes no difference between a file and a directory, since a directory is just a file containing names of other files. Programs, services, texts, images, and so forth, are all files. Input and output devices, and generally all devices, are considered to be files, according to the system.
In order to manage all those files in an orderly fashion, man likes to think of them in an ordered tree-like structure on the hard disk, as we know from MS-DOS (Disk Operating System) for instance. The large branches contain more branches, and the branches at the end contain the tree's leaves or normal files. For now we will use this image of the tree, but we will find out later why this is not a fully accurate image.

The -l option to ls displays the file type, using the first character of each input line:
jaime:~/Documents> ls -l
total 80
-rw-rw-r--   1 jaime   jaime   31744 Feb 21 17:56 intro Linux.doc
-rw-rw-r--   1 jaime   jaime   41472 Feb 21 17:56 Linux.doc
drwxrwxr-x   2 jaime   jaime    4096 Feb 25 11:50 course

Symbol	Meaning
-	Regular file
d	Directory
l	Link
c	Special file
s	Socket
p	Named pipe
b	Block device

Table 3-1. File types in a long list

Why partition?

Most Linux systems use fdisk at installation time to set the partition type.The fdisk utility has built-in help, should you forget these values.
On a server, system data tends to be separate from user data. Programs that offer services are kept in a different place than the data handled by this service. Different partitions will be created on such systems:

a partition with all data necessary to boot the machine
a partition with configuration data and server programs
one or more partitions containing the server data such as database tables, user mails, an ftp archive etc.
a partition with user programs and applications
one or more partitions for the user specific files (home directories)
one or more swap partitions (virtual memory)

On a running system, information about the partitions can be displayed using the df command
The df command only displays information about active non-swap partitions. These can include partitions from other networked systems
freddy:~> df -h
Filesystem          Size  Used Avail Use% Mounted on
/dev/hda8           496M  183M  288M  39% /
/dev/hda1           124M  8.4M  109M   8% /boot
/dev/hda5            19G   15G  2.7G  85% /opt
/dev/hda6           7.0G  5.4G  1.2G  81% /usr
/dev/hda7           3.7G  2.7G  867M  77% /var
fs1:/home           8.9G  3.7G  4.7G  44% /.automount/fs1/root/home

Figure 3-1. Linux file system layout

Table 3-2. Subdirectories of the root directory

Directory	Content
/bin	Common programs, shared by the system, the system administrator and the users.
/boot	The startup files and the kernel, vmlinuz. In some recent distributions also grub data. Grub is the GRand Unified Boot loader and is an attempt to get rid of the many different boot-loaders we know today.
/dev	Contains references to all the CPU peripheral hardware, which are represented as files with special properties.
/etc	Most important system configuration files are in /etc, this directory contains data similar to those in the Control Panel in Windows
/home	Home directories of the common users.
/initrd	(on some distributions) Information for booting. Do not remove!
/lib	Library files, includes files for all kinds of programs needed by the system and the users.
/lost+found	Every partition has a lost+found in its upper directory. Files that were saved during failures are here.
/misc	For miscellaneous purposes.
/mnt	Standard mount point for external file systems, e.g. a CD-ROM or a digital camera.
/net	Standard mount point for entire remote file systems
/opt	Typically contains extra and third party software.
/proc	A virtual file system containing information about system resources. More information about the meaning of the files in proc is obtained by entering the command man proc in a terminal window. The file proc.txt discusses the virtual file system in detail. If you don't have a Linux machine at hand on which to locate this file, it is included in Appendix E for your convenience.
/root	The administrative user's home directory. Mind the difference between /, the root directory and /root, the home directory of the root user.
/sbin	Programs for use by the system and the system administrator.
/tmp	Temporary space for use by the system, cleaned upon reboot, so don't use this for saving any work!
/usr	Programs, libraries, documentation etc. for all user-related programs.
/var	Storage for all variable files and temporary files created by users, such as log files, the mail queue, the print spooler area, space for temporary storage of files downloaded from the Internet, or to keep an image of a CD before burning it.

 How can you find out which partition a directory is on? Using the df command with a dot (.) as an option shows the partition the current directory belongs to, and informs about the amount of space used on this partition:
sandra:/lib> df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda7             980M  163M  767M  18% /
As a general rule, every directory under the root directory is on the root partition, unless it has a separate entry in the full listing from df (or df -h with no other options).

The file system in reality

Every partition has its own file system.throughout a system with multiple partitions, files with the same inode number can exist.
Every partition has its own set of inodes;Each inode describes a data structure on the hard disk, storing the properties of a file, including the physical location of the file data.

At the time a new file is created, it gets a free inode. In that inode is the following information:

Owner and group owner of the file.
File type (regular, directory, ...)
Permissions
Date and time of creation, last read and change.
Date and time this information has been changed in the inode.
Number of links to this file
File size
An address defining the actual location of the file data.

The only information not included in an inode, is the file name and directory. These are stored in the special directory files. By comparing file names and inode numbers, the system can make up a tree-structure that the user understands. Users can display inode numbers using the -i option to ls. The inodes have their own separate space on the disk.
#ls -i

Orientation in the file system

In the next example, a user wants to call on the wc (word count) command to check the number of lines in a file, but nothing happens and he has to break off his action using the Ctrl+C combination:

jumper:~> wc -l test
(Ctrl-C)
jumper:~> which wc
wc is hashed (/home/jumper/bin/wc)
jumper:~> echo $PATH
/home/jumper/bin:/usr/local/bin:/usr/local/sbin:/usr/X11R6/bin:
/usr/bin:/usr/sbin:/bin:/sbin

由于首次查找到匹配的是自己HOME目录下的WC，而非/usr/bin/wc ，所以并未得到我们想要的结果。

jumper:~> /usr/bin/wc -l test
       10 test

If the user uses programs in the other directories more frequently, he can change his path to look in his own directories last:

jumper:~> export PATH=/usr/local/bin:/usr/local/sbin:/usr/X11R6/bin:
/usr/bin:/usr/sbin:/bin:/sbin:/home/jumper/bin

  Changes are not permanent! 

       Note that when using the export command in a shell, the changes are temporary and only valid for this session (until you log out). Opening new sessions, even while the current one is still running, will not result in a new path in the new session. 
 
Absolute and relative paths

     In relative paths we also use the . and .. indications for the current and the parent directory.
    
The most important files and directories

The kernel
The kernel is the heart of the system. It manages the communication between the underlying hardware and the peripherals.
The kernel also makes sure that processes and daemons (server processes) are started and stopped at the exact right times.

The file /etc/shells gives an overview of known shells on a Linux system:
mia:~> cat /etc/shells
/bin/bash
/bin/sh
/bin/tcsh
/bin/csh

Fake Bourne shell(伪造SHELL)

Note that /bin/sh is usually a link to Bash, which will execute in Bourne shell compatible mode when called on this way.
Your default shell is set in the /etc/passwd file, like this line for user mia:
mia:L2NOfqdlPrHwE:504:504:Mia Maya:/home/mia:/bin/bash

Which shell am I using?
If you don't know which shell you are using, either check the line for your account in /etc/passwd or type the command
echo $SHELL

correct path to your home directory is stored in the HOME environment variable, in case some program needs it. With the echo command you can display the content of this variable:
orlando:~> echo $HOME
/nethome/orlando

 You can see for yourself if a limit is set using the quota command:
pierre@lamaison:/> quota -v
Diskquotas for user pierre (uid 501): none

If your system can not find the quota, then no limitation of file system usage is being applied.
 
The most important configuration files
most configuration files are stored in the /etc directory. Content can be viewed using the cat command, which sends text files to the standard output (usually your monitor). The syntax is straight forward:           cat file1 file2 ... fileN

Table 3-3. Most common configuration files

 

File	Information/service
aliases	Mail aliases file for use with the Sendmail and Postfix mail server. Running a mail server on each and every system has long been common use in the UNIX world, and almost every Linux distribution still comes with a Sendmail package. In this file local user names are matched with real names as they occur in E-mail addresses, or with other local addresses.
apache	Config files for the Apache web server.
bashrc	The system-wide configuration file for the Bourne Again SHell. Defines functions and aliases for all users. Other shells may have their own system-wide config files, like cshrc.
crontab and the cron.* directories	Configuration of tasks that need to be executed periodically - backups, updates of the system databases, cleaning of the system, rotating logs etc.
default	Default options for certain commands, such as useradd. （？）
filesystems	Known file systems: ext3, vfat, iso9660 etc.
fstab	Lists partitions and their mount points.
ftp*	Configuration of the ftp-server: who can connect, what parts of the system are accessible etc.
group	Configuration file for user groups. Use the shadow utilities groupadd, groupmod and groupdel to edit this file. Edit manually only if you really know what you are doing.
hosts	A list of machines that can be contacted using the network, but without the need for a domain name service. This has nothing to do with the system's network configuration, which is done in /etc/sysconfig.
inittab	Information for booting: mode, number of text consoles etc.
issue	Information about the distribution (release version and/or kernel info).
ld.so.conf	Locations of library files.
lilo.conf, silo.conf, aboot.conf etc.	Boot information for the LInux LOader, the system for booting that is now gradually being replaced with GRUB.
logrotate.*	Rotation of the logs, a system preventing the collection of huge amounts of log files.
mail	Directory containing instructions for the behavior of the mail server.
modules.conf	Configuration of modules that enable special features (drivers).
motd	Message Of The Day: Shown to everyone who connects to the system (in text mode), may be used by the system admin to announce system services/maintenance etc.
mtab	Currently mounted file systems. It is advised to never edit this file.
nsswitch.conf	Order in which to contact the name resolvers when a process demands resolving of a host name.
pam.d	Configuration of authentication modules.
passwd	Lists local users. Use the shadow utilities useradd, usermod and userdel to edit this file. Edit manually only when you really know what you are doing.
printcap	Outdated but still frequently used printer configuration file. Don't edit this manually unless you really know what you are doing.
profile	System wide configuration of the shell environment: variables, default properties of new files, limitation of resources etc.
rc*	Directories defining active services for each run level.
resolv.conf	Order in which to contact DNS servers (Domain Name Servers only).
sendmail.cf	Main config file for the Sendmail server.
services	Connections accepted by this machine (open ports).
sndconfig or sound	Configuration of the sound card and sound events.
ssh	Directory containing the config files for secure shell client and server.
sysconfig	Directory containing the system configuration files: mouse, keyboard, network, desktop, system clock, power management etc. (specific to RedHat)
X11	Settings for the graphical server, X. RedHat uses XFree, which is reflected in the name of the main configuration file, XFree86Config. Also contains the general directions for the window managers available on the system, for example gdm, fvwm, twm, etc.
xinetd.* or inetd.conf	Configuration files for Internet services that are run from the system's (extended) Internet services daemon (servers that don't run an independent daemon).

# ls -a
# ls -l
# ls -ltr
按修改时间排序，最近修改的文件在最底部

To find out more about the kind of data we are dealing with, we use the file command
# file cv.txt

Copying files and directories is done with the cp command. A useful option is recursive copy (copy all underlying files and subdirectories), using the -R option to cp. The general syntax is
# cp [-R] fromfile tofile

commands can be activated using the -i option. In that case the system won't immediately act upon request. Instead it will ask for confirmation, so it takes an additional click on the Enter key to inflict the damage:

# rm -ri archive/
rm: descend into directory `archive'? y

A very simple way of looking up files is using the which command, The which command is useful when troubleshooting "Command not Found" problems.
# which acroread
/usr/bin/which: no acroread in (/bin:/usr/bin:/usr/bin/X11)
The problem can be solved by giving the full path to the command to run, or by re-exporting the content of the PATH variable:
# export PATH=$PATH:/opt/acroread/bin
# echo $PATH
/bin:/usr/bin:/usr/bin/X11:/opt/acroread/bin

Using the which command also checks to see if a command is an alias for another command:
# which -a ls
ls is aliased to `ls -F --color=auto'
ls is /bin/ls

The find tool, known from UNIX, is very powerful, The most common use is for finding file names:

# find -name
# find . -size +5000k
psychotic_chaos.mp3

#  find . -name "*.tmp" -exec rm {} ; 清空临时文件

Later on ，locate was developed. This program is easier to use, but more restricted than find, since its output is based on a file index database that is updated only once every day. On the other hand, a search in the locate database uses less resources than find and therefore shows the results nearly instantly.
locate is a symbolic link to the slocate program:
# ls -l /usr/bin/locate
lrwxrwxrwx 1 root slocate  7 Oct 28 14:18 /usr/bin/locate -> slocate*

A simple but powerful program, grep is used for filtering input lines and returning certain patterns to the output。
uses grep to see how he did the thing with find:
查看命令历史记录
# grep -a find .bash_history
find . -name userinfo
man find
find ../ -name common.cfg
另一种方法
Also useful in these cases is the search function in bash, activated by pressing Ctrl+R at once, such as in the example where we want to check how we did that last find again:

# ^R
(reverse-i-search)`find': find `/home/thomas` -name *.xml（最近一条记录），输入要找目标命令第一字母即可，按^R 自动回滚，找到后按ESC退出，ENTER执行即可

找用户的HOME目录另一方法
# grep gdbruyne /etc/passwd
gdbruyne:x:981:981:Guy Debruyne, tel 203234:/home/gdbruyne:/bin/bash

find and locate are often used in combination with grep to define some serious queries.

转义符
# less *         display the file "*" instead of all the files in a directory
#cat This File   The same goes for filenames containing a space

A little bit of UNIX history explains this:

First there was cat. Output was streamed in an uncontrollable way.
The more command is still available on every Linux system.
less is the GNU version of more and has extra features allowing highlighting of search strings, scrolling back etc. The syntax is very simple:
# less file

Head and tail
These two commands display the n first/last lines of a file respectively. To see the last ten commands entered:
# tail -10 .bash_history

Linking files

Hard link: Associate two or more file names with the same inode. Hard links share the same data blocks on the hard disk, while they continue to behave as independent files.There is an immediate disadvantage: hard links can't span partitions, because inode numbers are only unique within a given partition.
相同的inode，则相同的文件内容，即一个Inode对应多个文件名，不能跨分区，因为Inode相对于分区才有意义；保护误删除。
Soft link or symbolic link (or for short: symlink): a small file that is a pointer to another file. A symbolic link contains the path to the target file instead of a physical location on the hard disk. Since inodes are not used in this system, soft links can span across partitions.
不同的Inode值，两个大小不同的文件。软件接是符号连接OR快捷方式，其内容是一个文件的物理路径，因此可以跨分区使用

Linux/Unix 档案系统中，有所谓的连结(link)，我们可以将其视为档案的别名，而连结又可分为两种 : 硬连结(hard link)与软连结(symbolic link)，
硬连结的意思是一个档案可以有多个名称，硬连结是存在同一个档案系统中。
而软连结的方式则是产生一个特殊的档案，该档案的内容是指向另一个档案的位置，而软连结却可以跨越不同的档案系统。
ln source dist 是产生一个连结(dist)到 source，至于使用硬连结或软链结则由参数决定。
不论是硬连结或软链结都不会将原本的档案复制一份，只会占用非常少量的磁碟空间。
The command to make links is ln. In order to create symlinks, you need to use the -s option:
ln -s targetfile linkname
example:
将档案 yy 产生一个 symbolic link : zz
#ln -s yy zz
#将档案 yy 产生一个 hard link : zz
ln yy xx 
 
File security Access rights: Linux's first line of defense

You should know what your user name is. using the id command, which also displays the default group you belong to and eventually other groups of which you are a member:  # id
uid=504(tilly) gid=504(tilly) groups=504(tilly),100(users),2051(org)
Your user name is also stored in the environment variable USER:
# echo $USER

Access mode codes

Code	Meaning
0 or -	The access right that is supposed to be on this place is not granted.
4 or r	read access is granted to the user category defined in this place
2 or w	write permission is granted to the user category defined in this place
1 or x	execute permission is granted to the user category defined in this place

 User group codes

Code	Meaning
u	user permissions
g	group permissions
o	permissions for others

The chmod command

File protection with chmod

Command	Meaning
chmod 400 file	To protect a file against accidental overwriting.（只读）
chmod 500 directory	To protect yourself from accidentally removing, renaming or moving files from this directory.（防止自己意外删除、修改、移动）
chmod 600 file	A private file only changeable by the user who entered this command.
chmod 644 file	A publicly readable file that can only be changed by the issuing user.
chmod 660 file	Users belonging to your group can change this file, others don't have any access to it at all.
chmod 700 file	Protects a file against any access from other users, while the issuing user still has full access.
chmod 755 directory	For files that should be readable and executable by others, but only changeable by the issuing user.（只能创建者自己才能修改，其它人只能读与执行）
chmod 775 file	Standard file sharing mode for a group.（标准的组共享）
chmod 777 file	Everybody can do everything to this file.

An example:
#  id
uid=501(asim) gid=501(asim) groups=100(users),501(asim),3400(web)
#  grep asim /etc/passwd
asim:x:501:501:Asim El Baraka:/home/asim:/bin/bash
#  grep 501 /etc/group
asim:x:501:

User private group scheme 
In order to allow more flexibility, most Linux systems follow the so-called user private group scheme, that assigns each user primarily to his or her own group. This group is a group that only contains this particular user, hence the name "private group". Usually this group has the same name as the user login name, which can be a bit confusing.
 
use the newgrp to log into any of these groups. In the example, asim needs to create files that are owned by the group web.

asim:/var/www/html$ newgrp web

asim:/var/www/html> id
uid=501(asim) gid=3400(web) groups=100(users),501(asim),3400(web)
When asim creates new files now, they will be in group ownership of the group web instead of being owned by the group asim:
asim:/var/www/html> touch test
asim:/var/www/html> ls -l test
-rw-rw-r--  1 asim web   0 Jun 10 15:38 test

 The file mask
#  umask
0002
The umask value is subtracted(-) from these default permissions after the function has created the new file or directory. Thus, a directory will have permissions of 775 by default, a file 664, if the mask value is (0)002. This is demonstrated in the example below:

创建的目录权限=缺省目录权限777减去此用户的umask(0002)即为：775
创建的文件权限=缺省目录权限666减去此用户的umask(0002)即为：664 因为文件缺省无执行权限，需用chmod a+x filename；
The root user usually has stricter default file creation permissions:
[root@estoban root]# umask
022

#  chown newuser file
Linux/Unix 是多人多工操作系统，所有的档案皆有拥有者。利用 chown 可以将档案的拥有者加以改变。一般来说，这个指令只有是由系统管理者(root)所使用，一般使用者没有权限可以改变别人的档案拥有者，也没有权限可以自己的档案拥有者改设为别人。只有系统管理者(root)才有这样的权限。
A example:
将档案 file1.txt 的拥有者设为 users 群体的使用者 jessie :
chown jessie:users file1.txt 

Linux/Unix 的档案调用权限分为三级 : 档案拥有者、群组、其他。利用 chmod 可以藉以控制档案如何被他人所调用。
B example:

将目前目录下的所有档案与子目录的拥有者皆设为 users 群体的使用者 lamport :
chmod -R lamport:users * 

范例 :将档案 file1.txt 设为所有人皆可读取 :
chmod ugo+r file1.txt
将档案 file1.txt 设为所有人皆可读取 :
chmod a+r file1.txt
将档案 file1.txt 与 file2.txt 设为该档案拥有者，与其所属同一个群体者可写入，但其他以外的人则不可写入 :
chmod ug+w,o-w file1.txt file2.txt
将 ex1.py 设定为只有该档案拥有者可以执行 :
chmod u+x ex1.py
将目前目录下的所有档案与子目录皆设为任何人可读取 :
chmod -R a+r *
此外chmod也可以用数字来表示权限如 chmod 777 file
语法为：chmod abc file

若用chmod 4755 filename可使此程序具有root的权限

3.4.2.5. Special modes
For the system admin to not be bothered solving permission problems all the time, special access rights can be given to entire directories, or to separate programs. There are three special modes:

Sticky bit mode: After execution of a job, the command is kept in the system memory. Originally this was a feature used a lot to save memory: big jobs are loaded into memory only once. But these days memory is inexpensive and there are better techniques to manage it, so it is not used anymore for its optimizing capabilities on single files. When applied to an entire directory, however, the sticky bit has a different meaning. In that case, a user can only change files in this directory when she is the user owner of the file or when the file has appropriate permissions. This feature is used on directories like /var/tmp, that have to be accessible for everyone, but where it is not appropriate for users to change or delete each other's data. The sticky bit is indicated by a t at the end of the file permission field:该位可以理解为防删除位. 设置sticky bit位后，只能由文件创建者删除相应的文件或目录。别人即使有写权限也不能删节除

mark:~> ls -ld /var/tmp
drwxrwxrwt   19 root     root         8192 Jan 16 10:37 /var/tmp/（）

The sticky bit is set using the command chmod o+t directory. The historic origin of the "t" is in UNIX' save Text access feature.

SUID (set user ID) and SGID (set group ID): represented by the character s in the user or group permission field. When this mode is set on an executable file, it will run with the user and group permissions on the file instead of with those of the user issuing the command, thus giving access to system resources. We will discuss this further in Chapter 4.setuid: 在执行时具有文件所有者的权限.
SGID (set group ID) on a directory: in this special case every file created in the directory will have the same group owner as the directory itself (while normal behavior would be that new files are owned by the users who create them). This way, users don't need to worry about file ownership when sharing directories:设置目录. 一个目录被标上setgid位,此目录下创建的文件继承该目录的属性.
mimi:~> ls -ld /opt/docs
drwxrws---  4 root    users          4096 Jul 25 2001 docs/

rwsrw-r-- 表示有setuid标志 (rwxrw-r--:rwsrw-r--)
rwxrwsrw- 表示有setgid标志 (rwxrwxrw-:rwxrwsrw-)
rwxrw-rwt 表示有sticky标志 (rwxrw-rwx:rwxrw-rwt)

文件的权限应该用四个八进制来表示，不过用 ls -l 命令时，只显示后三个罢了。如在该位上原有x, 则这些特殊标志显示为小写字母 (s, s, t). 否则, 显示为大写字母 (S, S, T)