At remount-time, the options are interpreted in the given order, e.g. left to right, except dlgt. The 'dlgt' option is disabled in interpreting.
If there is the same named file on the lower branch (larger index), aufs will hide the lower file. You can only see the highest file. You will be confused if the added branch has whiteouts (including diropq), they may or may not hide the lower entries.
If a process have once mapped a file by mmap(2) with MAP_SHARED and the same named file exists on the lower branch, the process still refers the file on the lower(hidden) branch after adding the branch. If you want to update the contents of a process address space after adding, you need to restart your process or open/mmap the file again. (cf. Branch Syntax).
If a process is referencing the file/directory on the deleting branch (by open, mmap, current working directory, etc.), aufs will return an error EBUSY.
If the branch permission is been changing 'rw' to 'ro, and a process is mapping a file by mmap(2) on the branch, the process may or may not be able to modify its mapped memory region after modifying branch permission flags. (cf. Branch Syntax).
The file is created per branch filesystem, and unlinked. So you cannot find this file, but it exists and is read/written frequently by aufs. (cf. External Inode Number Table).
If you use this option, Some applications will not work correctly. (cf. External Inode Number Table).
If the target dir which is being removed or renamed (destination dir) has a huge number of whiteouts, i.e. the dir is empty logically but physically, the cost to remove/rename the single dir may be very high. It is required to unlink all of whiteouts internally before issuing rmdir/rename to the branch. To reduce the cost of single systemcall, aufs renames the target dir to a whiteout-ed temporary name and invokes a pre-created kernel thread to remove whiteout-ed children and the target dir. The rmdir/rename systemcall returns just after kicking the thread.
When the number of whiteout-ed children is less than the value of dirwh, aufs remove them in a single systemcall instead of passing another thread. This value is ignored when the branch is NFS. The default value is AUFS_DIRWH_DEF].
Those threads stay in the system while the aufs module is loaded, and handle the special I/O requests from aufs. The default value is AUFS_NWKQ_DEF].
The special I/O requests from aufs include a part of copy-up, lookup, directory handling, pseudo-link, xino file operations and the delegated access to branches. For example, Unix filesystems allow you to rmdir(2) which has no write permission bit, if its parent directory has write permission bit. In aufs, the removing directory may or may not have whiteout or 'dir opaque' mark as its child. And aufs needs to unlink(2) them before rmdir(2). Therefore aufs delegates the actual unlink(2) and rmdir(2) to another kernel thread which has been created already and has a superuser privilege.
If you enable CONFIG_SYSFS, you can check this value through <sysfs>/module/aufs/parameters/nwkq.
So how many threads is enough? You can check it by <sysfs>/fs/aufs/stat. It shows the maximum number of the enqueued work at a time per a thread. Usually they are all small numbers or 0. If your workload is heavy and you feel the response is low, then check these values. If there are no zero and any of them is larger than 2 or 3, you should set 'nwkq' module parameter greater then the default value. But the reason of the bad response is in your branch filesystem, to increase the number of aufs thread will not help you.
The last number in <sysfs>/fs/aufs/stat after comma is the maximum number of the 'no-wait' enqueued work at a time. Aufs enqueues such work to the system global workqueue called 'events', but doesn't wait for its completion. Usually they does no harm the time-performance of aufs.
If the number of your branches is large or their path is long and you meet the limitation of mount(8), /etc/mtab or /proc/mount, you need to enable CONFIG_AUFS_SYSAUFS and set aufs module parameter brs=1.
When this parameter is set as 1, aufs doesn't show 'br:' (or dirs=) mount option through /proc/mounts, and /sbin/mount.aufs doesn't put it to /etc/mtab. So you can keep yourself from the page limitation of mount(8), /etc/mtab and /proc/mounts. Aufs shows 'br:' option through <sysfs>/fs/aufs/brs.
The default is brs=0, which means <sysfs>/fs/aufs/brs does not exist and 'br:' option will appear in /proc/mounts and /etc/mtab if you install /sbin/mount.aufs. If you didn't enable CONFIG_AUFS_SYSAUFS, this parameter will be ignored.
dir_path is a directory path. The keyword after 'dir_path=' is a permission flags for that branch. Comma, colon and the permission flags string (including '=')in the path are not allowed. Any filesystem can be a branch, but aufs and unionfs. If you specify aufs or unionfs as a branch, aufs will return an error saying it is overlapped or nested.
The maximum number of branches is configurable at compile time. The current value is AUFS_BRANCH_MAX].
When an unknown permission or attribute is given, aufs sets ro to that branch silently.
Those files are always opened and read/write by aufs frequently. If your writable branch is on flash memory device, it is recommended to put xino files on other than flash memory by specifing 'xino=' monut option. The maximum file size will be 'max inode number on the branch x size of an inode number'. For example,
$ df -i /branch_fs /dev/hda14 2599968 203127 2396841 8% /branch_fs
and /branch_fs is an branch of the aufs, the maximum xino file size for /branch_fs will be 2,599,968 x 8 bytes = about 20 MB. But it might not be allocated all of disk blocks.
The xino files are always hidden, i.e. removed. So you cannot do 'ls -l xino_file'. Instead, you can check it through /proc/fs/aufs/xino. (not implemented). Once you unmount aufs, the xino files for that aufs are totally gone. It means that the inode number is not permanent.
The xino files should be created on the filesystem except NFS. If your first writable branch is NFS, you will need to specify xino file path other than NFS. Also if you are going to remove the branch where xino files exist or change the branch permission to readonly, you need to use xino option before del/mod the branch.
If you do not want to use xino, use noxino mount option. Use this option with care, since the inode number may be changed silently and unexpectedly anytime. For example, rmdir failure, recursive chmod/chown/etc to a large and deep directory or anything else. And some applications will not work correctly. If you want to change the xino default path, use xino mount option.
After you add branches, the persistence of inode number is not guaranteed. At remount time, cached but unused inodes are discarded. And the newly appeared inode may have different inode number at the next access time. The inodes in use have the persistent inode number.
When aufs assigned an inode number to a file, and if you create the same named file on the upper branch directly, then the next time you access the file, aufs may assign another inode number to the file even if you use xino option. Some applications may treat the file whose inode number has been changed as totally different file.
When you have files named fileA and fileB which are hardlinked on a readonly branch, if you write something into fileA, aufs copies-up fileA to a writable branch, and write(2) the originally requested thing to the copied-up fileA. On the writable branch, fileA is not hardlinked. But aufs remembers it was hardlinked, and handles fileB as if it existed on the writable branch, by referencing fileA's inode on the writable branch as fileB's inode.
Once you unmount aufs, the plink info for that aufs kept in memory are totally gone. It means that the pseudo-link is not permanent. If you want to make plink permanent, try 'auplink' script just before one of these operations, unmounting your aufs, using 'ro' or 'noplink' mount option, deleting a branch from aufs, adding a branch into aufs, or changing your writable branch to readonly.
This script will reproduces all real hardlinks on a writable branch by linking them, and removes pseudo-link info in memory and temporary link on the writable branch. Since this script access your branches directly, you cannot hide them by
If you are willing to rebuild your aufs with the same branches later, you should use auplink script before you umount your aufs. If you installed both of /sbin/mount.aufs and /sbin/umount.aufs, and your mount(8) and umount(8) support them, and /etc/default/auplink is configured,
The /etc/default/auplink is a simple shell script which does nothing but defines $FLUSH. If your aufs mount point is set in $FLUSH, 'auplink' flushes the pseudo-links on that mount point. If $FLUSH is set to "ALL", 'auplink' will be executed for every aufs.
The 'auplink' script uses 'aulchown' binary, you need to install it too. The 'auplink' script executes 'find' and 'mount -o remount', they may take a long time and impact the later system performance. If you didn't install /sbin/mount.aufs, /sbin/umount.aufs or /sbin/auplink, but you want to flush pseudo-links, then you need to execute 'auplink' manually. If you installed and configured them, but do not want to execute 'auplink' at umount time, then use '-i' option for umount(8).
# auplink /your/aufs/root flush # umount /your/aufs/root or # auplink /your/aufs/root flush # mount -o remount,mod:/your/writable/branch=ro /your/aufs/root or # auplink /your/aufs/root flush # mount -o remount,noplink /your/aufs/root or # auplink /your/aufs/root flush # mount -o remount,del:/your/aufs/branch /your/aufs/root or # auplink /your/aufs/root flush # mount -o remount,append:/your/aufs/branch /your/aufs/root
The plinks are kept both in memory and on disk. When they consumes too much resources on your system, you can use the 'auplink' script at anytime and throw away the unnecessary pseudo-links in safe.
Additionally, the 'auplink' script is very useful for some security reasons. For example, when you have a directory whose permission flags are 0700, and a file who is 0644 under the 0700 directory. Usually, all files under the 0700 directory are private and no one else can see the file. But when the directory is 0711 and someone else knows the 0644 filename, he can read the file.
Basically, aufs pseudo-link feature creates a temporary link under the directory whose owner is root and the permission flags are 0700. But when the writable branch is NFS, aufs sets 0711 to the directory. When the 0644 file is pseudo-linked, the temporary link, of course the contents of the file is totally equivalent, will be created under the 0711 directory. The filename will be generated by its inode number. While it is hard to know the generated filename, someone else may try peeping the temporary pseudo-linked file by his software tool which may try the name from one to MAX_INT or something. In this case, the 0644 file will be read unexpectedly. I am afraid that leaving the temporary pseudo-links can be a security hole. It make sense to execute 'auplink /your/aufs/root flush' periodically, when your writable branch is NFS.
When your writable branch is not NFS, or all users are careful enough to set 0600 to their private files, you don't have to worry about this issue.
If you don't want this feature, use 'noplink' mount option and you don't need to install 'auplink' script and 'aulchown' binary.
none on /dev/shm/u type aufs (rw,xino=/dev/shm/rw/.aufs.xino,br:/dev/shm/rw=rw:/dev/shm/ro=ro) $ ls -li ../r?/f_src_linked* ./f_src_linked* ./copied ls: ./copied: No such file or directory 15 -rw-r--r-- 2 jro jro 2 Dec 22 11:03 ../ro/f_src_linked 15 -rw-r--r-- 2 jro jro 2 Dec 22 11:03 ../ro/f_src_linked2 22 -rw-r--r-- 2 jro jro 2 Dec 22 11:03 ./f_src_linked 22 -rw-r--r-- 2 jro jro 2 Dec 22 11:03 ./f_src_linked2 $ echo abc >> f_src_linked $ cp f_src_linked copied $ ls -li ../r?/f_src_linked* ./f_src_linked* ./copied 15 -rw-r--r-- 2 jro jro 2 Dec 22 11:03 ../ro/f_src_linked 15 -rw-r--r-- 2 jro jro 2 Dec 22 11:03 ../ro/f_src_linked2 36 -rw-r--r-- 2 jro jro 6 Dec 22 11:03 ../rw/f_src_linked 53 -rw-r--r-- 1 jro jro 6 Dec 22 11:03 ./copied 22 -rw-r--r-- 2 jro jro 6 Dec 22 11:03 ./f_src_linked 22 -rw-r--r-- 2 jro jro 6 Dec 22 11:03 ./f_src_linked2 $ cmp copied f_src_linked2 $ none on /dev/shm/u type aufs (rw,xino=/dev/shm/rw/.aufs.xino,noplink,br:/dev/shm/rw=rw:/dev/shm/ro=ro) $ ls -li ../r?/f_src_linked* ./f_src_linked* ./copied ls: ./copied: No such file or directory 17 -rw-r--r-- 2 jro jro 2 Dec 22 11:03 ../ro/f_src_linked 17 -rw-r--r-- 2 jro jro 2 Dec 22 11:03 ../ro/f_src_linked2 23 -rw-r--r-- 2 jro jro 2 Dec 22 11:03 ./f_src_linked 23 -rw-r--r-- 2 jro jro 2 Dec 22 11:03 ./f_src_linked2 $ echo abc >> f_src_linked $ cp f_src_linked copied $ ls -li ../r?/f_src_linked* ./f_src_linked* ./copied 17 -rw-r--r-- 2 jro jro 2 Dec 22 11:03 ../ro/f_src_linked 17 -rw-r--r-- 2 jro jro 2 Dec 22 11:03 ../ro/f_src_linked2 36 -rw-r--r-- 1 jro jro 6 Dec 22 11:03 ../rw/f_src_linked 53 -rw-r--r-- 1 jro jro 6 Dec 22 11:03 ./copied 23 -rw-r--r-- 2 jro jro 6 Dec 22 11:03 ./f_src_linked 23 -rw-r--r-- 2 jro jro 6 Dec 22 11:03 ./f_src_linked2 $ cmp copied f_src_linked2 cmp: EOF on f_src_linked2 $
If you add a branch which has fileA or fileB, aufs does not follow the pseudo link. The file on the added branch has no relation to the same named file(s) on the lower branch(es). If you use noxino mount option, pseudo link will not work after the kernel shrinks the inode cache.
This feature will not work for squashfs before version 3.2 since its inode is tricky. When the inode is hardlinked, squashfs inodes has the same inode number and correct link count, but the inode memory object is different. Squashfs inodes (before v3.2) are generated for each, even they are hardlinked.
Aufs has a mount option named 'udba' which specifies the test level at access time (at d_revalidate time) whether UDBA was happened or not.
It is recommended to use this option only when you are sure that nobody access a file on a branch. It might be difficult for you to achieve real 'no UDBA' world when you cannot stop your users doing 'find / -ls' or something. If you really want to forbid all of your users to UDBA, here is a trick for it. With this trick, users cannot see the branches directly and aufs runs with no problem, except 'auplink' script. But if you are not familiar with aufs, this trick may make yourself confused.
# d=/tmp/.aufs.hide # mkdir $d # for i in $branches_you_want_to_hide > do > mount -n --bind $d $i > done
When you unmount the aufs, delete/modify the branch by remount, or you want to show the hidden branches again, unmount the bound /tmp/.aufs.hide.
# umount -n $branches_you_want_to_unbound
This rule leads to some unexpected situation, but I hope it is harmless. Those are totally depends upon cache. Here are just a few examples.
If your outer modification (UDBA) is rare and you can ignore the temporary and minor differences between virtual aufs world and real branch filesystem, then try this mount option.
When a user access the file which was notified UDBA before, the cached data about the file will be discarded and aufs re-lookup. So the data will be updated. To use this option, you need linux-2.6.18 and later, and need to enable CONFIG_INOTIFY and CONFIG_AUFS_UDBA_INOTIFY.
----------------------------------------------------------------------
It is recommended to export your branch filesystems once before exporting aufs. By exporting once, the branch filesystem internal pointer named find_exported_dentry is initialized. After this initialization, you may unexport them. Additionally, this initialization should be done per the filesystem type. If your branches are all the same filesystem type, you need to export just one of them once. If you have never export a filesystem which is used in your braches, aufs will initialize the internal pointer by the default value, and produce a warning. While it will work correctly, I am afraid it will be unsafe in the future.
Additionally, there are several limitations or requirements.
If you want to discard cache within a certain filesystem, try 'mount -o remount /your/mntpnt'. Some filesystem may return an error of EINVAL or something, but VFS discards the unused dentry/inode caches on the specified filesystem.
Ignoring 'delete' option, and to keep filesystem consistency, aufs tries writing something to only one branch in a single systemcall. It means aufs may copyup even if the copyup-src branch is specified as writable. For exmaple, you have two writable branches and a large regular file on the lower writable branch. When you issue rename(2) to the file on aufs, aufs may copyup it to the higher writable branch. If this behaviour is not what you want, then you should rename(2) it on the lower branch directly.
And there is a simple shell script 'unionctl' which is compatible with unionctl(8) in Unionfs Version 1.x series, except --query action. This script executes mount(8) with 'remount' option and uses add/del/mod aufs mount options. If you are familiar with Unionfs Version 1.x series and want to use unionctl(8), you can try this script instead of using mount -o remount,... directly. Aufs does not support ioctl(2) interface. This script is highly depending upon mount(8) in util-linux-2.12p package, and you need to mount /proc to use this script. If your mount(8) version differs, you can try modifying this script. It is very easy.
Aufs uses the external inode number table by default.
The default branch permission for the first branch is 'rw', and the rest is 'ro'.
The whiteout is for hiding files on lower branches. Also it is applied to stop readdir going lower branches. The latter case is called 'opaque directory.' Any whiteout is an empty file, it means whiteout is just an mark. In the case of hiding lower files, the name of whiteout is AUFS_WH_PFX]<filename>.' And in the case of stopping readdir, the name is "AUFS_WH_PFX]AUFS_WH_PFX].opq" or "AUFS_WH_PFX]__dir_opaque." The name depends upon your compile configuration CONFIG_AUFS_COMPAT. All whiteouts are hardlinked, including '<writable branch top dir>/AUFS_WH_PFX]AUFS_WH_BASENAME].'
The hardlink on an ordinary (disk based) filesystem does not consume inode resource newly. But in linux tmpfs, the number of free inodes will be decremented by link(2). It is recommended to specify nr_inodes option to your tmpfs if you meet ENOSPC. Use this option after checking by 'df -i'.
When you rmdir or rename-to the dir who has a number of whiteouts, aufs rename the dir to the temporary whiteouted-name like AUFS_WH_PFX]<dir>.<random hex>.' Then remove it after actual operation. cf. mount option 'dirwh'.
statfs(2) returns the first branch info except namelen. The namelen is decreased by the whiteout prefix length.
Remember, seekdir(3) and telldir(3) are not defined in POSIX. They may not work as you expect. Try rewinddir(3) or re-open the dir.
The whiteout prefix (AUFS_WH_PFX]) is reserved on all branches. Users should not handle the filename begins with this prefix.
If you dislike the difference between the aufs entries in /etc/mtab and /proc/mounts, and if you are using mount(8) in util-linux package, then try ./mount.aufs script. Copy the script to /sbin/mount.aufs. This simple script tries updating /etc/mtab. If you don't care about /etc/mtab, you can ignore this script. Remember this script is highly depending upon mount(8) in util-linux-2.12p package, and you need to mount /proc.
Since aufs uses its own inode and dentry, your system may cache huge number of inodes and dentries. It can be as twice as all of the files in your union. It means that unmounting or remounting readonly at shutdown time may take a long time, since mount(2) in VFS tries freeing all of the cache on the target filesystem.
When you open a directory, aufs will open several directories internally. It means you may reach the limit of the number of file descriptor. And when the lower directory cannot be opened, aufs will close all the opened upper directories and return an error.
The sub-mount under the branch of local filesystem is ignored. For example, if you have mount another filesystem on /branch/another/mntpnt, the files under 'mntpnt' will be ignored by aufs. It is recommended to mount the sub-mount under the mounted aufs. For example,
# sudo mount /dev/sdaXX /ro_branch
# d=another/mntpnt
# sudo mount /dev/sdbXX /ro_branch/$d
# mkdir -p /rw_branch/$d
# sudo mount -t aufs -o br:/rw_branch:/ro_branch none /aufs
# sudo mount -t aufs -o br:/rw_branch/${d}:/ro_branch/${d} none /aufs/another/$d
I have never tested NFSv4 as aufs branch.
There are several characters which are not allowed to use in a branch directory path and xino filename. See detail in Branch Syntax and Mount Option.
The file-lock which means fcntl(2) with F_SETLK, F_SETLKW or F_GETLK, flock(2) and lockf(3), is applied to virtual aufs file only, not to the file on a branch. It means you can break the lock by accessing a branch directly. TODO: check 'security' to hook locks, as inotify does.
The fsync(2) and fdatasync(2) systemcalls return 0 which means success, even if the given file descriptor is not opened for writing. I am afraid this behaviour may violate some standards. Checking the behaviour of fsync(2) on ext2, aufs decided to return success.
# mount -v -t aufs br:/day0:/base none /u
none on /u type aufs (rw,xino=/day0/.aufs.xino,br:/day0=rw:/base=ro)
# mount -v -o remount,\
prepend:/day1,\
xino=/day1/xino,\
mod:/day0=ro,\
del:/day0 \
/u
none on /u type aufs (rw,xino=/day1/xino,br:/day1=rw:/base=ro)
# mount -t aufs br:/rw none /u # mount -o remount,append:/ro /u different uid/gid/permission, /ro # mount -o remount,del:/ro /u # mount -o remount,nowarn_perm,append:/ro /u # (there is no warning)
And there is a good sample which is for network booted diskless machines. See sample/ in detail.
When you mount or remount your union without -o ro common mount option and without writable branch, aufs will warn you that the first branch should be writable.