|
Automount md5 keys: a way to mount various resources. |
Download location (HTTP): http://linux.bononline.nl/linux/create_workspace/src |
Location: Index / mount.md5key
To start, a screenshot from K3b:
INTRODUCTION
The reason for me to develop this construction, is that I need a simple construction which
mounts on demand, to use in combination with the FUSE module fuse-workspace.
The aim of this module is to create a directory where all the resources (network and local) are
direct available to the user, without the need for the user to go to "foreign" places like /media/..
or /mnt/... Now, without making it too technical here, to mount these resources, like a Samba share,
a mounting facility (with root privileges) is necessary. There is a lot possible with FUSE, but to make that
work from within a FUSE module is too complicated (at least for me), and why trying if there is already a
service providing this service. The only complication is to make it work with the FUSE module,
and that's what this howto is about.
More information you'll find at :
- fuse-workspace: http://linux.bononline.nl/projects/fuse-workspace.
WHY THIS CONSTRUCTION
With the FUSE fs fuse-workspace it's possible to make a symlink look like a directory.
Under some circumstances fuse-workspace presents a symbolic link into a directory to
the calling process. If this symlink now is pointing to a key in an autofs managed directory,
configured to mount Windows/Linux Samba shares, it's very possible to make this share available
somewhere in the directory where fuse-workspace is mounted the SMB server name is ADMIN,
workgroup BONONLINE, shares public, sbon, admin and documentation):
A "Windows Network" tree may look like:
/home/sbon/Network/Windows Network/BONONLINE/ADMIN/admin -> /mnt/mount.md5key/sbon/mount/50b89942f697f095ac66359eae1b3a8f
/documentation -> /mnt/mount.md5key/sbon/mount/68141c6ce3077c8e438f7b31c3002053
/public -> /mnt/mount.md5key/sbon/mount/795e5d59a7901e91296cf6aae0987c94
/sbon -> /mnt/mount.md5key/sbon/mount/8236f6c5eae783204a24f447c4ea8ec3
As you see, the targets of the symbolic links are keys, which may look complicated.
They play an important role in this construction. These keys are linked to an "resource record"
(one to one so unique) through a hash table. The mount command used in this construction reads
this information, and launches the right mount command to do the real mount. In this case mount.cifs.
Futher here you'll find more details about how this construction exactly works.
Presenting this view of the contents of directory ADMIN activates the automounter to mount all targets,
cause almost every command want's to find out what the target *is* where a symlink is pointing to.
(to check this: mount | grep /mnt/mount.md5key/sbon will show all targets are mounted)
For example the basic command ls. On most systems this is defined as alias:
alias ls='ls --color=auto'
The option --color=auto makes ls to look at the target, as it wants to present every type in a different color.
Try the command /bin/ls -l on the commandline, and you'll see that this does not activate the targets.
Other clients as Dolphin (filebrowser in KDE) and other applications do something simular, and look for
the presence of a ".directory" file in the target. This file contains information how this directory has to
be presented in KDE (does Gnome work the same way??). One is the icon for this item.
It's obvious that this action activates the automounter to mount the target associated with the key
the symbolic link is pointing to.
The "Windows Network" tree above uses symlinks to the right targets. This works, but it not ideal.
Apart from mounting targets unnecessarily as described above, most applications "forget" the path
they are coming from, and the user ends up browsing a tree in /mnt/mount.md5key/..
which is NOT userfriendly. That's the reason I've developed the FUSE module fuse-workspace-union,
which is able to translate these symbolic link into directories. This looks like:
(assuming you have write permission to /var/lib/workspace/sbon/{bind,conf})
mkdir -p /var/lib/workspace/sbon/bind
mkdir -p /var/lib/workspace/sbon/conf
mv /home/sbon/Network /var/lib/workspace/sbon/bind
mkdir /home/sbon/Workspace
fuse-workspace-union --bind-directory=/var/lib/workspace/sbon/bind \
--conf-directory=/var/lib/workspace/sbon/conf \
/home/sbon/Workspace
Fuse-workspace-union mirrors the directory /var/lib/workspace/sbon/bind in
the mountpoint, just like the example module fusexmp in the source of FUSE does with the root (/).
Now without any configuration, and called like this, fuse-workspace-union will translate
no symlink, leave every symlink as is:
cd /home/sbon/Workspace/Network/Windows Network/BONONLINE/ADMIN ls -l lrwxrwxrwx 1 root root 61 2009-12-31 21:17 admin -> /mnt/mount.md5key/sbon/mount/50b89942f697f095ac66359eae1b3a8f lrwxrwxrwx 1 root root 61 2009-12-31 21:17 documentation -> /mnt/mount.md5key/sbon/mount/68141c6ce3077c8e438f7b31c3002053 lrwxrwxrwx 1 root root 61 2009-12-31 21:17 public -> /mnt/mount.md5key/sbon/mount/795e5d59a7901e91296cf6aae0987c94 lrwxrwxrwx 1 root root 61 2009-12-31 21:17 sbon -> /mnt/mount.md5key/sbon/mount/8236f6c5eae783204a24f447c4ea8ec3
(again you can check this behaviour with mount | grep /mnt/mount.md5key/sbon)
The first way to make fuse-workspace-union translate these symlinks is to create the right
directory in the conf tree:
install --directory /var/lib/workspace/sbon/conf/Network/Windows\ Network/BONONLINE/ADMIN/admin install --directory /var/lib/workspace/sbon/conf/Network/Windows\ Network/BONONLINE/ADMIN/documentation install --directory /var/lib/workspace/sbon/conf/Network/Windows\ Network/BONONLINE/ADMIN/public install --directory /var/lib/workspace/sbon/conf/Network/Windows\ Network/BONONLINE/ADMIN/sbon
When fuse-workspace-union finds a symbolic link when reading the contents of a directory, it
checks for the existence of the same directory in the conf directory. If present, it will make
it look like a directory, if not it will not change it.
Asking for the same overview:
cd /home/sbon/Workspace/Network/Windows Network/BONONLINE/ADMIN ls -l drwxr-xr-x 2 sbon netgroup 61 2009-12-31 21:17 admin drwxr-xr-x 2 sbon netgroup 61 2009-12-31 21:17 documentation drwxr-xr-x 2 sbon netgroup 61 2009-12-31 21:17 public drwxr-xr-x 2 sbon netgroup 61 2009-12-31 21:17 sbon
(again check this with mount | grep /mnt/mount.md5key/sbon, wait for a while for the automounter to unmount
the shares first... and you'll see that the targets are not mounted)
Apart from presenting the contents of a directory (task of readdir) different, the permissions are changed
(task of getattr).
Later I added the options --autofs-mount-directory and --autofs-cache-directory. The reason or this is that with these
options fuse-workspace-union is able to detect the link is pointing to a md5key in an autofs managed directory. By comparing the
dirname of the target of the symlink with the value of the --autofs-mount-directory option, and checking the basename of the target exists,
is clear that the symlink is pointing to a md5key in an autofs managed directory, resulting the link is translated in a
directory. So with these options it's not necessary to create the "conf" directories.
INSTALLATION
Autofs
normal installation of the automounter for linux, Version does not matter, this construction is making use of a very simple setup
of autofs.
The only thing is that were making use of a seperate auto.master, and a special instance of the automounter. I've chosen to do so,
cause in this construction the automounter gets a reload signal signal (to reread the auto.master file), and I've had not so good
experiences with other maps, more complicated than this one. If the mountpoints used here are managed by an autofs daemon, which does
also manages other mountpoints, the results may be unpredictable.
I assume autofs is installed.
First download the tarball from here:
http://linux.bononline.nl/linux/mount.md5key/src/mount.md5key-1.1.tar
Create the following directories:
# replace %USERID% and %GROUPID% by the values of your account # do this as root cd ..path to mount.md5key-1.0 package.. # first the utilities mount.md5key and mount.md5key.status cp sbin/mount.md5key /sbin cp usr/bin/mount.md5key.status /usr/bin # the mount.md5key dir and everything in it cp --recursive etc/mount.md5key /etc # the session.d dir install --directory /etc/session.d # copy the session-scripts.conf file, with locations of various files and directories cp session.d/session-scripts.conf /etc/session.d # copy the scripts directory, with everything in it cp --recursive session.d/scripts /etc/session.d install --directory /etc/autofs # if is does not exist already. # the autofs map file, which maps the keys to the virtual md5key fs cp etc/autofs/auto.md5key /etc/autofs # the directory (cache) with all the resource records install --directory /var/cache/mount.md5key
Now, everything is installed you to make use of this construction. There are some more scripts
in the tarball, but they follow later. Some are about mounting of local USB devices, others about the automation
of this construction.
Next is to start the automounter:
/etc/session.d/scripts/start/launch_session_automount.sh $USER
This will create an auto.master file in /var/run/autofs, looking like:
/mnt/mount.md5key/%USERID%/mount /etc/autofs/auto.md5key MD5KEY_USER=%USERID%
and start the automounter using this file.
The relevant directories (/var/run/autofs and /mnt/mount.md5key/%USERID%/mount) are created.
You can test this bij viewing the /proc/mounts file:
cat /proc/mount.md5key | grep /mnt/mount.md5key /etc/autofs/auto.md5key /mnt/mount.md5key/%USERID%/mount autofs rw,relatime,fd=6,pgrp=8269,timeout=600,minproto=5,maxproto=5,indirect 0 0
I've chosen for the /var/run/autofs location because it contains only runtime data, which should not be preserved between
different runs.
Next the directory where all the mounts will go in. Note that the (auto)mounting will not take place directly in this map,
but in a subdirectory: /mnt/mount.md5key/%USERID%/mount. The reason for me to create this subdirectory is to ensure that
other users cannot access mounts made on behalf of this user (here:sbon), so every user has her/his own directory containing mounts.
Futher there is the subdirectory mount, cause the automounter overwrites the permissions of the directory it's working with,
causing the just configured permissions (above) to disappear.
Following is the starting of the fuse module fuse-workspace-union. You have to compile it first:
cd ..path to mount.md5key-1.0 package../src # compile the package, you'll get a lot of warnings, but it should compile gcc -Wall `pkg-config fuse --cflags --libs` -lulockmgr fuse-workspace-union.c -o fuse-workspace-union mv fuse-workspace-union /usr/bin
Fuse-workspace-union is now installed. Start it with:
/etc/session.d/scripts/start/mount_workspace.sh %USERID%
This will create the directories:
/var/lib/workspace/%USERID%/bind /var/lib/workspace/%USERID%/conf %HOME of USERID%/Workspace
and run fuse-workspace-union under user %USERID% with parameters like:
/usr/bin/fuse-workspace-union %HOME of USERID%/Workspace \
--bind-directory=/var/lib/workspace/%USERID%/bind \
--conf-directory=/var/lib/workspace/%USERID%/conf \
--autofs-mount-directory=/mnt/mount.md5key/%USERID%/mount \
--autofs-cache-directory=/var/cache/mount.md5key \
-o allow_root,default_permissions,use_ino
The directory /var/lib/workspace/%USERID%/bind is empty, so there will also be nothing in the directory %HOME of USERID%/Workspace.
You can test it works with:
(if ~/Documents does not exists already create it first)
ln -sf %HOME of USERID%/Documents /var/lib/workspace/%USERID%/bind/Documents
mkdir /var/lib/workspace/%USERID%/conf/Documents
You should see now in the Workspace map the folder Documents appear, just like it is there, while in fact it is a symlink.
Without any resource record in /var/cache/mount.md5key nothing happens.
To create these for the SMB resources, edit the mount.md5key.conf
file in /etc/mount.md5key. Look especially at :
... GLOBAL_NETWORK_NAME=Network GLOBAL_NETWORK_ICON=.... SMB_USERID_BASE=local or remote SMB_AUTH_METHOD=guest, public, private or kerberos SMB_CRED_FILE=... SMB_NETWORK_NAME="Windows Network" SMB_NETWORK_ICON=.... ...
The different names you can choose what suites you, the most important here are SMB_USERID_BASE and SMB_AUTH_METHOD. The first is about
the accounts are only local or distributed. If the userid's (and groupid's) reported by the SMB server do have a meaning on this machine,
you have to choose "remote" here. For example an account database in LDAP, which is a source for the local machine via nss_ldap and pam_ldap
as well is used by the SAMBA server, or directly via Winbind.
If you don't know what this is about, take "local" here.
Next is the SMB_AUTH_METHOD. If you do not have credentials for the SMB servers, just select "guest". This is also a good start.
When you have credentials, but used globally, so for every user the same, select "public". If these credentials are personal,
thus differ between (local) users, select "private". Public or private credentials, if one of these two is used, the parameter
SMB_CRED_FILE must be set. This may be an function. Variables like homedir,userid are available. Look at the configuration file
for details. Last but not least you can select "kerberos", when in such an environment. I've not tested this, so if anything does not
work here please contact me (and please a patch).
When ready, run the script to detect all the SMB resources in your network: /etc/session.d/scripts/start/discover-smb-resources.sh.
This script uses one parameter, the userid of the user for who the resources are to be found. If omitted, the script will take all the users
for which there is an autofs managed mountpoint in /mnt/mount.md5key is running by looking at the auto.master file in
/var/run/autofs.
So, you can just run it:
/etc/session.d/scripts/start/discover-smb-resources.sh
If everything works, there should be some records in /var/cache/mount.md5key, containing information about the SMB resources found.
Note that it tries to use the utility nbtscan. If it is not found, it will use nmblookup instead. Nbtscan is preferred, it an excellent utility.
Important to say that nmblookup "needs" the smb.conf file to be installed (default: /etc/samba/smb.conf).
Last but not least, this script calls the manage_workspace.sh script, which will create the various directories in
/var/lib/workspace/%USERID%/{bind,conf}, according to the resources found. For example, if the resource record
for SMB server ADMIN in workgroup MYCOMPANY and share public is created, this script creates the link
/var/lib/workspace/%USERID%/bind/Network/Windows Network/MYCOMPANY/ADMIN/public ->
/mnt/mount.md5key/%USERID%/mount/...the right md5 key...
At the moment of writing, manage_workspace.sh supports SMB, FTP and USB (devices).
MOUNTING OF LOCAL DEVICES like USB
to be written...udev configuration...disable handling of these devices by HAL...extra scripts to handle adding/remving devices
.... detection of all already plugged in USB devices when a session starts...
Just like described here:
http://en.gentoo-wiki.com/wiki/Autofs#UDEV_with_Autofs
it's possible to mount USB devices with autofs. This is not suprising. In this approach it very easy to describe an USB device as
a resource, it's mountable with the standard mount commands. On my system a resource record for and USB device looks like:
cat /var/cache/mount.md5key/6e3744ccdd878f01a7e71088c8438ee5/config RECORD_type=usb RECORD_security=public RECORD_resource_uri="usb://4A09-FE39" USB_filesystem=vfat USB_model=USB_DISK_28X USB_vendor_id=13fe USB_model_id=1a00 USB_uuid=4A09-FE39
As you can see I've created the resource URI from the uuid, since this is an unique identifier. Futher notice the absence of the explicit
device path like /dev/sdb1 or /dev/sdc1. This device path is not always the same: it depends on the other devices already present, so this makes the record
change every time, and the directory it's in, making thing unnecessary complicated. Now, this path is not necessary, cause the exact path is
always reproduceable: /dev/disk/by-uuid/$USB_uuid is on Linux a symlink to the right device, in this case /dev/sdb1.
Just like in the documentation of Gentoo described an extra action is required, which makes udev run a script when a device is added and one
when it's removed:
.... snip ....
# by-label/by-uuid links (filesystem metadata)
ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}", \
RUN+="/lib/udev/udev.change.add", ENV{REMOVE_CMD}="/lib/udev/udev.change.remove"
Activating this mounting via autofs requires some steps:
- the kernel notices a device is plugged in
- udev is triggered, and rules are applied, and the script /lib/udev/udev.change.add is run (with several environment variables set)
- this script creates the right md5key record, and the calls:
- manage_workspace_sh, which updates the various workspaces in use by adding this device. Names chosen by the user are applied.
Above decribes the steps when a device is plugged in when users are logged in. The other way around is also possible: when an user
logs in when the device is already plugged in:
- when user logs in, a script is run (by ConsoleKit or Upstart) to check all USB devices present on the system, to create the resource
record. You maybe say, well it's plugged in, so there must be a resource record already, udev must have detected it earlier. This is
not always true, when a device is plugged in when the machine is off, and the switched on the udev rules are not applied. I do not know
this happens on all machines, it works this way on mine (Cross Linux From Scratch oktober 2009).
When done it also calls manage_workspace_sh:
- manage_workspace.sh, which distributes the changes into the active workspace directories.
When a device is removed, the /lib/udev/udev.change.remove script is run. The only it does is not removing the resource record, but
creating a notvalid file in the md5key cache. In the case of the example above:
touch /var/cache/mount.md5key/6e3744ccdd878f01a7e71088c8438ee5/notvalid
This marks the record as unvalid. Later, when this record is not used anymore it's safe to remove it.
(compare the "deletion" flag at database records).
At this moment udev notices HAL about the device. There is nothing wrong with that, but HAL on his turn notices the desktop environment
(in my case KDE), and providing information how to mount/unmount and access it. The mountpoint is at /media, which leads obviously
to conflicts/confusion when using this md5key/autofs construction. So disable HAL by adding the following to the preferences.fdi:
|
I've been looking for a way to make HAL cooperate with my
construction, but this is not easy. A major issue is the location (=directory) where the device is attached/available. With HAL
this is /media/%MODEL%, with my construction it's mounted at /mnt/mount.md5key/%USERID%/mount/%md5key%, and for the user available
at %HOME of USER%/Workspace/Devices/%MODEL%. So,. per user different maps where the device is available. HAL is not able to export
more than one "mountpoint".
AUTOMATION: make everything just done for you
to be written...with help of ConsoleKit...with Upstart everything becomes simpler...
INSTALL SCRIPT
...to be written...
WHAT WORKS
- locking
with the help of the fuse lockmanager, locking of files on the target filesystem, for example a document on a share mounted with cifs,
works. The command smbstatus on the remote samba server reports the lock.
- extended attributes and posix acl's
As long as the underlying target filesystem supports it, extended attributes are supported.
WHAT DOES NOT WORK
- inotify
At this moment the fusemodule does nothing with changes in the (target) underlying filesystem. I'm trying to find out how to enable it,
look at this message at the fuse-devel maillist:
[fuse-devel] How does inotify work with fuse.
It's not so easy for me to understand inotify, and especially with FUSE. First, what is a INOTIFY request?? I think that it is a request
to monitor a directory (and it's contents, not recursive) or a single file. For example, Dolphin, when opened, will want to monitor the
directory it's showing. So Dolphin will do a syscall to monitor this directory, and the system (kernel) will pass this through to the
relevant fs, in this case FUSE, which on his turn will pass it through to the module.
To experiment with inotify, the websites:
general info : http://inotify.aiken.cz/
inotify tools : http://wiki.github.com/rvoicilas/inotify-tools
are very usefull.
Now, my module has to recieve and manage the inotify requests coming from the clients, and pass them through to the various
underlying fs's. So, it has to manage a list with these requests. With Linux/inotify (and maybe others) filedescriptors play an
important role. Via poll/select/epoll a list a these fd's is watched. Note the following mail:
[fuse-devel] Echoing poll()/select() calls to underlying filesystems
Here is supposed the filesystem itself to be represented as a single filedescriptor, along with the other fd's, like inotify calls/requests.
This idea is very usefull, it makes the whole fs with a seperate list of fd's very manageable.
- interaction with the automounter
I would like that the fuse module has information which resource is mounted by the automounter, and which not. Also on this topic
I've posted a message at the autofs maillist:
[autofs] How to determine a path is a mountpoint
I cannot oversee what is more needed to make this construction work as it should. I've heard something about direct mmap support.
I only know this is a topic being worked on.
SCREENSHOTS
...some screenshots...
Automate this process... with the help of ConsoleKit, fcron (and in future UpStart) and manage-workspace.sh
futher ideas: mount the homedir to somewhere else, and use this as input again for fuse-workspace-union, which results in a
userfriendly map, containing links to all kind of places....
This software and the construction is distributed under the terms of the GNU General Public License. A copy you'll find at the src directory.