Linux

Download location (HTTP):       http://linux.bononline.nl/linux/mount.md5key.new/mount.md5key/src

Automount with md5 keys.

Here I describe a setup of the automounter, intended to make local and remote mountable by the automounter, as
a mounthelper for a fuse module (fuse-workspace-union) which presents this resources in a userfriendly
format in the homedirectory of the user.

To make this all work, a simple setup of the automounter is necessary. It should work with a md5sum as key,
and it's really a good thing, this is posssible with the automounter in Linux.
Now, I've been working a lot with different setups for creating multi-mount and executable maps,
and do not have very good experiences with the automounter when it comes to handling more complex
maps than one level deep. The construction should work with more than one user logged in, where users
are logging in and out, this means that autofs managed directories for these users are added and removed,
while the automounter is still running, and other users do not notice any difference.

This means that when a usersession starts, a line (with mountpoint and map) is added to an auto.master file,
and that this line is removed when this usersession ends. To make these changes visible, a reload signal is necessary.

Now this construction does not need complicated executable maps, just a mapping between keys is sufficient.
Futher the mounpoint does not have to be in the homedirectory, but can be anywhere. I've chosen

/mnt/mount.md5key/%USER%/mount	/etc/autofs/auto.md5key    MD5KEY_USER=%USER%

The auto.md5key map is a very simple map, using the non existent filesystem md5key:

*    -fstype=md5key :/&

WOW!! That's all?? Yes, this map instructs the automounter to pass every key to the mount command, which
on his turn will look for the command mount.md5key, and pass the mountpoint and key to it
(and the option MD5KEY_USER=%USER%). Now all intelligence is in the mount command, which will check
the key, lookup the information belonging to this key, and execute the right mount command with the right
parameters.

This all is possible because the automounter allows the filesystem (md5key) be virtual, not a real filesystem,
which also counts for the mount command. This just looks at the name of the filesystem and looks for the associated
mount.md5key and passes the parameters to it:

mount -t md5key

makes the mount command look for

mount.md5key

without testing md5key is a *real* filesystem!
The mount.md5key command is a so-called wrapper, which executes the right mount command for the resource, for example
mount.cifs for SMB shares, sshfs (is FUSE module) for SSH directories, curlftpfs (is FUSE module) for FTP servers, and
the traditional mount command for local resources like an USB disk with an EXT2 or VFAT filesystem.

RESOURCE RECORDS


As mentioned above this construction relies havily on "resource records", a table of records, each record containing
\information about a resource (remote or local), for example a share on a SMB (Windows or Samba) server, a FTP host or a SSH
directory:

These records contain the right information to make the right command do the mount with the right parameters.
This record is stored in a directory as md5key. This looks like:

/var/cache/mount.md5key/e4c43703d22a84c7f0a03859626cb76e contains

			.directory
			config
			permissions

Because the data is temporarly, but should preserved between different runs, the location /var/cache/mount.md5key
is logical.
To explain the different files:

.directory - this file is used in KDE to present the layout of a directory. This is not used by the mounting self,
but by fuse-workspace-union.
.directory.mounted - .directory file used when the resource is mounted.
.config - this file contains the actual data about the resource, like ipnumber, name, credentials etc.
permissions - a directory also used by fuse-workspace-union, to determine the permissions of the resource,
to prevent unnec. mounting.

#> cat /var/cache/mount.md5key/e4c43703d22a84c7f0a03859626cb76e/config
RECORD_type=smb
RECORD_security=private
RECORD_creator=discover-smb-resources.sh
LOCAL_user=sbon
SMB_workgroup=BONONLINE
SMB_name=LFS20060812
SMB_share=bononline
SMB_ip=192.168.0.2
SMB_auth_method=private
SMB_credential_file=/home/sbon/.smb/mount.cred

The command

ls -l /mnt/mount.md5key/sbon/mount/e4c43703d22a84c7f0a03859626cb76e

will mount the share bononline of server LFS20060812 at this directory, using CIFS.

The records above is an example of one resource, one "shared" directory on a SMB server. The utility
discover-smb-resources.sh finds all the SMB servers and all the shares on them, on my system this results in
22 records:

ls -l /var/cache/mount.md5key
total 96
drwxr-xr-x 3 root root 4096 2010-01-04 23:20 03c074d3b1c45ef7a68acc4ba7b6ad1b
drwxr-xr-x 3 root root 4096 2010-01-04 21:39 09ba8782fd28a51b37ab395fc5d91b64
drwxr-xr-x 3 root root 4096 2010-01-04 21:16 14d58d7d3de667bf5425857bf5a61148
drwxr-xr-x 3 root root 4096 2010-01-04 23:20 20a58a18773c06e7e175c49dfb6fd8c2
drwxr-xr-x 3 root root 4096 2010-01-04 23:20 2503f4c43c2bc924430fde9dcd80aa2f
drwxr-xr-x 3 root root 4096 2010-01-04 21:16 25d358e4d3a32d47b56f3f650ea8f5b1
drwxr-xr-x 3 root root 4096 2010-01-04 23:20 30ec6181b62efe5a86b9ecefbda59a07
drwxr-xr-x 3 root root 4096 2010-01-04 23:20 3f1603f9ed9e9d02630b16f7e38d0118
drwxr-xr-x 3 root root 4096 2010-01-04 21:16 4d29afc7294c40153f866b6c6f42abd9
drwxr-xr-x 3 root root 4096 2010-01-04 23:20 5555c2700db7f421bae84078095a2233
drwxr-xr-x 3 root root 4096 2010-01-04 23:20 60f15423102d56f7cc62f5aba8532d34
drwxr-xr-x 3 root root 4096 2010-01-04 21:16 6a51fd96447854caa87cd425c883b5e4
drwxr-xr-x 3 root root 4096 2010-01-04 21:16 911aa0702c265f24ffa95d64bc35a579
drwxr-xr-x 3 root root 4096 2010-01-04 23:20 a2bc963e16bf346633fa5cdd067dca13
drwxr-xr-x 3 root root 4096 2010-01-04 21:16 b4a0859ac795f50baf7e43ff4f49f314
drwxr-xr-x 3 root root 4096 2010-01-04 23:20 bbe4d6609adf145da58b8fab695c3446
drwxr-xr-x 3 root root 4096 2010-01-04 21:16 ca96cabb2a0733ce3b4eb1f547807101
drwxr-xr-x 3 root root 4096 2010-01-04 23:20 d9804c035eb8bea8541bedae3d542268
drwxr-xr-x 3 root root 4096 2010-01-04 21:16 dc6cd0ab7250fd30687517838c1c89c4
drwxr-xr-x 3 root root 4096 2010-01-04 21:16 ddf037550cfb92894350becc0b33a598
drwxr-xr-x 3 root root 4096 2010-01-04 21:16 e4c43703d22a84c7f0a03859626cb76e
drwxr-xr-x 3 root root 4096 2010-01-04 23:20 f7121ac71167a601d9b6967a2df92fa7

At this moment they are all SMB records, but this construction can also handle other services.
In fact anything is possible, as long it's mountable, direct or with the help of FUSE. For example
it's possible to mount a directory with SSH using sshfs, or FTP servers using curlftpfs.
I've not worked it out, but the format of the record is simular to the one shown above.


FORMAT RECORDS


The format I'm using at this moment is:

RECORD_type - the type of service this record is about. Example ftp, smb, ipx (Novell Netware), ssh, disk (local resource).

RECORD_security - for who is this record, is this for everybody ( security = public) or for one user (security = private)
If using no password and username (guest/anonymous), or one set of credentials for everyone, I call the security
public. If different users are using different passwords (or tickets with kerberos), the security is private.
When public, everybody can use the record, if private the record can only be used by one user, and that user
is set in the field LOCAL_user

RECORD_creator - name of the script which created this record

LOCAL_user - name of the user for who this record is for. Only set when RECORD_security is set to private.

SMB related fields

SMB_workgroup - name of the workgroup the server is part of
SMB_name - name of the server (this is the netbiosname)
SMB_share - name of the share
SMB_ip - ipnumber, required by mount.cifs. At this moment (january 2010) only ipv4.
SMB_auth_method - the authentification method. Possible choices are guest, public, private and kerberos
In this case (because security is set to private) only a private credential file of a kerberos ticket are valid.
SMB_credential_file - the file containing the credentials. In this case the file contains private credentials.


DISK related fields

DISK_bus=usb - the type of disk (2010-04-08: only USB supported)
DISK_filesystem - the filesystem on the partition
DISK_model - the modelname
DISK_vendor_id - id of vendor
DISK_model_id - id of model
DISK_uuid - UUID of disk


The field used I'm not sure about yet, developing still. For example a field for the creation time (RECORD_time)
for example, and records for other services. But they will be pretty simular to the SMB_ fields, like
FTP_server, FTP_directory, FTP_auth_method.

Note futher that some combinations are not possible. When that when SMB_auth_method is set to private (credentials)
the RECORD_security cannot be public.

The mount warpper mount.md5sum will look at these records, and translates them into the right mountcommand options
like "guest" when RECORD_security is set to public, and SMB_auth_method is set to guest, and
"guest,sec=krb5" when RECORD_security is private, and the SMB_auth_method is kerberos.


HOW IS THIS MD5 VALUE COMPUTED?

To make this construction work a simple key is necessary, only one level deep, where the name does not has to
represent the resource. The user does not have direct "contact" with these keys, they are used behind the scenes.
I've chosen for the md5 value of the record self. The md5sum of the config file is equal to the name of the directory
it is in:

(suppose a record is created in TMPDIR)

md5key=$(cat $TMPDIR/config | md5sum | awk '{ print $1 }')

install --directory /var/cache/mount.md5key/$md5key
mv $TMPDIR/config /var/cache/mount.md5key/$md5key

WHY MD5?

A short answer: it works. With this type of keys (unique, allways 32 char long) the mount command can determine
the right record very easy. Futher the FUSE module fuse-workspace-union can also work with these keys in a
very non complicated way, when determing the .directory file and the cached permissions of the mountpoint.
But then the question, why records anyway, and not let all the records determine the resources directly? You're
Introducing an extra layer, which makes things much more complicated.

Well, having a central place with all the resources, to be used by all kind of applications/processes/users
is a good thing. Keeping this database up to date is another issue. For SMB this is possible by using the utilities
nbtscan, nmblookup and smbclient, but other utilities like Service Location Providers (Avahi) and entering values
manual, are also possible.


MAINTENANCE

Note futher that there are only two changes are possible: a record is added or removed. A record cannot change,
because the record md5key is directly connected to it's content. A change in this content this leads directy to
another md5 value, which will result in removal of the old record, and creation of a new record.

Futher, a change in the resources does only affect this "database" of resource records, not the construction
with the automounter, or the FUSE module. This change in resources, for example a new SMB server, will result in
new records. The new records will be propagated to all users with a Windows Network tree. When this change is visible
to the user, he/she can make use of it. When he/she does so, the link to the new record will activate the automounter,
mount.md5key will lookup the new key, which of course is present, and will perform the requested action.

As you see, no reloading of the automounter is necessary, which is a good thing. Earlier I had a construction
where such a change in resource was only visible after a reload, which lead to a lot of instabilities...

For the smb protocol, there are several ways to discover network resources (workgroups/domains, servers and shares).
There are utilities like nbtscan and nmblookup, which detect the smb hosts, and smbclient which finds all shares per host (per user).
Futher there are service locators like Avahi and OpenSLP.

I've written a script (discover-smb-resources.sh) which uses nbtscan (if not present it takes nmblookup).

DIFFERENT WAYS of AUTHENTIFICATION and SHARING of ACCOUNTS at SMB

There are much more than one way a user (or processes part of the session started by the user) on a machine can authenticate
him/herself to an other process on an other machine. Futher the two hosts can share the same accounts database
(via LDAP or Winbind) or do not.
All these different situations result in different mount options.

Overview of different ways to authenticate:

A: guest authentification, the same for every user on this machine. Examples: for cifs shares the "guest" option, for FTP sites
the "anonymous" option. The record is shared (public).

B: one set of username and password, the same for every user. Example a file smb.mount.cred in /etc/mount.md5key, the same for
every user. Important is that the credentialsfile in no way depends on the user. Like the guest auth., the record is public.

C: every user own private set of username and password. The credentials depend on the user, like $HOME/.smb/mount.cred.
The record is private, and belonging to the user.

D: every user own private kerberos ticket (by definition). Same as with private credentials, record is private.

Note that applications do not have to deal with the authentification, when trying to access a resource, this construction does that. When
a record is created, the way how to deal with authentification and the credentials/tickets/auth.tokens (if needed) have to be known.
This information is used by the relevant mount command. At this moment there is no need for dialogs popping up asking access tokens..


Not sharing the accounts with the target server you're connecting with, means for example that the options uid and gid
are set to the user using the mount.

You can say that there is a strong relation between shared accounts and private auth. mechanism, but this is not always true.
Sometimes I've got a setup where the user has a private set of credentails, but the user (owning the credentails) self has no
meaning on the target server.

DIFFERENT RECORDS for the SAME RESOURCE available for the USER

It's very possible that different records for the same resource are available to a user ate the same time. For example a
share on a SMB server, where the possible authentification methods are guest and private credentials, so it's also very
possible that more than one record exist in the cache for one resource. It's very logic to make the construction offer the
records with the "best" auth. method to the user. But there are some situations where this is not the preferred method.
For example the records for SMB shares on a server, part of a "foreign" domain. When your credentials do not have a meaning
in this domain (eg there is no trust relationship) you are actually a guest, so you should make use of this authentification.

To summarize the notes:

- always try to take the "best" auth. method. The order is: guest, public credentials, private credentials, keberos.

- there are situations to take not the "best" auth. method.

- when resources form a group (in SMB world: a domain) the auth method for all resources part of this group should be the same

ANYTHING MOUNTABLE WORKS

Everything which is mountable (via Kernel or Fuse) works. At this moment the script mount.md5key is only capable of mounting
SMB shares (by passing the paramters to mount.cifs), but extending it with other services is possible. Mounting of SSH with
the FUSE module sshfs works, mounting of FTP servers is still problematic, Novell Netware should also work, but I've not tested
this. For FUSE filesystems the system needs of course support of FUSE, in case of Novell Netware the kernel should support
the IPX networkservice.

MOUNTING of LOCAL DEVICES

It's very possible to mount USB and other local devices. Udev provides ways to run a script when a device is added.
The only thing this script should do is to add a md5 record for this resource, and adjust all the workspace directories
of the users, making the device visible for the users.
This works, but there are some points:

- now udev triggers HAL, which on his turn triggers the user environment (KDE/Gnome). This should not happen.

- the graphical environment should use the directory offered by this construction.

- How to deal with "policykit"??

KIO slaves

I've posted about this construction at KDE Brainstorm

idea to get easy access to local and remote resources

In the reactions on this idea speak of problems with applications not knowing a file is local or remote of origin.
We'll it's not so hard to determine this origin, given the path. By comparing the path of this file to the valid
paths to remote resources (for this user):

	URI						resource record			path for the user (relative to mounpoint workspace)
smb://LFS20060812/bononline			e4c43703d22a84c7f0a03859626cb76e	Network/Windows Network/BONONLINE/LFS20060812/bononline
smb://LFS20060812/public			25d358e4d3a32d47b56f3f650ea8f5b1	Network/Windows Network/BONONLINE/LFS20060812/public
..
..
smb://ROUTER/sbon				b4a0859ac795f50baf7e43ff4f49f314	Network/Windows Network/BONONLINE/ROUTER/sbon
ftp://192.168.0.2				911aa0702c265f24ffa95d64bc35a579	Network/FTP servers/192.168.0.2
..
usb://8b056374-3cae-44dc-a87b-c5217e7a9e0e	dc6cd0ab7250fd30687517838c1c89c4	Devices/USB_FLASH_MEMORY_1

So it's not that hard to find the resource record belonging to path, if there is any. If there isn't one, the file is local
(well according to this construction).
A few notes here: I've added here an URI for an USB device, constructing it from the uuid. I do not know there is already a
way to determine the URI of a partition on an USB device, so for now I'm taking this one.
Futher, I've already added a field in the resource record for the URI, like:

RECORD_resource_uri="smb://ROUTER/sbon/"

This value is redundant, it's very easy to contruct it from the other field, but with it it's possible to do a (text) search
to this value.

Apart from being remote or not, additional information is there to be used. For example the underlying filesystem
(cifs, sshfs, curlftpfs for network, ext2, ntfs for local), and it's characteristics and the way it's mounted..

Licence

This software and the construction is distributed under the terms of the GNU General Public License. A copy you'll find at the src directory.