Linux

Download location (HTTP):

Introduction

This howto is about the easy user access to various resources (local and remote), based upon a list of records containing
information about the resource, the automounter able to mount any of these resources and a fuse module which presents all the
the resource in a browseable map:

A table with "resource records"

This construction is based upon what I call "resource records". These records contain information about the resource
like SMB (=Windows and/or Samba share), USB (=local usb disk), FTP (remote ftp host), ATA disk (=local harddisk) and
SSH (=access to remote ssh host), and information specific for that service.
To find and identify records uniquely these records are indexed using the md5sum value of their contents.

An example of such a record:

RECORD_type=smb
RECORD_security=private
RECORD_creator=20.discover-smb-shares.sh
RECORD_resource_uri="smb://SCLFS20091030/sbon/"
LOCAL_user=sbon
SMB_workgroup=BONONLINE
SMB_name=SCLFS20091030
SMB_share=sbon
SMB_ip=192.168.0.2
SMB_auth_method=private
SMB_credential_file=/home/sbon/.smb/mount.cred

The md5sum value is 601d329b6fd5c9d801e620ed7ac41606, and the path of the record is:

/var/cache/mount.md5key/601d329b6fd5c9d801e620ed7ac41606/config

Automounting records using their md5sum value as key

Part of this construction is a configuration of the automounter being able to mount the resource, for example
the Samba share smb://SCLFS20091030/sbon, only having the md5sum of the record.

The directory where resources are mounted this way depend on the userid, in my case is that sbon:

/mnt/mount.md5key/sbon/mount

When I access the path by doing:

cd /mnt/mount.md5key/sbon/mount/601d329b6fd5c9d801e620ed7ac41606

the share is automatically mounted.

Make these resources available in the users homedirectory

With the mounting facility available, the resources should be available in the users homedirectory, and nowhere else.

To make this possible, I've developed the Fuse module fuse-workspace. With Fuse it's possible to write your own
filesystem, and make it behave like you want it to do on commands like readdir, getattr, open and write.

The Fuse module I've written, creates a browseable tree with userfriendly names like:

Network/Windows Network/BONONLINE/SCLFS20091030/sbon/ ... contents of share ...
 |                                           |  |  | |                         |
 ---------------------------------------------  ---- ---------------------------
                     A                           B               C

The first part A is an userfriendly path to the resource. The names used here are configurable.

The second part B is the actual link to the resource. It's actually a symbolic link, but presented by the
Fuse module as directory. This makes it look like the share is mounted here, but in fact it's mounted somewhere else:

/mnt/mount.md5key/sbon/mount/601d329b6fd5c9d801e620ed7ac41606

The third part is the contents of the resource, in this case the SMB share smb://SCLFS20091030/sbon.

Features

Launched automatically

Through the use of ConsoleKit and the at daemon as launchmanager all the scripts and programs necessary
are started parallel to the current session. At this moment I'm using "at" for that, I know there are (better?)
alternatives like Upstart and System.d. I haven't worked on this issue, but I guess that integration with these
init alternatives is very possible.


Not System and not session related

About sessions as defined by dbus: this service is not part of a session, but started in parallel, and - important -
it's only started once for a user. So if a user logs in again, resulting in two sessions for one user, this
service not started again or something like that. The scripts which are taking care for the starting and the stopping
when a session is added or removed, check it's the first session or not.

At this moment (2 september 2010) there is no dbus integration at all.

I've tried to add "messages" to the construction, notifying the user an USB device is plugged in for example, but that
was not possible, cause the scripts taking care of the detection of the hardware and the adding it to the the
workspace, are not part of the session. I wanted to make use of the service provided by "org.freedesktop.Notifications",
but that service does not has any connection with system/user related messages. By design only apps part of the same
session are able to make use of that. It's ideal when every user logged in (and using a dbus aware environment)
recieves a "personal" message. At this moment that's not possible, and I've been wondering about this for some time now.
Soon I will post a message at the right maillist about this issue.

At this moment I'm using the utility "wall" to send a message.

Same functionality as Gnome VFS, but...

This construction offers the same functionality as Gnome VFS, but there are some important differences:

- Gnome VFS only works in an Gnome environment, and if the application you're using is linked to the right libraries.
My construction works with every environment, KDE, Gnome, LXDE and even a console (text) login. The construction is started
by ConsoleKit, and it does not make any difference the session is about.

- my construction is making use of native filesystems, like cifs for mounting SMB shares, where Gnome VFS has it's own way
to connect to SMB servers, cifs is the best filesystem for that purpose. It's a project of it's own, and very active. A lot
of people are working on it.


Automatically up to date

This construction tries to stay up to date. For local resources (=hardware) this is easy, using udev.
For remote resources that's not easy. This construction makes use of the utilities nbtscan (or nmblookup) and smbclient
to "poll" (pulling) the network regular, but it would be better when using some mechanism which pushes the changes to
from a remote host to this machine....something like OpenSLP..

Various issues

DETECTION of RESOURCES - local and remote

To make this work I had to setup a system to detect all kinds of resources, local and remote.
The localresources are relativly easy, using udev.
Detection of remote resources are more difficult, how to find a FTP server for example. To detect
smb resources I'm using the utilities nmblookup (or nbtscan) and smbclient.
I'm working on the use of service location providers like OpenSLP. Futher I'm testing how to detect
resources when in a Novell network using OpenEnterprise server.

To make "polling the network" work I had to develop a construction which manages the scheduling and
rescheduling and dynamically adding and removing of scripts. While there are tools like cron and fcron,
none of them really was capable to do this, reason for me to use the at daemon (and some extra scripts).


QUEUE MANAGER

To make this all work, I needed construction to run scripts (for example to add an autofs managed mountpoint,
and to start the fuse module fuse-workspace) when a session starts and when it ends. There was not such a
"launch manager" which worked for graphical AND text logins. I had to add some code to the default
installation of ConsoleKit, which has the ability to run a script when a session is removed, added or changed.


INOTIFY and FUSE - work to be done

Inotify does not work yet. Just like other filesystems like cifs, the VFS is not aware of the changes in the fs.
As a result, inotify does not work properly. It's not easy to make this work. A new structure in the Fuse
kernel module, as well as in the Fuse library to detect a "inotify request" is necessary, and mapping of this request
to an event in userspace by the fuse fs is necessary.


PROVIDING INFORMATION to apps / desktop

Providing information about the underlying fs/resource is possible through extended attributes. At this moment only working
is the call:

getfattr --name="system.md5key" ..some file..

gives the md5key of the resource(as explained above). It's very possible to extend this information, or to provide
information another way, for example a plain file containg the various resources available (and the path to it).

At this moment (19 july 2010) this already works, like:

sbon [ ~/Workspace/Network/Windows Network/BONONLINE ]$ getfattr --name=system.md5key SCLFS20091030/
# file: SCLFS20091030/
system.md5key="a2b3b4b67cef2df3ce13abd22c419637

sbon [ ~/Workspace/Network/Windows Network/BONONLINE ]$ cat /var/cache/mount.md5key/a2b3b4b67cef2df3ce13abd22c419637/config
RECORD_type=smb
RECORD_security=public
RECORD_creator=10.discover-smb-servers.sh
RECORD_function=secondary
SMB_function=server
SMB_workgroup=BONONLINE
SMB_name=SCLFS20091030
SMB_ip=192.168.0.2

So asking the fs information about the kind of resource (system.md5key) gives the md5key, the index of the record stored in
/var/cache/mount.md5key.
Here we're dealing with the SMB server SCLFS20091030, ip 192.168.0.2 and part of the workgroup (or domain?) BONONLINE.

Note one double quote in the answer. I do not know why this happens. This is one of the open issues.

Going into one share of this server:

sbon [ ~/Workspace/Network/Windows Network/BONONLINE ]$ cd SCLFS20091030/video
sbon [ ~/Workspace/Network/Windows Network/BONONLINE/SCLFS20091030/video ]$ getfattr --name=system.md5key .             
# file: .
system.md5key="aeb8537f8c347da7dcfcc2e0dd48749d

sbon [ ~/Workspace/Network/Windows Network/BONONLINE/SCLFS20091030/video ]$ cat /var/cache/mount.md5key/aeb8537f8c347da7dcfcc2e0dd48749d/config 
RECORD_type=smb
RECORD_security=private
RECORD_creator=20.discover-smb-shares.sh
RECORD_resource_uri="smb://SCLFS20091030/video/"
LOCAL_user=sbon
SMB_workgroup=BONONLINE
SMB_name=SCLFS20091030
SMB_share=video
SMB_ip=192.168.0.2
SMB_auth_method=private
SMB_credential_file=/home/sbon/.smb/mount.cred

This looks good. All values used here are exactly right!


IDEA: mount fuse-workspace at the homedirectory

I'm having the idea to mount the fuse module direct at an existing homedirectory. Then the directories Devices, Harddisks and
Network will be available direct in the users homedirectory, making it more intuitive.
I've also have some (wild) idea's to create a complete different environment using chroot and the module, like GoboLinux. Then
the user will not have a traditional Unix like environment with the directories /bin, /etc, /lib, /home etcetera, but only
the directories in the homedirectory. A call to a program in /bin for example will be redirected to original environment.


IDEA: make the filesystem remove unused old inodes and entries

At this moment inodes and entries are created when the user just uses the filesystem, and they will exist till the fs is
unmounted, or the kernel VFS sends a forget signal. This latest call is only done when the kernel needs space, en does that by
freeing inodes, not because they are not used for a long time.
It's necessary to modify FUSE to make it do this when otherwise idle.


IDEA: add dns domain information to the resource records which are networkrelated

I've used this construction on my laptop on another location then my home. The various script doing the discovery
of the networkresources and those for creating and maintaining the various paths in the workspace (like the dir to the workgroup
Network/Windows Network/BONONLINE and the servers in it) detect that the records for these resources were not valid anymore.
Adding extra information to the record regarding the DNS domainname, makes that process go more smootly. This changes for
example the status of a record from "notvalid", which is now used to mark records which do not apply, to "notvailable".


IDEA: present unformatted USB and others disks

At this moment partitions of disks (USB/SATA/..) which are not formatted, or if a disk does not have partitions at all,
they are ignored. It would be very nice to present them also, and offer the user the choice in the action list for example
partitionmanager. That would be very nice!
Required is a method to make the choice of partitionmanager available to Dolphin somehowe. This construction is capable of
manipulating the .directory file. I'm looking it's possible to add an extra "action" here.
The desktop I'm using works with mimetypes, and maps a list of actions to it. That does not work here. An unformatted USB
drive is just a directory here.
Futher I'm thinking to create an extra map for unformmated disks and partitions (like: Unformatted disks) or present these
in existing directories (like Harddisks and USBdevices) with an modified name, like Flask_Disk_Unformatted.
And what to do after the disk is formatted? It's a very nice to think about how to do that.


IDEA: present the user a message through the new inotify message system

This is not so hard. I've found a script which sends a message through dbus, and it works perfectly.


IDEA: add cdrom drives, with or without a cdrom.

At this moment cdroms are not handled yet.

Contacting me

If you would like to contact me, send an email to stef on this domain, bononline.nl.

Licence

This software and the construction is distributed under the terms of the GNU General Public License. A copy you'll find at the src directory.