Saturday, June 21, 2008

Python + inotify = Pyinotify [ how to watch folders for file activity ]

Sometimes it just might be handy to be able to watch a folder on a hard disk for changes. For example: A client app might drop small files on a shared folder. A server app might be watching the folder for just such an event. Once the file is created, the server will kick into action and perform whatever tasks are required.

This all comes from my CD burning application. I am currently thinking that the client apps will drop small xml files containing information about what to burn onto a folder the webserver has access to and the cdburner service will be watching...

The linux kernel provides inotify. This from wikipedia:
notify is a Linux kernel subsystem that provides file system event notification. It was written by John McCutchan with help from Robert Love and later Amy Griffis to replace dnotify. It was included in the mainline kernel from release 2.6.13 (2005-06-18), and could be compiled into 2.6.12 and possibly earlier releases by use of a patch. Its function is essentially an extension to filesystems to notice changes to the filesystem, and report those changes to applications.
Pyinotify is a python module that exposes the inotify api in python. This from http://pyinotify.sourceforge.net/:
pyinotify is a Python module for watching filesystems changes. pyinotify can be used for various kind of fs monitoring. pyinotify relies on a recent Linux Kernel feature (merged in kernel 2.6.13) called inotify. inotify is an event-driven notifier, its notifications are exported from kernel space to user space through three system calls. pyinotify binds these system calls and provides an implementation on top of them offering a generic and abstract way to use inotify from Python. Pyinotify doesn't requires much detailed knowledge of inotify. Moreover, it only needs few statements for initializing, watching, handling (optionnaly trough a new separate thread), and processing events notifications through subclassing. The only things to know is the path of items to watch, the kind of events to monitor and the actions to execute on these notifications. Note: pyinotify requires Python 2.3 and above, and Linux 2.6.13 at least.
I went ahead and gave it a go. I did find though, that on my fedora system ( fedora 9 x86_64 ) that the tutorial on the above wiki didn't quite work. Here is the tutorial code for the non threaded example that does work on my system.


#!/usr/bin/python
import os
import pyinotify

wm = pyinotify.WatchManager()
mask = pyinotify.IN_DELETE | pyinotify.IN_CREATE

class PTmp(pyinotify.ProcessEvent):
def process_IN_CREATE(self, event):
print "Create: %s " % os.path.join(event.path, event.name)
def process_IN_DELETE(self, event):
print "Delete: %s " % os.path.join(event.path, event.name)


notifier = pyinotify.Notifier(wm, PTmp())

wdd = wm.add_watch('/home/dave/projects', mask, rec=True)

while True:
try:
notifier.process_events()
if notifier.check_events():
notifier.read_events()
except KeyboardInterrupt:
notifier.stop()
break

Unfortunately inotify is a Linux Kernel technology that is not currently available on windows. I guess Windows has some other kind of API for filesystem event monitoring but if you are like me and want to keep things simple with python then I am afraid inotify on windows is not possible. If you feel that you would like to give this stuff a go then I suggest getting yourself a LiveCD of one of the Linux Distros. They can be bought here.
Post a Comment