03
Mar
09

python recursive directory walker

By John 6 Comments
Categories: files, python and scripting


I don’t like python’s os.walk, so every time I need to do some work with a lot of files/directories, I end up writing my own ( because I don’t save the code ). So, this time, I decided I’ll post the code here so that I don’t have to rewrite every time :


import os,sys

class DirWalker(object):

	def walk(self,dir,meth):
      """ walks a directory, and executes a callback on each file """
		dir = os.path.abspath(dir)
		for file in [file for file in os.listdir(dir) if not file in [".",".."]]:
			nfile = os.path.join(dir,file)
			meth(nfile)
			if os.path.isdir(nfile):
				self.walk(nfile,meth)

It’s simple, and gets the job done.

6 Responses to “python recursive directory walker”

Feed for this Entry Trackback Address
  1. 1 Sébastien
    March 5, 2009 at 10:05

    I didn’t know you could give functions as parameter to another function, this is great 🙂
    (I’m beginning to learn python atm).

    Reply
  2. 2 geo
    March 5, 2009 at 11:04

    Yeah, it’s pretty great to be able to have callbacks like this, and not in the Java way 🙂

    Reply
  3. 3 Meloman
    September 3, 2009 at 21:48

    Thank you for the code – should be helpful since I got into the Python project without any Python experience, so reusable pieces like that are great.
    Question. I’m importing this class and trying to run a test call like so:

    DirWalker.walk(‘c:\\testpyt’, os.open(file, ‘r’))

    The files inside of c:\testpyt are several flat html files, nothing crazy.

    I’m getting an error:

    Traceback (most recent call last):
    File “”, line 1, in
    TypeError: coercing to Unicode: need string or buffer, type found

    Is it this enough info for you to chuckle and tell me where I’m doing something wrong?

    Thanks in advance 🙂

    Reply
    • 4 geo
      September 4, 2009 at 06:06

      I think your error refers to the file variable. Maybe it gets interpreted as the ‘file’ type, I can’t say for sure. From you’re example, you’re using the DirWalker class incorectly. Walk is not a static method, it’s actually an instance one, so for it to work, you need to create an instance of it. Here’s an example that may get you started :

      
if __name__ == "__main__":
        # this is the callback we're executing on each found file
	def cb(file):
		print file
	DirWalker().walk("d:\\",cb) # notice that by saying DirWalker(), a new instance gets created .
      Reply
  4. 5 Cedric
    April 12, 2010 at 07:20

    A super nice way to do! I used it to look for specific file type. Here is the one I modified.
    Enjoy! 🙂

    import os,sys

    class DirWalkr(object):

    def walk(self,dir,meth, fileType=”.xml”):
    “”” walks a directory, and executes a callback on each file “””

    dir = os.path.abspath(dir)
    for file in [file for file in os.listdir(dir) if not file in [“.”,”..”]]:
    nfile = os.path.join(dir,file)

    if(os.path.isfile(nfile)):

    basename, extention = os.path.splitext(nfile)
    if(extention == fileType):
    meth(nfile)
    if os.path.isdir(nfile):
    self.walk(nfile,meth)

    elif(os.path.isdir(nfile)):
    self.walk(nfile,meth)

    Reply
    • 6 geo
      April 12, 2010 at 08:27

      Nice 🙂

      However, I think it would be better to use a function for selecting the file based on extension/content, or whatever criterion you want.
      That way the class would support a finer level of selection … or, if not, maybe supply the fileType as an array ( because sooner or later you’ll want to select multiple file types ) 🙂

      Reply

Leave a comment

« summer is approaching fast
script for associating a subtitle to a movie file »

Blog Stats

  • 281,707 hits