py.path

The ‘py’ lib provides a uniform high-level api to deal with filesystems and filesystem-like interfaces: py.path. It aims to offer a central object to fs-like object trees (reading from and writing to files, adding files/directories, examining the types and structure, etc.), and out-of-the-box provides a number of implementations of this API.

py.path.local - local file system path

basic interactive example

The first and most obvious of the implementations is a wrapper around a local filesystem. It’s just a bit nicer in usage than the regular Python APIs, and of course all the functionality is bundled together rather than spread over a number of modules.

Example usage, here we use the py.test.ensuretemp() function to create a py.path.local object for us (which wraps a directory):

>>> import py
>>> temppath = py.test.ensuretemp('py.path_documentation')
>>> foopath = temppath.join('foo') # get child 'foo' (lazily)
>>> foopath.check() # check if child 'foo' exists
False
>>> foopath.write('bar') # write some data to it
>>> foopath.check()
True
>>> foopath.read()
'bar'
>>> foofile = foopath.open() # return a 'real' file object
>>> foofile.read(1)
'b'

reference documentation

class py._path.local.LocalPath(path=None, expanduser=False)[source]

object oriented interface to os.path and other local filesystem related information.

exception ImportMismatchError[source]

raised on pyimport() if there is a mismatch of __file__’s

LocalPath.samefile(other)[source]

return True if ‘other’ references the same file as ‘self’.

LocalPath.remove(rec=1, ignore_errors=False)[source]

remove a file or directory (or a directory tree if rec=1). if ignore_errors is True, errors while removing directories will be ignored.

LocalPath.computehash(hashtype='md5', chunksize=524288)[source]

return hexdigest of hashvalue for this file.

LocalPath.new(**kw)[source]

create a modified version of this path. the following keyword arguments modify various path parts:

a:/some/path/to/a/file.ext
xx                           drive
xxxxxxxxxxxxxxxxx            dirname
                  xxxxxxxx   basename
                  xxxx       purebasename
                       xxx   ext
LocalPath.dirpath(*args, **kwargs)[source]

return the directory path joined with any given path arguments.

LocalPath.join(*args, **kwargs)[source]

return a new path by appending all ‘args’ as path components. if abs=1 is used restart from root if any of the args is an absolute path.

LocalPath.open(mode='r', ensure=False, encoding=None)[source]

return an opened file with the given mode.

If ensure is True, create parent directories if needed.

LocalPath.listdir(fil=None, sort=None)[source]

list directory contents, possibly filter by the given fil func and possibly sorted.

LocalPath.size()[source]

return size of the underlying file object

LocalPath.mtime()[source]

return last modification time of the path.

LocalPath.copy(target, mode=False)[source]

copy path to target.

LocalPath.rename(target)[source]

rename this path to target.

LocalPath.dump(obj, bin=1)[source]

pickle object into path location

LocalPath.mkdir(*args)[source]

create & return the directory joined with args.

LocalPath.write_binary(data, ensure=False)[source]

write binary data into path. If ensure is True create missing parent directories.

LocalPath.write_text(data, encoding, ensure=False)[source]

write text data into path using the specified encoding. If ensure is True create missing parent directories.

LocalPath.write(data, mode='w', ensure=False)[source]

write data into path. If ensure is True create missing parent directories.

LocalPath.ensure(*args, **kwargs)[source]

ensure that an args-joined path exists (by default as a file). if you specify a keyword argument ‘dir=True’ then the path is forced to be a directory path.

LocalPath.stat(raising=True)[source]

Return an os.stat() tuple.

LocalPath.lstat()[source]

Return an os.lstat() tuple.

LocalPath.setmtime(mtime=None)[source]

set modification time for the given path. if ‘mtime’ is None (the default) then the file’s mtime is set to current time.

Note that the resolution for ‘mtime’ is platform dependent.

LocalPath.chdir()[source]

change directory to self and return old current directory

LocalPath.as_cwd(*args, **kwds)[source]

return context manager which changes to current dir during the managed “with” context. On __enter__ it returns the old dir.

LocalPath.realpath()[source]

return a new path which contains no symbolic links.

LocalPath.atime()[source]

return last access time of the path.

LocalPath.chmod(mode, rec=0)[source]

change permissions to the given mode. If mode is an integer it directly encodes the os-specific modes. if rec is True perform recursively.

LocalPath.pypkgpath()[source]

return the Python package path by looking for the last directory upwards which still contains an __init__.py. Return None if a pkgpath can not be determined.

LocalPath.pyimport(modname=None, ensuresyspath=True)[source]

return path as an imported python module.

If modname is None, look for the containing package and construct an according module name. The module will be put/looked up in sys.modules. if ensuresyspath is True then the root dir for importing the file (taking __init__.py files into account) will be prepended to sys.path if it isn’t there already. If ensuresyspath==”append” the root dir will be appended if it isn’t already contained in sys.path. if ensuresyspath is False no modification of syspath happens.

LocalPath.sysexec(*argv, **popen_opts)[source]

return stdout text from executing a system child process, where the ‘self’ path points to executable. The process is directly invoked and not through a system shell.

classmethod LocalPath.sysfind(name, checker=None, paths=None)[source]

return a path object found by looking at the systems underlying PATH specification. If the checker is not None it will be invoked to filter matching paths. If a binary cannot be found, None is returned Note: This is probably not working on plain win32 systems but may work on cygwin.

LocalPath.basename

basename part of path.

LocalPath.bestrelpath(dest)

return a string which is a relative path from self (assumed to be a directory) to dest such that self.join(bestrelpath) == dest and if not such path can be determined return dest.

LocalPath.chown(user, group, rec=0)

change ownership to the given user and group. user and group may be specified by a number or by a name. if rec is True change ownership recursively.

LocalPath.common(other)

return the common part shared with the other path or None if there is no common part.

LocalPath.dirname

dirname part of path.

LocalPath.ensure_dir(*args)

ensure the path joined with args is a directory.

LocalPath.ext

extension of the path (including the ‘.’).

LocalPath.fnmatch(pattern)

return true if the basename/fullname matches the glob-‘pattern’.

valid pattern characters:

*       matches everything
?       matches any single character
[seq]   matches any character in seq
[!seq]  matches any char not in seq

If the pattern contains a path-separator then the full path is used for pattern matching and a ‘*’ is prepended to the pattern.

if the pattern doesn’t contain a path-separator the pattern is only matched against the basename.

classmethod LocalPath.get_temproot()[source]

return the system’s temporary directory (where tempfiles are usually created in)

LocalPath.load()

(deprecated) return object unpickled from self.read()

LocalPath.mklinkto(oldname)

posix style hard link to another name.

LocalPath.mksymlinkto(value, absolute=1)

create a symbolic link with the given value (pointing to another name).

LocalPath.move(target)

move this path to target.

LocalPath.parts(reverse=False)

return a root-first list of all ancestor directories plus the path itself.

LocalPath.purebasename

pure base name of the path.

LocalPath.read(mode='r')

read and return a bytestring from reading the path.

LocalPath.read_binary()

read and return a bytestring from reading the path.

LocalPath.read_text(encoding)

read and return a Unicode string from reading the path.

LocalPath.readlines(cr=1)

read and return a list of lines from the path. if cr is False, the newline will be removed from the end of each line.

return value of a symbolic link.

LocalPath.relto(relpath)

return a string which is the relative part of the path to the given ‘relpath’.

LocalPath.visit(fil=None, rec=None, ignore=<class 'py._path.common.NeverRaised'>, bf=False, sort=False)

yields all paths below the current one

fil is a filter (glob pattern or callable), if not matching the path will not be yielded, defaulting to None (everything is returned)

rec is a filter (glob pattern or callable) that controls whether a node is descended, defaulting to None

ignore is an Exception class that is ignoredwhen calling dirlist() on any of the paths (by default, all exceptions are reported)

bf if True will cause a breadthfirst search instead of the default depthfirst. Default: False

sort if True will sort entries within each directory level.

classmethod LocalPath.mkdtemp(rootdir=None)[source]

return a Path object pointing to a fresh new temporary directory (which we created ourself).

classmethod LocalPath.make_numbered_dir(prefix='session-', rootdir=None, keep=3, lock_timeout=172800)[source]

return unique directory with a number greater than the current maximum one. The number is assumed to start directly after prefix. if keep is true directories with a number less than (maxnum-keep) will be removed.

py.path.svnurl and py.path.svnwc

Two other py.path implementations that the py lib provides wrap the popular Subversion revision control system: the first (called ‘svnurl’) by interfacing with a remote server, the second by wrapping a local checkout. Both allow you to access relatively advanced features such as metadata and versioning, and both in a way more user-friendly manner than existing other solutions.

Some example usage of py.path.svnurl:

.. >>> import py
.. >>> if not py.test.config.option.urlcheck: raise ValueError('skipchunk')
>>> url = py.path.svnurl('http://codespeak.net/svn/py')
>>> info = url.info()
>>> info.kind
'dir'
>>> firstentry = url.log()[-1]
>>> import time
>>> time.strftime('%Y-%m-%d', time.gmtime(firstentry.date))
'2004-10-02'

Example usage of py.path.svnwc:

.. >>> if not py.test.config.option.urlcheck: raise ValueError('skipchunk')
>>> temp = py.test.ensuretemp('py.path_documentation')
>>> wc = py.path.svnwc(temp.join('svnwc'))
>>> wc.checkout('http://codespeak.net/svn/py/dist/py/path/local')
>>> wc.join('local.py').check()
True

Common vs. specific API, Examples

All Path objects support a common set of operations, suitable for many use cases and allowing to transparently switch the path object within an application (e.g. from “local” to “svnwc”). The common set includes functions such as path.read() to read all data from a file, path.write() to write data, path.listdir() to get a list of directory entries, path.check() to check if a node exists and is of a particular type, path.join() to get to a (grand)child, path.visit() to recursively walk through a node’s children, etc. Only things that are not common on ‘normal’ filesystems (yet), such as handling metadata (e.g. the Subversion “properties”) require using specific APIs.

A quick ‘cookbook’ of small examples that will be useful ‘in real life’, which also presents parts of the ‘common’ API, and shows some non-common methods:

Searching .txt files

Search for a particular string inside all files with a .txt extension in a specific directory.

>>> dirpath = temppath.ensure('testdir', dir=True)
>>> dirpath.join('textfile1.txt').write('foo bar baz')
>>> dirpath.join('textfile2.txt').write('frob bar spam eggs')
>>> subdir = dirpath.ensure('subdir', dir=True)
>>> subdir.join('textfile1.txt').write('foo baz')
>>> subdir.join('textfile2.txt').write('spam eggs spam foo bar spam')
>>> results = []
>>> for fpath in dirpath.visit('*.txt'):
...     if 'bar' in fpath.read():
...         results.append(fpath.basename)
>>> results.sort()
>>> results
['textfile1.txt', 'textfile2.txt', 'textfile2.txt']

Working with Paths

This example shows the py.path features to deal with filesystem paths Note that the filesystem is never touched, all operations are performed on a string level (so the paths don’t have to exist, either):

>>> p1 = py.path.local('/foo/bar')
>>> p2 = p1.join('baz/qux')
>>> p2 == py.path.local('/foo/bar/baz/qux')
True
>>> sep = py.path.local.sep
>>> p2.relto(p1).replace(sep, '/') # os-specific path sep in the string
'baz/qux'
>>> p2.bestrelpath(p1).replace(sep, '/')
'../..'
>>> p2.join(p2.bestrelpath(p1)) == p1
True
>>> p3 = p1 / 'baz/qux' # the / operator allows joining, too
>>> p2 == p3
True
>>> p4 = p1 + ".py"
>>> p4.basename == "bar.py"
True
>>> p4.ext == ".py"
True
>>> p4.purebasename == "bar"
True

This should be possible on every implementation of py.path, so regardless of whether the implementation wraps a UNIX filesystem, a Windows one, or a database or object tree, these functions should be available (each with their own notion of path seperators and dealing with conversions, etc.).

Checking path types

Now we will show a bit about the powerful ‘check()’ method on paths, which allows you to check whether a file exists, what type it is, etc.:

>>> file1 = temppath.join('file1')
>>> file1.check() # does it exist?
False
>>> file1 = file1.ensure(file=True) # 'touch' the file
>>> file1.check()
True
>>> file1.check(dir=True) # is it a dir?
False
>>> file1.check(file=True) # or a file?
True
>>> file1.check(ext='.txt') # check the extension
False
>>> textfile = temppath.ensure('text.txt', file=True)
>>> textfile.check(ext='.txt')
True
>>> file1.check(basename='file1') # we can use all the path's properties here
True

Setting svn-properties

As an example of ‘uncommon’ methods, we’ll show how to read and write properties in an py.path.svnwc instance:

.. >>> if not py.test.config.option.urlcheck: raise ValueError('skipchunk')
>>> wc.propget('foo')
''
>>> wc.propset('foo', 'bar')
>>> wc.propget('foo')
'bar'
>>> len(wc.status().prop_modified) # our own props
1
>>> msg = wc.revert() # roll back our changes
>>> len(wc.status().prop_modified)
0

SVN authentication

Some uncommon functionality can also be provided as extensions, such as SVN authentication:

.. >>> if not py.test.config.option.urlcheck: raise ValueError('skipchunk')
>>> auth = py.path.SvnAuth('anonymous', 'user', cache_auth=False,
...             interactive=False)
>>> wc.auth = auth
>>> wc.update() # this should work
>>> path = wc.ensure('thisshouldnotexist.txt')
>>> try:
...     path.commit('testing')
... except py.process.cmdexec.Error, e:
...     pass
>>> 'authorization failed' in str(e)
True

Known problems / limitations

  • The SVN path objects require the “svn” command line, there is currently no support for python bindings. Parsing the svn output can lead to problems, particularly regarding if you have a non-english “locales” setting.
  • While the path objects basically work on windows, there is no attention yet on making unicode paths work or deal with the famous “8.3” filename issues.