Python pathlib: Working with Files and Folders

By Engr. Henry Akpan

Working with Files and Folders the Modern Way

Real programs constantly deal with files and folders: reading a data file, saving results, finding every image in a directory, organising downloads, or making backups. Older Python code did this with string manipulation and functions scattered across the os and os.path modules. Since Python 3.4, the pathlib module offers a cleaner, object-oriented way. This Python pathlib tutorial covers building paths that work on Windows, macOS and Linux, inspecting path parts, checking whether files exist, creating and deleting folders, reading and writing files, finding files with glob patterns, and copying or moving files with shutil, ending with a small project that organises files by type. Every example runs in a fresh, empty folder, and the outputs shown are real.

Creating Path Objects

A Path object represents a location in the file system. Join parts with the / operator; pathlib uses the right separator for your operating system (backslash on Windows, forward slash elsewhere):

from pathlib import Path

project = Path("project")
data_file = project / "data" / "scores.csv"
print(data_file.as_posix())
print(data_file.name, data_file.stem, data_file.suffix)
print(data_file.parent.as_posix())
print(data_file.with_suffix(".json").name)
print([p for p in data_file.parts])

Output:

project/data/scores.csv
scores.csv scores .csv
project/data
scores.json
['project', 'data', 'scores.csv']

Printing data_file directly shows backslashes on Windows; as_posix() always uses forward slashes, which is handy for display. Other useful starting points are Path.cwd() (the current working directory), Path.home() (your home folder) and Path(__file__).parent (the folder containing the running script).

Creating Folders and Writing Files

(project / "data").mkdir(parents=True, exist_ok=True)
data_file.write_text("name,score\nAda,85\nMusa,72\n", encoding="utf-8")
(project / "notes.txt").write_text("Remember to back up.\n", encoding="utf-8")
(project / "data" / "photo.jpg").write_bytes(b"\xff\xd8\xff")  # a fake JPEG header
print(data_file.exists(), data_file.is_file(), project.is_dir())
print(data_file.read_text(encoding="utf-8").splitlines())
print(len(data_file.read_text(encoding="utf-8")), "characters")

Output:

True True True
['name,score', 'Ada,85', 'Musa,72']
26 characters
  • mkdir(parents=True, exist_ok=True) creates any missing parent folders and doesn't complain if the folder already exists.
  • write_text and read_text open, write or read, and close the file in one step. Always pass encoding="utf-8" so the code behaves the same on every system.
  • path.stat().st_size gives the size in bytes. On Windows it is larger than the character count here, because text mode writes each line ending as two bytes (carriage return and line feed).
  • For large files or line-by-line processing, use with path.open() as f: just as with the built-in open.

Listing and Finding Files with Python pathlib

iterdir() lists the items directly inside a folder. glob() matches a pattern, and rglob() searches all subfolders too:

for p in sorted(project.iterdir()):
    kind = "folder" if p.is_dir() else "file"
    print(kind, p.name)

print(sorted(p.name for p in project.glob("*.txt")))
print(sorted(p.as_posix() for p in project.rglob("*.*")))

Output:

folder data
file notes.txt
['notes.txt']
['project/data/photo.jpg', 'project/data/scores.csv', 'project/notes.txt']

Glob patterns use * for any characters, ? for one character and [abc] for a set. The order in which files are returned isn't guaranteed, so sort the results when order matters.

Renaming, Moving, Copying and Deleting

pathlib can rename and delete; the shutil module adds copying and moving whole folders:

import shutil

backup = Path("backup")
shutil.copytree(project, backup)                 # copy a whole folder
renamed = (backup / "notes.txt").rename(backup / "readme.txt")
shutil.copy2(data_file, backup / "scores_copy.csv")  # copy one file with its timestamps
(backup / "data" / "photo.jpg").unlink()          # delete a file
print(sorted(p.relative_to(backup).as_posix() for p in backup.rglob("*")))
shutil.rmtree(backup)                             # delete a folder and everything in it
print(backup.exists())

Output:

['data', 'data/scores.csv', 'readme.txt', 'scores_copy.csv']
False

Be careful: unlink() and shutil.rmtree() delete permanently, without using the Recycle Bin. When writing a script that deletes things, print what it would delete first ("dry run"), and only switch on the real deletion once you have checked the list.

To delete an empty folder use path.rmdir(). unlink(missing_ok=True) avoids an error if the file is already gone.

os and os.path Equivalents

You will meet older code that uses os. The common equivalents are:

  • os.path.join(a, b) → Path(a) / b
  • os.path.exists(p) → Path(p).exists()
  • os.path.basename(p) → Path(p).name
  • os.path.splitext(p) → Path(p).stem and Path(p).suffix
  • os.listdir(d) → Path(d).iterdir()
  • os.makedirs(d, exist_ok=True) → Path(d).mkdir(parents=True, exist_ok=True)
  • os.remove(p) → Path(p).unlink()

Most functions that accept a file name, including open(), pandas.read_csv() and shutil functions, accept Path objects directly. If a library insists on a string, use str(path).

Project: Organise a Downloads Folder by File Type

This script sorts the files in a folder into subfolders by extension, which is a common real-world automation task:

downloads = Path("downloads")
downloads.mkdir()
for name in ["cv.pdf", "notes.docx", "beach.jpg", "song.mp3", "logo.PNG", "data.csv"]:
    (downloads / name).write_text("x", encoding="utf-8")

GROUPS = {
    "Documents": {".pdf", ".docx", ".txt", ".csv"},
    "Images": {".jpg", ".jpeg", ".png"},
    "Audio": {".mp3", ".wav"},
}

def organise(folder, dry_run=True):
    moves = []
    for f in sorted(folder.iterdir()):
        if not f.is_file():
            continue
        ext = f.suffix.lower()
        group = next((g for g, exts in GROUPS.items() if ext in exts), "Other")
        target = folder / group / f.name
        moves.append((f.name, group))
        if not dry_run:
            target.parent.mkdir(exist_ok=True)
            f.rename(target)
    return moves

print(organise(downloads))               # dry run first: just report
organise(downloads, dry_run=False)
for g in sorted(downloads.iterdir()):
    print(g.name, sorted(p.name for p in g.iterdir()))

Output:

[('beach.jpg', 'Images'), ('cv.pdf', 'Documents'), ('data.csv', 'Documents'), ('logo.PNG', 'Images'), ('notes.docx', 'Documents'), ('song.mp3', 'Audio')]
Audio ['song.mp3']
Documents ['cv.pdf', 'data.csv', 'notes.docx']
Images ['beach.jpg', 'logo.PNG']

Notice f.suffix.lower(), so that logo.PNG is still recognised as an image, and the dry-run option that shows the plan before anything moves. A useful extension is to handle name clashes: if the target already exists, add a number to the file name before moving.

Summary

  • Python pathlib represents paths as objects; join them with / and they work on every operating system.
  • name, stem, suffix and parent give the parts of a path.
  • mkdir, write_text, read_text, exists and unlink cover everyday file tasks; always pass encoding="utf-8".
  • iterdir, glob and rglob find files; sort the results when order matters.
  • shutil copies and moves files and folders; deleting is permanent, so do a dry run first.

Related Tutorials