Key Concepts

"Pythonic" code leans on the language's own tools to say things briefly and clearly. This chapter covers the machinery behind that style: iterators, generators, comprehensions and decorators, followed by the habits that make code feel natural to experienced Python developers.

Iterators

A for loop works on anything iterable: lists, strings, dicts, files, ranges and more. Under the hood, the loop calls iter() to get an iterator, then calls next() repeatedly until the iterator raises StopIteration.

numbers = [10, 20, 30]
it = iter(numbers)
next(it)   # 10
next(it)   # 20
next(it)   # 30
# one more next(it) raises StopIteration

# zip and enumerate return iterators as well
names = ["mina", "joon"]
scores = [90, 85]
for rank, (name, score) in enumerate(zip(names, scores), start=1):
    print(rank, name, score)

Because iterators produce values on demand, they can walk through huge data sets without loading everything into memory. The flip side is that an iterator is single-use: once exhausted, it stays exhausted.

Generators

Any function containing yield is a generator function. Calling it does not run the body; it returns a generator object that executes up to the next yield each time a value is requested, then pauses. It is by far the easiest way to write a custom iterator.

def countdown(n: int):
    while n > 0:
        yield n
        n -= 1


list(countdown(3))   # [3, 2, 1]


def read_lines(path: str):
    with open(path, encoding="utf-8") as f:
        for line in f:
            if line.strip():
                yield line.rstrip("\n")


def numbered(lines):
    for i, line in enumerate(lines, start=1):
        yield f"{i}: {line}"


# stream a large file through a pipeline, one line at a time
for row in numbered(read_lines("log.txt")):
    print(row)

Use yield from to delegate to another iterable. The standard itertools module adds building blocks such as islice, chain and groupby for composing iterators.

Comprehensions

Comprehensions build a list, dict or set from a loop and an optional condition in a single expression. Swap the brackets for parentheses and you get a generator expression, which produces values lazily instead of building a collection up front.

nums = range(10)

squares = [n * n for n in nums]                 # list
evens = [n for n in nums if n % 2 == 0]         # with a filter
lengths = {w: len(w) for w in ["py", "rust"]}   # dict
initials = {w[0] for w in ["apple", "avocado"]} # set {'a'}

total = sum(n * n for n in nums)                # generator expression
pairs = [(x, y) for x in range(2) for y in range(2)]

Comprehensions shine when they are short. Once you are stacking several loops and conditions and the line no longer reads at a glance, an ordinary loop is the better choice.