Iterators & Generators

11. Iterators & Generators#

Iterators and generators are the backbone of Python’s memory-efficient data processing.

The iterator protocol: __iter__ and __next__
Built-in lazy iterators
Custom iterator classes
Generator functions and generator expressions
Lazy pipelines for streaming data

Chapter Video

This overview introduces generator functions and lazy value production with yield.

Python Tutorial: Generators - How to use them and the benefits you receive (11:14) by Corey Schafer

Hosted on YouTube; property of its original creator, embedded here for educational reference only.

Learning Objectives

By the end of this chapter, you will be able to:

Explain Python’s iterator protocol and use iter() and next() manually
Use built-in lazy iterators such as enumerate(), zip(), map(), filter(), and reversed()
Build custom iterator classes that implement the iterator protocol
Write generator functions with yield
Design multi-step data pipelines using chained generators

Chapter Flow

Glossary

#	Term	Explanation
1	Iterable	An object that can provide its items one at a time.
2	Iterator	An object that returns one item at a time and remembers its position.
3	Iterator protocol	The `__iter__` and `__next__` methods that make an object work as an iterator.
4	`iter`	A built-in function that returns an iterator from an iterable.
5	`next`	A built-in function that asks an iterator for its next item.
6	`StopIteration`	The exception that signals an iterator has no more items.
7	Lazy iterator	An iterator that produces values on demand instead of building a full list first.
8	`enumerate`	A lazy iterator that pairs each item with its index.
9	`zip`	A lazy iterator that pairs items from multiple iterables.
10	`map`	A lazy iterator that applies a function to each item.
11	`filter`	A lazy iterator that keeps items for which a function returns true.
12	Generator function	A function that uses `yield` to produce a sequence of values lazily.
13	`yield`	A statement that produces a value from a generator and pauses its execution.
14	Generator expression	A compact expression that creates a lazy generator.
15	Infinite sequence	A sequence that can keep producing values without a fixed end.
16	`itertools`	A standard-library module with tools for efficient iterator pipelines.
17	`islice`	An `itertools` function that takes selected positions from an iterator.
18	`chain`	An `itertools` function that connects multiple iterables into one stream.
19	`yield from`	Syntax for delegating part of a generator to another iterable.
20	Async iteration	Iteration over values that may arrive asynchronously, commonly using `async for`.
21	Generator pipeline	A chain of generators in which each stage lazily consumes the output of the previous stage.
22	Iterator exhaustion	The state of an iterator that has produced all its items and yields nothing on later passes.