Ask Python to loop over a list, a file, or a dictionary, and it plays the same trick every time: it asks the object for a Python iterator, then pulls values one by one until StopIteration says stop. This post unpacks that protocol, __iter__ and __next__ included, and shows you how to build a custom iterator of your own.

“Simple is better than complex. Complex is better than complicated.”

Tim Peters, The Zen of Python (PEP 20)

Last Updated: July 2026 | Tested on: Python 3.14.6 | Difficulty: Intermediate | Reading Time: 16 minutes

Every for loop you have ever written is quietly doing something more interesting than it looks. When you write for x in my_list, Python does not just jump to index 0, then index 1, and so on. It first calls iter(my_list) to get a Python iterator, then calls next() on that iterator again and again until the iterator says “I am done” by raising StopIteration.

Think of a vending machine. The machine is full of snacks (that is your data), but it only ever hands you one item at a time, when you press the button. You cannot grab the whole stock at once, and once a row is empty, pressing again gets you nothing. A Python iterator works the same way: it hands back one value per next() call and eventually runs out. Once you get this one idea, generators, lazy evaluation, and half of Python’s most elegant patterns suddenly make sense. The rest of this post shows exactly how the iterator protocol works, how to build your own iterator from scratch, and why it keeps your programs light on memory.

What a for Loop Actually Does

Tip: click the diagram to open it full screen, then use the zoom and pan controls for a closer look.

The diagram lays out the whole Python iterator protocol in one picture. You start with an iterable on the left (a list, tuple, string, dict, or set). Calling iter() on it hands you back an iterator, and every next() call asks that iterator for one more value. This repeats until the iterator raises StopIteration, which is its polite way of saying the vending machine is empty. The same loop powers every for loop, list comprehension, and unpacking line you write.

Later, when you give a class of your own an __iter__ and a __next__ method, it plugs straight into all of that syntax for free. To see it in real code, let us loop over a small list holding the names of three developers, Rahul, Niranjan, and Viraj, first the normal way and then the way Python secretly does it.

📄 for_loop_desugared.py: the truth behind for x in lst

team = ["Rahul", "Niranjan", "Viraj"]

# What you write:
print("=== Normal for loop ===")
for name in team:
    print(name)

# What Python actually does:
print("\n=== Desugared version ===")
iterator = iter(team)         # Step 1: Get an iterator
while True:
    try:
        name = next(iterator)  # Step 2: Get next value
        print(name)            # Step 3: Run loop body
    except StopIteration:      # Step 4: Stop when exhausted
        break

▶ Output

=== Normal for loop ===
Rahul
Niranjan
Viraj

=== Desugared version ===
Rahul
Niranjan
Viraj

What happened here: The for loop is just friendly shorthand for the iterator protocol. iter(team) calls team.__iter__(), which hands back a list_iterator object. Each next(iterator) then calls iterator.__next__() and returns one name at a time. When the names run out, __next__() raises StopIteration, the while True version catches it and breaks, and the real for loop ends the same way. Both blocks print the identical three names because they are doing the identical work. Every iterable in Python follows this exact pattern, so once you understand one, you understand them all.

Iterable vs Iterator: They Are Not the Same

📄 iterable_vs_iterator.py: the distinction that trips people up

scores = [95, 88, 72, 91]

# scores is an ITERABLE (has __iter__, but NOT __next__)
print(f"scores has __iter__? {hasattr(scores, '__iter__')}")
print(f"scores has __next__? {hasattr(scores, '__next__')}")

# iter(scores) returns an ITERATOR (has BOTH __iter__ AND __next__)
it = iter(scores)
print(f"\niterator has __iter__? {hasattr(it, '__iter__')}")
print(f"iterator has __next__? {hasattr(it, '__next__')}")
print(f"Type: {type(it)}")

# You can call next() on an iterator
print(f"\nnext(it) = {next(it)}")  # 95
print(f"next(it) = {next(it)}")    # 88
print(f"next(it) = {next(it)}")    # 72
print(f"next(it) = {next(it)}")    # 91

# But the iterator is now EXHAUSTED
try:
    next(it)
except StopIteration:
    print("StopIteration raised, iterator exhausted!")

# The original list is fine, create a new iterator
it2 = iter(scores)
print(f"\nFresh iterator: {next(it2)}")  # 95 again

▶ Output

scores has __iter__? True
scores has __next__? False

iterator has __iter__? True
iterator has __next__? True
Type: <class 'list_iterator'>

next(it) = 95
next(it) = 88
next(it) = 72
next(it) = 91
StopIteration raised, iterator exhausted!

Fresh iterator: 95

What happened here: Picture a book and a bookmark. The iterable is the book: it holds all the data and it has an __iter__ method. The iterator is the bookmark: it remembers exactly where you are and it has both __iter__ and __next__. The book itself does not track your place, so you can hand it to ten different readers and each one gets their own bookmark. That is what iter(scores) does.

It clips a fresh bookmark onto the list. Once that bookmark reaches the last page, it is spent, which is why the fifth next(it) raised StopIteration. The list was never touched, so iter(scores) happily gave us a brand new bookmark starting at 95. This is also why a single for loop never “uses up” your list: every loop quietly grabs a new iterator.

Building a Custom Iterator

Think of the final seconds before a rocket launch. Mission control calls out 5, 4, 3, 2, 1, one number at a time, and then stops. Nobody printed the whole countdown on paper first; each number is produced right when it is needed, and after “1” there is simply nothing left to say. That is exactly what we are about to build: a class that produces one value per request and knows when to stop. All it takes is two methods.

📄 countdown.py: a custom iterator from scratch

class Countdown:
    """Iterator that counts down from a number to 1."""

    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self  # The iterator returns itself

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        value = self.current
        self.current -= 1
        return value

# Use it in a for loop, just like any built-in iterable
print("Countdown:")
for num in Countdown(5):
    print(f"  {num}...")
print("  Launch!")

# Or use next() manually
countdown = Countdown(3)
print(f"\nManual: {next(countdown)}, {next(countdown)}, {next(countdown)}")

▶ Output

Countdown:
  5...
  4...
  3...
  2...
  1...
  Launch!

Manual: 3, 2, 1

What happened here: We built a working iterator with just two methods. __iter__ returns self, which tells Python “I am my own bookmark, start here.” __next__ hands back the current number, drops the counter by one, and raises StopIteration the moment it hits zero. When Countdown(5) goes into a for loop, Python calls iter() on it (gets the same object back), then calls next() over and over until that StopIteration fires. No list of numbers is ever stored. Each value is computed right when it is asked for. That is the entire protocol, and you just wrote it by hand.

Separating Iterable and Iterator

The Countdown class above is both the book and the bookmark in one object, so it can only be looped over once. After that, its counter is sitting at zero and it is done forever. If you want something you can loop over again and again, you split the two jobs apart: one class holds the data, a separate class tracks the position. Below, we build a roster for a small dev team of four members named Pravin, Sardar, Prathamesh, and Vinay, and loop over it twice to prove the point.

📄 team_roster.py: an iterable that creates fresh iterators

class TeamRoster:
    """Iterable team roster, can be iterated multiple times."""

    def __init__(self, members):
        self.members = members

    def __iter__(self):
        return TeamIterator(self.members)

class TeamIterator:
    """Iterator that walks through team members."""

    def __init__(self, members):
        self.members = members
        self.index = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.index >= len(self.members):
            raise StopIteration
        member = self.members[self.index]
        self.index += 1
        return member

team = TeamRoster(["Pravin", "Sardar", "Prathamesh", "Vinay"])

# Can iterate multiple times, each for loop gets a fresh iterator
print("First loop:")
for member in team:
    print(f"  {member}")

print("\nSecond loop (works again!):")
for member in team:
    print(f"  {member}")

▶ Output

First loop:
  Pravin
  Sardar
  Prathamesh
  Vinay

Second loop (works again!):
  Pravin
  Sardar
  Prathamesh
  Vinay

What happened here: Now the two jobs live in two classes. TeamRoster is the iterable, the book. Its __iter__ builds a brand new TeamIterator every time it is asked. TeamIterator is the iterator, the bookmark. It carries an index and walks through the members one by one. Because each for loop calls __iter__ and gets its own fresh TeamIterator starting at index 0, the second loop prints the full roster again. This is exactly how lists, dicts, tuples, and every other built-in container behave under the hood.

Edge Cases: The Surprising Parts

An iterator is like a movie ticket: once it has been scanned at the gate, showing it again gets you nowhere. Every surprise in this section traces back to that one rule. Below are the three catches that bite real code most often. Catch 2 uses a small dict of exam scores for three students named Anvi, Anvay, and Aditi.

📄 edge_cases.py: three iterator catches that bite real code

# Catch 1: Iterators are exhausted after one pass
nums = [1, 2, 3]
it = iter(nums)
first_pass = list(it)   # [1, 2, 3]
second_pass = list(it)  # [] EMPTY! Iterator is spent
print(f"First pass: {first_pass}")
print(f"Second pass: {second_pass}")

# Catch 2: Modifying a dict while iterating
scores = {"Anvi": 95, "Anvay": 88, "Aditi": 72}
try:
    for name in scores:
        if scores[name] < 80:
            del scores[name]  # RuntimeError!
except RuntimeError as e:
    print(f"\nError: {e}")

# Fix: iterate over a copy
scores = {"Anvi": 95, "Anvay": 88, "Aditi": 72}
for name in list(scores):  # list() creates a copy of keys
    if scores[name] < 80:
        del scores[name]
print(f"Filtered: {scores}")

# Catch 3: zip() returns an iterator (one pass only!)
pairs = zip([1, 2, 3], ["a", "b", "c"])
print(f"\nFirst list(): {list(pairs)}")
print(f"Second list(): {list(pairs)}")  # Empty!

▶ Output

First pass: [1, 2, 3]
Second pass: []

Error: dictionary changed size during iteration
Filtered: {'Anvi': 95, 'Anvay': 88}

First list(): [(1, 'a'), (2, 'b'), (3, 'c')]
Second list(): []

What happened here: All three catches come from the same root cause: an iterator is single use. In Catch 1, the first list(it) drains the iterator completely, so the second list(it) finds nothing left and returns an empty list. In Catch 2, deleting a key while looping changes the dict’s size mid-walk, and Python refuses with dictionary changed size during iteration rather than silently skip items. The fix is to loop over list(scores), which makes a throwaway snapshot of the keys so you can safely edit the real dict.

Catch 3 catches almost everyone: zip() does not return a list, it returns a one-pass iterator, so the second list(pairs) is empty for the very same reason as Catch 1. When in doubt, if a built-in feels “lazy,” assume it is an iterator and wrap it in list() once if you need the values more than one time.

Built-in Iterables You Use Every Day

Here is the fun part: you have been using this protocol all along, the way you use electricity every day without ever thinking about the wiring behind the wall. Strings, dicts, files, ranges, and half the built-in functions all speak the same iterator language.

📄 builtin_iterators.py: how much of Python is already iterable

# Strings are iterable (character by character)
for char in "Python":
    print(char, end=" ")
print()

# Dicts iterate over keys by default
config = {"host": "localhost", "port": 8080, "debug": True}
for key in config:
    print(f"  {key} = {config[key]}")

# Files are iterable (line by line), the most memory-efficient way to read
# with open("data.txt") as f:
#     for line in f:          # Reads one line at a time, not the whole file
#         process(line)

# range() returns an iterable (NOT a list, it is lazy)
r = range(1_000_000)
print(f"\nrange(1M) size in memory: {r.__sizeof__()} bytes")
print(f"list(range(1M)) would be: ~8,000,000 bytes")

# enumerate(), zip(), map(), filter() all return iterators
mapped = map(str.upper, ["hello", "world"])
print(f"Type of map(): {type(mapped)}")

▶ Output

P y t h o n 
  host = localhost
  port = 8080
  debug = True

range(1M) size in memory: 48 bytes
list(range(1M)) would be: ~8,000,000 bytes
Type of map(): <class 'map'>

What happened here: The protocol you learned is not some niche feature, it is everywhere. Strings hand you one character at a time, dicts hand you one key at a time, and a file object hands you one line at a time, which is why you can read a giant log file without loading it all into memory. The range(1_000_000) result is the headline: it occupies just 48 bytes because it never builds the million numbers, it computes each one on demand. The equivalent list would weigh in around 8 MB.

And map() reports its type as map, not list, a quiet reminder that map, zip, enumerate, and filter are all lazy iterators waiting to be consumed once.

When You’ll Use This

• Processing large files: Reading a 10GB log file line by line uses constant memory, because a file object is an iterator that keeps only one line in memory at a time.
• Database result sets: Object-Relational Mappers (ORMs) return iterators that fetch rows lazily from the database, not all 10 million rows at once.
• Custom data structures: When you build your own tree, graph, or linked list, adding __iter__ and __next__ lets it work with for loops, list(), sum(), and every other Python function that expects an iterable.

Common Misconceptions

Two wrong ideas trip up almost everyone the first time they meet iterators. Clear these up now and the rest of the topic stays simple.

❌ Misconception 1: “A list is an iterator”

# A list is an ITERABLE, not an iterator
# It creates a NEW iterator each time you loop over it
# That's why you can loop over a list multiple times

nums = [1, 2, 3]
# next(nums)  # TypeError: 'list' object is not an iterator
# You must do: next(iter(nums))

A list is the book, not the bookmark. It knows how to make iterators (it has __iter__), but it has no __next__, so calling next() on it directly raises TypeError: 'list' object is not an iterator. If you really want to step through it by hand, ask for an iterator first with next(iter(nums)).

❌ Misconception 2: “Iterators can be reset”

# There is no .reset() method on iterators
# Once exhausted, they're done. Create a new one.
# If you need multiple passes, use the iterable (list, tuple), not the iterator

An iterator has no rewind button. Once it raises StopIteration, it stays empty for good. There is no .reset() and no way to send it back to the start. When you need to loop more than once, keep a handle on the original iterable and ask it for a fresh iterator each time, or convert the values to a list up front with list(...).

Conclusion

So here is the whole story in plain words. Every for loop in Python is quietly calling __iter__() to get an iterator and then __next__() to pull values one at a time, until StopIteration says “that is the last one.” Iterables are the books, iterators are the bookmarks, and a bookmark only moves forward. Once you can see that, you understand why a list loops fine again and again, why a zip object goes empty after one pass, and how to give your own classes the same superpower.

Writing __iter__ and __next__ by hand works, but it is a fair bit of boilerplate for something this common. That is exactly the problem generators solve. In the generators tutorial you will get the same lazy, one-value-at-a-time behavior from a single yield keyword, with a fraction of the code. And if you want to see everything this series covers, from first steps to AI/ML, visit the Python + AI/ML tutorial series home.

Frequently Asked Questions

Try It Yourself

Build a FibonacciIterator class that yields Fibonacci numbers up to a given limit. Implement __iter__ and __next__ properly. Test it with list(FibonacciIterator(100)) to get all Fibonacci numbers under 100.

The iterator protocol you just learned is a foundation you will keep coming back to. In the generators tutorial you will see how yield writes all this boilerplate for you. Later, the asyncio tutorial uses async iterators for concurrent I/O, and in Part 4 the Pandas introduction shows DataFrames leaning on the same protocol to process million-row datasets one row at a time.

Interview Questions on Python Iterators

Scenario questions, not trivia: this is the form this topic takes in a real interview.

Q: Why must an iterator’s __iter__ method return self?

Because a for loop always starts by calling iter() on whatever you hand it. If that object is already an iterator, returning self lets the loop pick up from the current position instead of failing. This is what makes the common pattern of skipping a header with next(f) and then looping over the remaining lines of the same file work seamlessly.

Q: Your service parses a 5 GB log file and memory usage climbs to nearly 5 GB before the process gets killed. What do you check first?

Look for f.read() or f.readlines(), both of which load the entire file into memory at once. A file object is already an iterator, so for line in f: keeps only one line in memory at a time and usage stays flat no matter how big the file gets. Swapping the read-everything call for direct iteration is usually the entire fix.

Q: A teammate builds pairs = zip(ids, names), logs list(pairs) for debugging, and then the real processing loop receives zero items. What happened?

zip() returns a single-use iterator, and the debug list(pairs) call drained it completely. When the processing loop runs afterwards, it gets the same exhausted iterator and hits StopIteration immediately, so it sees nothing. The fix is to materialize once with pairs = list(zip(ids, names)) and reuse that list, or build a fresh zip object for each pass.

Q: What does next(it, default) do, and when would you reach for it?

Passing a second argument to next() makes it return that value instead of raising StopIteration when the iterator is empty. So next(it, None) means “give me the next item, or None if there is none,” with no try/except block needed. It is the cleanest way to express “first match or fallback” when pulling from a filter or generator expression.

Q: iter() has a two-argument form, iter(callable, sentinel). What does it do?

It builds an iterator that calls callable() on every next() and stops the moment the return value equals sentinel. The classic use is reading a file in fixed-size chunks: for chunk in iter(lambda: f.read(4096), ''): keeps yielding 4096-character chunks and stops cleanly when read() returns an empty string at end of file. It turns any repeated function call into a loopable stream without writing a class.

Q: Your custom container works fine in a single for loop, but a nested loop over the same object misbehaves: the inner loop runs once and then everything stops early. What is the design flaw?

The container’s __iter__ returns self, so the outer and inner loops share one iterator and one position counter. The inner loop exhausts it on the first outer pass, leaving nothing for the outer loop to continue with. The fix is the split shown in this post: have the container’s __iter__ return a brand new iterator object on every call, so each loop gets its own independent bookmark.

Further reading: for the full reference, see the official Python documentation.

Related Posts

Previous: Python: 50 Libraries Every Developer Should Know (The Ecosystem Map)

Next: Python: Generators with yield, Expressions, Pipelines

Series Home: Python + AI/ML Tutorial Series