Generator functions allow you to declare a function that behaves like an iterator. They allow programmers to make an iterator in a fast, easy, and clean way.

Iterators

What’s an iterator, you may ask?

An iterator is an object that can be iterated (looped) upon. An iterator is also an object that implements the iterator protocol.

It is used to abstract a container of data to make it behave like an iterable object. You probably already use a few iterable objects every day: strings, lists, and dictionaries to name a few.

An iterator is defined by a class that implements the Iterator Protocol. This protocol looks for two methods within the class: __iter__ and __next__.

Why Iterators?

• Iterators don’t compute the value of each item when instantiated. They only compute it when you ask for it. This is known as lazy evaluation.
• Iterators can only be iterated over once.

Iterator Protocol

An iterator protocol is nothing but a specific class in Python which further has the __next()__ method. Which means every time you ask for the next value, an iterator knows how to compute it. It keeps information about the current state of the iterable it is working on. The iterator calls the next value when you call next() on it. An object that uses the __next__() method is ultimately an iterator.

The built-in function iter takes an iterable object and returns an iterator.

>>> x = iter([1, 2, 3])
>>> x
<listiterator object at 0x1004ca850>
>>> x.next()
1

Iterables

An iterable is any object, not necessarily a data structure that can return an iterator. Its main purpose is to return all of its elements. Iterables can represent finite as well as infinite source of data. An iterable will directly or indirectly define two methods: the __iter__() method, which must return the iterator object and the __next()__ method with the help of the iterator it calls.

Itertools Module

Itertools is an built-in Python module that contains functions to create iterators for efficient looping.

Generators

Recall that generator functions allow us to create iterators in a more simple fashion. Generators introduce the yield statement to Python. It works a bit like return because it returns a value.

The difference is that it saves the state of the function. The next time the function is called, execution continues from where it left off, with the same variable values it had before yielding.

Why Generators?

• Memory Efficient
• Easy to Implement
• Represent Infinite Stream
• Pipelining Generators

Generator Expressions

This is the list comprehension equivalent of generators. It works exactly in the same way as a list comprehension, but the expression is surrounded with () as opposed to [].

Summary

• Generators allow you to create iterators in a very pythonic manner.
• Iterators allow lazy evaluation, only generating the next element of an iterable object when requested. This is useful for very large data sets.
• Iterators and generators can only be iterated over once.
• Generator Functions are better than Iterators.
• Generator Expressions are better than Iterators (for simple cases only).

Reference: Programiz and Meduim