Python: Iterators and generators

Got it! This site "www.robinsnyder.com" uses cookies. You consent to this by clicking on "Got it!" or by continuing to use this website. Note: This appears on each machine/browser from which this site is accessed.

We have see ways to iterate over lists, sets and dictionaries using the in construct. Here is the Python code [#1]

colorDict1 = {
	"red" : "FF0000",
	"green" : "00FF00",
	"blue" : "0000FF"
	}
colorList1 = [
	["red", "FF0000"],
	["green", "00FF00"],
	["blue", "0000FF"]
	]
colorSet1 = {
	"red",
	"green",
	"blue"
	}
print("List:")
for color1, hex1 in colorList1:
print("\t{0:s} : {1:s}".format(hex1,color1))
print("Dict:")
for (color1, hex1) in colorDict1.items():
print("\t{0:s} : {1:s}".format(hex1,color1))
print("Set:")
for color1 in colorSet1:
	print("\t{0:s}".format(color1))

Here is the output of the Python code.

The repeat iterator can return something many times but need not take a lot of memory.

Here is the Python code [#2]

import sys
from itertools import repeat
repeat3 = repeat(3,times=8)
size3 = sys.getsizeof(repeat3)
print("Repeat: size={0:d}".format(size3))
for pos1,i1 in enumerate(repeat3):
	print("\t{0:d}. {1:d}".format(pos1,i1))
repeat4 = repeat(3,times=1_000_000)
size4 = sys.getsizeof(repeat4)
print("")
print("Repeat: size={0:d}".format(size3))
print("\t(enumeration omitted)")

Note the use of 1_000_000 for 1000000 as Python allows the underscore to be used for grouping to make digits of numbers easier to read.

A file in Python is a class that is an iterator.

No matter how big the file, one can read lines one by one without ever reading the entire file (unless one iterates to the end).

On the other hand, the method readall will read all the lines in the file. Here is the output of the Python code.

Repeat: size=48
	0. 3
	1. 3
	2. 3
	3. 3
	4. 3
	5. 3
	6. 3
	7. 3
Repeat: size=48
	(enumeration omitted)

One can make a custom iterator for any collection over which it makes sense to iterate/loop.

Here is a custom iterator that returns the even numbers, in order. Here is the Python code [#3]

class evens():
	def __init__(self, start1=0):
		self.pos1 = start1
	def __iter__(self):
		return self
	def __next__(self):
		pos1 = self.pos1
		self.pos1 = pos1+2
		return pos1
evens1 = evens(2)
print("Evens:")
for i1 in range(8):
	even1 = next(evens1)
	print("\t{0:d}. {1:d}".format(i1,even1))

Here is the output of the Python code.

Evens:
	0. 2
	1. 4
	2. 6
	3. 8
	4. 10
	5. 12
	6. 14
	7. 16

If we try something like the following, the loop will never end.

for i1 in evens1(): # do something with i1 # end for

Iterators do not work well when the underlying sequences do not end. Instead, a generator may be more appropriate.

A generator function in Python is a function that contains one or more yield statements. Here is the Python code [#4]

def evens(start1=0):
	pos1 = start1
	while True:
		yield pos1
		pos1 += 1
evens1 = evens(2)
print("Evens:")
for i1 in range(8):
	even1 = next(evens1)
	print("\t{0:d}. {1:d}".format(i1,even1))

Here is the output of the Python code.

Evens:
	0. 2
	1. 3
	2. 4
	3. 5
	4. 6
	5. 7
	6. 8
	7. 9

A generator expression can be created as needed as an expression rather than using a named function.

A similar concept is a lambda function which is a way to make a function as an expression rather than using a named function. Here is the Python code [#5]

from itertools import count
evens1 = (2*pos1 for pos1 in count(1))
print("Evens:")
for i1 in range(8):
	even1 = next(evens1)
	print("\t{0:d}. {1:d}".format(i1,even1))

Here is the output of the Python code.

Evens:
	0. 2
	1. 4
	2. 6
	3. 8
	4. 10
	5. 12
	6. 14
	7. 16