Python Closures
Summary: in this tutorial, you’ll learn about Python closures and their practical applications.
Introduction to the Python closures
In Python, you can define a function from the inside of another function. And this function is called a nested function. For example:
def say():
greeting = 'Hello' def display():
print(greeting)
display()
Code language: Python (python)
In this example, we define the display
function inside the say
function. The display
function is called a nested function.
Inside the display
function, you access the greeting
variable from its nonlocal scope.
Python calls the greeting
variable a free variable.
When you look at the display
function, you actually look at:
- The
display
function itself. - And the free variable
greeting
with the value'Hello'
.
So the combination of the display
function and greeting
variable is called a closure:
By definition, a closure is a nested function that references one or more variables from its enclosing scope.
Returning an inner function
In Python, a function can return a value which is another function. For example:
def say():
greeting = 'Hello' def display():
print(greeting)
return display
Code language: Python (python)
In this example, the say
function returns the display
function instead of executing it.
Also, when the say
function returns the display
function, it actually returns a closure:
The following assigns the return value of the say
function to a variable fn
. Since fn
is a function, you can execute it:
fn = say()
fn()
Code language: Python (python)
Output:
Hello
The say
function executes and returns a function. When the fn
function executes, the say
function already completes.
In other words, the scope of the say
function was gone at the time the fn
function executes.
Since the greeting
variable belongs to the scope of the say
function, it should also be destroyed with the scope of the function.
However, you still see that fn
displays the value of the message
variable.
Python cells and multi-scoped variables
The value of the greeting
variable is shared between two scopes of:
- The
say
function. - The closure
The label greeting
is in two different scopes. However, they always reference the same string object with the value 'Hello'
.
To achieve this, Python creates an intermediary object called a cell
:
To find the memory address of the cell object, you can use the __closure__
property as follows:
print(fn.__closure__)
Code language: Python (python)
Output:
(<cell at 0x0000017184915C40: str object at 0x0000017186A829B0>,)
Code language: HTML, XML (xml)
The __closure__
returns a tuple of cells.
In this example, the memory address of the cell is 0x0000017184915C40
. It references a string object at 0x0000017186A829B0
.
If you display the memory address of the string object in the say
function and closure
, you should see that they reference the same object in the memory:
def say():
greeting = 'Hello'
print(hex(id(greeting))) def display():
print(hex(id(greeting)))
print(greeting)
return display
fn = say()
fn()
Code language: Python (python)
Output:
0x17186a829b0
0x17186a829b0
When you access the value of the greeting
variable, Python will technically “double-hop” to get the string value.
This explains why when the say()
function was out of scope, you still can access the string object referenced by the greeting
variable.
Based on this mechanism, you can think of a closure as a function and an extended scope that contains free variables.
To find the free variables that a closure contains, you can use the __code__.co_freevars
, for example:
def say(): greeting = 'Hello'
def display():
print(greeting)
return display
fn = say()
print(fn.__code__.co_freevars)
Code language: Python (python)
Output:
('greeting',)
Code language: JavaScript (javascript)
In this example, the fn.__code__.co_freevars
returns the greeting
free variable of the fn
closure.
When Python creates the closure
Python creates a new scope when a function executes. If that function creates a closure, Python also creates a new closure as well. Consider the following example:
First, define a function called multiplier
that returns a closure:
def multiplier(x):
def multiply(y):
return x * y
return multiply
Code language: Python (python)
The multiplier
function returns the multiplication of two arguments. However, it uses a closure instead.
Second, call the multiplier
function three times:
m1 = multiplier(1)
m2 = multiplier(2)
m3 = multiplier(3)
Code language: Python (python)
These function calls create three closures. Each function multiplies a number with 1, 2, and 3.
Third, execute functions of the closures:
print(m1(10))
print(m2(10))
print(m3(10))
Code language: Python (python)
Output:
10
20
30
The m1, m2, and m3 have different instances of closure.
Python closures and for loop
Suppose that you want to create all three closures above at once and you might come up with the following:
multipliers = []
for x in range(1, 4):
multipliers.append(lambda y: x * y)m1, m2, m3 = multipliers
print(m1(10))
print(m2(10))
print(m3(10))
Code language: PHP (php)
How it works.
- First, declare a list that will store the closures.
- Second, use a lambda expression to create a closure and append the closure to the list in each iteration.
- Third, unpack the closures from the list to the m1, m2, and m3 variables.
- Finally, pass the values 10, 20, and 30 to each closure and execute it.
The following shows the output:
30
30
30
This doesn’t work as you expected. But why?
The x
starts from 1 to 3 in the loop. After the loop, its value is 3.
Each element of the list is the following closure:
lambda y: x*y
Python evaluates x
when you call the m1(10)
, m2(10)
, and m3(10)
. At the moment the closures execute, x
is 3.
That’s why you see the same result when you call m1(10)
, m2(10)
, and m3(10)
.
To fix this, you need to instruct Python to evaluate x
in the loop:
def multiplier(x):
def multiply(y):
return x * y
return multiplymultipliers = []
for x in range(1, 4):
multipliers.append(multiplier(x))
m1, m2, m3 = multipliers
print(m1(10))
print(m2(10))
print(m3(10))
Code language: Python (python)
Summary
- A closure is a function and an extended scope that contains free variables.