NumPy - Views vs Copies#
Hey everyone! I can’t believe we’re half way through the year already. We have been extremely busy working on seminars for the rest of the year, as well as putting together some special events for our VIPs and alumni network.
We held VIP session at the end of last month wherein we challenged James to live-code the game of UNO from scratch. While UNO ended up being a much more complex game than any of us originally anticipated, James was able to accurately recreate the game, and he shared many helpful tips-and-tricks along the way.
This week, I wanted to change gears a bit and discuss one of my favorite tools in the Python ecosystem: NumPy. Throughout the years, my usage of NumPy has vastly grown and evolved. One concept that really helps you write effective NumPy code is familiarizing yourself with the idea of views & copies.
The NumPy packages revolves primarily around the numpy.ndarray object. Similar to the Python list, the numpy.ndarray is a container-like object and can be positionally indexed & sliced. However, importantly, the numpy.ndarray efficiently stores unboxed values as it does not contain PyObjects. This is what enables numpy to perform mathematical operations so much more quickly than the Python list- however I’ll save further discussion on this topic for another time.
What are Views & Copies?#
Views and copies are the two ways NumPy can derive a sub-array from a parent array.
a view is a portion of an numpy.ndarray that refers back to some parent array. Changes made to this array will propagate back to its parent and all other children that are referenced by that parent.
This idea is similar to Python’s use of references to track new variables instead of copying data to create new ones. Views also enable us to perform in place of mutations of arrays easily.
from numpy import zeros
x = zeros((3,3))
y = x[:] # simple slicing forces a view, so y and x refer to the same array
y += 1 # updating y in-place will also update x since y is a view of x
y = y + 1 # this is not an in-place operation, so y becomes a different array from x
print(
x,
y,
sep='\n{}\n'.format('\N{box drawings light horizontal}' * 40),
)
[[1. 1. 1.]
[1. 1. 1.]
[1. 1. 1.]]
────────────────────────────────────────
[[2. 2. 2.]
[2. 2. 2.]
[2. 2. 2.]]
We can apply the same idea with Python lists, having y refer to the same list as x, and mutating y in place to propagate changes back to x.
x = [0, 1, 2]
y = x
y[0] += 20
print(
x,
y,
sep='\n{}\n'.format('\N{box drawings light horizontal}' * 40),
)
[20, 1, 2]
────────────────────────────────────────
[20, 1, 2]
By utilizing views we exert fine-grained control over the memory use of our NumPy code. Additionally if speed is of concern, inplace operations do not need to spend time allocating new blocks of memory thus making the slightly faster as well.
You may also note in my comment in the code above I mentioned that “simple slicing returns a view,” let’s discuss how slicing and views/copies are related.
Controlling Views & Copies#
The easiest way to create views of an array is to use simple indexing. This refers to array slices that are created from valid Python slice objects/slice notation and numpy.newaxis. Additionally all .reshape based operations will produce views.
In contrast, numpy will create a copy of an array whenever fancy indexing is used. This refers to array slices created from boolean masking or multiple integer accession.
from numpy import arange
x = arange(5)
# Simple indexing examples - returns views
x[0]
x[:3]
x[1::2]
x[None, :2]
x.reshape(-1, 5)
# Fancy indexing - returns copies
x[x < 2]
x[[True, False, False, False, True]]
x[[0, 2, 4]]
array([0, 2, 4])
While all of the above operations return slices of the original array x, using simple indexing vs fancy indexing determines whether or not numpy returns a view into the original array or a copy of the original array.
from numpy import arange
x = arange(5)
y = x[:3] # y is a view of the first 3 values of x
y += 10
print(
x,
y,
sep='\n{}\n'.format('\N{box drawings light horizontal}' * 40),
)
[10 11 12 3 4]
────────────────────────────────────────
[10 11 12]
from numpy import arange
x = arange(5)
y = x[x < 3] # y is a copy of all values less than 3 in x
y += 10
print(
x,
y,
sep='\n{}\n'.format('\N{box drawings light horizontal}' * 40),
)
[0 1 2 3 4]
────────────────────────────────────────
[10 11 12]
Forcing a View#
This idea of views and copies takes us a little deeper into numpy by thinking about how the underlying arrays are structured we can make further sense of when a view or copy will be returned.
In its most simplified form, a numpy array is an abstraction on top of a block of contiguous memory. Metadata, such as the datatype, inform NumPy how to view into this block of memory to create the structured looking arrays that we’re used to interacting with. So whenever we perform a reshape or simple indexing operation, NumPy can simply create a new view into this array by altering some metadata.
Since NumPy arrays are views into contiguous blocks of memory, elements of that array hold a constant size, which is dictated by the datatype of the array. When we use positional access on a NumPy array (e.g. array[10], NumPy uses the datatype information to calculate the specific location in this reserved memory block to skip directly to this value without needing to perform any iteration.
Furthermore, NumPy stores more metadata about how to access and interact with the array in its strides, which dictate how many bytes to skip over to access the next value of an array.
x = arange(10)
print(
x.astype('uint8').data.strides,
x.astype('uint64').data.strides,
sep='\n{}\n'.format('\N{box drawings light horizontal}' * 40),
)
(1,)
────────────────────────────────────────
(8,)
From the above we use astype to instruct NumPy to use 8 times as much memory in the 'uint8' array compared to the 'uint64'. This fact is also represented in the strides of the array, which instruct NumPy that the start of each value in our contiguous block of memory is exactly 1 (or 8) bytes away from one another depending on the data type.
You can change these metadata directly via numpy.lib.stride_tricks.as_strided. By doing so, we are changing how NumPy views into the aforementioned reserved block of contiguous memory and are allocating a new block of contiguous memory.
This means that stride tricks are guaranteed to return a view.
from numpy import arange
from numpy.lib.stride_tricks import as_strided
xs = arange(12, dtype='uint8')
# as_strided(array, new shape, new stride)
ys = as_strided(xs, (6,), (2,))
ys += 20
print(
f'{xs = }',
f'{xs.data.strides = }',
f'{ys = }',
f'{ys.data.strides = }',
sep='\n{}\n'.format('\N{box drawings light horizontal}' * 40),
)
xs = array([20, 1, 22, 3, 24, 5, 26, 7, 28, 9, 30, 11], dtype=uint8)
────────────────────────────────────────
xs.data.strides = (1,)
────────────────────────────────────────
ys = array([20, 22, 24, 26, 28, 30], dtype=uint8)
────────────────────────────────────────
ys.data.strides = (2,)
The above snippet uses as_strided to create a new view into xs that has a shape of (6,) a stride of 2 bytes. This means that this new array ys views into the parent array starting at the beginning of the contiguous block of memory, then to get to the next value we skip 2 bytes (instead of skipping 1 byte as specified by original stride of xs).
Then What’s a Copy?#
A copy is simply any NumPy array that has to allocate a new contiguous block of memory. For example, when using fancy indexing there is no possible way to manipulate the array metadata such that we can simply work with the results as a view. If we think about the above examples with as_strided: a view can only be returned if it can be expressed in terms of a new stride or a new shape. This limits views to patterned access- meaning that views are somewhat limited to operating when there is a consistent pattern to element access that can be expressed in the shape of the output array and number of bytes to skip when advancing from one element in the contiguous block to another.
Since fancy indexing operations such as x[[1, 5, 7]] can not be expressed by changing these metadata, then numpy inspects the indexer [1, 5, 7] and then intelligently allocates a new block of memory with the same number of necessary slots as indicated by the indexer. It will then iterate over these indexers and fill in each slot in the new block of memory with the values at these positions of array x.
Copies can be very beneficial to work with, which is why many operations do not occur inplace in NumPy. More often than not focusing too much on views and inplace operations to keep a low memory overhead can lead to unexpected propagation bugs (e.g. I changed an array in one place and accidentally changed that array in many other places in my code), and a great way to work around this is by using copies.
Wrap Up#
There is a strict ruleset for when NumPy returns a view of underlying data vs a copy of underlying data. In order to make sense of these rules we need to understand just a little bit of the underlying mechanics of this library. However, once you do so you can make better sense of which inplace operations will propagate back to other arrays, and which will not.
If you’re worried about running out of memory while using NumPy arrays, ensuring that you work with views and preallocating output/intermediate arrays is extremely beneficial. By understanding views, copies, and NumPy internals you can ensure not only that your code is efficient, but also that you are being extra careful about how you operate on your arrays.