When learning and working with Python on machine learning it is important to make sure that Numpy arrays have the proper dimensions. Using improper dimensions may cause issues / bugs that are hard to track yet it is simple to prevent and we will see in this post.
The advice here presented, with my additions, came from a course I took on Machine Learning (ML) by Andrew Ng. Hope you find it useful as I did.
I will go over all the cells in a Python notebook. You can find it in my GitHub repository.
In cell #1 I just import Numpy. All operations are performed using Numpy arrays. They are very useful when working in ML using Python.
import numpy as np
In cell #2 we create a numpy array with 5 entries / cells. The array is populated with pseudo random numbers. The array is named ‘a’. There are two things to note in the output for this cell. First is the fact that the values are bracketed by a single set of [] brackets. Second is the shape of the array. We only have a single dimension. We might have expected [5, 1] or [1, 5]. What we declared was a rank one array.
# **** create a numpy array with 5 values ****
a = np.random.rand(5)
print("a:\n", str(a))
# **** rank one array (not a row or a column vector) ***
print("a.shape: " + str(a.shape))
In cell #3 we compute the transpose of a. Again, the results do not seem to be what we might be expecting. In general a transpose operation swaps the values in the rows into the columns. It seems that the operation made no changes in the array.
# **** create a numpy array with 5 values ****
print("a.T:\n" + str(a.T))
at = a.T;
print("at.shape: " + str(at.shape))
In cell #4 we display the results of a.T and it seems that we just got a single value.
# **** should be a vector (returns a number)****
print("np.dot(a, at): " + str(np.dot(a, a.T)))
Cell #5 declares an array named ‘b’ and initializes it to a set of five pseudo random numbers. Note that instead of declare a one dimensional array as we did before, we are now explicitly declaring a [5, 1] array. The array, as expected, ended up having one column with five values. The shape is exactly what we were looking for.
# **** instead use: ****
b = np.random.randn(5, 1)
print("b:\n" + str(b))
print("b.shape: " + str(b.shape))
In cell #6 we generate the transpose of ‘b’ and assign it to ‘bt’. From a row vector we now have a column vector. Life is good.
# **** ****
bt = b.T
print("bt: ", bt)
print("(bt).shape: " + str(bt.shape))
In the next cell, we perform the dot product of ‘b’ [5, 1] and ‘bt’ [1, 5]. As expected the result is a [5, 5] matrix with the proper values.
# **** ****
print("np.dot(b, bt):\n" + str(np.dot(b, bt)))
In cell #8 we explicitly declare a [5, 1] column vector and populate it with pseudo random numbers. The shape matches. Note that when the vector is displayed the values are enclosed by ‘[[‘ and ‘]]’. This was not the case when we declared the rank one array.
# **** column vector ****
c = np.random.randn(5,1)
print("c:\n" + str(c))
print("c.shape: " + str(c.shape))
assert(c.shape == (5,1))
In cell #9 we declare a row vector with five random entries. The shape is as expected. We also added a check in the form of an assert. If the shape of the resulting array does not match our expectation the notebook would thrown an exception. Since all is well, no exception is thrown.
# **** row vector ****
c = np.random.randn(1, 5)
print("c:\n" + str(c))
print("c.shape: " + str(c.shape))
assert(c.shape == (1,5))
To contrast, cell # 10 shows another example of declaring an array using a single dimension.
# **** do NOT use ****
c = np.random.rand(5)
print("c:\n" + str(c))
print("c.shape: " + str(c.shape))
We can now check if we got what we desired, but not explicitly asked for it. By checking with an assert, we get an exception. Using this approach will check that we are getting what we expect.
# **** to make sure c has the expected shape *****
assert(c.shape == (5,1))
Finally in cell # 12 we reshape a new array that we declared with a single dimension and six elements. We then reshape the array as a row vector and then as a column vector.
# **** you can also reshape an array ****
c = np.random.rand(6)
print("c.shape: " + str(c.shape))
print("c: " + str(c))
print()
c = c.reshape(1, 6)
print("c.shape: " + str(c.shape))
print("c: " + str(c))
print()
c = c.reshape(6, 1)
print("c.shape: " + str(c.shape))
print("c:\n" + str(c))
The takeaway of this exercise is to always declare vectors the way we need them to be. When in doubt, we can always check for the expected shape and if needed reshape them.
I recall when I was in high school and dealt with vector operations in linear algebra. Given that we are using a programming language, we need to get used to the idiosyncrasies and be able to direct our programs to do what we need to accomplish.
I uploaded my Jupyter notebook to my GitHub repository. You can find it here.
Experiment with the notebook and make sure you clearly understand what needs to be done. By developing good habits from the start, you will not have to change to avoid bad habits.
Keep on learning;
John
Please follow me on Twitter: @john_canessa