1D Tensors
title: Torch Tensors in 1D author: Juma Shafara date: "2023-09" date-modified: "2024-07-30" description: In this lab, you will learn the basics of tensor operations. you will compare them to vectors and numpy arrays. keywords: [ tensors, Types and Shape, Indexing and Slicing, Tensor Functions, Tensor Operations, Device_Op Operations, ]
Objective
- How tensor operations work in pytorch.
Table of Contents
In this lab, you will learn the basics of tensor operations. Tensors are an essential part of PyTorch; there are complex mathematical objects in and of themselves. Fortunately, most of the intricacies are not necessary. In this section, you will compare them to vectors and numpy arrays.
Estimated Time Needed: 25 min
Preparation
Import the following libraries that you'll use for this lab:
Check PyTorch version:
This is the function for plotting diagrams. You will use this function to plot the vectors in Coordinate system.
# Plot vecotrs, please keep the parameters in the same length
# @param: Vectors = [{"vector": vector variable, "name": name of vector, "color": color of the vector on diagram}]
def plotVec(vectors):
ax = plt.axes()
# For loop to draw the vectors
for vec in vectors:
ax.arrow(0, 0, *vec["vector"], head_width = 0.05,color = vec["color"], head_length = 0.1)
plt.text(*(vec["vector"] + 0.1), vec["name"])
plt.ylim(-2,2)
plt.xlim(-2,2)
Types and Shape
You can find the type of the following list of integers [0, 1, 2, 3, 4] by applying the constructor torch.tensor():
As a result, the integer list has been converted to a long tensor.
The Python type is still torch.Tensor:
You can find the type of this float list [0.0, 1.0, 2.0, 3.0, 4.0] by applying the method torch.tensor():
The float list is converted to a float tensor.
Note!
The elements in the list that will be converted to tensor must have the same type.
From the previous examples, you see that torch.tensor() converts the list to the tensor type, which is similar to the original list type. However, what if you want to convert the list to a certain tensor type? torch contains the methods required to do this conversion. The following code converts an integer list to float tensor:
You can also convert an existing tensor object (tensor_obj) to another tensor type. Convert the integer tensor to a float tensor:
The tensor_obj.size() helps you to find out the size of the tensor_obj. The tensor_obj.ndimension() shows the dimension of the tensor object.
The tensor_obj.view(row, column) is used for reshaping a tensor object.
What if you have a tensor object with torch.Size([5]) as a new_float_tensor as shown in the previous example?
After you execute new_float_tensor.view(5, 1), the size of new_float_tensor will be torch.Size([5, 1]).
This means that the tensor object new_float_tensor has been reshaped from a one-dimensional tensor object with 5 elements to a two-dimensional tensor object with 5 rows and 1 column.
Note that the original size is 5. The tensor after reshaping becomes a 5X1 tensor analog to a column vector.
Note!
The number of elements in a tensor must remain constant after applying view.
What if you have a tensor with dynamic size but you want to reshape it? You can use -1 to do just that.
You get the same result as the previous example. The -1 can represent any size. However, be careful because you can set only one argument as -1.
You can also convert a numpy array to a tensor, for example:
Converting a tensor to a numpy is also supported in PyTorch. The syntax is shown below:
back_to_numpy and new_tensor still point to numpy_array. As a result if we change numpy_array both back_to_numpy and new_tensor will change. For example if we set all the elements in numpy_array to zeros, back_to_numpy and new_tensor will follow suit.
Pandas Series can also be converted by using the numpy array that is stored in pandas_series.values. Note that pandas_series can be any pandas_series object.
consider the following tensor
The method item() returns the value of this tensor as a standard Python number. This only works for one element.
this_tensor=torch.tensor([0,1, 2,3])
print("the first item is given by",this_tensor[0].item(),"the first tensor value is given by ",this_tensor[0])
print("the second item is given by",this_tensor[1].item(),"the second tensor value is given by ",this_tensor[1])
print("the third item is given by",this_tensor[2].item(),"the third tensor value is given by ",this_tensor[2])
we can use the method tolist() to return a list
Practice
Try to convert your_tensor to a 1X5 tensor.
Indexing and Slicing
In Python, the index starts with 0. Therefore, the last index will always be 1 less than the length of the tensor object. You can access the value on a certain index by using the square bracket, for example:
# A tensor for showing how the indexs work on tensors
index_tensor = torch.tensor([0, 1, 2, 3, 4])
print("The value on index 0:",index_tensor[0])
print("The value on index 1:",index_tensor[1])
print("The value on index 2:",index_tensor[2])
print("The value on index 3:",index_tensor[3])
print("The value on index 4:",index_tensor[4])
Note that the index_tensor[5] will create an error.
The index is shown in the following figure:
Now, you'll see how to change the values on certain indexes.
Suppose you have a tensor as shown here:
Assign the value on index 0 as 100:
As you can see, the value on index 0 changes. Change the value on index 4 to 0:
The value on index 4 turns to 0.
If you are familiar with Python, you know that there is a feature called slicing on a list. Tensors support the same feature.
Get the subset of tensor_sample. The subset should contain the values in tensor_sample from index 1 to index 3.
As a result, the subset_tensor_sample returned only the values on index 1, index 2, and index 3. Then, it stored them in a subset_tensor_sample.
Note!
The number on the left side of the colon represents the index of the first value. The number on the right side of the colon is always 1 larger than the index of the last value. For example, tensor_sample[1:4]means you get values from the index 1 to index 3 (4-1).
As for assigning values to the certain index, you can also assign the value to the slices:
Change the value of tensor_sample from index 3 to index 4:
The values on both index 3 and index 4 were changed. The values on other indexes remain the same.
You can also use a variable to contain the selected indexes and pass that variable to a tensor slice operation as a parameter, for example:
# Using variable to contain the selected index, and pass it to slice operation
selected_indexes = [3, 4]
subset_tensor_sample = tensor_sample[selected_indexes]
print("The inital tensor_sample", tensor_sample)
print("The subset of tensor_sample with the values on index 3 and 4: ", subset_tensor_sample)
You can also assign one value to the selected indexes by using the variable. For example, assign 100,000 to all the selected_indexes:
The values on index 1 and index 3 were changed to 100,000. Others remain the same.
Note!
You can use only one value for the assignment.
Practice
Try to change the values on index 3, 4, 7 of the following tensor to 0.
Tensor Functions
For this section, you'll work with some methods that you can apply to tensor objects.
Mean and Standard Deviation
You'll review the mean and standard deviation methods first. They are two basic statistical methods.
Create a tensor with values [1.0, -1, 1, -1]:
Here is the mean method:
The standard deviation can also be calculated by using tensor_obj.std():
Max and Min
Now, you'll review another two useful methods: tensor_obj.max() and tensor_obj.min(). These two methods are used for finding the maximum value and the minimum value in the tensor.
Create a max_min_tensor:
Note!
There are two minimum numbers as 1 and two maximum numbers as 5 in the tensor. Can you guess how PyTorch is going to deal with the duplicates?
Apply tensor_obj.max() on max_min_tensor:
The answer is tensor(5). Therefore, the method tensor_obj.max() is grabbing the maximum value but not the elements that contain the maximum value in the tensor.
Use tensor_obj.min() on max_min_tensor:
The answer is tensor(1). Therefore, the method tensor_obj.min() is grabbing the minimum value but not the elements that contain the minimum value in the tensor.
Sin
Sin is the trigonometric function of an angle. Again, you will not be introducedvto any mathematic functions. You'll focus on Python.
Create a tensor with 0, π/2 and π. Then, apply the sin function on the tensor. Notice here that the sin() is not a method of tensor object but is a function of torch:
The resultant tensor sin contains the result of the sin function applied to each element in the pi_tensor.
This is different from the previous methods. For tensor_obj.mean(), tensor_obj.std(), tensor_obj.max(), and tensor_obj.min(), the result is a tensor with only one number because these are aggregate methods.
However, the torch.sin() is not. Therefore, the resultant tensors have the same length as the input tensor.
Create Tensor by torch.linspace()
A useful function for plotting mathematical functions is torch.linspace(). torch.linspace() returns evenly spaced numbers over a specified interval. You specify the starting point of the sequence and the ending point of the sequence. The parameter steps indicates the number of samples to generate. Now, you'll work with steps = 5.
Assign steps with 9:
Use both torch.linspace() and torch.sin() to construct a tensor that contains the 100 sin result in range from 0 (0 degree) to 2π (360 degree):
Plot the result to get a clearer picture. You must cast the tensor to a numpy array before plotting it.
If you know the trigonometric function, you will notice this is the diagram of the sin result in the range 0 to 360 degrees.
Practice
Construct a tensor with 25 steps in the range 0 to π/2. Print out the Maximum and Minimum number. Also, plot a graph showing the diagram that shows the result.
# Practice: Create your tensor, print max and min number, plot the sin result diagram
pi_tensor = torch.linspace(0, np.pi/2, steps = 25)
print("Maximum: ", pi_tensor.max())
print("Minimum: ", pi_tensor.min())
sin_result = torch.sin(pi_tensor)
plt.plot(pi_tensor.numpy(), sin_result.numpy())
# Type your code here
Tensor Operations
In the following section, you'll work with operations that you can apply to a tensor.
Tensor Addition
You can perform addition between two tensors.
Create a tensor u with 1 dimension and 2 elements. Then, create another tensor v with the same number of dimensions and the same number of elements:
Add u and v together:
The result is tensor([1, 1]). The behavior is [1 + 0, 0 + 1].
Plot the result to to get a clearer picture.
Try
Implement the tensor subtraction with u and v as u-v.
Tensors must be of the same data type to perform addition as well as other operations.If you uncomment the following code and try to run it you will get an error as the two tensors are of two different data types. NOTE This lab was created on a older PyTorch version so in the current version we are using this is possible and will produce a float64 tensor.
You can add a scalar to the tensor. Use u as the sample tensor:
The result is simply adding 1 to each element in tensor u as shown in the following image:
Tensor Multiplication
Now, you'll review the multiplication between a tensor and a scalar.
Create a tensor with value [1, 2] and then multiply it by 2:
The result is tensor([2, 4]), so the code 2 * u multiplies each element in the tensor by 2. This is how you get the product between a vector or matrix and a scalar in linear algebra.
You can use multiplication between two tensors.
Create two tensors u and v and then multiply them together:
The result is simply tensor([3, 4]). This result is achieved by multiplying every element in u with the corresponding element in the same position v, which is similar to [1 * 3, 2 * 2].
Dot Product
The dot product is a special operation for a vector that you can use in Torch.
Here is the dot product of the two tensors u and v:
The result is tensor(7). The function is 1 x 3 + 2 x 2 = 7.
Practice
Convert the list [-1, 1] and [1, 1] to tensors u and v. Then, plot the tensor u and v as a vector by using the function plotVec and find the dot product:
# Practice: calculate the dot product of u and v, and plot out two vectors
u = torch.tensor([-1, 1])
v = torch.tensor([1, 1])
plotVec([
{"vector": u.numpy(), "name": 'u', "color": 'r'},
{"vector": v.numpy(), "name": 'v', "color": 'b'},
{"vector": (u / v).numpy(), "name": w, "color": 'g'}
])
# Type your code here
About the Author
Hi, My name is Juma Shafara. Am a Data Scientist and Instructor at DATAIDEA. I have taught hundreds of peope Programming, Data Analysis and Machine Learning. I also enjoy developing innovative algorithms and models that can drive insights and value. I regularly share some content that I find useful throughout my learning/teaching journey to simplify concepts in Machine Learning, Mathematics, Programming, and related topics. Besides these technical stuff, I enjoy watching soccer, movies and reading mystery books.