Daniel Voigt Godoy - Deep Learning with PyTorch Step-by-Step A Beginner’s Guide-leanpub
Outputtensor([[1., 2., 1.],[1., 1., 1.]])tensor([[1., 3., 1., 1., 1., 1.]])OutputUserWarning: To copy construct from a tensor, it isrecommended to use sourceTensor.clone().detach() orsourceTensor.clone().detach().requires_grad_(True),rather than tensor.new_tensor(sourceTensor)."""Entry point for launching an IPython kernel.It seems that PyTorch prefers that we use clone()—together withdetach()—instead of new_tensor(). Both ways accomplish exactly the sameresult, but the code below is deemed cleaner and more readable.# Let's follow PyTorch's suggestion and use "clone" methodanother_matrix = matrix.view(1, 6).clone().detach()# Again, if we change one of its elements...another_matrix[0, 1] = 4.# The original tensor (matrix) is left untouched!print(matrix)print(another_matrix)Outputtensor([[1., 2., 1.],[1., 1., 1.]])tensor([[1., 4., 1., 1., 1., 1.]])You’re probably asking yourself: "But, what about the detach()method—what does it do?"It removes the tensor from the computation graph, which probably raises morequestions than it answers, right? Don’t worry, we’ll get back to it later in thischapter.PyTorch | 75
Loading Data, Devices, and CUDAIt is time to start converting our Numpy code to PyTorch: We’ll start with thetraining data; that is, our x_train and y_train arrays."How do we go from Numpy’s arrays to PyTorch’s tensors?"That’s what as_tensor() is good for (which works like from_numpy()).This operation preserves the type of the array:x_train_tensor = torch.as_tensor(x_train)x_train.dtype, x_train_tensor.dtypeOutput(dtype('float64'), torch.float64)You can also easily cast it to a different type, like a lower-precision (32-bit) float,which will occupy less space in memory, using float():float_tensor = x_train_tensor.float()float_tensor.dtypeOutputtorch.float32IMPORTANT: Both as_tensor() and from_numpy() return atensor that shares the underlying data with the original Numpyarray. Similar to what happened when we used view() in the lastsection, if you modify the original Numpy array, you’re modifyingthe corresponding PyTorch tensor too, and vice-versa.76 | Chapter 1: A Simple Regression Problem
- Page 50 and 51: Also, let’s say that, on average,
- Page 52 and 53: There is one exception to the "alwa
- Page 54 and 55: Random Initialization1 # Step 0 - I
- Page 56 and 57: Batch, Mini-batch, and Stochastic G
- Page 58 and 59: Outputarray([[-2. , -1.94, -1.88, .
- Page 60 and 61: one matrix for each data point, eac
- Page 62 and 63: Sure, different values of b produce
- Page 64 and 65: Output-3.044811379650508 -1.8337537
- Page 66 and 67: each parameter using the chain rule
- Page 68 and 69: What’s the impact of one update o
- Page 70 and 71: gradients, we know we need to take
- Page 72 and 73: Very High Learning RateWait, it may
- Page 74 and 75: true_b = 1true_w = 2N = 100# Data G
- Page 76 and 77: Let’s look at the cross-sections
- Page 78 and 79: Zero Mean and Unit Standard Deviati
- Page 80 and 81: Sure, in the real world, you’ll n
- Page 82 and 83: computing the loss, as shown in the
- Page 84 and 85: • visualizing the effects of usin
- Page 86 and 87: If you’re using Jupyter’s defau
- Page 88 and 89: Notebook Cell 1.1 - Splitting synth
- Page 90 and 91: Step 2# Step 2 - Computing the loss
- Page 92 and 93: Output[0.49671415] [-0.1382643][0.8
- Page 94 and 95: Notebook Cell 1.2 - Implementing gr
- Page 96 and 97: # Sanity Check: do we get the same
- Page 98 and 99: Outputtensor(3.1416)tensor([1, 2, 3
- Page 102 and 103: dummy_array = np.array([1, 2, 3])du
- Page 104 and 105: n_cudas = torch.cuda.device_count()
- Page 106 and 107: back_to_numpy = x_train_tensor.nump
- Page 108 and 109: I am assuming you’d like to use y
- Page 110 and 111: Outputtensor([0.1940], device='cuda
- Page 112 and 113: print(error.requires_grad, yhat.req
- Page 114 and 115: Output(tensor([0.], device='cuda:0'
- Page 116 and 117: 56 # need to tell it to let it go..
- Page 118 and 119: computation.If you chose "Local Ins
- Page 120 and 121: Figure 1.6 - Now parameter "b" does
- Page 122 and 123: There are many optimizers: SGD is t
- Page 124 and 125: 41 optimizer.zero_grad() 34243 prin
- Page 126 and 127: Notebook Cell 1.8 - PyTorch’s los
- Page 128 and 129: Outputarray(0.00804466, dtype=float
- Page 130 and 131: Let’s build a proper (yet simple)
- Page 132 and 133: "What do we need this for?"It turns
- Page 134 and 135: 1 Instantiating a model2 What IS th
- Page 136 and 137: In the __init__() method, we create
- Page 138 and 139: LayersA Linear model can be seen as
- Page 140 and 141: There are MANY different layers tha
- Page 142 and 143: We use magic, just like that:%run -
- Page 144 and 145: • Step 1: compute model’s predi
- Page 146 and 147: RecapFirst of all, congratulations
- Page 148 and 149: Chapter 2Rethinking the Training Lo
Loading Data, Devices, and CUDA
It is time to start converting our Numpy code to PyTorch: We’ll start with the
training data; that is, our x_train and y_train arrays.
"How do we go from Numpy’s arrays to PyTorch’s tensors?"
That’s what as_tensor() is good for (which works like from_numpy()).
This operation preserves the type of the array:
x_train_tensor = torch.as_tensor(x_train)
x_train.dtype, x_train_tensor.dtype
Output
(dtype('float64'), torch.float64)
You can also easily cast it to a different type, like a lower-precision (32-bit) float,
which will occupy less space in memory, using float():
float_tensor = x_train_tensor.float()
float_tensor.dtype
Output
torch.float32
IMPORTANT: Both as_tensor() and from_numpy() return a
tensor that shares the underlying data with the original Numpy
array. Similar to what happened when we used view() in the last
section, if you modify the original Numpy array, you’re modifying
the corresponding PyTorch tensor too, and vice-versa.
76 | Chapter 1: A Simple Regression Problem
Change language