Computational speed for PyTorch tensors using cuda().squeeze() and squeeze().cuda()

I am wondering which of the following codes in PyTorch work faster:

my_tensor.cuda().squeeze() vs. my_tensor.squeeze().cuda()

Check with the code:

time_start_vocoder = time.perf_counter()          

a = my_tensor.cuda().squeeze()

            

time_elapsed_test = (time.perf_counter() - time_start_vocoder)

print('a time=',time_elapsed_test)

            

time_start_vocoder = time.perf_counter()

            

b = my_tensor.squeeze().cuda()

            

print('b time=',time_elapsed_test)

Where

my_tensor.shape = torch.Size([1, a_large_number])

Result:

Run 1:

a time= 0.00013689976185560226

b time= 9.429734200239182e-05

Run 2:

a time= 0.0001775706186890602

b time= 0.00010665040463209152

Run 3:

a time= 0.0001584356650710106

b time= 6.485264748334885e-05

Swap the order of a and b in code:

Run 1:

b time= 0.0001363120973110199

a time= 0.00011534709483385086

Run 2:

b time= 0.00021496228873729706

a time= 7.021520286798477e-05

Run 3:

b time= 0.00020176637917757034

a time= 6.876792758703232e-05

Conclusion:

The execution time for my_tensor.cuda().squeeze() and my_tensor.squeeze().cuda() are similar, and my_tensor.cuda().squeeze() might be a little bit faster.

Pi (π) in Torch Tensor - torch.pi

There is no pi (π) function in PyTorch with some version, so some people suggested to use math.pi or np.pi for conversion into torch tensor. For me, torch.pi works with a notebook without the GPU, but not with a computer with the GPU.

Here is the code to compare different implementations of the Pi function.

import torch

import numpy as np

import time

import math

#test pi

time_start = time.perf_counter()

pi_math = torch.tensor(math.pi)

time_elapsed_test = (time.perf_counter() - time_start)

print('math pi time =',time_elapsed_test)

time_start = time.perf_counter()

pi_np = torch.tensor(np.pi)

time_elapsed_test = (time.perf_counter() - time_start)

print('np pi time =',time_elapsed_test)

time_start = time.perf_counter()

pi_torch = torch.pi

time_elapsed_test = (time.perf_counter() - time_start)

print('torch pi time =',time_elapsed_test)

time_start = time.perf_counter()

PI = torch.acos(torch.Tensor([-1]))

time_elapsed_test = (time.perf_counter() - time_start)

print('PI time =',time_elapsed_test)

Results for computation time:

math pi time = 3.6502000057225814e-05

np pi time = 1.9006000002264045e-05

torch pi time = 8.009999419300584e-07

PI time = 0.00035780000007434865

Hence torch.pi is the fastest.

Without torch.pi, you may use torch.tensor(np.pi)

For an accurate pi, cast the type to float64:

torch.tensor(np.pi, dtype=torch.float64)

Example:

(Pdb) torch.tensor(np.pi)*100000

tensor(314159.2812)

(Pdb) torch.tensor(np.pi, dtype=torch.float64)*100000

tensor(314159.2654, dtype=torch.float64)

MATLAB:

pi = 3.141592653589793

Reference:

Np.pi equivalent in Pytorch

Is there the Pi greek (3.1415…) defined somewhere?

Python: Short-Time Fourier Transform (STFT) and its inverse transform (ISTFT) using librosa

Below shows an example using STFT to transform speech into the frequency domain and ISTFT to transform the spectral data back to waveforms in the time domain. The Librosa package is used.

import librosa

import soundfile as sf

x, sr = librosa.load('1.wav', sr=None)

n_fft = 256

hop_length = 128

win_length = 256

X = librosa.stft(x, n_fft=n_fft, hop_length=hop_length, win_length=win_length)

y = librosa.istft(X, n_fft=n_fft, hop_length=hop_length, win_length=win_length)

sf.write('1_istft.wav', y, 16000)

Results for x and y:

References:

Matlab: Short-Time Fourier Transform (STFT) and its inverse transform (ISTFT) (StudyEECC)

librosa.stft

librosa.istft

Matlab: Short-Time Fourier Transform (STFT) and its inverse transform (ISTFT)

For sound processing in realtime scenarios, it is not possible to wait for a complete sound file. In this case, short-time processing is important.

Here is an example using STFT to transform speech into the frequency domain and inverse transform the spectral data back to waveforms in the time domain:

[S,F,T] = stft(x, fs,'Window',win2,'OverlapLength',128,'FFTLength',256); 

[y,ti] = istft(S,fs,'Window',win2,'OverlapLength',128,'FFTLength',256);

References:

stft - Short-time Fourier transform (MathWorks)

istft - Inverse short-time Fourier transform (MathWorks)

Matlab: Discrete Cosine Transform (DCT) for speech processing

The DCT and inverse DCT may be used to convert a speech signal into the transform domain using real values and back to the speech waveform.

Example with Matlab:

%load speech

load mtlb

x = mtlb;

X = dct(x);

y = idct(X);

Results:

The values for x and y look the same, but MATLAB consider them as different.

>> isequal(x,y)

ans =

  logical

   0

For real-time processing, short-time DCT is required.

References:

Discrete Cosine Transform (Wikipedia)

dct (MathWorks)

Math for Deep Learning: arg min

arg min f(x) = the value x with the minimum value of f(x)

這個arg min f(x)的意思是，當f(x)是最小值的時候，x的數值

e.g. 以下的例子

f(0) = 3

f(0.9) = 2.1

f(1) = 2

f(1.1) = 2.5

f(2) = 5

f(3) = 46

因為最小的f(x) ＝ 2
則arg min f(x) = 1

Reference:

Explanation on arg min

Spyder: Show Numpy array in variable explorer

By default, Spyder does not show some Numpy arrays in variable explorer.

To show them, simply unselect "Exclude all-uppercase references".

Reference:

why converted numpy array is NOT showing in the variable exloporer in Spyder 4? (StackOverflow)