What is the exploding gradient problem while using the back propagation technique?
- The exploding gradient problem occurs in deep learning when gradients during backpropagation become very large, leading to numerical instability and slow convergence.
- It can be mitigated using gradient clipping or selecting appropriate initialization methods.
- Gradient Descent (GD) computes gradients using the entire dataset in each iteration.
- Stochastic Gradient Descent (SGD) updates the model's parameters using only one randomly selected data point (or a small batch) in each iteration. It's computationally efficient but can have more erratic convergence.