Linear Discriminant Analysis
To overcome the shortcomings of high dimensional models around us, various dimensionality reduction methods have been introduced, which are useful in converting a high dimensional model into a low dimensional model, thus facilitating easy analysis. Of which, feature extraction is one the most important ones. Feature extraction rebuilds the original features to form newer features that are more informative, generalised forms of the original features, thus preventing overfitting, non-redundant meaning they do not consist of correlated or inter-related features. PCA (Principal Component Analysis) and LDA (Linear Discriminant Analysis) are two different feature extraction methods.
LDA (Linear Discriminant Analysis): This is a supervised type of learning. It is a dimensionality reduction technique used in applications of pattern classification. The maximizing separation between different classes is the prime aim of this technique. Linear combination of features facilitates classification or, more commonly, dimensionality reduction.
Advantages of LDA:
- Linear decision boundary: As the features increase, the dimensions also increase and thus, defining classifiers for such a dataset becomes difficult. Hence, dimensionality reduction proves to be helpful in such a scenario. With dimensionality reduction, the original dataset with high dimensions is reduced to 2 dimensions, and thus a linear classifier can be defined.
- Fast classification: After dimensionality reduction, classification of the data points becomes easier. Previously with the enormous amount of data, specific features. Defining classifiers is a challenge, and without defining a classifier, classification of the given points is not possible. Hence LDA proves to be of great help in complex classification problems.
- Easy data visualization: A dimension in which data visualization, data interpretation, the separation between the classes is maximized is chosen. This is done by trial and error method. The dataset is plotted in 1 dimension, then in 2, the dimension in which separation is maximum is chosen, and then the new generalized features are computed.
Disadvantages of LDA:
- Increased time requirements: Due to classification usage, a labelled training dataset has to be given to the algorithm. This increases the time required for training the model. Also requires a certain number of examples (data points) in the training dataset with well-separated classes to give stable outputs. This also accounts for a considerable amount of time.
- Binary classification: LDA is intended for two-class or binary classification problems. It can be used for multi-classification problems as well but might do well in such an environment. Hence, it is rarely used for multi-classification problems.
January, 16, 2021