1. Introduction

Sentiment analysis is the process of using text analysis, natural language processing, and statistics to analyze customer reviews. It is an automated process to determine whether reviews express a positive, <note: There should be no space between a comma and the preceding word> negative, or neutral opinion about any topic or product. The research on sentiment analysis analyzes the polarity of the opinions of customers. Sentiment analysis helps brands and companies monitor their reputation on social media, gain feedback from customers, and much more. Opinions of previous customers or other people have a substantial influence on our daily decision making. These decisions range from choosing a daycare to buying a new refrigerator. Sentiment analysis plays a vital role in almost every field, like e-commerce, health, entertainment, education, banking, etc. Sentiment analysis is also important for enterprises and organizations to enhance the quality of products, which results in better marketing. Sentiment analysis is gaining importance in both science and industry.

Nowadays, the use of social networking sites such as Facebook, Twitter, Instagram, etc., has increased tremendously. These sites fabricate much opinion data about anything and everything. They enable users to post comments and information. Then, knowledge is extracted from this user-generated content. Important information analyzed from the data is known as opinions or sentiments of a particular user.

Sentiment analysis detects and extracts sentiments from the text and classifies them. This classification can be done at document level, sentence level, and aspect level as well. In document-level classification, the complete document is analyzed and classified as expressing positive or negative views. In the sentence level, the objective and subjective sentences are separated and then analyzed for whether the sentence has a positive, negative, or neutral opinion. Aspect-level sentiment analysis is also called feature-level sentiment analysis and is based on feature-based opinion mining. This type of sentiment analysis is very important when we want to know the exact likes and dislikes about the product.

Initially, sentiment analysis was limited to text data only, but now, data are available in various formats on various social media sites. Now, the reviews or data are available in the form of audio, video, pictures, emoticons, etc. To find the verified sentiments such as anger, happiness, sadness, frustration, etc., monitoring of such emoticons or pictures is also necessary. For sentiment analysis, the raw data or reviews have to be preprocessed. Feature vectors are extracted, then the model is trained, and finally, polarity classification is done. Fig. 1 shows a clear picture of the machine learning process.

Fig. 1. Machine learning process.

2. Approaches of Sentiment Analysis

Sentiment analysis can be done using techniques. There are two main approaches that can be used for sentiment analysis: lexicon/rule-based sentiment analysis and machine learning-based sentiment analysis.

A. Lexicon/Rule-Based Sentiment Analysis - In this approach, we calculate the sentiment score or polarity based on predefined manually created rules. This is a simple approach but has many problems, such as not having the ability to understand the meaning of long and short sentences and focusing on words or phrases of the text. In the lexicon approach, the text is considered as a bag of words. The positive, negative, or neutral polarity is calculated for all the words. Finally, the sum or average polarity is calculated to find the overall polarity of the review, tweet, etc.

B. Machine Learning Approach - In sentiment analysis using a machine learning approach, the polarity of the text is analyzed using various machine learning algorithms. Machine learning techniques use a training set and a test set for classification. The training set consists of input feature vectors and class labels corresponding to them. This training set is used to develop a classification model, which classifies the input feature vector with their corresponding class labels.

The test set validates the model by predicting the class labels of unseen text. The advantage of this technique is that it adapts and creates a trained model for a specific purpose ^[14]. There are many machine learning techniques, which can be broadly classified as supervised learning techniques and unsupervised learning techniques ^[5].

a. Unsupervised Learning Technique - In an unsupervised approach, no previous assumptions and definitions about the variables are provided to the model. The model learns the structure of data itself. There are no labeled data available. These techniques do not require prior training data ^[15].

b. Supervised Learning Technique - This approach is used when there is a finite set of classes. This technique uses labeled data to train classifiers. Automatic classifiers use a training set to learn various characteristics of a document, and to verify the performance of automatic classifier, a test set is used. Supervised learning algorithms generate functions that map input to their corresponding output ^[16]. Some of the supervised machine learning algorithms are linear regression, Bayesian network, support vector machine, maximum entropy, logistic regression, boosting, random forest, and many more.

3. Literature Review

There has been vast research available on sentiment analysis using various machine learning algorithms. One paper ^[1] compares the performance of two supervised machine learning techniques for sentiment analysis in terms of accuracy, precision, and recall. Dey et al. compared the naive Bayes’ algorithm and K-nearest neighbor for sentiment classification of movie reviews and hotel reviews. It was noted that the classifiers gave better results for the movie reviews with the Naive Bayes’ approach. Above 80% accuracy was obtained, and Naive Bayes surpassed the K-nearest neighbor approach.

Similar results were obtained for the hotel reviews. The accuracy yielded by both the classifiers was lower than the accuracy obtained in the case of movie reviews. As different datasets show different behavior with the same classifier, it is estimated that the performance of classifiers is dependent on the datasets.

In another paper ^[2], the performance of a supervised machine learning classifier , a support vector machine was studied for sentiment classification. Performance was analyzed, and a comparison was done using the WEKA tool by Ahmad et al. Precision, recall, and F-measure were the terms used to measure results. According to WEKA, the correctness of SVM during opinion classification is different in different datasets. The results clearly showed the dependency of the support vector machine performance on the input dataset.

An ensemble technique for sentiment analysis of reviews was used in another paper ^[3]. In this approach, the range from negative to positive scoring was from 1 to 5. An ensemble classification technique of random forest, multinomial naive bayes algorithm and multilayer perceptron methods gave the best performance. A random tree model, the best single classifier, produced an accuracy of 78.6%. The best ensemble technique produced a classification accuracy of 91.02%.This shows a vast increase in the accuracy.

Another paper ^[4] was on earlier research work and deals with the classification of text, emoticons, or images separately. It was observed that emoticons along with text have been neglected. Ullah et al. analyzed both the text and emoticon mode of data together and separately using both deep learning algorithms and machine learning algorithms. The dataset used to find sentiments was Twitter-based airline review data. Various feature extraction techniques like TF-IDF, bag of words, N-gram, and emoticon lexicons were used. Better results were yielded when various machine learning and deep learning classifiers were used on the combined text and emoticon data. In the case of LSTM and CNN, the accuracy achieved for combined text and emoticon data was 81% and 89%, respectively, as compared to 88% and 79% accuracy of the two classifiers for text-only data.

A paper ^[5] analyzed emotions and opinion of reviewers by using various machine learning techniques. Three supervised machine learning algorithms, namely support vector machine, naive Bayes, and maximum entropy, were analyzed and compared in terms of evaluating parameters: recall, F-measure, accuracy, and precision. This paper states that in the case of a support vector machine classifier, the work of Bac Le et al. gives better performance in precision, recall, and f-measure, whereas the work of Geetika Gautam et al. gives better accuracy. As far as Naive Bayes classifiers are concerned, the work of Bac Le et al. gives better performance in precision and f-measure score, but the work of Geetika Gautam et al. gives better performance in recall and accuracy. Similarly, for maximum entropy, the performance of the work of Neethu et al. is better than that of Geetika Gautam et al..

In another paper ^[6], machine learning techniques were used with a dictionary-based approach, which is a type of lexicon-based approach. Sentiment analysis was done for all product reviews, and then classification was done using machine learning algorithms, namely Naive Bayes and support vector machine. The classifiers calculate the accurate measurement for the same dataset in this paper. The naive Bayes classifier achieved 98.17% accuracy, and the support vector machine showed 93.54% accuracy. In this paper, it was also concluded that aspect-level sentiment analysis also helps in improving results.

A paper ^[7] introduced two novel deep learning approaches for sentiment classification, namely a convolutional neural network and long short-term memory. LSTM and CNN were designed separately to encode sentiment details to the corresponding feature vectors. A 3-layer neural network combined with the advantages of LSTM and CNN vectors was used to achieve better results in terms of accuracy of sentiment classification. Tein et al. have also proposed an approach of clustering documents or sentences according to their similarities.

In another paper ^[8], a text classification framework was presented for sentiment analysis based on the multinomial Naive Bayes classification algorithm and the TF - IDF method. This study shows that when using a simple multinomial naive Bayes classifier, the accuracy obtained was 91%, but it was increased to 93% when logistic regression was applied along with the naive Bayes classifier. Sci-kit learn's train-test split was used to split a dataset into training and testing data.

In another paper ^[9], for performance evaluation of logistic regression, all three cases of unigram, bigram, and trigram were considered, and it was stated that trigram provides exactness with 93.4%, and perceptron precision is larger. Also, bag of words is an entirely proficient technique that compromises with a little exactness.

In another paper ^[10], few basic classifiers were used by a proposed framework for sentiment analysis, such as naive Bayes, support vector machine, and logistic regression. Subjectivity filtering was used for better accuracy in the case of small datasets. Usernames, emoticons, punctuations, and URLs were removed from reviews or tweets. Stemming was done to decrease the size of the feature space. After all the preprocessing steps, the polarity score was calculated, ranging from -5 to 5. Naive Bayes and support vector machine classifiers were used to classify the sentiments in this study, and logistic regression was utilized for highlighting the features. Also, the effective score of the word was used for defining labels.

In another paper ^[17], Lee et al. explore various preprocessing techniques, including segmentation, conjunction rules, stop words and punctuation elimination, negation, and keyword-based comparison. They demonstrate that accuracy of classifying sentiments can be increased by using these pre- processing steps. The work shows that when applying only the first and mandatory stage, i.e., segmentation, the accuracy was 81.65%, but it increases to 83.25% when other pre-processing techniques are applied separately. Also, when these techniques are applied in stages, the accuracy increases to as high as 87.05%.

Ramadhan et al. conclude that the most leading parameter to achieve better accuracy, precision, or recall is the composition of training and testing data. It was observed in a paper ^[18] that the higher the amount of training data is, the more accuracy is achieved. However, while using a K-fold cross validation process, the number of folds does not have any impact on the performance of any method.

In another study ^[19], sentiment analysis was applied on the phase level rather than the document level. Key graph keyword extraction technique was used to extract keywords with high frequency terms. Also, by measuring the strength, the intensity of opinion polarity was obtained. The K-means clustering algorithm was used to compare the normal results with the star rating of the same data. It was observed that the results were better in the case of star rating data. This work can be extended to the attribute level for increased accuracy and precision of sentiment polarity.

In another paper ^[20], the performance of a clustering-based approach was compared to supervised learning and symbolic techniques. It was observed that the accuracy of supervised learning is higher as compared to a clustering-based approach, although it is at an acceptable level. Also, due to the use of balanced data, accuracy is accepted to be low.

Supervised learning technique requires much time for training. Although extra time is taken by a clustering-based approach to cluster data multiple times, this approach takes less time as compared to that taken by supervised learning for training on a dataset of the same size. Thus, the efficiency of a cluster-based approach is observed to be better than supervised learning.

In other research ^[21], Troussas et al. targeted Facebook , as Twitter updates are limited to 140 characters only. Facebook allows 5000 characters for status updates. Thus, more clear sentence construction is expected. In this study, three classifiers, namely the Naive bayes algorithm, Rocchio, and perceptron methods, were used to compare whether Facebook statuses are positive or negative. Various methods of data representation have been explored, and improvements to unigram models were reported in the research. The paper finally concludes that the naive Bayes algorithm performed better for precision and F-score values, but the recall value was lower as compared to the Rocchio classifier and higher than the perceptron classifier.

Mubarak et al. ^[22] concluded that the naive Bayes algorithm performs better for aspect-based sentiment analysis. In this study, a part-of-speech tagging method and chi squared method were used for feature selection. These methods have helped speed up the computation time, but degradation in system performance is noted.

Sridshar et al. ^[23] proposed a hybrid neural architecture consisting of bi-directional LSTM and an artificial neural network for classification of text data. Bi - LSTM identifies the semantic orientation of the reviews, whereas the spatial characteristics of hypernym features are mapped to temporal features of the reviews. The hybrid model determines the polarity of movie reviews with an accuracy of 83.16%. Hypernym-based feature engineering is used to pull out features from the abstract.

Hoang et. al ^[24] used the bidirectional encoder representation from transformer (BERT), a pre-trained language model, to train bidirectional representations from unlabeled data. The study proposed a new abstract-based sentiment analysis to identify sentiments related to each aspect. Further research was done to observe the association between the aspect and text. An ensemble model that can classify both sentiments and aspects was also used.

Kanakraj et. al ^[25] used various bagging and boosting ensemble methods to track the performance against various parameters such as precision, recall, and F-score. For conducting such experiments, data were subjected to various data processing modules, such as data cleaning, set finding, and feature vector formation. They proposed that the ensemble methods outperform the conventional classification models. Also, extremely randomized tree methods perform better than other classification techniques. The chronological summary of various machine learning techniques and their accuracy for sentiment analysis are presented in Table 1.

4. Conclusion

In this paper, a review of various machine learning techniques for sentimental analysis has been presented. This study concludes that various machine learning algorithms can be used for sentiment analysis. The literature review shows that some algorithms show different performances on different datasets. Also, we can achieve better accuracy or performance of classifiers when ensemble techniques or hybrid techniques are used. The results also depend on the preprocessing techniques used for feature extraction. Multiple preprocessing techniques help in more accurate sentiment analysis. Much research work has been done on text sentiment analysis, but emotions also play a vital role in sentiment classification of customer reviews or tweets.