## Can an off the shelf EEG reader, and a short piece of Matlab code, be enough to detect emotion?

This is a write up of a project I did as part of a graduate class at SUTD, course 30.502 (Research Methods). This course, taught by Prof Robert Simpson, it a mandatory module as part of the PhD program, within which students preform some experiments on brainwaves using Electroencephalography (EEG) readers. The students design their own study using the reads, and are judged on the design of experiment, use of data, manuscript describing the research, and novelty of the study.

For our experiment, me and several other students used an EEG reader to examine which brainwave frequencies were most prevalent during exposure to different types of music. We aimed to determine if there exists any statistically significant correlation between the amplitudes associated with certain brainwave frequencies and different types of auditory inputs. Our method used the fast fourier transform to isolate relevant wavelength bands in order to compare the powers between different stimuli.
We found that although there seems to be some degree of the above menioned correlation, it is not statistically significant. However, no conclusions could be drawn as the number of test subjects were limited, and the equipment used to measure brain waves may have had certain limitations which might have diminished the quality of our measurements.

We used Mindwave which is a low budget EEG and requires minimal calibration and setup. The device connects wirelessly via bluetooth link to a single computer. The mindwave uses its own bluetooth adapter that transmits and receives signals. Custom code provided by Chen Lujie parses the raw data into a single vector of signal power data that is outputted to a .CSV file.

The Mindwave is unique from most EEG devices largely because of its low cost. However, the unit does have some limitations. Unlike most commercial EEG units Mindwave has only one sensor that is located at the center of the forehead. Literature suggests that different brainwave spectrum may be stronger or more present in different areas of the brain. Thereby it was expected that the unit will be most sensitive to delta waves as they occur in the frontal lobes of the brain. Signals from the other waves may be more difficult to measure as they propagate from spatially distant localities.

### Wheres the report?

So this article is a brief description of the experiment. If you can't to view the actually report, which contains more points on the theory on emotion and brain waves, and has all the data and code theres a link to the final report here, and link to all the files from the project here.

### What are Brainwaves?

Brain waves or neural oscillations are rhythmic and repetitive electrical impulses generated by the neural activity in the central nervous system. The capturing and analysing of these signals is paramount to many fields such as neuroscience, cognitive psychology and cognitive linguistics to study several psychological and physiological phenomena. The brain functions as network of many highly connected neurons which fire off electrical signals in order to communicate. If many fire at the same time then the induced change in electric potential may be measured from outside the skull. Brain waves are based on these electro-potential signal changes and are induced by different functions of the brain. These are entirely caused by firing neurons within the brain network with preforming various tasks, with different patterns relating to different behaviour. It has been long known that the average neuron firing patterns may oscillate at various frequencies - and that these frequencies in turn may be related to various states of consciousness. EEG readers record the changes in this electric potential and record the data electronically so the waveforms may be observed by outputting a serial stream of voltage measurements.

Although there are several methods for breaking the continuous frequency ranges of the brain into groups - we have followed the standard convention of breaking the complete signal into 5 major groups. The frequency ranges examined in this report are as follows:

• Delta waves
• Delta waves occur in frequency range of 0-4 ~ Hz. These tend to be the highest in amplitude recorded via EEG readers and are normally recorded throughout during infancy and in 'slow wave sleep' in adults. In adults they are primarily produced in the sub-cortical regions - near the front of the brain, although they are also found within the diffuse regions, and mid-line regions of the brain as well.
• Theta waves
• Theta waves occur in frequency range of 4-7 ~ Hz. These are usually associated with drowsiness in adults - however it may also be associated with meditation and states of intense creativity \cite{meditation}.They are also mostly associated with infancy and an excess of theta waves is associated with brain abnormality.
• Alpha waves
• Alpha waves occur in the frequency range of 7-14 ~ H. Primarily recorded in the posterior regions of the brain and typically higher in amplitude on the dominant side (associated in the side opposite to the dominant writing hand). Increases in amplitude are correlated with states of relaxation and the closing of the eyes. Attenuations in amplitude are correlated with either opening of the eyes or periods of mental fatigue.
• Beta waves
• Beta waves occur in the frequency range of 14-30 ~ Hz. These are produced primarily on the frontal regions of the brain and distributed evenly between both hemispheres. Generally these are associated with muscular movement orders and to a lesser extend periods of active concentration \cite{beta}. Absence or lower amplitude beta waves are correlated with cortical damage and periodic changes in amplitude may be caused by various benzodiazepines and halluionogentics. Beta waves are the found to be the dominant waves in patients who have their eyes open or are nervous.
• Gamma waves
• Gamma waves occur on the frequency range of 30-100 ~ Hz. These are associated with the linking signals between collective bunches on neurons to perform motor or cognitive functions.

### Song choice

As we wish to compare collected brainwaves between varying audio stimuli, songs with a large hypothetical difference in emotional response are selected. Russel's circumplex model is used. The circumplex model splits common mood descriptions over a 2D plane with the axis of Arousal and Valence. Using this emotional assignment of songs, we selected songs with a large separation in emotional space; ”Hallelujah - Jeff Buckley” and ”Smells like teen spirit - Nirvana”. The songs lie in the first and third quadrant of the model respectively.

One-minute samples are taken from each of these tracks and separated with white noise to create one complete track. Code was prepared to sync the EEG reader with the track so recording would begin with the track and end 10 seconds after completion in case there was in lag in the track's initialization - allowing all of the data to be used without a need to cut any parts away.
The long periods of music were chosen to both ensure that the participants had time to adjust to the different audio stimuli and that brainwaves in low frequencies would be recorded. It was also hoped that this would reduce the effect of noise on the total sample.

### The Experiment

A single experiment with nine trials was completed. The protocol was as follows:

• A participant is placed in a quiet room with no external auditory stimuli. The Mindwave EEG reader is fitted. The participant is informed that he or she will be listening to three audio tracks in succession. They are then instructed to rest their hands on the desk and close their eyes to mitigate noise from blinking or movement.
• Jeff Buckley's Hallelujah is played while the subject remains motionless. This continues for 60 seconds.
• White noise is played for 60 more seconds.
• Nirvana’s Smells Like Teen Spirit is then played for 60 seconds.
• This experiment is repeated for several participants. Data is split into the various components relating to different tracks.
• Power of the brainwave bandwidths is then compared between different auditory inputs.

### The Data

In the current section we will first describe the raw data (EEG) collected during the experiment, and the preliminary treatment of outliers. We then decompose the original signals into its frequency components using the Fast Fourier Transform algorithm and identify the presence of the different wave forms. We conclude performing statistical hypothesis testing.

The experiment has been performed on 9 volunteers (adults), from whom EEG data has been recorded.

The original EEG signals from 8 subjects studied (data collected on one of the 9 participants has been excluded). Preliminary examination of the raw data reveals that there exists three clusters of subjects, namely, those corresponding to relatively clean data, with certain apparent patterns clearly visible, those with clean data with some sections of apparent noise, and those with very noisy data. It may be the case that in participants whose data was significantly noisy, the electrodes of the EEG may not have been in good contact with the skin.

Following data collection, it was found that despite the data collection being taken over an equal time period for all participants, a different number of data points were recorded. In order to handle this anomaly, it was assumed that the EEG sensor being used recorded samples at different sampling rates during different recordings. The collected data was scaled accordingly.

Certain noticeable outliers were also manually removed prior to data analysis. Also, data from one of the participants was excluded due to the presence of large amount of noise in the recordings. The source of this noise was concluded to have arisen from the facial movements that were observed during data collection on this participant.

In order to transform the raw time varying sensor data into the frequency domain, the Fast Fourier Transform (FFT) algorithm was used. The FFT of a signal x is defined as indicated below: \begin{align} X_k\ \stackrel{\text{def}}{=}\ \sum_{n=0}^{N-1} x_n \cdot e^{-i 2 \pi k n / N}, \quad k\in\mathbb{Z}\, \end{align} The raw time series data was divided into three parts, each corresponding to a particular type of sound (song A, white noise or song B), and each lasting for 60 seconds. The FFT was perfored on each of these parts separately to reveal the relative squared amplitudes or powers corresponding to each frequency. Due to some inconsistencies in the sampling time of the sensor, it was assumed that the sensors transmit at a constant rate throughout a particular recording. In order to incorporate the above mentioned assumption, a scaling factor was used to scale the horizontal axis of the time series data according to its sampling rate. Also, since the actual time duration of each part in a particular recording was 60 seconds, the horizontal axis of the frequency spectrum had to be scaled by a factor of 60 in order to obtain the true frequencies on the horizontal axis.
Once the frequency spectrum of each part was obtained, the frequencies were binned into delta, theta, alpha, beta and gamma wave frequency bands based on neuroscience literature.

The power associated with each frequency band was then determined by integrating the discrete power values within that particular band. Later on, these integrated power values were compared using a Wilcoxon test in order to determine whether there is any statistically significant difference between them for different types of sounds. Following the Fourier Transform procedure, amplitudes of the different wave bands could be contrasted between different audio stimuli.

Boxplot showing the comparisons across each wave band across all audio stimuli. R-statistics was utilised to produce the box plot - bar indicate median, box edges for +-25 percentiles and lines to +-50 percentiles. Points outside 1.5 times the 25-75 percentile range were taken to be anomalous. Each point shows data from a single participant.

we compare the power values for each particular frequency range across the participants and across the different audio stimuli. On the y-axis are reported the value of the powers, while the top and bottom horizontal axis report the different categories over which the power has been measured (audio stimuli and frequency range). The boxes indicate the first, second (median) and third quartiles of the data observed. Vertical lines spans until the maximum and minimum observed values, while an isolated point might be considerate as a potential outlier (1.5 standard deviation further form the median).

Firstly, we observe that the amplitude of the low frequency waves is significantly high as compared to other frequencies. A potential explanation for this observation is that the low frequency delta waves are known to be both of the highest amplitude and are also most prominent in the pre-frontal cortex in adults -\ which included the entirety of the participants (the EEG used is comprised only of a single sensor placed upon the forehead of the participant).

Interestingly, the results do seem to show an increase in the amplitudes across all frequency bands when the white noise is played between the two musical tracks. Although this was not expected, it may be that brain activity during this intermediate period of white noise is associated with deep thought (the participants were instructed to sit in silence without movement - in a meditation like state.). This was not initially considered by the authors however, the increase of brainwave amplitudes in when audio stimuli is removed is a point of interest.

In order to test for the difference in mean intensity of the delta waves induced by the first song, the second song and the white noise, we design the hypothesis testing and run them using the Wilcoxon test. In particular, we wish to test the following three hypotheses, where $\mu_1, \mu_2, \mu_{wn}$ are respectively the mean intensity of the delta waves generated during the first song, the second song and the white noise: \begin{align} \label{f1} H_0: \quad \mu_1 = \mu_2 , \qquad H_1: \quad \mu_1 \neq \mu_2 \end{align} \begin{align} \label{f2} H_0: \quad \mu_1 = \mu_{wn} , \qquad H_1: \quad \mu_1 \neq \mu_{wn} \end{align} \begin{align} \label{f3} H_0: \quad \mu_2 = \mu_{wn} , \qquad H_1: \quad \mu_2 \neq \mu_{wn} \end{align} The Wilcoxon test is a non-parametric statistical test for paired samples (the samples collected are paired since EEG data is collected on the same subject for different musical stimulus). The choice of a non-parametric test is justified by the small sample size, for which normal distribution of the underlying population cannot be assumed. The Wilcox test pool together the two samples considered (of size $n$ and $m$) and compute for each observed value its respective rank. For $Z_i, i=1...n$ being the observed value from the first sample, and for $Z_{n+j}, j=1,...m$ being the observed values from the second sample, we compute the respective ranks $Ri := rank(Z_i)$. The test statistic is then: \begin{align} T^{Wilcoxon} = \sum_i^{n} R_i \end{align} Large values of $T^{Wilcoxon}$ mean that the observed value of the first group are generally larger than the ones from the second group, and hence indicate evidence against $H_0$.

### Conclusions

In this work, a general framework for the study of the relationship between music and brain activity has been proposed, and a procedure to collect neural oscillations induced by audio stimuli was described. In addition, several drawbacks of the sensor, including its variable sampling time and its bias for delta wave frequencies was mentioned. The methods for taking these inconsistencies into consideration and for removing outliers before transforming the data into its frequency domain was also explained. The bias towards delta wave frequencies was confirmed by their significantly larger squared amplitudes in the frequency spectrum. Based on established guidelines of binning different frequency components, the strength of the different frequency components of the brain waves was obtained. Finally, the transformed data was analysed using the statistical non-parametric Wilcoxon test in order to determine the existence of any significant differences in the strengths of the delta frequency components for the different types of sounds. It was found that although certain patterns could be observed from the raw data qualitatively, the high p-values obtained from the Wilcoxon test on the transformed were too large to conclude the existence of any statistically significant differences between the different sounds.

This however, may not imply the non existence of any considerable differences in the transformed data, as it can be observed in the plot of the raw signals. It must be acknowledged that the sensor used for data collection had certain limitations, and the sample size was too small to arrive at any conclusions.