DeePay: deep learning decodes EEG to predict consumer’s willingness to pay for neuromarketing

Figures

• 

  FIGURE 1. Experimental design. Subjects first received instructions, were mounted with an 8-electrode EEG cap, and received their 50 NIS endowment. Then, they performed the BDM task, for 72 different products, each repeating three times in a random order for a total of 216 trials. On each trial, subjects watched a screen with a plus icon for 1–2 s, then observed the product for 3.5 s, and finally had 15 s to mark how much they were willing to pay for the observed product, from 0 to 50 NIS. Importantly, the starting point of the tick marker on the bid scale (in red) was randomly located in each trial. Every 20 trials a reminder screen appeared to remind subjects to think on their bid as they observe a product. At the end of the task, one trial was chosen at random, and a random price was given to the product at that trial. If the bid was larger than the price, the subject won the product and the remainder of the endowment (50 NIS minus the price of the product). If the bid was lower than the price, the subject did not win the product, but kept the full endowment.

• 

  FIGURE 2. Behavioral exclusion criteria. These graphs demonstrate the WTP values for all three blocks, sorted by the second block (lowest to highest value), for four example subjects. (A) Subject 12 was excluded since she bid less than 5 NIS over 85% of her trials, representing very low variability in WTP across products. We excluded 27 subjects based on this criterion. (B) Subject 40 was not excluded and is considered a good subject. Note that she bid above 45 NIS in only 47% of her trials and was therefore not excluded. No subjects were excluded based on this criterion. (C) For subject 149, the correlation between the second block and third block was 0.41, representing very high variability in WTP values. This might suggest that this subject did not report its “true” valuations but answered randomly or that her actual valuations change very quickly on a block-by-block basis. Two subjects were excluded based on this criterion. (D) Subject 172 exhibited consistent behavior across the three repetitions of the BDM blocks and, in addition, highly varied valuation across products. Her correlation coefficient between blocks 2 and 3 was 0.97, her bid rate below 5 NIS was only 12%, and her bid…

• 

  FIGURE 3. WTP values per product. The above shows the average WTP values across subjects and blocks as “filled” large dots, and the actual values scattered around them as semi-transparent smaller dots, so a more popular value for a product will appear as more saturated and denser dots. The right of the graph shows the frequency of each WTP value across all products and subjects, binned to a width of 1 NIS. The percentages signify the propensity of all WTP values binned to a width of 10 NIS each, from 0 to 50. Error bars signify the standard error of the mean (SEM).

• 

  FIGURE 4. Correlation of valuations between blocks. The correlation between valuations of the same products between every two blocks, per subject. Correlations between blocks 2 and 3 were significantly larger than correlations between blocks 1 and 2 [t(181) = 8.39, p < 0.001] and blocks 1 and 3 [t(181) = 10.15, p < 0.001].

• 

  FIGURE 5. Bandpass filters. (A) Top–3 examples of filters from the trained network; bottom–the Fourier transform of each filter. (B) Aggregation of all sinc bandpass filters from the network. Each horizontal line represents a single bandpass filter, such that the line starts and ends at the filter’s cut-off frequencies. The histogram in opaque gray shows the overall proportion of each frequency across all filters.

• 

  FIGURE 6. Spatial frequency illustration. The interpolated spatial distribution of weights from the second layer plotted as topographic maps. Note that artifacts appeared in areas further away from the existing electrodes, due to limitations on the interpolation procedure, and should be ignored. (A) There are five spatial filters for each sinc bandpass filter, shown as five rows of maps in the figure. Each column represents a corresponding sinc filter, from left to right, as shown in Figure 5A. (B) The average spatial weights per frequency band. Spatial weights from the second layer of the network were divided into their respective band (or nearest band), according to the bandpass filter to which they belong from the previous layer. Then, all spatial weights averaged within each band to yield eight final weights per frequency band.

• 

  FIGURE 7. Distribution of weights from the attention layer. When the input reaches the attention layer, it is comprised of 75 time points (due to average pooling), each with its individual weight that is generated per input. We passed all samples through the final network and saved all weights. Thereon, we plotted a violin distribution of all weights per time point to generate this figure. Between time points –1 to 0 s (red area), subjects watched a black screen, between 0.9–1.3 s from stimulus onset (orange area), there was a slight increase in importance, and from time point 2.5 onward, we can observe a substantial increase in importance (green area).

• 

  FIGURE 8. Average ERP per quartile of WTP values. EEG recordings of electrode FPz (A), F7 (B), and Cz (C), after averaging for each WTP quartile, across subjects and products. The product image appeared at time zero, marking the start of a trial. Opaque bounds around a line signify the SEM at each data point. Cleary, each electrode recorded different responses and required diverse feature extraction procedures, further demonstrating the need for optimized and automatic feature extraction. The shaded areas mark time points where no difference between WTP quartiles was found. Dotted lines indicate the 150, 220, and 300-ms mark, respectively.

• 

  FIGURE 9. Spectrogram differences for WTP quartiles. (A) The difference between the average spectrogram of the top and bottom average spectrogram quartiles. (B) The difference between the average spectrogram of the mid-high and mid-low average spectrogram quartiles. The product image appeared at time zero, marking the start of a trial. The dashed pink circle surrounded a substantial difference in the log PSDs.