In part I of this essay I argue a mind-centered view of BCI-controlling brain signals (or brain signal “targets”) is more optimal than a task-based view, or single component of mind-view such as intention. One needs to consider, define, and then target the user’s mind as a whole, in order to work with the brain signal to which it corresponds.
The user’s state of mind is a largely untapped resource for the BCI community. So far, the vast majority of the machine learning community has worked on improving and robustifying the EEG signal decoding, without considering the human in the loop (Lotte, Jeunet, Mladenovic et al., 2018). I’ll now describe a view of BCI with the user’s mind as its centerpiece. Let’s start with how the mind — and brain signal — can be more optimally defined.
Consider a specific example: a user’s intention to “move the cursor.” Every time a user activates this intention, it is done within a specific context: environment, situation (lab, home, work…), level of experience & expertise, etc. These variables cause considerable variation in the user’s state of mind. They can affect the user’s perception, thought, emotion, executive function, goals, attention, motivation, fatigue etc. In other words, “move the cursor” occurs within a larger mind context that is quite variable, or noisy.
Now let’s imagine the cursor has 5 degrees of freedom: left, right, up, down, and stop. Potential brain signature targets, used to control the cursor’s movement, would be “cursor perception (size and shape),” “cursor movement” (left, right…), and the perception of a computer screen. These are predominately signatures of visual perception. Goal targets might be “attend to the cursor,” and “move the cursor left.” An emotional target state might be “feeling calm and focused.” And so on.
The bad news is, a “move the cursor” intention, and brain signal, has a lot of variability. The good news is, this variability can be used. Once defined, it can be accounted for — to be either filtered out of the signal (as noise) or to be included as PART OF the signal.
For example, let’s say the user has trouble moving the cursor down. The emotion “frustration” and perhaps “anxiety” would become part of the “down” intention signal. If these emotions and brain signals are defined, they can be filtered out of the “down intention” signal. Conversely, if they happen to be a consistent part of the user’s emotional state while learning the BCI device, they could also be targeted as part of the “down intention” signal. The target would be “down while feeling frustrated and anxious.”
Look closely and you will see the above aspects of mind — in this case movement intentions and related mind components — are all memories. In fact almost all states of mind during routine tasks can be defined as memories. “Cursor visual characteristics,” a “watch the cursor” goal, and even “feeling frustrated” are all memories.
The idea mind = memories allows the BCI designer, tester, trainer, and user to become keenly aware of the target states of mind. State of mind/brain signal targets could be researched, and selected, based on desirability, ease of activation, and uniqueness. The user could then endeavor to strongly, selectively, and consistently activate these. In other words, the user could activate — with conscious intent — her own memories as “targets” to hit, to more effectively and easily control the device. Memories and brain signal targets are one and the same.
Brain signal noise would also mirror her state of mind and active memory. Potential noise would include emotions associated with success or failure, strong desire, fatigue, frustration, and the meaning of the task. The latter would include why the user wants to move the cursor in a given context: environmental/social settings of lab, work, or home; and life context such as a job promotion, relocation, or loss of a loved one.
In short, user state of mind, brain signal, and state of memory activation are one and the same. A brain signal target = a set of general memories. Brain signal targets are optimally defined as a state of mind within a particular environmental/social/life experience/task context. To the extent this memory set is accurately defined, connected, and weighted, it represents a tool for enhancing BCI design, testing, learning, and everyday use.
Lotte, F., Jeunet, C., Mladenovic, J., N’Kaoua, B., Pilette, L. (2018). A BCI Challenge for the signal processing community: considering the user in the loop. Signal Processing and Machine Learning for Brain-Machine Interfaces, IET 1-33.