The mind’s contents can be thought of as that of experience: perception, thought, emotion, goals, attention, intention, motor control etc., etc. However it is defined, it’s clear the mind or mental processes are the whole point of the brain. Without a mind what use would a brain be? The main function of the brain is to enable the mind’s expression.
Brain models which ignore the mind make no sense. This is because, in my opinion, they leave out that which makes the brain interesting and useful in the first place. It makes little sense to model “voluntary movement” without also modeling “movement perception (of one’s body & what it is acting on), movement intention, related emotions and intentional states while moving,” etc.
Further, if you want to model the brain in a practical way – to enhance an applied neuroscience project for example — you’ll need to get more specific. The details of the mind become critical. For example, to model “movement intention” for a motor neuroprosthetic, you’ll need to model “reach for a cup,” “fearful feelings during reaching,” and thoughts such as “careful” and “don’t knock the cup over.”
The mind is currently the invisible elephant in the room in applied neuroscience. But there’s good news! We can not only recognize it exists and is important, but learn to harness its power! It all starts with the idea that the mind is the key to understanding the brain.
I have argued in parts I, II, III that the brain signal is a direct reflection of the mind. I have also claimed a better definition of the latter would yield better brain signal “targets” for designer, tester, and user of a BCI device. I now wish to explain further how these targets might help the user.
First, state of mind targets could be defined by the user, with conscious intent. This gives the user more control over the whole process. She can decide what she wants to do with her mind, during device operation, with a full and clear awareness. What states of mind might she wish to strive toward, and express, during her daily life? Or for that matter, during her time in the lab. She can research, discuss and arrive at a personalized, ideal target states of mind. These targets may take into account her lifestyle, occupation, social habits, and personal goals. She could define her most desired, realistic and easily-to-attain states of mind.
These personalized or user-defined states of mind would be the targets for her to try to “hit.” For example, if she meditates regularly, “mindfulness during BCI movement” might be one target.
The user could do their own research in designing her targets. For example, the user might learn it’s difficult to maintain a constant level of excitement/dopamine brain signal, when one is both failing and succeeding at artificial movement. Social situations would magnify this effect. In this case, the user could define a state of mind that allows for emotional flexibility and variability.
On the other hand, the user might be a devoted Buddhist who is (or strives to be) unattached to results. In this case, she could define her target as “an even level of excitement while moving.”
Overall, personalized mind/brain targets could be researched, considered, and defined by the user — in conjunction with her clinical and professional team, as well as family and friends. The user could construct fairly rigid/consistent, or more flexible targets — according to the demands of the situation, the physical environment, etc. These targets would be hit more strongly, consistently and precisely since (1) the targets more accurately reflect her state of mind, (2) she has a clear idea of what the targets are, and (3) she is motivated to hit them.
In Parts I and II of this essay I argue the mind is a overlooked, yet potentially very useful, resource for improving BCI performance. In particular, I argue (a) the mind controls the brain signal, (b) is most accurately defined as a set of general memories, based on experience (not processing), and therefore (c) general memories are optimal brain signal “targets” for designer, teacher, learner, and user to shoot for.
In other words, brain signal targets — when defined accurately (as a set of general memories) — become powerful BCI tools.
Let’s now look at this in more detail. How would one build an optimal mind, or general memory, BCI target? A movement intention is a good starting point. For example, consider a “cursor moves left” intention. It would be comprised of a memory set featuring “cursor,” “left,” and “cursor moves left.” These are perceptual memories comprised of a range of cursor sizes and shapes, left directions (straight left, left but a little up, left and down 1 inch, etc.), and cursor movements (directions, speeds, distances).
Whenever the user attempts to move a cursor, in a particular way, a specific memory set will activate. These memories ARE definable. They have a particular (range of) perceptual content.
When moving a cursor, the user will have cognitive, emotional and other associations as part of their mind. Once this memory set is acknowledged and listed, it can then be connected and weighted.
It’s important to realize that human memory is much more than a brain process. From a subjective view, all memories have content. Can the reader think of a memory that does not have definable content? By definition, a memory set has content that, if not defined with 100% completeness and precision, can be summarized quite accurately.
Once a target memory set is defined, a corresponding set of functional neural network (FNN) ranges can be defined as well. And, a corresponding set of brain signal ranges. A general memory set = a set of FNN ranges = a set of brain signal ranges.
Why might defining BCI brain signal targets based on general memories be useful? Because, it leads to more accurate mind and brain signatures — of any (common and desired) movement, within any context. These signatures would be more accurate, comprehensive, and precise than task-based or other cognitive neuroscience-based signatures. They can be fit to the person, situation, task, and other context.
With signal classification for example, the range of signals interpretable as “move cursor left” would be broadened. They would include the range of (the most common, strongly-activated, and unique) COMBINATIONS of perception, emotion, intention, inner speech and other aspects of mind.
The basic idea is to recognize user “state of mind” exists, it occurs in the brain, it can be defined as a set of active general memories (based on past experience), is represented physically by a corresponding set of brain signal range activity, their variability across movements and context can be defined, and these brain signal ranges make ideal BCI targets for the designer, trainer, learner, and user to strive to “hit.”
In part I of this essay I argue a mind-centered view of BCI-controlling brain signals (or brain signal “targets”) is more optimal than a task-based view, or single component of mind-view such as intention. One needs to consider, define, and then target the user’s mind as a whole, in order to work with the brain signal to which it corresponds.
The user’s state of mind is a largely untapped resource for the BCI community. So far, the vast majority of the machine learning community has worked on improving and robustifying the EEG signal decoding, without considering the human in the loop (Lotte, Jeunet, Mladenovic et al., 2018). I’ll now describe a view of BCI with the user’s mind as its centerpiece. Let’s start with how the mind — and brain signal — can be more optimally defined.
Consider a specific example: a user’s intention to “move the cursor.” Every time a user activates this intention, it is done within a specific context: environment, situation (lab, home, work…), level of experience & expertise, etc. These variables cause considerable variation in the user’s state of mind. They can affect the user’s perception, thought, emotion, executive function, goals, attention, motivation, fatigue etc. In other words, “move the cursor” occurs within a larger mind context that is quite variable, or noisy.
Now let’s imagine the cursor has 5 degrees of freedom: left, right, up, down, and stop. Potential brain signature targets, used to control the cursor’s movement, would be “cursor perception (size and shape),” “cursor movement” (left, right…), and the perception of a computer screen. These are predominately signatures of visual perception. Goal targets might be “attend to the cursor,” and “move the cursor left.” An emotional target state might be “feeling calm and focused.” And so on.
The bad news is, a “move the cursor” intention, and brain signal, has a lot of variability. The good news is, this variability can be used. Once defined, it can be accounted for — to be either filtered out of the signal (as noise) or to be included as PART OF the signal.
For example, let’s say the user has trouble moving the cursor down. The emotion “frustration” and perhaps “anxiety” would become part of the “down” intention signal. If these emotions and brain signals are defined, they can be filtered out of the “down intention” signal. Conversely, if they happen to be a consistent part of the user’s emotional state while learning the BCI device, they could also be targeted as part of the “down intention” signal. The target would be “down while feeling frustrated and anxious.”
Look closely and you will see the above aspects of mind — in this case movement intentions and related mind components — are all memories. In fact almost all states of mind during routine tasks can be defined as memories. “Cursor visual characteristics,” a “watch the cursor” goal, and even “feeling frustrated” are all memories.
The idea mind = memories allows the BCI designer, tester, trainer, and user to become keenly aware of the target states of mind. State of mind/brain signal targets could be researched, and selected, based on desirability, ease of activation, and uniqueness. The user could then endeavor to strongly, selectively, and consistently activate these. In other words, the user could activate — with conscious intent — her own memories as “targets” to hit, to more effectively and easily control the device. Memories and brain signal targets are one and the same.
Brain signal noise would also mirror her state of mind and active memory. Potential noise would include emotions associated with success or failure, strong desire, fatigue, frustration, and the meaning of the task. The latter would include why the user wants to move the cursor in a given context: environmental/social settings of lab, work, or home; and life context such as a job promotion, relocation, or loss of a loved one.
In short, user state of mind, brain signal, and state of memory activation are one and the same. A brain signal target = a set of general memories. Brain signal targets are optimally defined as a state of mind within a particular environmental/social/life experience/task context. To the extent this memory set is accurately defined, connected, and weighted, it represents a tool for enhancing BCI design, testing, learning, and everyday use.
Lotte, F., Jeunet, C., Mladenovic, J., N’Kaoua, B., Pilette, L. (2018). A BCI Challenge for the signal processing community: considering the user in the loop. Signal Processing and Machine Learning for Brain-Machine Interfaces, IET 1-33.
What exactly are brain signal “targets,” and how can they enhance the field of brain computer interface?
The concept of using a brain signal as a target — to build into device design, improve trainer/user performance, and use as a classifier (to control a device) — depends on the mind. After all, the user’s state of mind most directly determines his or her brain signal. Brain signal reflects state of mind.
Although the mind/brain signal connection may seem obvious, it is an unrecognized issue within most of the BCI community. The mind is barely recognized as having anything to do with the brain signal. This is a huge problem, because the brain signal reflects not the task as much as the mind. The brain signal reflects the user’s overall state of mind, not just the task (though task is PART of the mind).
How best to elicit, analyze, classify, and teach the user how to generate a given brain signal? The answer is to focus on, understand, and define the mind that corresponds to it. Yet, the mind is all but ignored. In place of brain signal targets that are mind-based are sub-optimal stand-ins: mainly task-based and movement intention targets.
The reason task-based targets are sub-optimal (as are single mind targets such as intention) is that the mind during a task or intention is filled with many other components. These include (visual and somatosensory) perception, emotion, attitude, motivation, imagination, and many others. These off-target states of mind can come to dominate it during a task. Yet they are seen, inevitably, as undefined noise — rather than DEFINED noise that can be either a.) separated from the signal, or b.) converted to (part of) the signal.
The solution to sub-optimal brain signal targets is to include the mind as a whole. Targets based on the mind overall will yield not only more optimal brain signal classification. More importantly, they will allow the brain signal to be more consciously and skillfully used: by BCI designer, tester, teacher, and user alike. To do this, see the mind for what it is, define it accurately, and include all of it in relation to the brain signal.
Given target components of mind are reflected by corresponding brain signal targets, the question is how to optimally define the mind. This requires taking a step back to look at the mind: what is it, and how does it connects to the brain?
First, the current understanding of the mind, i.e. mental processes, is fundamentally correct in one important way. That is, it IS closely connected to the brain. However the nature of this connection is shaky and full of variation. There are a variety of conceptual frameworks ranging from full-blown materialism (the mind doesn’t exist — its all brain!) to hybrid understandings of mind and brain, to (rarely) frameworks that place the mind in a role equal to that of the brain. Among these conceptual frameworks there is no consensus. Yet, there is only one correct answer!
My view, one only occasionally expressed in the literature, is that mind and brain are 2 sides of the same coin. We all experience something throughout the day — both conscious experience and unconscious processes and content. From a subjective point of view, one continually experiences perception, creates recognition and meaning, feels emotion and somatosensation, forms goals, imagines and intends, and so on. This occurs every moment of the day.
The mind exist. Yet, it’s contents are located within the physical brain. Therefore, from an objective (brain) view, any mental phenomenon would have to be mirrored by some aspect of the brain.
Moreover, if the mind is expressed or manifest inside the brain, it would have a neural correlate. For example, consider an intention — the intention to “move my hand left forward.” This has to have a brain signal that corresponds to, or represents, it. Otherwise, that particular intention could not cause that movement to occur, via the motor cortex and spine. An intention creates movement from within the brain. If an intention were manifest anywhere else but besides the brain (basal ganglia & motor cortex) it would not be able to generate that particular movement.
The problem conceptually is that brain and mind are muddled together. Brain is seen as “the mind.” This makes sense in that, from a third person perspective, the mind is nowhere to be found. It can only be inferred. There IS no perception, thought, feeling, intention or any other aspect of mind to be seen within the neurons and neural substrate.
Because of this dominance of the third-person objective view, and because the mind lacks a neural correlate, it is seen alternately as a non-entity, a nuisance, a fact of the brain, and something to be ignored when possible. To understand the brain let’s ignore the mind and look to the brain; hopefully at some point its relationship to the mind will become clear.
What if the subjective mind, and brain, ARE two sides of the same coin? What would this have to do with brain signal “targets”? In principle, it means any aspect of the mind is — simultaneously — an aspect of brain activity. A given intention = a given brain signal. Thus when a BCI user is instructed to “intend to move your left arm forward,” she simultaneously activates a corresponding aspect of her brain.
Regarding the mind’s neural correlate, functional neural networks — regions of large scale neural activity acting in synchrony and coordination — play a dominant role in real-time brain function. Therefore it is safe to assume FNNs and the mind are closely connected, if not one and the same phenomenon.
As a particular intention/FNN is repeated over time, it would form a group or range of similar intentions/FNN activity. A “move my left hand” intention would come to represent a state of mind “signature” — and a corresponding FNN range signature.
Mind/FNN signatures are “targets” for the BCI user to try to “hit” with each intended movement. If mind is filled with that intention (and no other) the FNN activity will reflect this strong, low-noise FNN activity, and corresponding brain signal.
In short, the mind (ex: a movement intention & accompanying perception and emotion) = a set of (active) functional neural network ranges = a corresponding set of brain signal ranges = a potential BCI brain signal target.
To the extent the user’s state of mind is recognized as the optimal target, and defined accurately, brain signal “targets” for the BCI designer, tester, trainer, and user to shoot for will be more useful and accurate as well. The good news is, mind/brain signal targets can indeed be defined, with accuracy and precision.