Brain Signal Classification and the Mind

Accurately connecting cognition and the mind to brain activity — via brain signal neuroimaging, labeling or classification — is fraught with a number of very formidable challenges. Many of these relate to a lack of understanding the human mind, and being unable to categorize it accurately (Poldrack & Yarkoni, 2016).

That problems defining the mind creates signal classification problems makes perfect sense, given the brain signal is known to be closely connected to the mind. A subject’s state of mind — perception, cognition, emotion, motivation, intention etc. — as it changes (or not), will in turn affect their brain signal. Given the close mind/brain signal relationship, I argue the mind holds the key to more accurate, reliable and robust brain signal classification.

After all, a subject’s mind — perception, cognition, emotion, motivation, reward, fatigue, executive control, goals, attention, intention etc. — is the driver of their brain signal. Or at least, his or her brain signal corresponds to their mind. Therefore, if one could better-define the subject’s mind, their brain signal would be better-defined as well.

If a viable path toward better brain signal classification is clear, why isn’t it being acted on? The main problem is the mind – consciousness and unconscious mental processes — remains ill-defined, as does its connection to the brain (Varoquaux & Poldrack, 2018). Due to lack of theoretical understanding, this potential use of the mind remains underrealized.

But what if a method of defining the mind and mapping it to the brain, with much greater accuracy, were to exist? If brain signal essentially mirrors mind, defining the latter would dramatically enhance defining the former. Better mind — and brain activity — definitions are the basis of better brain signal labeling and classification.

One area in which improved mind/brain signal definitions, based on the mind not simply the task, could be applied is brain computer interface (BCI). A main component of BCI technology is brain signal classification. To operate an external device via brain signals, the signals need to be labeled or classified accurately, in order to activate a corresponding (user-desired) external action.

However, BCI technology is known to have problems with reliable and robust performance, especially outside the lab. Most BCI systems cannot be used independently for long periods of time (Chavarriaga et. al., 2016). One reason for this is mind and brain signal varies greatly: in accordance with the variation of real-world conditions (environment, situation — work, home, social — activity, and task). In addition, changes to the user’s internal mental state (emotion, motivation, goals, intention, attention, fatigue, frustration…) will affect the ongoing signal.

This variation in external and internal conditions is obviously a big problem in brain signal classification. But it could also be part of the solution. The first step is to recognize, understand, and define this variation — so one can then account for it (as noise) or take advantage of it (as part of the signal).

For example, consider signal classification within the context of a specific BCI and motor neuroprosthetic project. Imagine a research group wanted to classify a brain signal of a user’s intention to “reach my right arm forward” (to grasp a cup). To begin, there is no consensus on what aspects of mind are best to try to classify. Should it be the intention “(me) reach forward?” or “my arm reach forward?” Should associated motor imagery (of body, arm, hand) be the main target of classification? Should a prediction (I will soon reach forward?) be included? What about the accompanying perceptual experience (visual and somatosensory) and recognition — of “my arm,” its motion, what it is reaching toward, and why?

Or maybe, given this complexity, the mind should be ignored in favor of task. Maybe task not mind should be the classifier of the brain signal. In other words, maybe the best thing to do is to ignore the mind altogether and classify the signal based purely on the type of task being performed (task-based signal classification).

The main problem with task-based neuroimaging, and classification, however, is that the mind is shifted to the background. In fact, the mind is much more closely connected to the brain signal than the task in which the mind is (primarily) engaged. For example, a user could (a) imagine/intend to reach an artificial arm forward either with great calmness and confidence, or (b) with great frustration and anxiety. A and b might be the exact same task, performed under very similar conditions; yet are very different in terms of state of mind — and brain signal.

Seeing the brain signal as a reflection of the user’s overall mind (not just task), and defining the mind accurately, are two overlooked yet crucial components of brain signal classification. The mind can be an excellent classifier. Each movement task type (within a context type) will have its own state of mind “signature.” This is because each mental state during a task/context type has its own unique range of components and associations: of perception, emotion, motivation, imagination, intention, etc.

Despite the current lack of accuracy in defining the mind and mapping it to the brain, as well as corresponding shortcomings in signal classification, there is great news! The mind CAN be defined accurately — given a correct mind/brain theoretical paradigm. An accurate method of mapping mind to brain will significantly enhance brain signal classification — given the brain signal directly reflects the subject’s state of mind.


Chavarriaga, R., Fried, O., Kleih, S., Lotte, F., Scherer, R. (2016). Heading for new shores! Overcoming pitfalls in bci design. Brain-Computer Interfaces, 4, 60.

Poldrack, R.A., Yarkoni, T. (2016). From brain maps to cognitive ontologies: informatics and the search for mental structure. Annual Review of Psychology, 67, 587.

Varoquaux, G., & Poldrack, R. A. (2018). Predictive models avoid excessive reductionism in cognitive neuroimaging. Current Opinion in Neurobiology, Elsevier55, 1.

Comments are closed.