Brain-computer interface paradigms and neural coding

Brain signal patterns generated in the central nervous system of brain-computer interface (BCI) users are closely related to BCI paradigms and neural coding. In BCI systems, BCI paradigms and neural coding are critical elements for BCI research. However, so far there have been few references that clearly and systematically elaborated on the definition and design principles of the BCI paradigm as well as the definition and modeling principles of BCI neural coding. Therefore, these contents are expounded and the existing main BCI paradigms and neural coding are introduced in the review. Finally, the challenges and future research directions of BCI paradigm and neural coding were discussed, including user-centered design and evaluation for BCI paradigms and neural coding, revolutionizing the traditional BCI paradigms, breaking through the existing techniques for collecting brain signals and combining BCI technology with advanced AI technology to improve brain signal decoding performance. It is expected that the review will inspire innovative research and development of the BCI paradigm and neural coding.


Introduction
Brain-computer interface (BCI) is a revolutionizing human-computer interaction (Graimann et al., 2013), which directly bypasses peripheral nerves and muscles to establish a new communication and control channel between the brain and external devices (Wolpaw and Wolpaw, 2012).It has the potential to monitor, replace, improve/recover, enhance, and supplement damaged or impaired inputs or outputs of the central nervous system (CNS) (Ramsey and Millán José del, 2020).
In the BCI system, brain signal patterns generated in the CNS of the BCI user are closely related to BCI paradigms and neural coding (Allison et al., 2012), which are the foundation for decoding correctly the user's intent.Therefore, BCI paradigms and neural coding are critical in BCI research.To date, there have been many references on brain signal processing and classification algorithms in BCI systems.For instance, Lotte et al. (2007Lotte et al. ( , 2018) ) provided a comprehensive overview of the modern classification algorithms used in electroencephalogram (EEG) -based BCIs, and Bashashati et al. (2007) provided Tai et al. 10.3389/fnins.2023.1345961 a comprehensive review of the signal processing techniques in BCI systems.However, there have been few references focusing on the definition and design principles of the BCI paradigm as well as the definition and modeling principles of BCI neural coding.For example, Abiri et al. (2019) reviewed EEG-based BCI paradigms, and Xu et al. (2021) reviewed the EEG-based BCI brain coding and decoding mechanisms.In addition to EEGbased BCI paradigms and neural coding, there are also other BCI paradigms and neural coding based on brain imaging techniques, such as intracortical local field potentials (LFP) (Hochberg et al., 2012;Willett et al., 2021Willett et al., , 2023)), electroencephalogram (ECoG) (Luo et al., 2022;Branco et al., 2023;Metzger et al., 2023), functional near-infrared spectroscopy (fNIRS) (Abdalmalak et al., 2021;Paulmurugan et al., 2021;Eastmond et al., 2022), functional magnetic resonance imaging (fMRI) (Naselaris et al., 2011;Du et al., 2019), magnetoencephalography (MEG) (Xu et al., 2022;Bu et al., 2023), and hybrid brain-computer interface (hBCI) (Choi et al., 2017;Mussi and Adams, 2022).Therefore, we systematically elaborated on the definition and design principles of the BCI paradigm as well as the definition and modeling principles of BCI neural coding and introduced the existing main BCI paradigms and neural coding.Finally, the challenges and future research directions of the BCI paradigm and neural coding were discussed.It is expected that the review will inspire innovative research and development of the BCI paradigm and neural coding.
2 Definition and design principles of the BCI paradigm

Definition of BCI paradigm
Brain-computer interface paradigm is a set of specific mental tasks or external stimuli carefully selected/designed by the BCI developer to represent the user's intentions under specific brain imaging techniques.The purpose of the BCI paradigm is to "write" the user's intentions into brain signals, which represent or code the user's intentions.It is expected that the brain imaging technology used can detect the neural coding of the user's intentions, laying the foundation for subsequent "reading" or decoding of the user's intentions (Chavarriaga et al., 2017;Wolpaw et al., 2020).It is worth noting that it is difficult for BCI to decode the arbitrary or random mental activity of the user, as well as the arbitrary or random external stimulus received by the user.
Specific mental tasks are implicit mental activities, such as motor imagery (MI), visual imagery, speech imagery, mental arithmetic, and reasoning; specific external stimuli are explicit attentional tasks, such as visual, auditory, and tactile stimuli.Specific features of brain signals are induced by mental tasks or external stimuli, and identify specific mental tasks and specific stimuli, which provide the basis for subsequent BCI decoding.Mental tasks or external stimuli correspond to specific brain functions and brain activities, which are closely related to specific brain regions and brain networks/brain circuits.Figure 1 illustrates the relationship between the BCI paradigm, specific brain functions, and structures.Special attention should be paid to the fact that BCI paradigms are usually discussed in the context of specific brain imaging techniques, which means that BCI paradigms are closely related to specific brain imaging techniques.

Principles for designing BCI paradigm
There have been several BCI paradigms so far, such as MI, steady-state visual evoked potential (SSVEP), and P300 paradigms.These paradigms have their advantages and disadvantages, and many researchers are still improving these paradigms.The innovative design of BCI paradigms is one of the critical contents of BCI research.To translate the designed BCI paradigms into practical applications, the principles for designing the BCI paradigm are based on user-centered design (Kübler et al., 2014;Chavarriaga et al., 2017;Lyu et al., 2023) and human factors engineering of BCI (Allison et al., 2012;Kübler et al., 2013Kübler et al., , 2020;;Liberati et al., 2015;Martin et al., 2018;Kübler, 2020;Ramsey and Millán José del, 2020;Wolpaw et al., 2020;Branco et al., 2021) to evaluate it.The principles for designing the BCI paradigm are proposed in Table 1.

CNS signals evoked by BCI paradigm specific tasks should have good separability
Brain-computer interface paradigms require users to perform specific mental tasks or receive specific external stimuli.Brain signal features evoked by the user performing the designed BCI paradigm task are significantly differentiable for different mental tasks or external stimuli under specific brain imaging techniques, or the relevant CNS activity better coded the mental tasks or external stimuli designed by the BCI paradigm.Better separability is the basis for subsequently achieving higher BCI decoding accuracy.It is worth noting that specific brain imaging techniques need to be considered in the innovative design of BCI paradigms.
When screening various mental tasks and external stimuli for combination, the classification performance of various mental tasks combinations or external stimuli combinations (Jang et al., 2022) needs to be evaluated to determine the most suitable mental tasks combinations or external stimuli combinations for customized BCI.

BCI paradigm tasks are easy for users to perform
Some mental tasks are easy to perform, while others are not, usually choosing tasks that users are proficient in and natural in their daily life and work.Mental tasks are designed to be as simple as possible, suitable for users, approved, and even enjoyed by users.Easy to perform BCI paradigm tasks can increase user acceptance of BCI technology and promote its translation into practical applications.

BCI paradigm tasks are safe for the user
The brain imaging technology involved in the BCI paradigm is required to be safe for users and not harmful to their physical and mental health (Liberati et al., 2015).In addition, external stimuli have a lower risk of causing brain diseases in users, and mental tasks and external stimuli are less likely to cause excessive fatigue in users, to reduce mental load.The overall user satisfaction of BCI paradigm tasks is high level BCI paradigm tasks include specific mental tasks and/or external stimuli.

BCI paradigm tasks bring the user a good experience and comfort level
The user experience and comfort of the BCI paradigm are related to the comfort of the sensors used to collect brain signals, as well as the experience and comfort of mental tasks or external stimuli.They are also related to the decoding performance (stability, accuracy, and speed of decoding) under the BCI paradigm, which affects the user's acceptance of BCI.The BCI paradigm is required to have a high user rating for experience and comfort, which can be evaluated by an experience and comfort questionnaire.At present, the user experience and comfort of the existing BCI paradigm are not high, and the acceptance of BCI by users is not high.

Tasks specified by the BCI paradigm are consistent with tasks controlled by BCI
Brain-computer interface paradigm tasks are designed to avoid non-transparent mappings, which are inconsistencies between mental tasks and control commands, such as the use of left-handed MI to imagine the corresponding commands to control the robot's movement to the right.Non-transparent mappings may lead to changes in autonomous intentions, which may affect the user's performance during brain-computer interaction.For this reason, when designing a BCI paradigm task, it is important to consistent the mental task with the task controlled by the BCI.

BCI paradigm tasks are designed to the needs of specific user
During the screening of BCI paradigm tasks, the combination of mental tasks or external stimuli should be designed according to the specific needs of the application, the more mental tasks or external stimuli are not the better.It is necessary to just simply fulfill its requirements.In the case of rehabilitation training for movement disorders, it is appropriate to choose the MI of the corresponding limbs as the paradigm, but if it is to realize the simple communication of "YES" or "NO, " it is not appropriate to use the left or right limb MI as the mental task.

The overall user satisfaction of BCI paradigm tasks is high level
User satisfaction with BCI paradigm tasks is related to several factors that include the above-mentioned BCI paradigms should have good separability, be easy for the user to perform, be safe for the user, bring the user a good experience and comfort level, consistent with tasks controlled by BCI, and fit in the application needs of a specific user, which need to be considered and evaluated comprehensively to design a user-friendly BCI paradigm task.
The BCI paradigm task has a significant impact on whether potential BCI users accept and enjoy using the BCI system, and for this reason, the mental tasks that drive BCI need to be designed and optimized according to the user's ability characteristics.

Definition and mechanisms models of BCI neural coding
Based on the definition and design principles of the BCI paradigm elaborated above, the following sections elaborate on the definition and design principles of BCI neural coding, the relationship between BCI paradigm, BCI neural coding, and BCI neural decoding, and the relationship between BCI neural coding, brain neural coding, and computer information coding.The schematic diagram for the relationship between BCI paradigm, BCI neural coding, and BCI neural decoding.

Definition of BCI neural coding
It is worth noting that BCI neural coding is closely related to the chosen level and parameter settings of brain imaging technology, such that the level of technology used to record the ECoG signals and the hardware settings (including the sampling frequency and hardware filters) affect the types of features that can be extracted and analyses that can be performed (Hill et al., 2012;Shirhatti et al., 2016).

Principles for modeling BCI neural coding
Modeling BCI neural coding requires consideration of specific BCI paradigms, mechanisms of brain neural coding, neural signal features collected by different brain imaging techniques, and efficient decoding of the user's intentions.
(1) Modeling neural coding under a specific BCI paradigm.
Different BCI paradigms, such as SSVEP-BCI, P300-BCI, MI-BCI, and other paradigms have different neural coding models.(2) Modeling BCI neural coding based on the mechanisms of brain neural coding.The mechanisms of brain neural coding characterize the hypothesized relationships between external stimuli or mental tasks and the response of specific neuronal populations, as well as the relationships between the electrical activity of neurons within neuronal populations (Johnson, 2000;Brown et al., 2004).An implantable BCI neural coding model can be modeled based on these relationships within the mechanisms of brain neural coding.4) BCI neural coding model being beneficial for subsequent neural decoding.The purpose of modeling BCI neural coding is to efficiently decode the user's intentions.

Relationship between BCI paradigm, BCI neural coding, and BCI neural decoding
Brain-computer interface paradigm tasks are usually designed first, then the neural coding under the BCI paradigm is revealed, followed by the extraction of brain signal features from neural coding laws, and finally the neural decoding.The BCI paradigm with BCI neural coding is the basis or premise of BCI decoding.It should be emphasized that in BCI systems, there is no corresponding neural coding without the BCI paradigm, no neural decoding without BCI neural coding, or no highperformance neural decoding performance without good BCI paradigms and neural coding.Figure 3 illustrates the relationship between BCI paradigm, BCI neural coding, and BCI neural decoding.

Relationship between BCI neural coding, brain neural coding, and computer information coding
Brain neural coding is the basis of BCI neural coding, which characterizes the hypothesized relationships between external stimuli / mental tasks and the responses of specific neurons Tai et al. 10.3389/fnins.2023.1345961or populations of neurons (Johnson, 2000;Brown et al., 2004).According to the theory that sensory and other information is represented in the brain by a network of neurons, it is believed that neurons can encode both digital and analog information (Thorpe, 1990).Computer information coding is the process of converting information from one form or format to another and can be used to represent the relationship of things, which can be represented by numbers, letters, special symbols, or combinations of them, to convert data into codes or coded characters that can be translated into the original data form.Figure 4 illustrates the relationship between BCI neural coding, brain neural coding, and computer information coding.
Inspired by the brain neural coding model in Figure 4, BCI neural coding models can be proposed, as shown in Table 2, and will be reviewed in sections 3.5-3.11.These BCI neural codes will eventually be transformed into computer information coding to be processed by computers.

Frequency/rate coding for BCI
Frequency/rate coding for intracortical BCI can be inspired by traditional models of neuronal firing rate coding.In most sensory systems, the firing rate increases, generally nonlinearly, with increasing stimulus intensity (Kandel et al., 2000).Since the sequence of action potentials generated by a given stimulus varies from trial to trial, neuronal responses are typically treated statistically or probabilistically.The spike count rate for a single trial can be calculated from Equation (1) (Gerstner and Kistler, 2002).
The schematic diagram for the relationship between BCI neural coding, brain neural coding, and computer information coding.The D trial is the duration of a trial, with typical values of 100 ms or 500 ms (Gerstner and Kistler, 2002), and N spikes is the number of spikes occurring within the D trial .
The time-dependent rate of issuance in a timedependent stimulus can be calculated from Equation (2) (Gerstner and Kistler, 2002).
where N K is the number spike appearing on all repeated trials from time t and t+ t, K is the number of retrials, t is the start time relative to the stimulus sequence, and t is the time interval, usually in the range of a millisecond or a few milliseconds.The FR td applies to both resting and time-dependent stimuli, but it is unlikely to be the coding scheme used by neurons in the brain (Gerstner and Kistler, 2002).

Time coding for BCI
Intracortical BCI time coding can be inspired by time coding models in neural coding (Butts et al., 2007;Gollisch and Meister, 2008).The time resolution of neural coding is on the millisecond time scale, suggesting that precise spike timing is an important element in neural coding (Theunissen and Miller, 1995).For example, the ability of many organisms to discriminate between stimuli (such as visual stimuli, auditory stimuli, tactile stimuli, gustatory stimuli, and olfactory stimuli) on a millisecond time scale suggests that time coding is also a model that functions in sensory systems (Gollisch and Meister, 2008).
The spiking activity features that can be extracted by time coding are time-to-first-spike after the stimulus onset (Victor, 2005;Carleton et al., 2010), phase-of-firing for background oscillations, features based on the second and higher statistical moments of the ISI probability distribution (Kostal et al., 2007), spike randomness, or precisely timed groups of spikes (Thorpe, 1990;Jolivet et al., 2006;Chen et al., 2009).
Information on the time structure of stimulus-evoked spike trains or dispensing rates is determined by the dynamics of the stimulus, the properties of the stimulus, and the nature of the neural coding process (Montemurro et al., 2008).In Equation (2) the code is rate coding if FR td changes slowly with time and time coding if it changes rapidly.

Phase-of-firing coding for BCI
Intracortical BCI phase coding can be inspired by phase-offiring coding models in neural coding (Havenith et al., 2011).Phase-of-firing coding is a neural coding scheme that this type of code takes into account the spike count coding and a time label for each spike according to a time reference based on the phase of local ongoing oscillations at low (Montemurro et al., 2008) or high frequencies (Fries et al., 2007;Havenith et al., 2011).The feature of this code is that neurons adhere to a preferred order of spiking between a group of sensory neurons, resulting in a firing sequence (Havenith et al., 2011).For example, each neuron in the visual cortex has its own preferred/preferred relative firing time during the gamma oscillation cycle.

Intracortical neuronal population coding for BCI
Intracortical BCI neuron population coding can be inspired by the joint activity of multiple neurons to characterize stimuli or mental activity.In BCI intracortical neuronal population coding, each neuron has a distribution of responses over some set of inputs, and the responses of many neurons may be combined to determine corresponding stimulus or mental activity about the inputs.Neuronal population coding captures the essential features of neural coding (Wu et al., 2002).
Experimental studies have revealed that this coding paradigm is widely used in the sensor and motor areas of the brain.for example, Neurons are modulated to the moving/motor direction in the visual area medial time (Maunsell and Van Essen, 1983).However, Individual neurons fire fastest or slower in one direction depending on the distance of the target from the neuron's "preferred" direction, the vectors of all neurons, and the coding of motion signals.
Typical neuronal population coding involves neurons with Gaussian modulation curves whose mean value varies linearly with stimulus intensity.Positional coding in a population of neurons can be used to encode continuous variables such as joint position, eye position, color, or sound frequency.Rate coding of the entire population ensures higher fidelity and accuracy than rate coding of individual neurons (Mathis et al., 2012).

Correlation coding for BCI
Intracortical BCI correlation coding can be inspired by the correlation coding model of neuronal firing (Panzeri et al., 1999).This model suggests that correlations between action potentials or "spikes" within a spike sequence may carry additional information beyond the simple timing of the spikes (Ahmadi et al., 2021;Zhang and Constandinou, 2023).Correlations may also carry information that is not present in the average firing rate of the two pairs of neurons (Decharms and Merzenich, 1996).

Sparse coding for BCI
Intracortical biometric sparse coding can be inspired by the sparse coding model of neural coding, in which each activity of a stimulus or mental activity is encoded by the strong activation of a relatively small set of neurons, which do not use the full set of available neurons, but rather a subset of them.In the BCI decoding stage, algorithms for sparse signal representation and processing can be used.
The fact that only a few neurons in a population of neurons respond to a given stimulus and that each neuron responds to only a few stimuli out of all possible stimuli, may be a biological selective response.Theoretical studies of sparse distributed memory have shown that sparse coding increases the capacity of associative memory by reducing the overlap between representations (Kanerva, 1988).Experimentally, sparse representations of sensory information have been observed in several systems, including visual (Vision, 2000), auditory (Hromádka et al., 2008), tactile (Crochet et al., 2011), and olfactory (Ito et al., 2008).
Sparsity may be focused on time sparsity, for example, the features extracted from all frequency bands during motion imagery are not all well separable (Olshausen and Field, 1996); It may also focus on spatial sparsity, as in the sparsity of neuronal populations activated, or the sparsity of brain regions/brain networks activated, such that it is the sensory-motor-related brain regions or brain networks that are predominantly activated during MI.
Most sparse coding models are based on linear generative models (Rehn and Sommer, 2007), as shown in Equation (3) (Lee et al., 2006).
vectors and s j is the weight of each input vector b j are combined linearly by Eq. ( 3) to approximate the inputs.Other models are based on match tracking and dictionary learning (Pati et al., 1993;Davis et al., 1994;Needell and Tropp, 2009).

Hybrid coding for BCI
To better or more fully characterize the relationship between the stimulus / mental activity and the neural response, a combination of several neural coding methods can be considered, which is a hybrid neural coding approach.To more accurately decode the stimulus or mental activity from the neural response, intracortical BCI hybrid coding schemes can combine two or more of the above models (Choi et al., 2017;Mussi and Adams, 2022).For example, global features such as pitch or formant transition profiles can be represented by both rate coding and place coding (Miller and Sachs, 1983).

Existing main BCI paradigms and neural coding
Brain-computer interface paradigms and neural coding are directly related to specific brain imaging techniques.The existing main BCI paradigms and neural coding models involve brain imaging techniques including the acquisition of intracortical LFP, ECoG, fNIRS, fMRI, MEG, EEG, and hybrid brain imaging techniques, as shown in Table 3.

Intracortical LFP-BCI paradigms and neural coding
The wrapping of intracortical electrodes has a significant effect on the collection of individual neuronal Spikes but not on LFP (Milekovic et al., 2018(Milekovic et al., , 2019)).LFP is expected to be used for longterm cortical control of an artificial device and is a low-frequency signal (<250 Hz) consisting of the sum of all the electrical activity in the region adjacent to the tip of the electrodes implanted in the cortex.To date, most LFP-based BCI research has been carried out based on typical center-out arm movements (Rickert et al., 2005;Heldman et al., 2006;Wang, 2006), and some studies have employed point-to-point motor tasks (Ahmadi et al., 2021;Zhang and Constandinou, 2023).It has been shown that the M1 tune encodes movement direction (Georgopoulos et al., 1986) and velocity.There is a modest decrease in β-LFP amplitude beforehand movement and a large increase in HF-LFP spectral amplitude during hand movement.The time-domain characterization of LFP can be used to control computer cursors (Kennedy et al., 2004) and to encode motion parameters (Rickert et al., 2005;Ahmadi et al., 2021).Examples of the main existing LFP-BCI paradigms with neural coding studies are shown in Supplementary Table 1.

ECoG-BCI paradigms and neural coding
Electroencephalogram is the recording of the overall activity of a large population of neurons in a localized area by electrodes placed on the supernatural or subdural cortex, with time and spatial resolution of a few milliseconds and millimeters (ECoG is superior to MEG and EEG) (Katzner et al., 2009;Dubey and Ray, 2019), which is less affected by muscle activity and ocular artifacts (Ball et al., 2009), The ECoG has an excellent noise ratio.These advantages favor the coding of stimuli or mental tasks so that potential brain signal features are found to be well discriminated, Therefore, the brain imaging technique of ECoG is better suited for BCI.

fMRI-BCI paradigms and neural coding
Functional magnetic resonance imaging (Ogawa et al., 1990) has high spatial resolution (Logothetis et al., 2001), better robustness and user-friendliness, and individualized flexibility.The ability of fMRI to measure deep brain region structure and activity, map functional connectivity networks, and allow the use of amygdala and ventral striatum with BCI neurofeedback for user (Mehler et al., 2018) has become a core technique for mapping neuroplasticity (Seitz, 2010).
Spatial localization of brain functions using the fMRI-BCI space coding model to produce spatially distinct patterns of brain activation by engaging different combinations of brain regions during the time that subjects are receiving external stimuli or intentionally performing different mental activities (Yoo et al., 2004;Boly et al., 2007;Monti et al., 2010;Senden et al., 2019).Models of fMRI-BCI time coding reliably detect the onset, offset, and duration of single-trial fMRI responses evoked by various mental activities, which is the assignment of specific coding intervals for specific intentions (Monti et al., 2010).Models of fMRI-BCI amplitude coding using different fMRI signal levels within specific brain regions to encode different movement intentions (Yoo et al., 2001).The combination of fMRI-BCI hybrid coding models using the above signal features (spatial, temporal, and amplitude) can increase the degree of freedom to encode different units of information or increase the distinguishability of the evoked brain activation patterns, thus maximizing the decoding accuracy (Sorger et al., 2012).Examples of the main existing fMRI -BCI paradigms with neural coding studies are shown in Supplementary Table 3.

fNIRS-BCI paradigms and neural coding
Functional near-infrared spectroscopy measures the hemodynamic response of brain tissue during resting state (Abdalmalak et al., 2021;Paulmurugan et al., 2021;Eastmond et al., 2022), external stimulus, and mental activity, including changes in oxy-hemoglobin (HbO) and deoxyhemoglobin (HbR) concentration, with the main advantage of good portability, tolerating a certain degree of head movement of the subject, and a favorable ecological effect.fNIRS-BCI can also be used to facilitate the rehabilitation of patients with motor dysfunction and/or cognitive dysfunction, such as those suffering from stroke and spinal cord injury.
The fNIRS-BCI time coding model extracts time-domain features of hemodynamic responses associated with a specific event (such as external stimulus or mental task), for example, the mean, peak, and variance of HbO and HbR (Holper and Wolf, 2011;Hwang et al., 2014;Naseer et al., 2014;Hong et al., 2015).Models of fNIRS -BCI amplitude coding using different fNIRS signal levels within specific brain regions to encode different movement intentions (Coyle et al., 2007;Kaiser et al., 2014).Examples of the main existing fNIRS -BCI paradigms with neural coding studies are shown in Supplementary Table 4.

MEG-BCI paradigms and neural coding
Magnetoencephalography is a non-invasive neuroimaging technique for detecting weak magnetic field changes generated by the electrical activity of central neurons (Xu et al., 2022;Bu et al., 2023).The technique has high time (less than 1 ms) and spatial [2-5 µm (Cetin and Temurtas, 2021)] resolution and low sensitivity to artifacts generated by muscle activity, but its comfort, aesthetics, and ease of use leave much to be desired.
The MEG-BCI time coding model characterizes mental tasks or external stimuli by information such as the peaks of the time-domain waveforms of the MEG signals as well as the time points.The frequency coding model characterizes a specific event (mental activity or external stimulus) by the MEG power spectral features (Mellinger et al., 2007;Chen and Bai, 2009;Halme and Parkkonen, 2016).MEG signals are complex non-linear and nonstationary signals, which the single time or frequency coding model will lose some feature information, and a time-frequency coding model can be used to obtain the relationship between signal frequency over time (Chholak et al., 2019).MEG-BCI space coding models can be used to downsize the data using spatial filtering methods to differentiate between mental activities or external stimuli (Rathee et al., 2021).Examples of the main existing MEG-BCI paradigms with neural coding studies are shown in Supplementary Table 5.

EEG-BCI paradigms and neural coding 4.6.1 MI paradigms and neural coding
Mental tasks involved in MI paradigms include slow, non-fine, non-dexterous MI, fast, fine, and highly dexterous MI, MI involving a unilateral limb, coordinated MI involving multiple limbs, a single or repetitive MI, as well as kinematics or kinetics parameters imagery.The neural coding of MI paradigms can be encoded using (1) time-domain features such as movement-related potential (MRP) or movement-related cortical potential (MRCP) (movement preparation potentials, movement monitoring potentials, and endof-movement rebound potentials); (2) frequency-domain features coding, such as neural oscillatory power change features of µ/β/γ and other rhythms, which are commonly used event-relateddesynchronization/synchronization (ERD/ERS); and (3) spacedomain features coding, such as the primary motor area of the hemispheres, the auxiliary motor area, and the premotor area.In addition, the neural coding of some MI paradigms has yet to be studied in depth.

Slow, non-fine, non-dexterous MI
Slow, non-fine, and non-dexterous movements usually involve gross limb movements that do not require rapidity, fine coordination, and a high degree of dexterity.
Slow movements are slow and do not require rapid responses or high rates of execution, which include slow walking, strolling, and soothing stretches are all slow movements (Pfurtscheller et al., 2003b;Müller-Putz et al., 2007).Non-fine movements do not require a high degree of fine coordination or precise control of fine muscle groups.Comparatively, they favor holistic and basic movements.Such as simple hand lifting, striding, and simple dance movements (Pfurtscheller et al., 1997;Ramoser et al., 2000;Kaiser et al., 2014).Non-dexterous movements do not require high skills or complex combinations of movements.They focus more on the simplicity and ease of realization of the movements.For example, balancing simple objects, simple stretching, and bending movements (Neuper et al., 2009).

Fast, fine, and highly dexterous movement imagery
Fast, fine, and highly dexterous movements usually involve the movement of fine limbs, the execution of which requires fast, accurate movements with a high degree of skill and coordination These types of movements often require long periods of training and practice to achieve a high level of skill.
Rapid movements are executed at a high speed.It requires rapid reaction and movement execution.The users can react quickly in a short period and complete the movement at a high rate.Fine movements require a high degree of precision and care.The users need to control the movement accurately, including the coordination of small muscle groups and precise handling of details.Highly dexterous movements demonstrate exceptional skill and flexibility.The performer can perform the movement with grace and agility.Implantable acquisition of brain signals with high space resolution is usually used to encode and decode such movements, and scalp EEG makes it difficult to encode and decode such MI with stability and high precision.

MI involving the unilateral limb
In daily life, some movements involve only the unilateral limb, such as tapping movements of the right or left index finger, internal or external rotation of the wrist, flexion or extension of the wrist, clenching of the fist, pinching of the thumb against the other fingers, and extension of the arm.These movement exercises favor unilateral limb strength, balance, and coordination, and help improve symmetry and motor control (Pfurtscheller et al., 1997;Ramoser et al., 2000;Neuper et al., 2009;Hashimoto and Ushiba, 2013).The difficulty of recognizing different imagined movements in a single limb is greater compared to the recognition of imagined movements in different limbs.For example, recognizing various imagined movements in the affected limb of hemiparetic patients poses a greater challenge.

Coordinated MI involving multiple limbs
In daily life, coordinated movement usually involves synergistic movements of multiple limbs to cooperate in time and space to effectively accomplish a desired task (Zhang et al., 2022).Examples include walking and threading a needle.These types of movements usually require some training making good coordination and overall control between limbs (Müller-Putz et al., 2007; Frontiers in Neuroscience 08 frontiersin.orgHashimoto and Ushiba, 2013).The neural and decoding of coordinated motor involving at least two limbs is to be studied in depth.

A single or repetitive MI
A single MI BCI paradigm is different from the repetitive or continuous MI BCI paradigm, and the BCI coding is different.The Synchronous MI-BCI requires subjects to perform imagery tasks according to a temporal sequence designed by the researcher, and it is typically used to build classification models.The asynchronous MI-BCI is much more challenging, where subjects' imaginative mental activity can be self-paced rather than performing the imaginative task according to a temporal sequence designed by the researcher.

Kinematic or kinetic parameters imagery
The kinematic parameters of the limb include the velocity of movement, trajectory of movement, and time of movement.The kinetic parameters of the limb include the driving force of movement and acceleration of movement.For example, the speed of the right index finger tap (such as slow, medium, and fast), reaching and grasping processes, space navigation, and grip size (Flint et al., 2022).Compared to kinematic parameter imagery, kinetic parameter imagery coding and decoding have been relatively less studied and more difficult.
4.6.2External stimulus paradigms and neural coding 4.6.2.1 P300-BCI paradigms and neural coding In the P300-BCI paradigm, the probability of a target/target stimulus (novel stimulus with small probability) is no more than 20%, and the probability of a non-target stimulus (standard stimulus with large probability) is no less than 80%.When a user is exposed to a target stimulus during 220-500 ms (latency) a positive peak of 5-10 microvolts is induced, most significant at the midline location (Pz, Cz, and Fz in the 10/20 international system).This component characterizes the target stimulus.The visual P300-BCI speller was first implemented by Farwell and Donchin (1988), and subsequently, there have been many variants of the P300-BCI paradigm, mainly differences in visual stimulus characterization and presentation.In addition to the visual P300-BCI paradigm, there are also auditory P300-BCI (Furdea et al., 2009;Klobassa et al., 2009) and tactile P300-BCI (Müller-Putz et al., 2006;Brouwer and Van Erp, 2010).
Although the P300-BCI can provide effective input of characters, the practicality still faces challenges.The system's online transmission rate is low, which makes it difficult to meet the current real-time needs.The inseparability of external stimuli is tied to attention such as vision and hearing, and offline training tends to be prolonged, which causes fatigue to the users.

SSVEP-BCI paradigms and neural coding
In the SSVEP-BCI paradigm, when a subject gazes at a visual stimulus of a certain flicker frequency [low band (<12 Hz), middle band (12-30 Hz), and high band (>30 Hz)], a stabilizing potential component of the same frequency as the stimulus frequency or its higher harmonic frequencies is induced.The SSVEP-BCI paradigm can be traced back to as early as a 1995 conference report (McMillan et al., 1995), but is not the paradigm that is now commonly used; the paradigm that is now commonly used comes from a 1999 conference report (Ming and Shangkai, 1999).Subsequently, the SSVEP-BCI paradigm has many innovative designs such as frequency modulation combined with phase and amplitude modulation (Chen et al., 2015;Zhang et al., 2021).
Although SSVEP-BCI has high performance (for example, significant features, stable amplitude, high interference immunity, high information transfer rate, less training, and large instruction set), it requires highly accurate eye control (Herrmann, 2001;Chang et al., 2014), and may lead to subject fatigue when using low blinking frequency (Molina and Mihajlovic, 2010;Müller et al., 2011;Volosyak et al., 2011) the interaction of SSVEP is unnatural, and the user's satisfaction remains to be further improved.SSVEP-BCIs must essentially have flickering external visual stimuli, and thus cannot be separated from visual attention.Examples of the main existing EEG-BCI paradigms with neural coding studies are shown in Supplementary Table 6.

hBCI paradigms and coding
A hybrid Brain-Computer Interface (hBCI) aims to improve the usability or efficacy of BCI systems hBCI consists of a mix of a BCI system (the main system) and an add-on system that assists the BCI (Li et al., 2019).which can be a nonexternal stimulus or non-psychological task-driven system, or an AI system (e.g., a deep learning-based machine vision or computer vision system) can be mixed to improve the accuracy of target recognition by the main system BCI and increase the number of brain-controlled/other commands.As shown in Figure 5.
As can be seen from Figure 5, BCI paradigms of the main system can be subsets of different external stimuli and mental tasks.For example, the P300 paradigm can be designed by visual, auditory, tactile, or olfactory stimuli according to the Oddball paradigm.The P300 paradigm, the SSVEP paradigm, and the kinesthetic imagery paradigm can be combined.BCI paradigms of the main system induce electromagnetic/metabolic activity signals related to brain activity, and multiple brain activity patterns in these brain signals can encode external stimuli and mental tasks.Non-external stimuli and mental tasks such as eye movement, limb movement, or heartbeat in the additional system can be characterized by other non-brain activity physiological signal patterns.The AI system in the addon system can enhance the level of intelligence of the main BCI system.

Challenges and future research directions for BCI paradigms and neural coding
So far, existing BCI paradigms and neural codes have limitations that hinder the translational application of BCI.For this reason, the innovative design and improvement of BCI paradigms and neural codes is one of the key tasks in the development of BCI systems.

User-centered design and evaluation for BCI paradigms and neural coding
The end user of BCI is the user, while the user itself is the source of the CNS signals that drive the BCI.The user is the most complex, active, and highly adaptive subsystem essential to the BCI system.Therefore, BCI systems are the most typical humanin-the-loop systems (the human brain is directly connected or coupled to the machine, a closed-loop system with direct brainmachine interaction), and the design and evaluation (usability and satisfaction) of BCI paradigms and coding need to be usercentered, taking into account BCI human factors engineering (Lyu et al., 2023).
Brain-computer interface paradigms and neural coding are closely related to the neural mechanisms of the user's mental activities/tasks (Liberati et al., 2015).The performance of a BCI system (such as effectiveness and efficiency) is closely related to the user's mental activity, such that the performance of a movementimagery BCI system is largely dependent on the user's effectiveness or ability to perform MI (Kübler et al., 2014;Martin et al., 2018).
It is worth noting that to evaluate the first principle of BCI paradigm design proposed in section 2.2, which states that CNS signals evoked by BCI paradigm specific tasks should have good separability, any innovatively designed BCI paradigm, and neural coding model typically requires offline data analysis and model establishment, and ultimately must be validated and evaluated by neural decoding in an online BCI system.

Revolutionizing the traditional BCI paradigms
The BCI paradigm, from a perspective of communication principles and technology, is a coding protocol in which brain intentions are encoded into signals generated by neural activity through specific external stimuli or mental tasks.So far, BCI has been developed for more than 50 years.However, current BCI paradigms are more limited, and the transformation faces great challenges, which need to significantly improve, we need to break through the traditional classical BCI paradigms (such as SSVEP-BCI, P300-BCI, and MI-BCI), and add new BCI paradigms that are more natural and more effective to interact with the user.In recent years, many important advances have been made in the innovation of BCI paradigms (Willett et al., 2021(Willett et al., , 2023;;Metzger et al., 2023).

Speech-BCI paradigm
Speech is the primary mode of human communication, and the speech BCI paradigm is one of the more natural BCI paradigms.Speech BCI has the potential to decode the neural activity triggered by attempted speech into text or sound, thus promising to restore rapid communication for paralyzed patients (Metzger et al., 2023;Willett et al., 2023).

Handwriting imagery-BCI paradigm for input text
To date, a major focus of BCI research has been the restoration of motor skills to gross limbs such as reaching and grasping or typing with computer cursor clicks.Willett et al. (2021) developed an intracortical brain-computer interface that decodes attempted handwriting actions via neural activity in the hand junction area of the motor cortex and uses a recursive neural network decoding method to translate neural activity in the motor cortex into text in real-time.The schematic for combining BCI technology with advanced AI technology.

Breaking through the existing techniques for collecting brain signals
paradigm and neural coding, which is related to the quality of the collected signals and the final BCI control effect.With the continuous development of micro-nano processing technology and electrode materials, electrodes for invasive BCI tend to be flexible, miniaturized, high-throughput, and integrated.Currently, the research and development of hydrogel EEG electrodes are more active (Xue et al., 2023), stereotactic EEG (sEEG) (Herff et al., 2020), and in-ear EEG electrodes (Wang et al., 2023) have also made positive progress.In addition, minimally invasive endovascular stent-electrode techniques (Oxley et al., 2016(Oxley et al., , 2021)), minimally invasive local-skull electrophysiological modification methods (Sun et al., 2022), and other schemes have been proposed to innovate brain signal acquisition.

Combining BCI technology with advanced AI technology to improve brain signal decoding performance
Currently, classical machine learning still has an advantage in BCI neural decoding, but deep learning also has the potential to enhance BCI decoding performance.Some studies have introduced suitable deep learning algorithms in decoding brain signals, and these studies show that BCI technology combined with advanced AI techniques is expected to significantly improve brain signal decoding performance (Willett et al., 2021(Willett et al., , 2023;;Metzger et al., 2023).Figure 6 illustrates the introduction of AI into BCI to improve the intelligence of BCI and facilitate the clinical translational application of BCI.

Conclusion
In the BCI technology system, BCI paradigms and neural coding are some of the key and important contents of BCI research and development.In the paper, the definition of BCI paradigms and seven design principles, as well as the definition and coding model of BCI neural coding, including BCI frequency/rate coding, time coding, phase coding, intracortical neuron population coding, correlation coding, sparse coding, and hybrid coding model are shown more systematically and clearly.The existing main BCI paradigms and neural coding are presented, including intracortical LFP-BCI, ECoG-BCI, fNIRS-BCI, fMRI-BCI, MEG-BCI, EEG-BCI, and hybrid BCI paradigms and neural coding.Finally, user-centered design and evaluation for BCI paradigms and neural coding, revolutionizing the traditional BCI paradigms, breaking through the existing techniques for collecting brain signals and combining BCI technology with advanced AI technology to improve brain signal decoding performance are discussed.It is expected that this paper will inspire the innovative research and development of BCI paradigms and neural coding.

FIGURE 1
FIGURE 1Schematic diagram for the relationship between the BCI paradigm, specific brain functions, and structures.

FIGURE 2
FIGURE 2Schematic diagram for the process of BCI neural coding.

FIGURE 3
FIGURE 3 (3) Considering the features from the time domain, frequency domain, and spatial domain of neural signals collected using different brain imaging techniques when modeling BCI neural coding.Given the different temporal and spatial resolutions of brain imaging techniques, such as EEG, fNIRS, fMRI, MEG, ECoG, intracortical LFP, or Spikes, the measured brain activities (electrical activities of central neurons or metabolic activities of brain tissue) are different.(

FIGURE 5
FIGURE 5 The performances of the BCI paradigm and neural coding are directly related to the level of brain signal collection technology, which requires a breaking through of brain signal collection technology.How brain signals are acquired is crucial for the BCI Frontiers in Neuroscience 10 frontiersin.org

TABLE 1
The principles for designing the BCI paradigm.

TABLE 2
The BCI neural coding model.

Table 2 .
power changes can sometimes improve decoding performance.Examples of the main existing ECoG-BCI paradigms with neural coding studies are shown in Supplementary