EEG and EMG dataset for the detection of errors introduced by an active orthosis device

This paper presents a dataset containing recordings of the electroencephalogram (EEG) and the electromyogram (EMG) from eight subjects who were assisted in moving their right arm by an active orthosis device. The supported movements were elbow joint movements, i.e., flexion and extension of the right arm. While the orthosis was actively moving the subject's arm, some errors were deliberately introduced for a short duration of time. During this time, the orthosis moved in the opposite direction. In this paper, we explain the experimental setup and present some behavioral analyses across all subjects. Additionally, we present an average event-related potential analysis for one subject to offer insights into the data quality and the EEG activity caused by the error introduction. The dataset described herein is openly accessible. The aim of this study was to provide a dataset to the research community, particularly for the development of new methods in the asynchronous detection of erroneous events from the EEG. We are especially interested in the tactile and haptic-mediated recognition of errors, which has not yet been sufficiently investigated in the literature. We hope that the detailed description of the orthosis and the experiment will enable its reproduction and facilitate a systematic investigation of the influencing factors in the detection of erroneous behavior of assistive systems by a large community.


INTRODUCTION
Exoskeletons and orthoses are frequently used to assist or enable human movement (see Kirchner & Bütefür (2022)).They are even able to augment classical therapy approaches such as mirror therapy (Kirchner et al. 2013).The electroencephalogram (EEG) can be used not only to infer the intention to move but also to trigger the assistance provided by an exoskeleton (Kirchner & Bütefür 2022).This has been shown to be very important for successful neuro-rehabilitation (Noda et al. 2012, Hortal et al. 2015).It can also be used to infer the subjective correctness of the behavior of a robot that the human observes or interacts with, as shown in several works such as by Kim et al. (2017Kim et al. ( , 2020)), Chavarriaga et al. (2014).EEG activity can also be used to enable the teaching of subjective preferences to a prosthesis, as proposed by Iturrate et al. (2015).Furthermore, misbehavior of an assistive device can be detected using the error-related potentials (ErrPs) that occur when the brain observes errors (van Schie et al. 2004).
Inferring errors from EEG is challenging because it requires asynchronous classification of relevant patterns in the EEG, which often leads to many false positives due to long interaction times with the system or long task times for the system (Omedes et al. 2015, Lopes-Dias et al. 2021a, Spüler & Niethammer 2015, Lopes-Dias et al. 2021b).While most studies focus on how the brain evaluates erroneous behavior using visual stimuli, according to our literature review, there is only one study that uses tactile stimuli to elicit an ErrP (Schiatti et al. 2018).Moreover, it has only been shown that rather complex errors in behavior elicit the ErrP or correlated changes in the frequency domain.It has not yet been studied how the brain responds to simpler errors that are quite obvious and may not require extensive evaluation to assess correctness.
Based on the available literature, we believe that there is a gap in the research on error-related activity.Modalities, other than the visual modality, should also be studied more closely during the interaction or as a source of feedback in terms of what activities they induce in the brain.It is also necessary to better understand what complexity in task, response, or interaction errors is required to elicit the ErrP.Furthermore, methods for continuous classification of error-related activity and methods that allow to distinguish between different, partially overlapping EEG activities, should be explored more intensively.In this regard, there is a particular lack of openly accessible data that would allow a larger research community to contribute.Furthermore, the use of robots limits the number of research groups that can conduct research on such problems.For this reason, we recorded a dataset of 8 subjects wearing an active orthosis device that introduces simple errors in its behavior.In addition, details about active elbow orthosis are provided, as it is a fairly simple robotic system that could allow other groups to replicate the work or extend it.Furthermore, the experimental procedure has also been described which would give relevant information about the error complexity, subject instructions, and whether or not subjects should respond explicitly to errors.
We hope that this first open-access dataset will enable broader research on how assistive technology can be improved by using EEG to provide more natural and individualized support for activities of daily living.Such support is very important for rehabilitation (Kornhaber et al. 2018).
The rest of the paper is structured as follows.Section 2 provides detailed information about the experimental design and methods used to record the dataset.It also describes the data format and the folder structure for a better understanding of the dataset.Furthermore, Section 3 presents a preliminary quality analysis of the recorded data in the form of response-time analysis and eventrelated potential analysis.Finally, in Section 4, we provide an overview of this experiment and discuss future possibilities.

METHODS AND EXPERIMENTAL DESIGN
This section provides information about the experimental design including details about subjects' informed consent, experimental setup & procedure, methods used for data acquisition, and the formatting of the recorded dataset.

Participants
Eight healthy subjects (four male and four female; average age 21.8±2.4years; right-handed; students) voluntarily participated in the study.Some days before the start of experiments, all subjects were invited to the lab for a basic introduction and preliminary testing.This included checking the fit of the used orthosis and measuring the head circumference for determining their EEG cap size.All the subjects gave their written informed consent and were told that they could stop the experiment at any time without consequences.The experiment lasted for 4.9 ± 0.6 hours on average and all the subjects received a monetary compensation of 10C per hour.

Experimental Setup and Procedure
An overview of all the protocols followed throughout the experiment is provided in this section.

Subject Preparation
Before the start of the experiments, the subjects were prepared with a 64-channel EEG system and an eight-channel EMG system (see Section 2.3.1 and 2.3.2 for details).Furthermore, each subject was fitted with an active orthosis (see Section A) on their right arm as shown in Figure 1-(a), and held a small air-filled ball in their left hand.To trigger support from the orthosis, the subjects were required to express their intention to move by applying a force greater than a start threshold in the movement direction.The required force varied among subjects, depending on their strength.It was ensured that the start thresholds were large enough to prevent unintended starts (refer to Section C).After indicating their intention to move, the subjects were instructed to ease their arm muscles as the orthosis took control of the movement and applied adequate torque at the elbow joint.A comprehensive list of the different thresholds for each subject can be found in Table 1.

Experimental Procedure
The subjects' task was to identify errors in the behavior of the orthosis.These errors were deliberately introduced during flexion or extension movements.Here, the term error refers to a momentary change in the direction of orthosis movement for a short duration of time (see Table 2).Furthermore, the term movement trial will be used to indicate a complete range of flexion or extension movement.
In the first experimental run, subjects were asked to perform 30 movement trials (15 flexions and 15 extensions) with no errors to obtain a baseline.The experimental run began only after the subjects heard a start phrase from the experimenter.This was followed by a training session where the subjects got familiarized with how the errors felt and were instructed to squeeze the ball in their left hand as soon as they felt an error.In each of the following runs, six errors were introduced in a randomized sequence within 30 movement trials.The order of occurrence of errors was varied individually for each run (see Section 2.2.3 for more details).Overall, as part of the experiment, each subject performed 10 experimental runs with six errors each and one baseline run without any errors.Before each run, the subjects were reminded to stand still to avoid motion artifacts in the EEG and EMG data.They were also asked to fixate their eyes upon a black cross against the white wall in front of them to avoid eye artifacts in the EEG data.It was also brought to their notice that if, for some reason, they felt an error but forgot to squeeze the ball, they shall just move forward with the run.The subjects were not informed about missed errors during the experiment.At the end of a run, the orthosis motor automatically disabled itself and the subjects were informed about this via a stop phrase.After five runs, a 15-minute break was given during which the subjects could relax and take off the orthosis.A visual summary of the whole experimental procedure is provided in Figure 1-(b).

Error Introduction
At the start of each experimental run, a distinct list of six random values between 1 and 30 (number of errors) was generated.This list, termed as the error sequence, followed the following conditions: • Values 1 and 2 must not be included.
• There must be a gap of at least two between two consecutive numbers from the list.
• The list must be sorted in ascending order.
The values in the list corresponded to the trial numbers in which errors were introduced.The Python3 random library (Van Rossum & Drake 2009) was used to generate the random sequence.
Furthermore, whenever a movement trial began, the trial number was matched with all numbers from the generated error sequence.If a match was found, an error was deliberately introduced near the Mean error position within the movement trial.The error position varied for flexion and extension with a Maximum deviation of 0.3 • from the Mean error position as mentioned in Table 2.
In practice, if the orthosis were executing flexion before the introduction of the error, it would transition into extension for the specified duration of error (see Table 2) and then resume flexion until the end of the trial and vice-versa.

Data Acquisition
This section provides detailed information about the methods used for recording the EEG and EMG data.Additionally, it also describes the process of synchronization of these two types of data.

EEG Recording
The EEG data were recorded using the 64-channel LiveAmp64 system from Brain Products GmbH 1 and an ActiCap slim electrode system 2 with an extended 10-20 layout.The reference electrode was placed at FCz and the GND at AFz.Great efforts were made to record high-quality EEG data and minimize the noise in the data by keeping the impedances of all 64 electrodes below a threshold of 5 kΩ.This impedance check was performed both prior to and after each experimental run.The EEG data were recorded using the Recorder software 3 (version 1.25.0001)from Brain Products GmbH.The sampling rate was 500 Hz and the measurement system used hardware filters that limited the bandwidth of the data to a passband of 0.0 Hz -131.0Hz.

EMG Recording
To record bipolar EMG data, the ANT mini eego amplifier4 was used.The EMG data were recorded with a sampling rate of 1000 Hz using an adapted eego SDK5 for Python.Eight channels were used, each measuring the muscle activity of the following muscles on both the arms: Before placing the electrodes, the skin was prepared with Isopropyl alcohol (70% V/V).The electrodes were placed on the muscle belly in accordance with the SENIAM guidelines (Hermens et al. 2000).

Synchronization of EMG & EEG Data
To enable synchronization of EEG and EMG data for offline analysis, an Arduino Nano board and the Sensor & Trigger extension from the EEG system were used to mark start and end time points of the EMG data recordings within the EEG data.The EEG system was used as the main device to enable an alignment of both data streams with respect to each other.Despite the EEG data recordings starting before the EMG, the marked events serve as reference points to align both data streams.With this approach, an average time difference below 8.5 ms between both data streams was achieved after evaluating the synchronicity for all recorded data sets.This result was arrived at by comparing the amount of recorded EMG data against the marked events recorded by the EEG system.Please refer to section 2.4.3 for the specification of the marked events in the EEG data.

Dataset and Format
This section provides a description of the data format, along with detailed information about the dataset and the recorded events.

Data Format
The recorded EEG data follows the BrainVision Core Data Format 1.0, consisting of a binary data file (.eeg), a header file (.vhdr), and a marker file (.vmrk) 6 .For ease of use, the data can be exported into the widely adopted BIDS format as described in Gorgolewski et al. (2016).Furthermore, for data analysis, processing and classification, two popular options are available -MNE (Python) 7 and EEGLAB (MATLAB) 8 .In contrast, the EMG data is stored in the .txtformat, where each column represents a separate EMG channel.

Dataset Description
In this section, the dataset's folder structure is explained along with the convention used for naming the files.

Folder Structure
This section describes the hierarchical folder structure of the recorded dataset.At the highest level, there are three folders, namely EEG, EMG, and Metadata.The Metadata folder contains a .txtfile for each subject, segregated by a unique code, which consists of meta-information about the subject as well as the measurement sets.In addition to these files, there is also a short description.txtfile with some general information about the whole study.
Furthermore, within each of the modality folders (EEG or EMG), there is an additional level of folders segregated by subject codes.Inside the EEG folder, each subject sub-folder is further divided into two sub-folders viz.data and imp.The data folder consists of the actual measurement files as described in section 2.4.1.In total, there is one 6 https://www.brainproducts.com/support-resources/brainvision-core-data-format-1-0/ 7 https://mne.tools/stable/reading_raw_data.html 8 https://sccn.ucsd.edu/eeglab/index.phpbaseline set without any errors stored inside another sub-folder named baseline without error and 10 sets with deliberate errors introduced.Each header file (.vhdr) also contains the impedance values of every electrode before the set.Conversely, each header file inside the imp folder contains impedance values after each set.All in all, all impedance values, before and after the set, are available within the header files (.vhdr).It has to be noted that, for some subjects, an additional set was recorded for safety purposes and included in this dataset under a subfolder named additional sets.For more detailed information, please refer to the Metadata readme files included within the dataset.

Naming Convention for Data Files
A consistent naming convention was followed for all our files dividing the filename into five parts.The first part was the date of acquisition in yyyymmdd format (e.g.20230424), followed by the subject code (e.g.AC17D).The third part included the experiment identifier, in this case, orthosisErrorIjcai, followed by multi indicating that subjects could hear and see (multiple modalities) during the experiments.For baseline runs, the suffix baseline set plus the set number (e.g 1 or 2) was added while for experimental runs with errors, only the run number was appended at the end (e.g.set5).As an example, a filename would look like 20230424 AC17D orthosisErrorIjcai multi set1.txt.It is important to note that, the term set was used to represent the data files associated with the corresponding experimental run.

Recorded Events
To keep track of all the events occurring during the set, these events were recorded and stored in marker files (.vmrk).The marker files are located within the data folder of each subject under EEG (see section 2.4.2 for data structure).The first recorded event (after the start of a set) was named S1 and it marked the start of the EMG recording for synchronization purposes (see section 2.3.3 for detailed information).The event S1 also occurred at the end of the EMG measurement.The next recorded event was S64 which marked the start of flexion movement.Similarly, the start of an extension movement was marked by the event S32.In order to mark a trial without errors, the event S48 was added around the Mean error position as mentioned in Table 2.The event S96 occurred as soon as an error was introduced in the trial.Additionally, if the subject squeezed the ball, the recorded event S80 was written to the marker file.

ANALYSIS OF DATA QUALITY
In the following, we performed some basic analysis on the recorded data to give proof of data quality and to briefly describe evoked EEG activity as well as muscle activity.Furthermore, to ensure data quality, invalid measurement sets were excluded from the data repository.The excluded sets of all subjects are listed below in the form of subject code, set number followed by the reason for excluding the set.The excluded sets are as follows: • AA56D, set8: orthosis shutdown by max.current limit.
• AY63D, baseline set1: subject played with the air-filled ball during experiment.
Each of the rejected sets was excluded and supplemented by an additional measurement set (as mentioned in Section 2.4.2.1).Although the data quality was kept as high as possible, we observed a 50 Hz noise for some measurement sets and EEG channels, introduced by the orthosis.The 50 Hz noise was also observed in the EMG data.

Behavioral analysis
For response-time analysis, we analyzed the response times for the incorrect events (error events).As mentioned above, the subjects were instructed to squeeze an air-filled ball after recognizing an error.The time between the error event (S96, true label) and response to the event (S80) was calculated for all events.
According to the experimental design, we expected a total of 480 responses to error events (6 error events × 10 datasets × 8 subjects = 480 error events).However, we found 9 false negative cases (i.e., the ball was not squeezed, even after an error event occurred) and 5 false positive cases (i.e., the ball was squeezed, even when the error event did not occur).Hence, a total of 471 error event-response pairs (480 error events -9 false negatives) were used to compute response time.We obtained a median value of 0.72 s over 471 error event-response pairs.We also performed two additional analyses.First, we calculated the response time averaged over all 10 sets for each subject (see, Table 3-(A)).We also calculated the response time averaged over all 8 subjects for each set (see, Table 3-(B)).

Event Related Potentials (ERP) Analysis
For ERP analysis, we analyzed the EEG data using EEGLAB 9 .We preprocessed the data as follows.The raw EEGs were downsampled to 250 Hz, re-referenced to an average reference, and filtered between 0.1 Hz and 15 Hz.The FCz channel, used as a reference in the EEG recording, was recalculated as an EEG channel for ERP analysis.After preprocessing, the artifacts were rejected by visual inspections which means that only clean EEG data were used for EEG segmentation.The artifact-free EEG data were segmented into epochs from 0.1 s to 1 s after each event type (correct/incorrect).Epochs were averaged within each event type with a baseline correction (−0.1 s until stimulus onset).
For averaging epochs, we only used the error events with correct responses i.e., true positive cases (the ball was squeezed when error events occurred).Figure 2 shows the ERPs averaged over all epochs for each event type (S48: correct event, S96: incorrect event) for Subject AQ59D.The ERP morphology, i.e., the shape and distribution on the scalp suggest that the introduction of errors elicits a P300 component, more specifically a P3b (Polich 2007) component.This may be elicited by infrequently occurring odd events to which subjects respond, i.e., task-relevant events (Kirchner et al. 2016).In the case of this subject, we could not observe an error-related potential, which is usually evoked by the recognition of errors.

CONCLUSION
We presented and described an open-access dataset of EEG and EMG data obtained from eight subjects who were assisted by an active orthosis device in moving their right arm.Behavioral analysis showed that the errors were very well recognized by the subjects.Introduced errors were a momentary change in the direction of movement of the orthosis for a short amount of time.The errors were simple 9 https://sccn.ucsd.edu/eeglab/index.php and easy to detect.The appearance of an average event-related potential (ErrP) in the form of P3b indicates that the subject recognized the erroneous events as odd events.One reason that the error introduction did not elicit an error-related potential in the EEG could be due to the simplicity of the error.On the other hand, the very dominant P3b could also overlay the ErrP.These conclusions are very preliminary due to the analysis of only one subject.We hope that the dataset provided and the detailed information about the experimental setup would allow its replication.This would enable the research community to systematically investigate the relationship between odd-event detection and erroneous event evaluation, evoked in the brain.A better understanding of this relationship would help to develop future approaches that could allow automatic adaptation of an assistive device to a subject's subjective needs.Jan-Philipp Brettschneider for his work on the mechanical design of orthosis while interning at DFKI.

APPENDIX A Active Orthosis Device
The orthosis device (Figure 3) was mainly built from off-the-shelf components and in-house manufactured parts (e.g.3D printed or water jet cut) resulting in a simple and cost-effective design.The orthosis consists of an upper arm and a forearm structure connected and driven by an actuator.To compensate for the weight of the device and thereby enhance wearer comfort, the upper arm structure is attached to a strap that is sewn on a shoulder orthosis (see Figure 1-(a)).
To transmit forces to the wearer's arm, the elbow orthosis connects to the human body via an upper arm and a forearm interface.The cuffs consist of silicone pads cast on bent, water-jet-cut aluminum sheet parts.The sheet parts transmit the interaction forces between the orthosis and the wearer and can be deformed elastically to an individual arm shape.The silicone pads not only provide soft contact points to the human tissue but also friction between the arm and interfaces to keep the orthosis in place.
To accommodate different wearers, both upper arm and forearm structure lengths can be adjusted.While the upper arm length is fixed after adjustment, the forearm interface is attached to a slider.This prismatic joint adds a passive degree of freedom to the system that not only adjusts the forearm length but also compensates for misalignments between the axes of rotation of the elbow and orthosis.For further individual adjustment, the forearm interface can be inclined.
The actuator is a T-MOTOR AK80-6 with a rated torque of 6 N • m (12 N • m peak).This drive belongs to the category of so-called quasi-direct-drives or proprioceptive actuators (Wensing et al. (2017)).These comprise a high torque density motor combined with a low gear ratio transmission.Such actuators feature high intrinsic back-drivability and simultaneously allow for accurate high-bandwidth force control.The quasi-direct-drive actuation paradigm has been introduced in rehabilitation robotics throughout the last years, e.g., in Lv et al. (2018), Yu et al. (2020).For communication with the AK80-6, we used a USB to CAN interface and the driver software of Shubham Vyas (2023).We operated the drive in position control mode using the onboard low-level controller.In addition to the software limits, a mechanical stop constrains the range of motion of the orthosis device from the hard limit (upper arm and forearm structure in parallel) to a flexion of −130 • , for safety reasons.
The orthosis was powered by a laboratory power supply set to a voltage of 24 V and a current of up to 3 A.An emergency stop was connected in series with the actuator in case of malfunctions, and a 2400 µF, 100 V rated capacitor was also added in parallel with the actuator for transient voltage suppression.Cable glands were used as strain reliefs to protect the data and power cable connectors from damage.

B Response Listener and Event Trigger Board
Whenever the subject felt an error in the orthosis device, they would press the air-filled ball in their left hand.The state of this ball was recorded by the Response Listener, a microcontroller that continuously read 16-bit analog inputs and transmitted the pin states, through a serial connection, to the Python script.This script then converted the pin states into single-byte values and sent them serially to the Event Trigger Board which is an ATmega328-based microcontroller board (Arduino Nano12 ).
Additionally, other events such as the error introduction, start of flexion or extension, and no-error movement trial were also sent as unique single-byte arrays to the Event Trigger Board.This board read the byte values serially and mapped them into transistor-transistor logic (TTL) output signals.These signals were eventually recorded into the electroencephalogram (EEG) marker files, as described in the Recorded Events section of the paper.

C Calibration Sequence and Safety Measures
For calibrating the zero-reference position of the orthosis, the subjects were asked to fully extend their arms to the hardware limit of the elbow motor.After this calibration, the orthosis would move to a more comfortable start position which was termed the Fully Extended position (see Figure 4).As a safety measure, the range of motion of the orthosis was restricted between the Fully Extended position and Fully Flexed position (both inclusive).On reaching the range limits, the elbow motor stopped automatically and held the position until the subjects crossed the start threshold.Moreover, another layer of safety was added by restricting the maximum supply current and forcing motor torque limits.
Furthermore, the start thresholds for flexion and extension were calibrated individually.Initially, the threshold values were set to 1.0 N • m for flexion and 1.2 N • m for extension.To determine the appropriate threshold value for flexion, multiple flexion movement trials were performed by the subjects until they arrived at a force that felt neither too weak nor too strong.In contrast, to establish the optimal start threshold for each subject for extension, they were asked to rest their arms in the Fully Flexed position and the lowest value at which no unwanted extensions occurred was selected.

Figure 1 .
Figure 1.(a) Subject prepared with EEG and EMG electrodes wearing orthosis on their right arm (b) Visualization of the different steps in the experimental procedure.

Figure 3 .
Figure 3. Computer Aided Design (CAD) model side and front view of the active orthosis device.The actuator mount and mechanical stop are colored for better visibility.The connection to the shoulder orthosis can be seen in Figure 1-(a).

Figure 4 .
Figure 4. Different operating positions of the orthosis.(A) Zero-reference position for the experimental run.(B) Comfortable start position and extension limit during movement trial.(C) Flexion limit during movement trial.(D) Positions around which errors were introduced.

Table 1 .
Start thresholds for each subject.

Table 3 .
Results of response-time (RT) analysis.(A) Median RT for each subject over 10 datasets.(B) Median RT for each set over 8 subjects.µ ± σ : mean ± standard deviation.