BCI Competition III

- Final Results -

[ remarks | winners | true labels | organizers ]

[ tübingen:I | albany:II | graz:IIIa | graz:IIIb | berlin:IVa | berlin:IVb | berlin:IVc | martigny:V ]


The announcement and the data sets of the BCI Competition III can be found here.

Results for download: all results [ pdf ] or presentation from the BCI Meeting 2005 [ pdf ]

A Kind Request

It would be very helpful for the potential organization of further BCI competitions to get some feedback, criticism and suggestions, about this competition. We are interested to hear from participants about their experiences as well as from non-participants reporting why they refrained from taking part. Feedback may concern general issues or specific data sets (e.g. pros and contras for each data set).
Write your feedback to Benjamin Blankertz <benjamin.blankertz@tu-berlin.de>

[ top ]


Remarks

The results should not be taken too seriously. Of course they do not provide an objective ranking of quality: Nevertheless we hope that Watch out for the article on the competition in IEEE Transactions on Biomedical Engineering where each winning team will present a detailed description of their algorithms.

[ top ]


Data set I [Tübingen]


The performance measure is classification accuracy.
-> Note: The expected accuracy, if classification is made by chance, is 50%. <-

#. contributor  acc  research lab co-contributors
1. Qingguo Wei 91%  Tsinghua University, Beijing Fei Meng, Yijun Wang, Shangkai Gao
2. Paul Hammon 87%  University of California, San Diego
3. Michal Sapinski 86%  Warsaw University, Poland
3. Mao Dawei 86%  Zhejiang University, P.R.C. Ke Daguan, Xie Mingqiang, Ding Jichang, Zheng Kening, Zhou Jie, Murat
3. Alexander D'yakonov 86%  Moscow State University
3. Liu Yang 86%  National University of Defense Technology Changsha, P.R.C. Hu Dewen, Zhou Zongtan, Zang Guohua
7. Florian Knoll 84%  TU Graz Alois Schloegl, Martin Hieden, Carmen Vidaurre, Bernhard Graimann
7. Zhou Zongtan 84%  National University of Defense Technology Changsha, P.R.C. Liu Yang, Hu Dewen
9. Jianzhao Qin 83%  South China University of Technology, China and Institute for Infocomm Research, Singapore Yuanqing Li
10. Matthias Krauledat 82%  Fraunhofer FIRST, Berlin
11. Kiyoung Yang 81%  University of Southern California
11. Martin Hieden 81%  TU Graz, Austria Florian Knoll, Alois Schloegl, Carmen Vidaurre, Bernhard Graiman
13. Archis Gore 79%  Fergusson College, Pune
13. Elly Gysels 79%  CSEM, Neuchatel
15. Xiaomei Pei 69%  Xi'an Jiaotong University, P.R.C. Guangyu Bin
16. Ehsan Arbabi 67%  Sharif University of Technology, Tehran Mohammad Bagher Shamsollahi
17. Florian Popescu 66%  Fraunhofer FIRST, Berlin
18. Hyunjin Yoon 65%  University of Southern California
18. Guido Nolte 65%  Fraunhofer FIRST
20. Timothy Uy 60%  University of California, Irvine (?)
21. Wit Jakuczun 59%  Warsaw University of Technology
22. Ken Wong 58%  Stanford University
23. Xi-Chen Sun 54%  Peking University Jufu Feng, Lianwen Wu, Qiansheng Cheng
24. Nanying Liang 50%  Nanyang Technological University, Singapore
25. Bin An 48%  University of Science and Technology of China, Hefei Yan Ning, Qiang Chen, Zhaohui Jiang, Huanqing Feng
26. Miharu Nishino 44%  Univ. of Tokyo Osamu Fukayama, Takashi Sato
27. Yan Ning 22%  University of Science and Technology of China, Hefei Bin An, Qiang Chen, Zhaohui Jiang, Huanqing Feng

1. Qingguo Wei, Tsinghua University, Beijing

with Fei Meng, Yijun Wang, Shangkai Gao
Features: Combination of Bandpower, CSSD or Waveform Mean, Fisher Discriminant Analysis
Classification: linear SVM
some details [ html | doc ]

2. Paul Hammon, University of California, San Diego

Features: ICA then combination of AR coeffients, spectral power (0-45Hz), wavelet coefficients
Classification: regularized logistic regression
some details [ txt ]

3. Michal Sapinski, Warsaw University, Poland

Features: offset and spectral power of hand selected channels
Classification: logistic regression
some details [ txt ]

3. Mao Dawei, Zhejiang University, P.R.C.

with Ke Daguan, Xie Mingqiang, Ding Jichang, Zheng Kening, Zhou Jie, Murat
Features: Standard deviation of Hilbert-Huang Transform for time-frequency windows (5Hz*0.2S), 7 channels
Classification: Mahalanobis distance between classes
some details [ html | doc ]

3. Alexander D'yakonov, Moscow State University

Features: 7 channels visualized and chosen by hand
some details [ txt ]

3. Liu Yang, National University of Defense Technology Changsha, P.R.C.

with Hu Dewen, Zhou Zongtan, Zang Guohua
Features: Bandpass (2.5Hz to 25Hz), CSP, CWT, correlation between trials.
Classification: LDA
some details [ txt ]

7. Florian Knoll, TU Graz

with Alois Schloegl, Martin Hieden, Carmen Vidaurre, Bernhard Graimann
Features: AAR coeffients and band power (10-12 Hz and 16-24 Hz) from 10 channels
Classification: LDA
some details [ txt ]

7. Zhou Zongtan, National University of Defense Technology Changsha, P.R.C.

with Liu Yang, Hu Dewen
Features: bandpass (5-30Hz), CSP, 12 channels, CWT with coefficients corr. to 4-24Hz, weighting of coefficients by t-test
Classification: LDA
some details [ txt ]

9. Jianzhao Qin, South China University of Technology, China and Institute for Infocomm Research, Singapore

with Yuanqing Li
some details [ pdf ]

10. Matthias Krauledat, Fraunhofer FIRST, Berlin

Features: bandpass (8-12Hz), Gaussian Mixture Model for channel selection and clustering.
Classification: via cluster mean
some details [ txt ]

11. Kiyoung Yang, University of Southern California

Features: correlation and covariance matrix coefficients
Classification: SVM
some details [ txt ]

11. Martin Hieden, TU Graz, Austria

with Florian Knoll, Alois Schloegl, Carmen Vidaurre, Bernhard Graiman
Features: AAR coefficents and bandpower (7-10,1 0-12, 16-24 Hz), 11 channels chosen
some details [ txt ]

13. Archis Gore, Fergusson College, Pune

Features: bandpower
Classification: ensemble of neural networks and fisher discriminant functions
some details [ txt ]

13. Elly Gysels, CSEM, Neuchatel

Features: phase, PSD, and Empirical Mode Decomposition
Classification: Robust Discriminant Analysis
some details [ txt ]

15. Xiaomei Pei, Xi'an Jiaotong University, P.R.C.

with Guangyu Bin
Features: band power of 4 channels, selected by Fisher ratio, multi-variate AR coefficients
Classification: FDA
some details [ html | doc ]

16. Ehsan Arbabi, Sharif University of Technology, Tehran

with Mohammad Bagher Shamsollahi
Features: bandpass (0.5-45Hz), statistical and parametric models, various transforming functions
Classification: Bayesian classifier and SVM
some details [ pdf ]

17. Florian Popescu, Fraunhofer FIRST, Berlin

Features: bandpass (8-40Hz), AR coefficients and 'coupled fits'
Classification: sparse model from 'finite prediction error'
some details [ txt ]

18. Hyunjin Yoon, University of Southern California

Features: channel reduction via feature subset selection, correlation coefficients of all trials
Classification: SVM
some details [ txt ]

18. Guido Nolte, Fraunhofer FIRST

Method: probabilistic classification of cross-spectrum in alpha frequency range
some details [ txt ]

20. Timothy Uy, University of California, Irvine (?)

21. Wit Jakuczun, Warsaw University of Technology

Features: discriminative biorthogonal bases
Classification: discriminative biorthogonal bases
some details [ pdf ]

22. Ken Wong, Stanford University

Features: downsampling, subset of 30 channels
Classification: linear model with Tikhonov regularization
some details [ txt ]

23. Xi-Chen Sun, Peking University

with Jufu Feng, Lianwen Wu, Qiansheng Cheng
Features: time-frequency coefficients chosen by Rayleigh Coefficients
Classification: Attribute Clustering Network via Nearest Neighbors
some details [ pdf ]

24. Nanying Liang, Nanyang Technological University, Singapore

Features: AR coefficients for overlapping windows
Classification: neural network (single layer FF) with ELM training
some details [ html | doc ]

25. Bin An, University of Science and Technology of China, Hefei

with Yan Ning, Qiang Chen, Zhaohui Jiang, Huanqing Feng
Features: 2 CSP components, AR coeffients
Classification: SVM
some details [ txt ]

26. Miharu Nishino, Univ. of Tokyo

with Osamu Fukayama, Takashi Sato
Features: cross-correlation between test and training trials
Classification: 1-nearest neighbor
some details [ txt ]

27. Yan Ning, University of Science and Technology of China, Hefei

with Bin An, Qiang Chen, Zhaohui Jiang, Huanqing Feng
Features: AR coefficients
Classification: Nearest neighbor, SVM
some details [ txt ]

[ top ]


Data set II [Albany]


The performance measure is classification accuracy. The first row shows the result where data from all 15 repetitions have been used. For the results of the second row only the first 5 repetitions have been used.
-> Note: The expected accuracy, if classification is made by chance, is 2.8%. <-

#. contributor  acc (15)   acc (5)  research lab co-contributors
1. Alain Rakotomamonjy 96.5%  73.5%  PSI CNRS FRE-2645, INSA de Rouen, France V. Guigue
2. Li Yandong 90.5%  55.0%  Department of Automation Department of Biomedical Engineering, Tsinghua University, China Gao Xiaorong, Ma Zhongwei, Lin Zhonglin, Lu Wenkai, Hong Bo
3. Zhou Zongtan 90.0%  59.5%  Department of Automatic Control, National University of Defense Technology, China Liu Yang, Hu Dewen, Zang Guohu
4. Ulrich Hoffmann 89.5%  53.0%  Signal Processing Institute, Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland
5. Lin Zhonglin 87.5%  57.5%  Department of Automation, Tsinghua University, China Zhang Changshui, Gao Xiaorong, Zhou Jieyun
6. Renzhi Lu 83.0%  54.5%  Rensselaer Polytechnic Institute, NY
7. Gerardo Gentiletti 78.5%  46.0%  Neuroimaging Laboratory, Department of Electrical Engineering, Universidad Autonoma Metropolitana Iztapalapa, Mexico City Oscar Suarez, Veronica Banuelos
8. Gautam Kunapuli 75.0%  34.0%  Rensselaer Polytechnic Institute, NY
9. Qi Hongzhi 33.5%  27.5%  Tianjin University, China Zhang Qian, Wang Zhen, Wan Baikun
10. Kiyoung Yang 7.5%  7.0%  University of Southern California Cyrus Shahabi, Hyunjin Yoon

1. Alain Rakotomamonjy, PSI CNRS FRE-2645, INSA de Rouen, France

with V. Guigue
some details [ txt ]

2. Li Yandong, Department of Automation Department of Biomedical Engineering, Tsinghua University, China

with Gao Xiaorong, Ma Zhongwei, Lin Zhonglin, Lu Wenkai, Hong Bo
some details [ txt ]

3. Zhou Zongtan, Department of Automatic Control, National University of Defense Technology, China

with Liu Yang, Hu Dewen, Zang Guohu
some details [ txt ]

4. Ulrich Hoffmann, Signal Processing Institute, Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland

some details [ txt ]

5. Lin Zhonglin, Department of Automation, Tsinghua University, China

with Zhang Changshui, Gao Xiaorong, Zhou Jieyun
some details [ txt ]

6. Renzhi Lu, Rensselaer Polytechnic Institute, NY

some details [ txt ]

7. Gerardo Gentiletti, Neuroimaging Laboratory, Department of Electrical Engineering, Universidad Autonoma Metropolitana Iztapalapa, Mexico City

with Oscar Suarez, Veronica Banuelos
some details [ txt ]

8. Gautam Kunapuli, Rensselaer Polytechnic Institute, NY

some details [ txt ]

9. Qi Hongzhi, Tianjin University, China

with Zhang Qian, Wang Zhen, Wan Baikun
some details [ txt ]

10. Kiyoung Yang, University of Southern California

with Cyrus Shahabi, Hyunjin Yoon
some details [ txt ]

[ top ]


Data set IIIa [Graz]


The performance measure is kappa value. The first column shows the average across the three subjects, columns 2 to 4 show the results for the individual subjects.
-> Note: The expected kappa value, if classification is made by chance, is 0. <-

#. contributor  kappa   K3   K6   L1  research lab co-contributors
1. Cuntai Guan 0.7926  0.8222  0.7556  0.8000  Neural Signal Processing Lab Institute for Infocomm Research, Singapore Haihong Zhang, Yuanqin Li
2. Gao Xiaorong 0.6872  0.9037  0.4333  0.7111  Tsinghua University, Beijing, China Wu Wei, Wang Ruiping, Yang Fusheng
3. Jeremy Hill 0.6272  0.9481  0.4111  0.5222  Max Planck Institute for Biological Cybernetics, Tuebingen and Tuebingen University Michael Schroeder

1. Cuntai Guan, Neural Signal Processing Lab Institute for Infocomm Research, Singapore

with Haihong Zhang, Yuanqin Li
Method: Fisher ratios of channel-freqency-time bins, feature selection, designing mu- and beta passband, multiclass CSP, SVM
some details [ txt ]

2. Gao Xiaorong, Tsinghua University, Beijing, China

with Wu Wei, Wang Ruiping, Yang Fusheng
Method: surface laplacian, 8-30Hz filter, CSP (one-vs-rest), SVM+kNN+LDA, bagging
some details [ pdf ]

3. Jeremy Hill, Max Planck Institute for Biological Cybernetics, Tuebingen and Tuebingen University

with Michael Schroeder
Method: resampling 100Hz, detrending, Infomax ICA, Amplitude spectra (Welch), linear PCA, and SVM, scores are constant for each trial
some details [ txt ]

[ top ]


Data set IIIb [Graz]


The performance measure is maximal steepness (t0=3s) of mutual information [bits/s]. The first column shows the average across the three subjects, columns 2 to 4 show the results for the individual subjects.
-> Note: The expected steepness, if classification is made by chance, is 0. <-

#. contributor  MI/t   O3   S4   X11  research lab co-contributors
1. S. Lemm 0.3190  0.1698  0.4382  0.3489  Fraunhofer (FIRST) IDA, Berlin Germany
2. O. Burmeister 0.2506  0.1626  0.4174  0.1719  Forschungszentrum Karlsruhe, Germany M. Reischl, R. Mikut
3. Xiaomei Pei 0.1380  0.2030  0.0936  0.1173  Institute of Biomedical Engineering of Xian Jiaotong University, Xian, China Guangyu Bin
4. S. Parini 0.1184  0.1153  0.1218  0.1181  Politecnico di Milano, Italy L. Piccini, L. Maggi, G. Panfili, G. Andreoni
5. D. Coyle 0.1159  0.1039  0.1490  0.0948  University of Ulster, Northern Ireland G. Prasad, M. McGinnity
6. Dezhong Yao 0.1104  0.1184  0.1516  0.0612  University of Electronic Science and Technology of China (UESTC) Chengdu, China Yu Yin, Xiang Liao
7. K. Tavakolian 0.0474  0.0704  0.0229  0.0489  University of Northern British Columbia, Canada S. Rezaei

1. S. Lemm, Fraunhofer (FIRST) IDA, Berlin Germany

Method: ERP and ERD (MU and BETA), propabilistic classification model, accumulative classifier
some details [ txt ]

2. O. Burmeister, Forschungszentrum Karlsruhe, Germany

with M. Reischl, R. Mikut
Method: Bandbower (BP), ratios and differences of BP; MANOVA for feature selection; SVM and linear combiner
some details [ pdf ]

3. Xiaomei Pei, Institute of Biomedical Engineering of Xian Jiaotong University, Xian, China

with Guangyu Bin
Method: FFT with Hanning window of 1s-segments; Fisher Discriminant Analysis
some details [ txt | doc ]

4. S. Parini, Politecnico di Milano, Italy

with L. Piccini, L. Maggi, G. Panfili, G. Andreoni
Method: optimization of frequency bands (using Cauer Elliptic Filter and Bandpower using LDA) and time-interval (using amplitude modulation envelope) using LDA; Classification with AR(4) parameters (Burg) and PSD from 1s-window with boosted regularized LDA;
some details [ pdf ]

5. D. Coyle, University of Ulster, Northern Ireland

with G. Prasad, M. McGinnity
Method: Preprocessing with Prediction Neural Networks (pNN); Short-Time-Fourier-Transform (STFT); Linear Discriminant Analysis;
some details [ pdf ]

6. Dezhong Yao, University of Electronic Science and Technology of China (UESTC) Chengdu, China

with Yu Yin, Xiang Liao
Method: Scales of Morlet-Wavelet optimized by statistical method; wavelet-based bandpower, Support-Vector-Machines, normalized weighted summation for accumulative classification result.
some details [ txt | doc ]

7. K. Tavakolian, University of Northern British Columbia, Canada

with S. Rezaei
Method: AAR(6) parameters from 1s-segments; Bayesian Network classifier
some details [ pdf ]

[ top ]


Data set IVa [Berlin]


The performance measure is accuracy. The first column shows the overall accuracy for all five subjects, columns 2 to 4 show the results for the individual subjects.
-> Note: The expected accuracy, if classification is made by chance, is 50\%. <-

#. contributor  acc   aa   al   av   aw   ay  research lab co-contributors
1. Yijun Wang 94.17%  95.5%  100.0%  80.6%  100.0%  97.6%  Tsinghua University, Beijing Han Yuan, Dan Zhang, Xiaorong Gao, Zhiguang Zhang, Shangkai Gao
2. Yuanqing Li 85.12%  89.3%  98.2%  76.5%  92.4%  80.6%  Institute for Infocomm Research, Singapore Xiaoyuan Zhu, Cuntai Guan
3. Liu Yang 83.45%  82.1%  94.6%  70.4%  87.5%  88.1%  National University of Defense Technology, Changsha, Hunan Zhou Zongtan, Zang Guohua, Hu Dewen
4. Zhou Zongtan 72.62%  83.9%  100.0%  63.3%  50.9%  88.1%  National University of Defense Technology, Changsha, Hunan Hu Dewen, Liu Yang
5. Michael Bensch 69.17%  73.2%  96.4%  70.4%  79.9%  50.8%  University of Tuebingen and Max Planck Institute for Biological Cybernetics Jeremy Hill
6. Cedric Simon 68.57%  83.0%  91.1%  50.0%  87.9%  54.4%  Université Catholique de Louvain-la-Neuve (UCL-Belgium) Lehembre Remy
7. Elly Gysels 67.86%  69.6%  96.4%  64.3%  69.6%  61.9%  Swiss Center for Electronics and Microtechnology, Neuchatel, Switzerland
8. Carmen Viduarre 64.05%  66.1%  92.9%  67.3%  68.3%  50.4%  Dp IEE, State University of Navarra (UPNA) / Institut for Human-Computer Interfaces/BCI Lab, TU Graz Alois Schloegl, Martin Hieden, Florian Knoll, Rafael Cabeza
9. Le Song 63.69%  66.1%  100.0%  63.3%  64.3%  54.4%  NICTA and School of Information Technology
10. Ehsan Arbabi 62.74%  70.5%  94.6%  56.1%  63.8%  56.3%  Electrical Engineering Department, Sharif University of Technology, Tehran, Iran Emad Fatemizadeh
11. Cyrus Shahabi 59.52%  57.1%  76.8%  57.7%  64.3%  54.0%  University of Southern California Kiyoung Yang, Hyunjin Yoon
12. Kiyoung Yang 53.93%  52.7%  85.7%  61.2%  51.8%  43.7%  University of Southern California Hyunjin Yoon, Cyrus Shahabi
13. Wang Feng 51.90%  50.9%  53.6%  54.6%  56.2%  46.0%  Southeast University, Nanjing Liu Jianmin, Gong Kainan, Wang Xiaonan, Tang Liyan
14. Hyunjin Yoon 51.19%  50.0%  67.9%  52.6%  52.7%  45.6%  University of Southern California Kiyoung Yang, Cyrus Shahabi

1. Yijun Wang, Tsinghua University, Beijing

with Han Yuan, Dan Zhang, Xiaorong Gao, Zhiguang Zhang, Shangkai Gao
Features: Combination of CSP, AR and Temporal Waves of the readiness potential. Adaptation by former classified test samples as extended training samples for the datasets with small amount of training data.
Classification: LDA
some details [ pdf ]

2. Yuanqing Li, Institute for Infocomm Research, Singapore

with Xiaoyuan Zhu, Cuntai Guan
Preprocessing: common average reference. Features: CSP on mu and beta band (frequency band and time bins were chosen by cross-validation).
Classification: extended EM algorithm
some details [ pdf ]

3. Liu Yang, National University of Defense Technology, Changsha, Hunan

with Zhou Zongtan, Zang Guohua, Hu Dewen
Preprocessing: CSP on bandpass filtered signals (2.5-28 Hz), Continous Wavelet Transformation. Features: Rows of the correlation matrix.
Classification: linear with penalisation of energy.

4. Zhou Zongtan, National University of Defense Technology, Changsha, Hunan

with Hu Dewen, Liu Yang
Preprocessing: CSP on bandpass filtered signals (3-28 Hz). Features: sum of continous wavelet transformation values (4-24 Hz) weighted by the two samples t-Test statistic on every time-frequency point.
Classification: LDA
some details [ txt ]

5. Michael Bensch, University of Tuebingen and Max Planck Institute for Biological Cybernetics

with Jeremy Hill
Preprocessing: Detrending. Channel selection by ROC plots on coherence values. Common electrodes for all subjects were chosen. 4 electrode groups per subject and 6 frequency bands.
Classification: nu-SVM on subject's own trials

6. Lehembre Remy, Université Catholique de Louvain-la-Neuve (UCL-Belgium)

with Cedric Simon
Features: CSP (Common Spatial Patterns) and CSSD (Common Spatial Subspace decomposition). Increasing Training set by Mahalannobis distance.
Classification: SVM
some details [ txt ]

7. Elly Gysels, Swiss Center for Electronics and Microtechnology, Neuchatel, Switzerland

Features: PLV (Phase locking Value) features with Fast Correlation Based Filter.
Classification: SVM or DA
some details [ txt ]

8. Carmen Viduarre, Dp IEE, State University of Navarra (UPNA) / Institut for Human-Computer Interfaces/BCI Lab, TU Graz

with Alois Schloegl, Martin Hieden, Florian Knoll, Rafael Cabeza
Features: concatenation of logarithmic band power estimation in frequency bands 10-12 and 16-24 Hz and AAR parameters with order 3 and update coefficient 0.0055. Monopolar/Bipolar channel selection based on leave-one-out cross-validation with LDA.
Classification: LDA based Kalman filtering
some details [ pdf ]

9. Le Song, NICTA and School of Information Technology

Method: common spatial pattern on band (10-30hz) pass-filtered EEG (without re-referencing).
some details [ txt ]

10. Ehsan Arbabi, Electrical Engineering Department, Sharif University of Technology, Tehran, Iran

with Emad Fatemizadeh
Features: several features of different types (statistical features, features related to parametric models, coefficient of different transforming functions ...)
Classification: Bayesian classifier, leave-one-out to select features and to decide whether training from all or only from one subject are used.
some details [ pdf ]

11. Cyrus Shahabi, University of Southern California

with Kiyoung Yang, Hyunjin Yoon
Preprocessing: channel selection by Corona on the row differences. Features: correlation coefficiant matrix (upper triangle), aggregated by mean or max function. Channel were selected by a linear SVM with cross-validation,
Classification: linear SVM
some details [ txt ]

12. Kiyoung Yang, University of Southern California

with Hyunjin Yoon, Cyrus Shahabi
Features: correlation coefficiant matrix or covariance matrix (upper triangle), aggregated by mean or max function. Feature extraction method (corr vs cov, and mean vs max) and channels were selected by a linear SVM with cross-validation.
Classification: linear SVM
some details [ txt ]

13. Wang Feng, Southeast University, Nanjing

with Liu Jianmin, Gong Kainan, Wang Xiaonan, Tang Liyan
Preprocessing: Channel elimination by energy and selection by correlation coefficients. Features: CSP and CSSP on several frequency band.
Classification: linear and nonlinear SVM.
some details [ txt ]

14. Hyunjin Yoon, University of Southern California

with Kiyoung Yang, Cyrus Shahabi
Preprocessing: channel selection by a feature subset selection method
Features: correlation coefficiant matrix (upper triangle)
Classification: linear SVM
some details [ pdf ]

[ top ]


Data set IVb [Berlin]


-> There was only one submission to this data set. For that reason the competition is not evaluated for this data set.<-


Data set IVc [Berlin]


The performance measure is the mean square error with respect to the target vector (-1 for class left, 0 for relax and 1 for foot).
-> Note: The result for a submission with constant output of 0 is 2/3. <-

#. contributor  mse  research lab co-contributors
1. Dan Zhang 0.30  Department of Biomedical Engineering, Tsinghua University, Beijing Yijun Wang
2. Liu Yang 0.59  National University of Defense Technology, Changsha, Hunan Hu Dewen, Zhou Zongtan, Zang Guohua
3. Zhou Zongtan 0.60  National University of Defense Technology, Changsha, Hunan Hu Dewen, Liu Yang
4. Bo Hong 0.67  Johns Hopkins School of Medicine Soumyadipta Acharya, Nitish V. Thakor
5. Kiyoung Yang 0.74  Information Laboratory, University of Southern California Hyunjin Yoon, Cyrus Shahabi
6. Cyrus Shahabi 0.88  Information Laboratory, University of Southern California Kiyoung Yang, Hyunjin Yoon
7. Hyunjin Yoon 1.33  Information Laboratory, University of Southern California Kiyoung Yang, Cyrus Shahabi

1. Dan Zhang, Department of Biomedical Engineering, Tsinghua University, Beijing

with Yijun Wang
Features. ERD by CSSD(common spatial subspace decomposition).
Classification: Fisher Discriminant Analysis.
some details [ pdf ]

2. Liu Yang, National University of Defense Technology, Changsha, Hunan

with Hu Dewen, Zhou Zongtan, Zang Guohua
Preprocessing: CSP on bandpass-filtered signals (2.5-25 Hz).
Features. Rows of the correlation matrix.
Classification: linear with penalisation of energy.
some details [ txt ]

3. Zhou Zongtan, National University of Defense Technology, Changsha, Hunan

with Hu Dewen, Liu Yang
Preprocessing: CSP on bandpass filtered signals (2.5-25 Hz).
Features. sum of continous wavelet transformation values (4-24 Hz) weighted by the two samples t-Test statistic on every time-frequency point.
Classification: LDA
some details [ txt ]

4. Bo Hong, Johns Hopkins School of Medicine

with Soumyadipta Acharya, Nitish V. Thakor
Features. ERD (alpha and beta) identification in Independent Components (IC)
Classification: Committee Neural Network
some details [ txt ]

5. Kiyoung Yang, Information Laboratory, University of Southern California

with Hyunjin Yoon, Cyrus Shahabi
Preprocessing: (row) differences used instead of raw data and channel selection by a feature subset selection
Features. correlation coefficient matrix (upper triangle) of (row) differences.
Classification: linear SVM

6. Cyrus Shahabi, Information Laboratory, University of Southern California

with Kiyoung Yang, Hyunjin Yoon
Preprocessing: (row) differences used instead of raw data and channel selection by a feature subset selection method
Features. correlation coefficient matrix (upper triangle) of (row) differences.
Classification: linear SVM

7. Hyunjin Yoon, Information Laboratory, University of Southern California

with Kiyoung Yang, Cyrus Shahabi
Preprocessing: channel selection by a feature subset selection method
Features. correlation coefficient matrix (upper triangle)
Classification: linear SVM

[ top ]


Data set V [Martigny]

Remark: The original description of this data set stated that class labels should be calculated from segments of one second of raw data resp. from one precomputed feature vector. Later the organizers realized that it might not be clear enough that this statement meant the requirement that no other (i.e. also past) samples must be used. (The reason for this requirement is that the system should have a fast response time, although this property it not neccessary for the competition's test set.) Then this point was clarified on the competition web site. Unfortunately some competitors did not read the clarification and did not understand the original formulation in this sense. Their methods included smoothing consequtive classifier output on longer time windows. Since these results are not comparable to the other, we had to take them out of the regular scoring, see the second table below.

The performance measure is classification accuracy. The first column shows the average across the three subjects, columns 2 to 4 show the results for the individual subjects. The column titled 'psd' indicates whether the submission used the precomputed psd features (y) or not (n).
-> Note: The expected accuracy, if classification is made by chance, is 33.33%. <-

#. contributor psd  acc   s1   s2   s3  research lab co-contributors
1. Ferran Galan y 68.65  79.60  70.31  56.02  University of Barcelona Francesc Oliva, Joan Guardia
2. Xiang Liao y 68.50  78.08  71.66  55.73  University of Electronic Science and Technology of China (UESTC) Yu Yin, Dezhong Yao
3. Walter y 65.90  77.85  66.36  53.44  ???
4. Xiaomei Pei y 65.67  76.03  69.36  51.61  Institute of Biomedical Engineering of Xi'an Jiaotong University Guangyu Bin, Chongxun Zheng
5. Irene Sturm y 64.91  78.08  63.83  52.75  Fraunhofer FIRST (IDA), Berlin Guido Dornhege
6. Stephan Uray y 64.60  81.05  73.04  39.68  TU Graz
7. Julien Kronegg y 64.04  76.06  64.83  51.18  University of Geneva Douglas Rofes
8. John Q. Gan y 63.91  77.40  63.83  50.46  University of Essex, Colchester Louis C.S. Tsui
9. Shiliang Sun n 62.83  74.31  62.32  51.99  Tsinghua University, Beijing Changshui Zhang, Jie Pan
10. J. Ignacio Serrano M. D. del Castillo y 62.61  75.80  61.75  50.23  Instituto de Automatica Industrial. CSIC. Madrid
11. Changshui Zhang y 60.47  72.15  59.22  50.00  Tsinghua University, Beijing Shiliang Sun, Feiping Nie
12. Douglas Rofes y 59.81  72.52  59.85  46.99  University of Geneva
13. Alois Schloegl n 52.71  69.00  57.05  32.29  TU Graz Carmen Vidaurre
14. Ehsan Arbabi n 50.25  55.41  51.79  43.61  Sharif University of Technology Mohammad Bagher Shamsollahi
15. Remy Lehembre y 50.23  72.60  46.31  31.65  Universite Catholique de Louvain-la-Neuve (UCL-Belgium) Simon Cedric
16. Georgios Lappas y 45.72  71.78  33.81  31.39  Technological Educational Institution (TEI) of Western Macedonia, University of Hertfordshire Andreas Albrecht
17. Mohammad Bagher Shamsollahi y 44.97  71.46  32.52  30.76  Sharif University of Technology Ehsan Arbabi
18. Ikaro Silva y 30.68  38.98  27.45  25.55  ???
19. Ali Salehi n 27.97  26.54  32.84  24.53  ???
20. Ikaro Silva2 y 14.24  5.82  10.54  26.38  ???

1. Ferran Galan, University of Barcelona

with Francesc Oliva, Joan Guardia
some details [ pdf ]

2. Xiang Liao, University of Electronic Science and Technology of China (UESTC)

with Yu Yin, Dezhong Yao
some details [ htm | doc ]

3. Walter, ???

some details [ txt ]

4. Xiaomei Pei, Institute of Biomedical Engineering of Xi'an Jiaotong University

with Guangyu Bin, Chongxun Zheng
some details [ htm | doc ]

5. Irene Sturm, Fraunhofer FIRST (IDA), Berlin

with Guido Dornhege
some details [ txt ]

6. Stephan Uray, TU Graz

some details [ pdf ]

7. Julien Kronegg, University of Geneva

with Douglas Rofes
some details [ pdf ]

8. John Q. Gan, University of Essex, Colchester

with Louis C.S. Tsui; (re-submission without smoothing)
some details [ txt ]

9. Shiliang Sun, Tsinghua University, Beijing

with Changshui Zhang, Jie Pan
some details [ txt ]

10. J. Ignacio Serrano M. D. del Castillo, Instituto de Automatica Industrial. CSIC. Madrid

some details [ htm | doc ]

11. Changshui Zhang, Tsinghua University, Beijing

with Shiliang Sun, Feiping Nie
some details [ txt ]

12. Douglas Rofes, University of Geneva

some details [ txt ]

13. Alois Schloegl, TU Graz

with Carmen Vidaurre
some details [ txt ]

14. Ehsan Arbabi, Sharif University of Technology

with Mohammad Bagher Shamsollahi
some details [ pdf ]

15. Remy Lehembre, Universite Catholique de Louvain-la-Neuve (UCL-Belgium)

with Simon Cedric
some details [ txt ]

16. Georgios Lappas, Technological Educational Institution (TEI) of Western Macedonia, University of Hertfordshire

with Andreas Albrecht
some details [ txt ]

17. Mohammad Bagher Shamsollahi, Sharif University of Technology

with Ehsan Arbabi
some details [ pdf ]

18. Ikaro Silva, ???

some details [ txt ]

19. Ali Salehi, ???

some details [ txt | doc ]

20. Ikaro Silva2, ???

some details [ txt ]

[ top ]

Data Set V - Methods acting on Longer Time Segments


See the remark above. The first three submissions used some post-processing smoothing, the forth submission used longer time windows for classification.


#. contributor psd  acc   s1   s2   s3  research lab co-contributors
1. John Q. Gan y 80.66  95.98  79.49  67.43  University of Essex, Colchester Louis C.S. Tsui
2. David R. Hardoon y 78.98  90.18  79.03  67.66  University of Southampton Charanpal Dhanjal, Zakria Hussain
3. Louis C.S. Tsui n 76.88  96.60  75.79  58.49  University of Essex, Colchester John Q. Gan
4. Nanying Liang y 76.68  95.43  70.74  63.76  ???

1. John Q. Gan, University of Essex, Colchester

with Louis C.S. Tsui
some details [ txt ]

2. David R. Hardoon, University of Southampton

with Charanpal Dhanjal, Zakria Hussain
some details [ pdf ]

3. Louis C.S. Tsui, University of Essex, Colchester

with John Q. Gan
some details [ txt ]

4. Nanying Liang, ???

some details [ htm | doc ]

[ top ]


Competition Winners


data set research lab contributor(s)
I Tsinghua University, Beijing Qingguo Wei, Fei Meng, Yijun Wang, Shangkai Gao
II PSI CNRS FRE-2645, INSA de Rouen, France Alain Rakotomamonjy, V. Guigue
IIIa Neural Signal Processing Lab Institute for Infocomm Research, Singapore Cuntai Guan, Haihong Zhang, Yuanqin Li
IIIb Fraunhofer (FIRST) IDA, Berlin Germany Steven Lemm
IVa Tsinghua University, Beijing Yijun Wang, Han Yuan, Dan Zhang, Xiaorong Gao, Zhiguang Zhang, Shangkai Gao
IVc Department of Biomedical Engineering, Tsinghua University, Beijing Dan Zhang, Yijun Wang
V University of Barcelona Ferran Galan, Francesc Oliva, Joan Guardia

Winners are asked to submit an article (short communication, 2 to 4 pages) on their algorithm to IEEE Transactions on Neural Systems and Rehabilitation Engineering until July 19th 2005. We will contact the winners to give them more details on this.

[ top ]


True Labels of competition test sets

[ top ]


Organizers

Albany: Gerwin Schalk, Dean Krusienski, Jonathan R. Wolpaw

Berlin: Benjamin Blankertz, Guido Dornhege, Klaus-Robert Müller

Graz: Alois Schlögl, Bernhard Graimann, Gert Pfurtscheller

Martigny: Silvia Chiappa, José del R. Millán

Tübingen: Michael Schröder, Thilo Hinterberger, Thomas Navin Lal, Guido Widman, Niels Birbaumer

[ top ]



Contact:   Dr. Benjamin Blankertz <benjamin.blankertz@tu-berlin.de>
Fraunhofer FIRST (IDA)
Kekulestr. 7, D-12489 Berlin, Germany
Tel: +49.(0)30.6392-1875