NN3 HomeResultsMotivationInstructionsDatasetsSubmissionFAQ & HelpConferencesPublications

NN3 Results

Results on the Complete Dataset of 111 Time Series

This represents the actual benchmark of the NN3 competition, as the reduced dataset of 11 series is included in the 111. Congratulations to all of you that were able to forecast this many time series automatically! Please find the results for the top 50% of submissions released below by name and description. All other participants must contact the competition organisers via email to agree the disclosure of their name and method with their rank.

Please note that you must have the login-information to download the descriptions. If you have not yet registered, or have forgotten your login, please visit the [NN3 Homepage] and register your email. The access information will then be sent to you.

Rank on SMAPE

Participant

SMAPE

CONFERENCE
PRESENTATION

DESCRIPTION

-

Stat. Contender - Wildi

14,84%

 

-

Stat. Benchmark - Theta Method (Nikolopoulos)

14,89%


 
 description missing

1

Illies, Jäger, Kosuchinas, Rincon, Sakenas, Vaskevcius

15,18%

 

-

Stat. Benchmark - ForecastPro (Stellwagen)

15,44%

 

-

CI Benchmark - Theta AI (Nikolopoulos)

15,66%

presentation
missing
 description missing

-

Stat. Benchmark - Autobox (Reilly)

15,95%

 

2

Adeodato, Vasconcelos, Arnaud, Chunha, Monteiro

16,17%

 

3

Flores, Anaya, Ramirez, Morales

16,31%

presentation
missing

4

Chen, Yao

16,55%

presentation
missing

5

D'yakonov

16,57%

 

6

Kamel, Atiya, Gayar, El-Shishiny

16,92%

 

7

Abou-Nasr

17,54%

8

Theodosiou, Swamy

17,55%

 

-

CI Benchmark - Naive MLP (Crone)

17,84%

 

9

de Vos

18,24%

 

10

Yan

18,58%

-

CI Benchmark - Naive SVR (Crone, Pietsch)

18,60%

 

11

C49

18,72%


 
 not disclosed by author

12

Perfilieva, Novak, Pavliska, Dvorak, Stepnicka

18,81%


 

13

Kurogi, Koyama, Tanaka, Sanuki

19,00%

presentation
missing

14

Stat. Contender - Beadle

19,14%


 

15

Stat. Contender - Lewicke

19,17%


 

16

Sorjamaa, Lendasse

19,60%

17

Isa

20,00%


 

18

C28

20,54%


 
not disclosed by author

19

Duclos-Gosselin

20,85%


 

-

Stat. Benchmark - Naive

22,69%


 
not disclosed by author

20

Papadaki, Amaxopolous

22,70%


 

21

Stat. Benchmark - Hazarika

23,72%


 

22

C17

24,09%


 
not disclosed by author

23

Stat. Contender  - Njimi, Mélard

24,90%


 

24

Pucheta, Patino, Kuchen

25,13%


 

25

Corzo, Hong

27,53%


 

Submissions in RED are statistical methods that entered the competition as "benchmarks". They can either be existing and estabished statistical methods or novel methods entered to be evaluated through the competition (e.g. Wildi). Submissions in BLUE are CI methods that entered the competition as "benchmarks" but were computed by the organisers as points of reference (e.g. Theta AI, Naive MLP etc.).
Only original submissions with mthods from compuational Intelligence were eligible to win the competition (no benchmarks, no statistical methods and were in part calculated by the NN3 supervisors).

 

Results on the Reduced Dataset of 11 Time Series (subset of the complete)

Please find the results for the top 50% of submissions released below by name and description (plus the ones already disclosed on the complete dataset). All other participants must contact the competition organisers via email to agree the disclosure of their name and method with their rank.

 Rank on SMAPE

Participant 

SMAPE

CONFERENCE
PRESENTATION
DESCRIPTION
 

CI Benchmark - Theta AI (Nikolopoulos)

13,07%

   pending
 
 

Stat. Benchmark - Autobox (Reily)

13,49%

 
 

Stat. Benchmark - ForecastPro (Stellwagen)

13,52%

 

1

Yan

13,68%

 

Stat. Benchmark - Theta (Nikolopoulos)

13,70%


 

 description missing

2

llies, Jäger, Kosuchinas, Rincon, Sakenas, Vaskevcius

14,26%

 

3

Chen, Yao

14,46%

presentation
missing

4

Yousefi, Miromeni, Lucas

14,49%


 

5

Ahmed, Atiya, Gayar, El-Shishiny

14,52%

 

6

Flores, Anaya, Ramirez, Morales

15,00%

presentation
missing

7

Adeodato, Vasconcelos, Arnaud, Chunha, Monteiro

15,10%

 
 

Stat. Contender  - Wildi

15,32%

 

8

Luna, Soares, Ballini

15,35%

 

9

Theodosiou, Swamy

16,19%

 

10

Hwang, Song, Kasabov

16,31%

 

11

Duclos-Gosselin

16,37%

 

12

Kurogi, Koyama, Tanaka, Sanuki

16,49%

presentation
missing

13

White

16,56%

presentation
missing

14

Abou-Nasr

16,69%

 

Stat. Contender - Beadle

17,14%

 

15

Sorjamaa, Lendasse

17,16%

 
 

Stat. Contender  - Njimi, Mélard

17,19%

   

16

Rabie

17,24%

    

17

Jimenez, Rebuzzi Vellasco, Tanscheit

17,78%

   

18

Ruta, Gabrys

17,90%

 

19

Isa

18,07%

   
 

CI Benchmark - Naive SVR

18,37%

 

20

Fillon, Bartoli, Poloni

18,39%

 
 

CI Benchmark - Naive MLP

18,69%

 
 

Stat. Contender - Lewicke

19,51%

 

21

Perfilieva, Novak, Pavliska, Dvorak, Stepnicka

19,77%


 

22

Safavieh, Andalib, Andalib

20,04%


 

23

de Vos

20,26%

 

24

C52

20,35%


 
not disclosed by author

25

D'yakunov

20,45%

 

26

C32

20,63%


 
not disclosed by author

27

Weng, Liu, Cheng, Hwang

20,69%


 

28

C29

20,76%


 
not disclosed by author

29

C49

21,05%


 
not disclosed by author
 

Stat. Benchmark - X12 ARIMA (McElroy)

21,48%


 

30

C35

24,03%


 
not disclosed by author

31

C6

24,05%


 
not disclosed by author

32

C28

24,05%


 
not disclosed by author

33

Phienthrakul, Kijsirikul

25,69%


 
 

Stat. Benchmark - Naive

25,71%


 
not disclosed by author

34

C45

26,04%


 
not disclosed by author

35

Pucheta, Patino, Kuchen

27,39%


 

36

Papadaki, Amaxopolous

28,10%


 

37

Stat. Benchmark - Hazarika

28,62%


 

38

C39

29,14%


 
not disclosed by author

39

C17

30,81%


 
not disclosed by author

40

C56

32,15%


 
not disclosed by author

41

C14

33,42%


 
not disclosed by author

42

Carjabal

34,77%


 

43

Peralta, Gutiereez, Sanchis

36,71%


 

44

Kuremoto, Obayashi, Kobayashi

38,45%

 

45

Corzo, Hong

67,38%


 

Submissions in RED are statistical methods that entered the competition as "benchmarks". They can either be existing and estabished statistical methods or novel methods entered to be evaluated through the competition (e.g. Wildi). Submissions in BLUE are CI methods that entered the competition as "benchmarks" but were computed by the organisers as points of reference (e.g. Theta AI, Naive MLP etc.).
Only original submissions with mthods from compuational Intelligence were eligible to win the competition (no benchmarks, no statistical methods and were in part calculated by the NN3 supervisors).

 

©  2006 BI3S-lab - Hamburg, Germany - All rights reserved - Questions, Comments and Enquiries via eMail -  [Impressum & Disclaimer]
The Knowledge Portal on Forecasting with Neural Networks @ www.neural-forecasting.com -
last update: 18.10.2006