Hello:
I modified my SPRT simulator: it can display now the distribution of the length of simulations in an output Notepad:
Code: Select all
alpha: 0.0500
beta: 0.0500
drawelo_fixed: 240.0000
bayeselo_0: -1.5000
bayeselo_1: 4.5000
bayeselo_fixed: 1.5000
Elo: 0.9627
Simulations: 10000
========================
1/10000 Passes: 1 Fails: 0 <Games>/simulation: 75982
2/10000 Passes: 1 Fails: 1 <Games>/simulation: 60804
3/10000 Passes: 1 Fails: 2 <Games>/simulation: 41399
4/10000 Passes: 2 Fails: 2 <Games>/simulation: 33734
5/10000 Passes: 2 Fails: 3 <Games>/simulation: 34209
6/10000 Passes: 3 Fails: 3 <Games>/simulation: 34620
[...]
9995/10000 Passes: 4994 Fails: 5001 <Games>/simulation: 28597
9996/10000 Passes: 4994 Fails: 5002 <Games>/simulation: 28597
9997/10000 Passes: 4995 Fails: 5002 <Games>/simulation: 28602
9998/10000 Passes: 4995 Fails: 5003 <Games>/simulation: 28602
9999/10000 Passes: 4995 Fails: 5004 <Games>/simulation: 28600
10000/10000 Passes: 4995 Fails: 5005 <Games>/simulation: 28599
Shortest simulation: 1951 games (simulation 3850).
Longest simulation: 220405 games (simulation 3251).
Average number of games per simulation: 28599
Type I errors (false positives): 0.00 %
Type II errors (false negatives): 50.05 %
There are 1211 simulations with score > 50% that failed SPRT.
There are 21 simulations with score = 50% that failed SPRT.
Estimated elapsed time: 373.38 seconds.
Speed: 765957 games/second.
Code: Select all
Shortest simulation: 1951 games (+322 -423 =1206) ---> fail.
Longest simulation: 220405 games (+44338 -43623 =132444) ---> pass.
Code: Select all
From 1000 to 1999 games: 2 simulations ( 0.02 %); accumulated: 0.02 %.
From 2000 to 2999 games: 40 simulations ( 0.40 %); accumulated: 0.42 %.
[...]
From 19000 to 19999 games: 215 simulations ( 2.15 %); accumulated: 46.23 %.
From 20000 to 20999 games: 245 simulations ( 2.45 %); accumulated: 48.68 %.
From 21000 to 21999 games: 221 simulations ( 2.21 %); accumulated: 50.89 %.
From 22000 to 22999 games: 204 simulations ( 2.04 %); accumulated: 52.93 %.
From 23000 to 23999 games: 193 simulations ( 1.93 %); accumulated: 54.86 %.
From 24000 to 24999 games: 205 simulations ( 2.05 %); accumulated: 56.91 %.
From 25000 to 25999 games: 172 simulations ( 1.72 %); accumulated: 58.63 %.
From 26000 to 26999 games: 171 simulations ( 1.71 %); accumulated: 60.34 %.
From 27000 to 27999 games: 165 simulations ( 1.65 %); accumulated: 61.99 %.
From 28000 to 28999 games: 139 simulations ( 1.39 %); accumulated: 63.38 %.
From 29000 to 29999 games: 143 simulations ( 1.43 %); accumulated: 64.81 %.
[...]
From 127000 to 127999 games: 3 simulations ( 0.03 %); accumulated: 99.41 %.
From 128000 to 128999 games: 4 simulations ( 0.04 %); accumulated: 99.45 %.
[...]
From 219000 to 219999 games: 0 simulations ( 0.00 %); accumulated: 99.99 %.
From 220000 to 220999 games: 1 simulation ( 0.01 %); accumulated: 100.00 %.
Number of finished simulations: 10000.
------------------------
Code: Select all
alpha: 0.0500
beta: 0.0500
drawelo_fixed: 270.0000
bayeselo_0: 0.0000
bayeselo_1: 6.0000
bayeselo_fixed: 3.0000
Elo: 1.7284
Simulations: 10000
========================
1/10000 Passes: 1 Fails: 0 <Games>/simulation: 56247
2/10000 Passes: 1 Fails: 1 <Games>/simulation: 59612
3/10000 Passes: 1 Fails: 2 <Games>/simulation: 41418
4/10000 Passes: 2 Fails: 2 <Games>/simulation: 33629
5/10000 Passes: 3 Fails: 2 <Games>/simulation: 30384
6/10000 Passes: 3 Fails: 3 <Games>/simulation: 27274
[...]
9995/10000 Passes: 5006 Fails: 4989 <Games>/simulation: 30719
9996/10000 Passes: 5006 Fails: 4990 <Games>/simulation: 30720
9997/10000 Passes: 5006 Fails: 4991 <Games>/simulation: 30720
9998/10000 Passes: 5006 Fails: 4992 <Games>/simulation: 30718
9999/10000 Passes: 5007 Fails: 4992 <Games>/simulation: 30721
10000/10000 Passes: 5008 Fails: 4992 <Games>/simulation: 30720
Shortest simulation: 1412 games (simulation 5747).
Longest simulation: 300012 games (simulation 1052).
Average number of games per simulation: 30720
Type I errors (false positives): 0.00 %
Type II errors (false negatives): 49.92 %
There are 2741 simulations with score > 50% that failed SPRT.
There are 23 simulations with score = 50% that failed SPRT.
Estimated elapsed time: 419.15 seconds.
Speed: 732912 games/second.
Code: Select all
Shortest simulation: 1412 games (+196 -292 =924) ---> fail.
Longest simulation: 300012 games (+53212 -51820 =194980) ---> fail.
Code: Select all
From 1000 to 1999 games: 5 simulations ( 0.05 %); accumulated: 0.05 %.
From 2000 to 2999 games: 19 simulations ( 0.19 %); accumulated: 0.24 %.
[...]
From 21000 to 21999 games: 201 simulations ( 2.01 %); accumulated: 47.15 %.
From 22000 to 22999 games: 215 simulations ( 2.15 %); accumulated: 49.30 %.
From 23000 to 23999 games: 196 simulations ( 1.96 %); accumulated: 51.26 %.
From 24000 to 24999 games: 205 simulations ( 2.05 %); accumulated: 53.31 %.
From 25000 to 25999 games: 169 simulations ( 1.69 %); accumulated: 55.00 %.
From 26000 to 26999 games: 177 simulations ( 1.77 %); accumulated: 56.77 %.
From 27000 to 27999 games: 175 simulations ( 1.75 %); accumulated: 58.52 %.
From 28000 to 28999 games: 172 simulations ( 1.72 %); accumulated: 60.24 %.
From 29000 to 29999 games: 157 simulations ( 1.57 %); accumulated: 61.81 %.
From 30000 to 30999 games: 150 simulations ( 1.50 %); accumulated: 63.31 %.
From 31000 to 31999 games: 164 simulations ( 1.64 %); accumulated: 64.95 %.
[...]
From 127000 to 127999 games: 2 simulations ( 0.02 %); accumulated: 99.12 %.
From 128000 to 128999 games: 0 simulations ( 0.00 %); accumulated: 99.12 %.
[...]
From 299000 to 299999 games: 0 simulations ( 0.00 %); accumulated: 99.99 %.
From 300000 to 300999 games: 1 simulation ( 0.01 %); accumulated: 100.00 %.
Number of finished simulations: 10000.
------------------------
Code: Select all
alpha: 0.0500
beta: 0.0500
drawelo_fixed: 270.0000
bayeselo_0: 0.0000
bayeselo_1: 3.0000
bayeselo_fixed: 1.5000
Elo: 0.8642
Simulations: 10000
========================
1/10000 Passes: 0 Fails: 1 <Games>/simulation: 290883
2/10000 Passes: 1 Fails: 1 <Games>/simulation: 157972
3/10000 Passes: 1 Fails: 2 <Games>/simulation: 127151
4/10000 Passes: 1 Fails: 3 <Games>/simulation: 122171
5/10000 Passes: 1 Fails: 4 <Games>/simulation: 128136
6/10000 Passes: 1 Fails: 5 <Games>/simulation: 121786
[...]
9995/10000 Passes: 5035 Fails: 4960 <Games>/simulation: 122177
9996/10000 Passes: 5035 Fails: 4961 <Games>/simulation: 122170
9997/10000 Passes: 5035 Fails: 4962 <Games>/simulation: 122165
9998/10000 Passes: 5036 Fails: 4962 <Games>/simulation: 122159
9999/10000 Passes: 5037 Fails: 4962 <Games>/simulation: 122154
10000/10000 Passes: 5037 Fails: 4963 <Games>/simulation: 122159
Shortest simulation: 6475 games (simulation 4217).
Longest simulation: 933396 games (simulation 2780).
Average number of games per simulation: 122159
Type I errors (false positives): 0.00 %
Type II errors (false negatives): 49.63 %
There are 2737 simulations with score > 50% that failed SPRT.
There are 8 simulations with score = 50% that failed SPRT.
Estimated elapsed time: 1663.77 seconds.
Speed: 734231 games/second.
Code: Select all
Shortest simulation: 6475 games (+1252 -1029 =4194) ---> pass.
Longest simulation: 933396 games (+163750 -161637 =608009) ---> fail.
Code: Select all
From 6000 to 6999 games: 1 simulation ( 0.01 %); accumulated: 0.01 %.
From 7000 to 7999 games: 2 simulations ( 0.02 %); accumulated: 0.03 %.
[...]
From 91000 to 91999 games: 42 simulations ( 0.42 %); accumulated: 49.32 %.
From 92000 to 92999 games: 52 simulations ( 0.52 %); accumulated: 49.84 %.
From 93000 to 93999 games: 64 simulations ( 0.64 %); accumulated: 50.48 %.
From 94000 to 94999 games: 61 simulations ( 0.61 %); accumulated: 51.09 %.
From 95000 to 95999 games: 56 simulations ( 0.56 %); accumulated: 51.65 %.
From 96000 to 96999 games: 52 simulations ( 0.52 %); accumulated: 52.17 %.
From 97000 to 97999 games: 51 simulations ( 0.51 %); accumulated: 52.68 %.
From 98000 to 98999 games: 53 simulations ( 0.53 %); accumulated: 53.21 %.
From 99000 to 99999 games: 38 simulations ( 0.38 %); accumulated: 53.59 %.
From 100000 to 100999 games: 44 simulations ( 0.44 %); accumulated: 54.03 %.
From 101000 to 101999 games: 42 simulations ( 0.42 %); accumulated: 54.45 %.
From 102000 to 102999 games: 44 simulations ( 0.44 %); accumulated: 54.89 %.
From 103000 to 103999 games: 42 simulations ( 0.42 %); accumulated: 55.31 %.
From 104000 to 104999 games: 38 simulations ( 0.38 %); accumulated: 55.69 %.
From 105000 to 105999 games: 35 simulations ( 0.35 %); accumulated: 56.04 %.
From 106000 to 106999 games: 49 simulations ( 0.49 %); accumulated: 56.53 %.
From 107000 to 107999 games: 43 simulations ( 0.43 %); accumulated: 56.96 %.
From 108000 to 108999 games: 33 simulations ( 0.33 %); accumulated: 57.29 %.
From 109000 to 109999 games: 59 simulations ( 0.59 %); accumulated: 57.88 %.
From 110000 to 110999 games: 41 simulations ( 0.41 %); accumulated: 58.29 %.
From 111000 to 111999 games: 53 simulations ( 0.53 %); accumulated: 58.82 %.
From 112000 to 112999 games: 43 simulations ( 0.43 %); accumulated: 59.25 %.
From 113000 to 113999 games: 40 simulations ( 0.40 %); accumulated: 59.65 %.
From 114000 to 114999 games: 44 simulations ( 0.44 %); accumulated: 60.09 %.
From 115000 to 115999 games: 44 simulations ( 0.44 %); accumulated: 60.53 %.
From 116000 to 116999 games: 46 simulations ( 0.46 %); accumulated: 60.99 %.
From 117000 to 117999 games: 45 simulations ( 0.45 %); accumulated: 61.44 %.
From 118000 to 118999 games: 27 simulations ( 0.27 %); accumulated: 61.71 %.
From 119000 to 119999 games: 37 simulations ( 0.37 %); accumulated: 62.08 %.
From 120000 to 120999 games: 42 simulations ( 0.42 %); accumulated: 62.50 %.
From 121000 to 121999 games: 34 simulations ( 0.34 %); accumulated: 62.84 %.
From 122000 to 122999 games: 42 simulations ( 0.42 %); accumulated: 63.26 %.
From 123000 to 123999 games: 38 simulations ( 0.38 %); accumulated: 63.64 %.
[...]
From 255000 to 255999 games: 10 simulations ( 0.10 %); accumulated: 90.21 %.
From 256000 to 256999 games: 7 simulations ( 0.07 %); accumulated: 90.28 %.
[...]
From 932000 to 932999 games: 0 simulations ( 0.00 %); accumulated: 99.99 %.
From 933000 to 933999 games: 1 simulation ( 0.01 %); accumulated: 100.00 %.
Number of finished simulations: 10000.
------------------------
Why I do this? Just to bring some (useful?) numbers to SF testing framework. In typical SPRT(-1.5, 4.5) and SPRT(0,6) tests I get less than 1% of simulations over 128000 games, which is the default maximum number of games set in SF testing framework. But I recently saw some SPRT(0, 3) tests where the default maximum number of games is 256000... I get around 10% of those simulations over 256000 games (99% of my simulations were of less than 466000 games). I set the worst case (the longest expected average of games) of elo = (elo0 + elo1)/2, so my numbers should be considered as a maximum.
I noted a curious thing in my results: in each of these three cases, the average (arithmetic average) number of games per simulation correspond to 62.8% or 62.9% (more less) of the accumulated distribution; moreover, median/average ratios of these three cases are between 0.755 and 0.764 at first approximation. I mean, those values (percentile of the average and the ratio median/average) seem to vary in tiny intervals from my inexperienced POV.
I hope that more people can verify my results. Sorry for my long post.
Regards from Spain.
Ajedrecista.
