How do you spend time?

[Kempelen]

Hello,

I have a core duo to test my engine, and need about one full day to wait for 6000 games (and sametimes even I need more games/time). I am curious about how other spend time when you can not test nothing because you have to wait. Many of you maybe spend time in other places and only make little programming emprovements day to day, but sure many of you like to invest more time in this hobbie. how do you spend that dead time?

regards,
Fermin

[hgm]

Work on you GUI!

[velmarin]

Hello Fermin.
And with this crisis and so long time...
The truth is that the tests are depressing, do not like having commercial programs, will have a room full of hardware.
I entertain myself with other projects simpler and less technical than Bouquet.
I try to learn very much need, now with Heavychess with which I started tinkering, SlowChess interesting program, and a lot more, I build everything.

Entries in FICS and Jaquemate.org occasionally, some Internet
And the family, a walk from time to time and a couple of wines in the neighborhood.

[jdart]

I do some non-programming chess stuff, including working on the opening book, finding and validating test positions, looking at and analyzing games. If I'm waiting for a test to finish I may also be implementing a new idea that will be in the next test run.

But believe it or not I do some non-chess activities, too. I have been volunteering at the local animal shelter for the past year and enjoying that a lot.

--Jon

[tpetzke]

I do some sports, mostly running every other day and enjoy that a lot (in total about 100km per month).

What time control do you use for your tests and what depth do you get in the early middle game with queens still alive ?

I use something like 15 sec + 0.2 sec per move which lets me reach depth 9 sometimes 10 and hope the results will scale to longer TCs as well.

Thomas (whose engine has a perfect 0% score against yours ...

[Kempelen]

What time control do you use for your tests and what depth do you get in the early middle game with queens still alive ?

I run 30 sec games with no inc. I have no stats about what depth I reach, but I have have stats about average game duration, with is 47 secs. So in my two cores machine, it is 3676 games a day.
I usually run 7000 games with about two days, but same test I can see if progress in less than a day if errar bar say it.
My problem is I feel sametimes they are few games to test same things.

[tpetzke]

I somewhere read that from a statistics point of view it is incorrect to abort a test series as soon as the score moves outside the error bar. So I let the series usually finish (except in extreme cases).

The waiting time is annoying.

Thomas...

[hgm]

This is not incorrect, provided you realize that it affects the relation between standard deviation and error bar. If you decide only after the full number of games, you need 1.96 STD to get 95% confidence (= 5% false positives). If you would abort the test whenever the result exceeds 1.96 STD during the test, it would double the number of false positives, so your confidence would drop to 90%. But you can compensate for that by increasing your full-length confidence to 97.5%, which corresponds to 2.24 STD, i.e. 1.14 times as large as for the 95% confidence. This means you will have to make the STD 1.14 times smaller to get the same resolution in ELO, which means playing 1.14*1.14 = 1.30 times as many games.

So in the worst case (to reject a change as having any effect) it will take 30% more games for the same confidence. But in many cases you will abort the test before the end, when a change has a large effect. So it depends on how many effective vs ineffective changes you make whether you will benefit from this testing strategy.

Of course you can save lots of time with any stopping criterion by testing multiple changes simultaneously, in the same set of games.

[vincent]

I will soon be back from a week of vacation. During this time my quad core computer tested three modifications for my chess engine.

Code: Select all

$ occet-results $(grep "v2.1.0-\(8\|17\)-*" -l res/4/games-*.pgn) 2> /dev/null
Rank Name                                Elo    +    - games score oppo. draws
   1 Beowulf 2.4a                        570    4    3 69916   81%   299    8%
   2 RattateChess 1.0 Nosferatu          523    4    3 68517   76%   299    7%
   3 Sjaak 470                           344    4    4 68055   56%   299   13%
   4 Purple Haze v2.1.0-8-gdb6c1f0-r8    303    4    4 49459   46%   338    9%
   5 Purple Haze v2.1.0-17-gba30771-r2   303    4    4 49248   45%   340    9%
   6 Purple Haze v2.1.0-17-gba30771-r1   300    4    4 48877   46%   335    9%
   7 Purple Haze v2.1.0-8-gdb6c1f0-r3    300    3    4 49209   45%   338    9%
   8 Purple Haze v2.1.0-17-gba30771-r3   298    5    5 38608   36%   421    9%
   9 Purple Haze v2.1.0-8-gdb6c1f0-r1    297    4    4 49312   45%   337    9%
  10 Purple Haze v2.1.0-8-gdb6c1f0       294    4    4 48721   45%   337    9%
  11 Zeta Dva 0210                       239    3    4 69941   43%   299   10%
  12 TSCP 1.8.1                            0    4    4 57005   17%   300    6%

I am planning to build a cluster some day and an online shop even tested for me which motherboard (for the new AMD FX 8350) could run without graphic card. But I need to buy more than just one node and it is currently too expensive for my budget.

In the meantime I schedule my modifications with code that just introduce stylistic changes. I tend to be impatient but this is just a hobby so it's okay.

[smatovic]

my engine is still under construction and unstable,
so i need to watch most of the games played with debug output to
see how an modification behaves

--
Srdja