Fixed Depth Tournament

[FlavusSnow]

Would there be any reason to do a tournament with a fixed search depth and unlimited time? My thought is that a fixed depth rating list would indicate more accurate evaluation functions. The problem is that sometimes speed compensates for accuracy and those two things work in concert. Anyone else have thoughts on this? Anyone else interested in seeing a rating list like this?

[Martin Thoresen]

Sure, if the search depth 17 meant search depth 17 in all engines.

But unfortunately, it doesn't.

[Uri Blass]

I think that rating list may be interesting but fixed depth is not going to help to compare evaluations because different programs use different pruning.

[Matthias Gemuh]

My thought is that a fixed depth rating list would indicate more accurate evaluation functions.

No, it won't !!!

[FlavusSnow]

I guess the same could be said about a fixed node count... depends on what the author of the engine considers a node.

[hgm]

Indeed, but at least for node-count the definition of a single engine would be constant during the game.

Someone once tested Joker80 for me against Gothic Vortex, and insisted to do it at fixed depth (because it was running as background on a heavly loaded machine, so that playing by time made little sense).

The result turned out to be totally meaningless. Initialy both engines were set to 13 ply, and Gothic Vortex won every game, but also used 100 times as much wall-clock time per game on average. So we decided to set Vortex to 13 ply, but Joker to 15 ply. Then on average they used the same time. But in individual games, sometimes Joker used 10 times as much time as Vortex, and on other games it was just the other way around. There were very few games where they used approximately equal time. And almost always the engine that had used most time was the engine that would win.

[Albert Silver]

I guess the same could be said about a fixed node count... depends on what the author of the engine considers a node.

Yes. If you want to test an engine's pure ability, without allowing time management to be a factor, you should try a fixed time per move.

In fact, this is actually how they played blitz games a century ago. There are account of high level events with Capablanca and co. where the timer was set to 10 seconds, and when the bell rang, all the players had to play a move no matter what.

[michiguel]

I guess the same could be said about a fixed node count... depends on what the author of the engine considers a node.

Yes. If you want to test an engine's pure ability, without allowing time management to be a factor, you should try a fixed time per move.

In fact, this is actually how they played blitz games a century ago. There are account of high level events with Capablanca and co. where the timer was set to 10 seconds, and when the bell rang, all the players had to play a move no matter what.

In the River Plate Club (Argentina), this old tradition was kept until late 70's. Every Sunday, after the soccer match, the chess club (which was at the stadium) had always this traditional tournament. What was the prize? Honor! Lots of strong players took part of it. GM O. Panno, for instance.

It was a lot of fun. I was a kid and enjoyed every minute of it. Nothing better than playing chess after a shot of adrenaline caused by a soccer match. Oh good old times...

Miguel

[bhlangonijr]

I guess the same could be said about a fixed node count... depends on what the author of the engine considers a node.

Yes. If you want to test an engine's pure ability, without allowing time management to be a factor, you should try a fixed time per move.

In fact, this is actually how they played blitz games a century ago. There are account of high level events with Capablanca and co. where the timer was set to 10 seconds, and when the bell rang, all the players had to play a move no matter what.

In the River Plate Club (Argentina), this old tradition was kept until late 70's. Every Sunday, after the soccer match, the chess club (which was at the stadium) had always this traditional tournament. What was the prize? Honor! Lots of strong players took part of it. GM O. Panno, for instance.

It was a lot of fun. I was a kid and enjoyed every minute of it. Nothing better than playing chess after a shot of adrenaline caused by a soccer match. Oh good old times...

Miguel

Soccer and Chess must be realy exciting. Better than that only chessboxing...
http://www.youtube.com/watch?v=43Wcbd0dJpQ

[bob]

Would there be any reason to do a tournament with a fixed search depth and unlimited time? My thought is that a fixed depth rating list would indicate more accurate evaluation functions. The problem is that sometimes speed compensates for accuracy and those two things work in concert. Anyone else have thoughts on this? Anyone else interested in seeing a rating list like this?

WIll not show anything useful. If you limit the search to a specific depth, some programs do way more extensions than others. Some do clever things in the q-search. What you will end up with is a program playing at 1 second per move against a program playing at 30 seconds per move (or something similar) because one does way more extensions or q-search stuff. With a 30:1 time advantage, who should win?

And you also have the case where some programs lie about the depth, what to do there? Or the program that counts plies in units of 3 rather than 1 (Junior)?