Handicap poll

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

What handicap would you prefer to see in a Komodo vs GM match?

Poll ended at Sun Jul 19, 2015 6:14 am

A pawn (plus move or two) or two pawns
7
21%
Rook for Knight (plus move perhaps)
7
21%
Opening blunders (like 1e4 f5? or 1.e4 f6? 2d4 Kf7?)
0
No votes
Huge time handicap (like an hour to a minute)
8
24%
White pieces for GM, draw odds, and no book
6
18%
Every move can be retracted once until next move played.
4
12%
Other
1
3%
 
Total votes: 33

lkaufman
Posts: 6284
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Handicap poll

Post by lkaufman »

Let's assume we get a 2650 FIDE GM to play a 6 game match over 3 days at some standard time control like maybe 90' plus 30" or so. The handicap should be such that the match could go either way, and should not involve letting the GM use an engine, since then it's not a human vs engine game. What sort of handicap would you most enjoy watching?
Komodo rules!
User avatar
Nathanael Russell
Posts: 803
Joined: Sat Jan 31, 2015 11:50 pm
Location: Philadelphia, USA

Re: Handicap poll

Post by Nathanael Russell »

1. A pawn (plus move or two) or two pawns
2. Rook for Knight (plus move perhaps)

I would much rather watch how Komodo overcomes the above handicaps at full evaluation strength than handicapping the strength of the engine itself.
Core_Engine_Tester_CCRL
Image
JJJ
Posts: 1346
Joined: Sat Apr 19, 2014 1:47 pm

Re: Handicap poll

Post by JJJ »

Nathanael Russell wrote: I would much rather watch how Komodo overcomes the above handicaps at full evaluation strength than handicapping the strength of the engine itself.
Same here. I want engine at full power.
I voted pawn handicap, but white pieces and draw odds are fine.
lkaufman
Posts: 6284
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: Handicap poll

Post by lkaufman »

I see that so far the leading option is the White, draw odds, and no book option. I suppose that if we go this way, we would still need at least a two or three move book just to provide variety and to avoid prepared games as much as possible, and I also suppose we should also offer some moderate time odds. Of course for all of these handicaps details depend on the opponent.
Komodo rules!
lkaufman
Posts: 6284
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: Handicap poll

Post by lkaufman »

So far it seems the plurality preference is for material handicaps, with support split between the Exchange and pawn or pawns. The draw, move, and no (or minimal) book handicap is also popular, and might be a good idea if we have success with ideas to reduce draws with weaker opponents. The huge time handicap idea is pretty much the same as using only one core with a much smaller handicap. When I think about it, I doubt that even Carlsen with an hour would stand a chance against Komodo on my 24 core with a minute, so imagine what we would have to give to a 2650 to make an even match. Maybe an hour to a minute using only two cores on a a laptop might be reasonable, perhaps also with ponder off? Anyway, it's always possible to combine different ideas in one match, as in Rybka vs. Milov and Stockfish vs. Nakamura.
Komodo rules!
Ferdy
Posts: 4851
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Handicap poll

Post by Ferdy »

lkaufman wrote:Let's assume we get a 2650 FIDE GM to play a 6 game match over 3 days at some standard time control like maybe 90' plus 30" or so. The handicap should be such that the match could go either way, and should not involve letting the GM use an engine, since then it's not a human vs engine game. What sort of handicap would you most enjoy watching?
I prefer time handicap. Give the GM the standard time of 90+30 then make some calculation on the TC to be used by the engine.
Here is an idea.
1. Collect games of some players with that rating or close to it played on that std time.
2. Analyze the positions with the engine given time/pos, threads, hash and computer specs.
3. Calculate the average error of the player (for all positions where the engine suggested move is different from the player move in the actual game)
4. Calculate the average error of the engine too.
5. Given pos1 from actual game, a player played g1f3. Give pos1 to engine and let it analyze at given time/pos and save the bestmove and bestscore. If bestmove is also g1f3 then no error will be added to the player and engine.
6. If the bestmove is different from the player move, say pos1 (bestmove e4, bestscore 10cp) then verify which move is probably correct. Here is how it is done.
a. Verify what is the score of the player move. We already knew that the engine bestscore is 10cp from bestmove e4 on that pos1, but we don't know yet the score of g1f3. So give pos1 + g1f3 to the engine and let it analyze 2x the time/pos. Typically

Code: Select all

position fen pos1 moves g1f3
go movetime n
where n could be 10000 or 10sec.
Say the engine returns with bestmove d7d5 and score of -15cp. Now we change the sign of the score to get the side POV of the player, and that is
score 15cp. So what we have is
pos1, g1f3, +15cp (player)
pos1, e4, +10cp (engine)
engine error = 15-10 = 5cp.
b. In case where the engine verification search of the player move is say +30cp, that would be player POV score is -30cp. So
pos1, g1f3, -30cp (player)
pos1, e4, 10cp (engine)
player error = 10 - (-30) = 40cp.
c. ave error = total error/total pos
d. If the error is too big say 200cp or 2 pawns do not consider it in the calculation.
7. So at the end of analysis you will have say
engine ave error = 30cp
player1 ave error = 40cp
player2 aver error = 75cp
player3 ... and so on.

From that data you will have an idea of what the engine is capable of given the time/pos. So you can have
Engine TC = time/pos * 80 pos for example plus some other factors.
say time/pos = 5sec
Engine TC = 5*80 = 400sec = 6.667m say 10m + 5s inc.
The 30cp and 40cp is probably close. But if this is wider you may adjust the TC.

As we have known an engine is not capable of making blunders often, and that is the challenge for the player, to avoid big mistakes and create some plans to hopefully outplay the engine on time handicap. It is this plan of outplaying the engine that I want to see :).
Kohflote
Posts: 240
Joined: Wed Sep 19, 2007 11:07 am
Location: Singapore

Re: Handicap poll

Post by Kohflote »

Hi,
Human will get tired but machine will not. I suggest:

(1) longer time rest between games
(2) time handicap or no thinking of chess software while is player's move.

Best regards,
Koh, Kah Huat
Ferdy
Posts: 4851
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Handicap poll

Post by Ferdy »

Kohflote wrote:Hi,
Human will get tired but machine will not. I suggest:

(1) longer time rest between games
(2) time handicap or no thinking of chess software while is player's move.

Best regards,
Koh, Kah Huat
I agree, should be ponder off.
Vinvin
Posts: 5313
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: Handicap poll

Post by Vinvin »

Ferdy wrote:
Kohflote wrote:Hi,
Human will get tired but machine will not. I suggest:

(1) longer time rest between games
(2) time handicap or no thinking of chess software while is player's move.

Best regards,
Koh, Kah Huat
I agree, should be ponder off.
+1
lkaufman
Posts: 6284
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA
Full name: Larry Kaufman

Re: Handicap poll

Post by lkaufman »

Kohflote wrote:Hi,
Human will get tired but machine will not. I suggest:

(1) longer time rest between games
(2) time handicap or no thinking of chess software while is player's move.

Best regards,
Koh, Kah Huat
No ponder is a small handicap, maybe 25 elo or so. It is not useful by itself, but together with a large time handicap it makes sense, because it is illogical for the engine to have one second for one move and then one minute for the next if it happens to guess the opponent's move right. So I agree with those who say that with big time handicap there should be no ponder. I think we did that in the Ehlvest vs Rybka match.
Komodo rules!