duncan wrote:bob wrote:duncan wrote:Is it possible to test who does the better positional moves, by allowing the gm takebacks only in the case where he loses material to some tactical combination.
I think Mr Kaufman said that allowing one move takebacks is 'only' worth 150 elo and is confident rybka would win a gm under such conditions.
of course more may be necessary.
I think that is too subjective a mechanism. Perhaps it could work if it goes such that if on any move, after the GM moves, if the program sees a win of any material, then the GM will have to take back his previous move, unless he is making an intentional positional sacrifice of some sort. But I am not sure that is the best way to proceed, On the human's move, he might really like a move and when he sees that it loses a pawn, he might investigate further to see if he still wants to play it. I'd rather have him make his move on his own time/clock. Otherwise he would either get a time advantage if a move gets rejected and he thinks "off the clock" or he might tend to move too quickly to see if the opponent will reject the move, so that he has more time to find another candidate. I'd rather just see him work everything out, then make a move and have to stick with it.
Are you saying that If a game was played that after the GM moves, if the program sees a win of any material, then the GM will have to take back his previous move, unless he is making an intentional positional sacrifice of some sort.
and rybka won (which Mr kaufman is confident would happen) it is evidence that Rybka plays better positional moves than a gm.
while if it lost, it would not be in itself evidence that the gm has better positional understanding, as the conditions are advantageous to the human.
Let's list several hypotheses here, just for discussion:
1. Rybka is better in eval and tactics than GM players.
2. Rybka is better in tactics only when compared to a GM.
3. Rybka is better in eval only when compared to a GM.
4. Rybka is worse in both than a GM.
There is little doubt we can discard 3 and 4 outright, agreed? So we are trying to decide whether 1 or 2 is true. And I don't think it is so easy. Here's why:
1. For tactics, we can easily measure this by just looking at the number of times Rybka's eval takes a sudden upward jump after a tactically incorrect move is played by the GM. And we know this happens with enough regularity to be a serious issue.
2. For eval it is more complicated. I have absolutely no doubt that a GM has a better evaluation than Rybka. To test, make a human move in under one second, which limits the search to one or two nodes in total. For Rybka, limit the search to one ply, which will be way more nodes, but will force it to rely on its evaluation. Who do you think would win? I can beat crafty easily limiting it to a one ply search with me moving instantly. I will occasionally even hang a piece and lose it, but overall I win those kinds of games, because my eval is simply better and includes some pattern-matching that programs can't do.
3. But it is more sticky than that. If your eval covers a reasonable (even if small) subset of GM knowledge, the deep searches turn positional judgement into a tactical issue of sorts. For example, a 20 ply search to create a strong outpost knight, or a weakened pawn structure, etc.
So 3 above is the sticky issue. how much can a deep search with rudimentary positional understanding offset the human's much better understanding but much shallower (overall) search?
It is an interesting question. But if we choose to normalize on the human's 1 -2 node search, and equalize that with the computer, it is no contest. If we let the program use a deep search, then it changes. I am not sure by how much and believe the human GM would still be favored. But it is an interesting question to think about, and it would be more interesting to test it.