Knowing your opponents

[mjlef]

When a human plays, he/she often modifies the playing style based on his opponent, taking into accounts an opponent's weaknesses. I know in developing my program, I have come up with versions that do well against some computer opponents at the expense of other opponents. The UCI spec does not feed the name of the opponent to the chess engine. I think in most tournaments programmers are able to fine tune values between matches, change books, etc, so they could tune to the opponent. Would it make sense to expand UCI to feed the opponent program name to the engine to help automate this? Or do you feel that such tuning is "gaming" the system?

Mark

[Edmund]

When a human plays, he/she often modifies the playing style based on his opponent, taking into accounts an opponent's weaknesses. I know in developing my program, I have come up with versions that do well against some computer opponents at the expense of other opponents. The UCI spec does not feed the name of the opponent to the chess engine. I think in most tournaments programmers are able to fine tune values between matches, change books, etc, so they could tune to the opponent. Would it make sense to expand UCI to feed the opponent program name to the engine to help automate this? Or do you feel that such tuning is "gaming" the system?

Mark

There is a difference between UCI and GUI, the latter is the Graphical User Interface that has to provide the information to the engine. The UCI (=Universal Chess Interface) is just one form of protocol to communicate with the engine. The UCI actually supports a command to tell the engine the opponent, however I neither know any GUIs nor engines that use this command.

Code: Select all

		 *  = UCI_Opponent, type string
		   With this command the GUI can send the name, title, elo and if the engine is playing a human
		   or computer to the engine.
		   The format of the string has to be [GM|IM|FM|WGM|WIM|none] [|none] [computer|human] 
		   Example:
		   "setoption name UCI_Opponent value GM 2800 human Gary Kasparow"
		   "setoption name UCI_Opponent value none none computer Shredder"

[Uri Blass]

When a human plays, he/she often modifies the playing style based on his opponent, taking into accounts an opponent's weaknesses. I know in developing my program, I have come up with versions that do well against some computer opponents at the expense of other opponents. The UCI spec does not feed the name of the opponent to the chess engine. I think in most tournaments programmers are able to fine tune values between matches, change books, etc, so they could tune to the opponent. Would it make sense to expand UCI to feed the opponent program name to the engine to help automate this? Or do you feel that such tuning is "gaming" the system?

Mark

I think most humans do not modify the playing style based on the opponent.

I also doubt if it is possible to earn much by modifying the playing style based on the oponnent.

see the FRC rating list
http://www.computerchess.org.uk/ccrl/40 ... t_all.html

usually the performance of programs after a match of 100 games is very close to the rating of the programs(difference can be explained by statistical error and I do not need the tuning explanation to explain why a program performed better than expected or worse than expected).


When I click on rybka3 and look at the perf column I see numbers between +32 and -50

http://www.computerchess.org.uk/ccrl/40 ... a_3_64-bit


clicking on shredder12 gives numbers between +28 and -37

clicking on Naum4 gives numbers between +86 and -69

looking at many programs I found no number that is bigger than +86
and usually the numbers are even smaller than 50.

Uri

[Don]

When a human plays, he/she often modifies the playing style based on his opponent, taking into accounts an opponent's weaknesses. I know in developing my program, I have come up with versions that do well against some computer opponents at the expense of other opponents. The UCI spec does not feed the name of the opponent to the chess engine. I think in most tournaments programmers are able to fine tune values between matches, change books, etc, so they could tune to the opponent. Would it make sense to expand UCI to feed the opponent program name to the engine to help automate this? Or do you feel that such tuning is "gaming" the system?

Mark

I think most humans do not modify the playing style based on the opponent.

I also doubt if it is possible to earn much by modifying the playing style based on the oponnent.

see the FRC rating list
http://www.computerchess.org.uk/ccrl/40 ... t_all.html

usually the performance of programs after a match of 100 games is very close to the rating of the programs.


When I click on rybka3 and look at the perf column I see numbers between +32 and -50

http://www.computerchess.org.uk/ccrl/40 ... a_3_64-bit


clicking on shredder12 gives numbers between +28 and -37

clicking on Naum4 gives numbers between +86 and -69

looking at many programs I found no number that is bigger than +86
and usually the numbers are even smaller than 50.

Uri

Are you saying that there is not very much intransitive between most players? In other words, if A beat B and B beats C, then we can also expect A to beat C.

If that is what you are saying, I agree. I am not saying there are NO intransitive relationships, but I believe they are fairly weak and almost certainly much weaker than people imagine. Most people claim some kind of intransitive relationship after playing only a few hundred games and trying to draw conclusions where the error margins are too high. I think a lot of people would jump to the conclusion that a 60 out of 100 result is a proof of superiority when it isn't.

But I do stress this - I think there is some intransitivity between certain programs. If your program is particularly good at something and your opponent is particularly bad at that same thing, it MIGHT make your program score lower than your ELO ratings would predict.

ELO is a great system, but it is just a model and it's not a perfect model. It assumes a 1 dimensional relationship space that defines chess strength.

[BubbaTough]

Clearly engines should be told who they are playing. How useful this is can be debated (I think it can be very useful, and think it effects top human players greatly, particularly in opening selection and "contempt-like" issues such as offering a draw or choosing between safe and tactically messy situations).

The annoying thing is, its hard to get the key relevant input: Name, Rating, computer/human.

Name often has many variants, Rating and opponent type are often not input or accurate. So many simple tailoring of your engine you can do (such as setting contempt by name or rating) is hard to pull off, and in some cases can even be harmful.

-Sam

[Don]

When a human plays, he/she often modifies the playing style based on his opponent, taking into accounts an opponent's weaknesses. I know in developing my program, I have come up with versions that do well against some computer opponents at the expense of other opponents. The UCI spec does not feed the name of the opponent to the chess engine. I think in most tournaments programmers are able to fine tune values between matches, change books, etc, so they could tune to the opponent. Would it make sense to expand UCI to feed the opponent program name to the engine to help automate this? Or do you feel that such tuning is "gaming" the system?

Mark

It does open the door to "gaming", on the other hand it's certainly a big part of human play. A human always knows who his opponent is, whether computer or human unless of course it is intentionally hidden or there is deceit. I think over the years my programs have won more games due to the fact the human KNEW he was playing a computer and this adversely affected their play. So it's not really clear to me how much of an advantage you could get from this.

But I would say that since our normal human chess playing convention is that you get to know who your opponent is, it seems like this should be available to the computer to use in any way it wishes.

It's very clearly done by humans who save some special move for a certain opponent and in some cases they have waited years to spring it.

[Don]

Clearly engines should be told who they are playing. How useful this is can be debated (I think it can be very useful, and think it effects top human players greatly, particularly in opening selection and "contempt-like" issues such as offering a draw or choosing between safe and tactically messy situations).

The annoying thing is, its hard to get the key relevant input: Name, Rating, computer/human.

Name often has many variants, Rating and opponent type are often not input or accurate. So many simple tailoring of your engine you can do (such as setting contempt by name or rating) is hard to pull off, and in some cases can even be harmful.

-Sam

There should be some common authority on names of chess players - sort of like domain names for computer networks. At least for computer chess. Maybe we should ask Dann Corbitt, since he is a database expert to become the naming authority for computer chess. You have to register a name from him for $1000.00 per year

[mjlef]

When a human plays, he/she often modifies the playing style based on his opponent, taking into accounts an opponent's weaknesses. I know in developing my program, I have come up with versions that do well against some computer opponents at the expense of other opponents. The UCI spec does not feed the name of the opponent to the chess engine. I think in most tournaments programmers are able to fine tune values between matches, change books, etc, so they could tune to the opponent. Would it make sense to expand UCI to feed the opponent program name to the engine to help automate this? Or do you feel that such tuning is "gaming" the system?

Mark

It does open the door to "gaming", on the other hand it's certainly a big part of human play. A human always knows who his opponent is, whether computer or human unless of course it is intentionally hidden or there is deceit. I think over the years my programs have won more games due to the fact the human KNEW he was playing a computer and this adversely affected their play. So it's not really clear to me how much of an advantage you could get from this.

But I would say that since our normal human chess playing convention is that you get to know who your opponent is, it seems like this should be available to the computer to use in any way it wishes.

It's very clearly done by humans who save some special move for a certain opponent and in some cases they have waited years to spring it.

Here are some examples that are pretty consistant for my program. If I am playing a weaker opponent, if I go less selective, my win percentage improves. If I am playing a stronger opponent, my win percentage increases if I go more selective.

Another example: if I am playing a stronger opponent, my draw percentage increases if my side keeps a queen on the board (I assume this is since it increases the rep draw chance).

[mjlef]

When a human plays, he/she often modifies the playing style based on his opponent, taking into accounts an opponent's weaknesses. I know in developing my program, I have come up with versions that do well against some computer opponents at the expense of other opponents. The UCI spec does not feed the name of the opponent to the chess engine. I think in most tournaments programmers are able to fine tune values between matches, change books, etc, so they could tune to the opponent. Would it make sense to expand UCI to feed the opponent program name to the engine to help automate this? Or do you feel that such tuning is "gaming" the system?

Mark

There is a difference between UCI and GUI, the latter is the Graphical User Interface that has to provide the information to the engine. The UCI (=Universal Chess Interface) is just one form of protocol to communicate with the engine. The UCI actually supports a command to tell the engine the opponent, however I neither know any GUIs nor engines that use this command.

Code: Select all

		 *  = UCI_Opponent, type string
		   With this command the GUI can send the name, title, elo and if the engine is playing a human
		   or computer to the engine.
		   The format of the string has to be [GM|IM|FM|WGM|WIM|none] [|none] [computer|human] 
		   Example:
		   "setoption name UCI_Opponent value GM 2800 human Gary Kasparow"
		   "setoption name UCI_Opponent value none none computer Shredder"

Thanks! I wonder if we should enforce this and encourage GUIS to use these commands.

[Don]

When a human plays, he/she often modifies the playing style based on his opponent, taking into accounts an opponent's weaknesses. I know in developing my program, I have come up with versions that do well against some computer opponents at the expense of other opponents. The UCI spec does not feed the name of the opponent to the chess engine. I think in most tournaments programmers are able to fine tune values between matches, change books, etc, so they could tune to the opponent. Would it make sense to expand UCI to feed the opponent program name to the engine to help automate this? Or do you feel that such tuning is "gaming" the system?

Mark

It does open the door to "gaming", on the other hand it's certainly a big part of human play. A human always knows who his opponent is, whether computer or human unless of course it is intentionally hidden or there is deceit. I think over the years my programs have won more games due to the fact the human KNEW he was playing a computer and this adversely affected their play. So it's not really clear to me how much of an advantage you could get from this.

But I would say that since our normal human chess playing convention is that you get to know who your opponent is, it seems like this should be available to the computer to use in any way it wishes.

It's very clearly done by humans who save some special move for a certain opponent and in some cases they have waited years to spring it.

Here are some examples that are pretty consistant for my program. If I am playing a weaker opponent, if I go less selective, my win percentage improves. If I am playing a stronger opponent, my win percentage increases if I go more selective.

Another example: if I am playing a stronger opponent, my draw percentage increases if my side keeps a queen on the board (I assume this is since it increases the rep draw chance).

I think it's probably more important to know the relative strength of your opponent than his specific identify based on what you are saying.