Next Komodo vs. GM handicap match - "Standard chess&quo

[JJJ]

Question : I m 1770 player on 3 min or 1 min on lichess. Wich engines play the same level at me ? Wich engine should I pick on CCRL for exemple ?

[Jesse Gersenson]

Question : I m 1770 player on 3 min or 1 min on lichess. Wich engines play the same level at me ? Wich engine should I pick on CCRL for exemple ?

Try Komodo, start with depth 1. That should be easy. Then try depth 2. Depth 3 will be too tough. Also, you could try Queen odds and set contempt to 700. Or try rook odds with contempt set to 400.

[carldaman]

Question : I m 1770 player on 3 min or 1 min on lichess. Wich engines play the same level at me ? Wich engine should I pick on CCRL for exemple ?

Try Komodo, start with depth 1. That should be easy. Then try depth 2. Depth 3 will be too tough. Also, you could try Queen odds and set contempt to 700. Or try rook odds with contempt set to 400.

What contempt should be set for even-up, no-odds games between humans and Komodo?

Thanks,
CL

[lkaufman]

Question : I m 1770 player on 3 min or 1 min on lichess. Wich engines play the same level at me ? Wich engine should I pick on CCRL for exemple ?

Try Komodo, start with depth 1. That should be easy. Then try depth 2. Depth 3 will be too tough. Also, you could try Queen odds and set contempt to 700. Or try rook odds with contempt set to 400.

What contempt should be set for even-up, no-odds games between humans and Komodo?

Thanks,
CL

I recommend dividing the rating difference by 15. So if we call Komodo 3300 (it depends on hardware and tc though), and you are 2100, then Contempt should be 80 according to this.

[lkaufman]

Question : I m 1770 player on 3 min or 1 min on lichess. Wich engines play the same level at me ? Wich engine should I pick on CCRL for exemple ?

Try Komodo, start with depth 1. That should be easy. Then try depth 2. Depth 3 will be too tough. Also, you could try Queen odds and set contempt to 700. Or try rook odds with contempt set to 400.

I don't think Contempt settings should be that high; they will likely just cause bad chess moves to be played. Based on his rating, Contempt should be something like 100 to 120, but if Komodo is giving rook odds maybe 200 or so is reasonable. Queen odds is only for really weak players, unless you are talkiing about bullet (or near-bullet) chess.

The problem with playing Komodo at fixed depth is that if the setting is "fair" for a human, it will play much stronger than the human in the middle game and much weaker in the endgame. A super-fast time setting like game in one second plus .01" increment should in theory be better, but I don't think most GUIs will allow this, and even if they do overhead will mess it up. Maybe we should offer a setting that just spins idly for 99% of the time, for example. Does any engine/GUI do this?

[lkaufman]

Yes, I very much hope it is at least a relatively close contest. If it is, I think we could learn a bit more about the current relative level of play between the two species. We'd just need more of these contests to accumulate a base of games.

I'm certainly not hoping for a 3.5/0.5 result

All that said, I find all of your human/Komodo matches entertaining, and I don't want you to think I'm just complaining about your efforts. They are appreciated! I am just curious if we could better extrapolate where modern hardware at tournament time controls sit from your handicap matches. I think we can, if we do what is necessary to level the playing field and then adjust the rating to say a typical 6way with 3min/move.

Again, just my thoughts/wishes,
Mike

The problem is that although we have pretty good data on how much doubling speed or time is worth in engine vs engine play, we can only guess how this translates to elo points against humans. But we have some idea.

I'm curios about a notional metric I like to call "path-length-to-Elo" or PLTE. Or at what processor speed must a given program run to play at some arbitrary human strength, say 2200, 2400 or 2600? Each program probably has a unique PLTE or PLTE curve.

Questions related to this metric might be:

1) Knowing a program's PLTE curve and setting a processor speed for it to play at a human ELO of 2500, could grandmaster sparring at this level better contribute to further improving PLTE as opposed to self-play or sparring with other programs at full speed?

2) How much has this metric changed over time with the introduction of software innovations. For example, with current knowledge, could we necessarily decrease PLTE on legacy hardware (e.g. MC68000, ARM, 6502) used by period programs?

3) Or has stronger play required an overall higher PLTE at a some Elo threshold?

4) Does anyone else think this is interesting?

Yes, I find it interesting. Yes, I'm pretty sure that with current knowledge we could easily make stronger engines (decrease PLTE) on legacy hardware, though not by as much as on current hardware. Yes, playing games with GMs equalized in this way would indeed teach us new things or ways to improve the engines, though it might not be cost-effective. This weekend's match will be the first attempt to roughly equalize play in standard chess between a current top engine and a GM, although probably it will not be fully equal. Perhaps equal enough to learn something.

[mhull]

Yes, I very much hope it is at least a relatively close contest. If it is, I think we could learn a bit more about the current relative level of play between the two species. We'd just need more of these contests to accumulate a base of games.

I'm certainly not hoping for a 3.5/0.5 result

All that said, I find all of your human/Komodo matches entertaining, and I don't want you to think I'm just complaining about your efforts. They are appreciated! I am just curious if we could better extrapolate where modern hardware at tournament time controls sit from your handicap matches. I think we can, if we do what is necessary to level the playing field and then adjust the rating to say a typical 6way with 3min/move.

Again, just my thoughts/wishes,
Mike

The problem is that although we have pretty good data on how much doubling speed or time is worth in engine vs engine play, we can only guess how this translates to elo points against humans. But we have some idea.

I'm curios about a notional metric I like to call "path-length-to-Elo" or PLTE. Or at what processor speed must a given program run to play at some arbitrary human strength, say 2200, 2400 or 2600? Each program probably has a unique PLTE or PLTE curve.

Questions related to this metric might be:

1) Knowing a program's PLTE curve and setting a processor speed for it to play at a human ELO of 2500, could grandmaster sparring at this level better contribute to further improving PLTE as opposed to self-play or sparring with other programs at full speed?

2) How much has this metric changed over time with the introduction of software innovations. For example, with current knowledge, could we necessarily decrease PLTE on legacy hardware (e.g. MC68000, ARM, 6502) used by period programs?

3) Or has stronger play required an overall higher PLTE at a some Elo threshold?

4) Does anyone else think this is interesting?

Yes, I find it interesting. Yes, I'm pretty sure that with current knowledge we could easily make stronger engines (decrease PLTE) on legacy hardware, though not by as much as on current hardware. Yes, playing games with GMs equalized in this way would indeed teach us new things or ways to improve the engines, though it might not be cost-effective. This weekend's match will be the first attempt to roughly equalize play in standard chess between a current top engine and a GM, although probably it will not be fully equal. Perhaps equal enough to learn something.

If the engine yet proves too strong in this experiment, underclocking might be another option. PCSTATS (back in 2005) comfortably underclocked an Athlon 64 4000+ to 800 Mhz. So you wouldn't need to adapt to a 32-bit machine to get single-thread legacy performance.

http://www.pcstats.com/articleview.cfm?articleID=1798

[lkaufman]

Yes, I very much hope it is at least a relatively close contest. If it is, I think we could learn a bit more about the current relative level of play between the two species. We'd just need more of these contests to accumulate a base of games.

I'm certainly not hoping for a 3.5/0.5 result

All that said, I find all of your human/Komodo matches entertaining, and I don't want you to think I'm just complaining about your efforts. They are appreciated! I am just curious if we could better extrapolate where modern hardware at tournament time controls sit from your handicap matches. I think we can, if we do what is necessary to level the playing field and then adjust the rating to say a typical 6way with 3min/move.

Again, just my thoughts/wishes,
Mike

The problem is that although we have pretty good data on how much doubling speed or time is worth in engine vs engine play, we can only guess how this translates to elo points against humans. But we have some idea.

I'm curios about a notional metric I like to call "path-length-to-Elo" or PLTE. Or at what processor speed must a given program run to play at some arbitrary human strength, say 2200, 2400 or 2600? Each program probably has a unique PLTE or PLTE curve.

Questions related to this metric might be:

1) Knowing a program's PLTE curve and setting a processor speed for it to play at a human ELO of 2500, could grandmaster sparring at this level better contribute to further improving PLTE as opposed to self-play or sparring with other programs at full speed?

2) How much has this metric changed over time with the introduction of software innovations. For example, with current knowledge, could we necessarily decrease PLTE on legacy hardware (e.g. MC68000, ARM, 6502) used by period programs?

3) Or has stronger play required an overall higher PLTE at a some Elo threshold?

4) Does anyone else think this is interesting?

Yes, I find it interesting. Yes, I'm pretty sure that with current knowledge we could easily make stronger engines (decrease PLTE) on legacy hardware, though not by as much as on current hardware. Yes, playing games with GMs equalized in this way would indeed teach us new things or ways to improve the engines, though it might not be cost-effective. This weekend's match will be the first attempt to roughly equalize play in standard chess between a current top engine and a GM, although probably it will not be fully equal. Perhaps equal enough to learn something.

If the engine yet proves too strong in this experiment, underclocking might be another option. PCSTATS (back in 2005) comfortably underclocked an Athlon 64 4000+ to 800 Mhz. So you wouldn't need to adapt to a 32-bit machine to get single-thread legacy performance.

http://www.pcstats.com/articleview.cfm?articleID=1798

I could simply run Komodo 32 bit instead of 64 bit to effectively cut the speed in half again and to show performance on such systems, but there are too few users who still run 32 bit (except on cellphones) to justify doing so.

[Laskos]

If the engine yet proves too strong in this experiment, underclocking might be another option. PCSTATS (back in 2005) comfortably underclocked an Athlon 64 4000+ to 800 Mhz. So you wouldn't need to adapt to a 32-bit machine to get single-thread legacy performance.

http://www.pcstats.com/articleview.cfm?articleID=1798

Come one, to me these are hardly serious issues. Use SF with UCI "go nodes 1000" (Komodo for some reason doesn't have this UCI command) and you will get roughly SF playing on ZX Spectrum of 1982 at 1 second/move. SF with 1000 nodes is probably depth 4 or so (seldepth even higher), and probably close to 1800-1900 level. 10,000 nodes is an x386 1s/move and so on.

[mhull]

If the engine yet proves too strong in this experiment, underclocking might be another option. PCSTATS (back in 2005) comfortably underclocked an Athlon 64 4000+ to 800 Mhz. So you wouldn't need to adapt to a 32-bit machine to get single-thread legacy performance.

http://www.pcstats.com/articleview.cfm?articleID=1798

Come one, to me these are hardly serious issues. Use SF with UCI "go nodes 1000" (Komodo for some reason doesn't have this UCI command) and you will get roughly SF playing on ZX Spectrum of 1982 at 1 second/move. SF with 1000 nodes is probably depth 4 or so (seldepth even higher), and probably close to 1800-1900 level. 10,000 nodes is an x386 1s/move and so on.

Sure, yes. That's one way of doing it. HOWEVER those are hard limits which don't allow "full striving" so to speak. I think it better the program is unfettered in an identical fashion as it is at higher Mhz. Because at any Mhz, the program itself is managing available resources. Hard limits on nodes searched takes the program out of its natural resource management processes, which might not be desirable in this kind of experiment.