chess.com personality bots research

[lkaufman]

Hi guys

I've been researching several general questions related to chess.com personality bots.
1. Does engine run on front or back end?
2. What engine are they built on top of?
3. What settings were used to tune personalities?
4. How interesting to play with them?

Answering first question was probably the most trivial - switching of internet connection
didn't prevented engine to response hence it's running on the front end, but this
didn't yet satisfy me, so I've started inspecting requests within browser's developer tools network tab. There were lot's of
requests... I was expecting some sort of the REST API being available (imagine how cool it
could be to play with bots via API using HTTP requests!) but didn't find any.

Instead I saw this like URL endpoints being requested:
https://www.chess.com/bundles/app/js/ve ... 70d7b6.bin
So... we have some "komodo-lite" located in "jschessengine" folder. I'm not aware of a used
technology here, maybe it's a web assembly maybe not so if someone knows what it is please tell it to me)

The rest of experiment was to actually try to play with available bots (you need to pay to play with any of them).
The very first thing that is clear that bots are using different opening books with a different quality and length of opening lines.
When it comes to actual search I could distinguish between defensive/attacking style - I might be horribly wrong but
it feels like more/less aggressively tuned LMR - again like if someone knows it for sure please tell it to me)
On the other hand I guess either material weights/PST are tuned differently or maybe some other additional eval params.
Also from time to time especially with weak bots it seems like eval is randomized from time to time, I might be wrong though.

Finally - how interesting to play with them. Well, it's hard to say probably because all the time I was enjoying the work
of developers working on this project and tried to imagine how it could be done technically.

QUESTIONS:

To chess players:
Now due to the lack of understanding regarding the last point I'd like to ask chess players
whether you enjoy playing versus bots in general and chess.com bots in particular or not.
The reason behind why I ask that is simple - I'm now thinking about possible adding of
personality bots into my javascript engine WukongJS but before that I'd like to figure out
how requested this bot feature could be.

To engine developers:
I know that we're more likely about to be spending time on improving playing strength
and don't really care much about this like bot features but maybe someone has tried
something similar? Or may be these bots are just a commercial trick to attract new
players for potentially paid subscription based accounts?

So simply saying - does the work on features intended to potentially satisfy chess players make sense
if we're talking about free open source projects? Or all these "user pleasing" modes/settings are just
commercial tricks.

I would appreciate any feedback because currently apart from obvious strength improvement for my
engine I'm also thinking about some additional features but not sure how requested they are.

Thanks in advance!

The bots are based on Komodo using the Skill levels to control strength and various parameter settings to modify playing style. However I believe that the bots rated below about 2400 are using tiny NNs (like Dragon, but aimed to emulate amateurs, not to play super-strong). I think that the ratings are reasonable if you are playing quickly, something like 5' + 5" speed, but if you take your time the ratings are unrealistically high.

Very interesting, thank you! I'm surprised to know that NNs are used for bot as well.
All of NNs are trained by Dietrich Kappe I suppose?
Did you run tests to rate bots, e.g. 300 games with different bots/skills versions in the tournament or it's just a rough estimate regarding
rating based on human testing?
Didn't know komodo team collaborates with chess.com, cool, or are you a single company now?
Sorry for my curiosity, I'm just researching how people making money on chess programming)
Before I thought that it's possible only if you've created 3000+ engine but these bots are very basic as you said...

Larry, I'm really curious to know your personal opinion regarding the following:
Does it make sense trying to create "bots" for my online JS engine which is pretty basic at the moment (no more than 2000 Elo)
As an alternative to monsters like chess.com and similar?
I guess not but still want to know what you think.
I mean I would never ever be even close the the level of Komodo or whatever other top-engine team,
so does mean that I'm doomed to non-commercial chess programming only?

KomodoChess and chess.com are partners, but not a single company. Dietrich Kappe only works on nets for Komodo Dragon; the nets used by chess.com for the bots are by their own personnel. KomodoChess isn't directly involved with the bots; we just give chess.com the right to use Komodo software and modify it for their own needs. The NNs in the bots are supposed to make tactical errors similar to humans; if they don't play like (weak) humans it's not due to something basic about A/B, it just means that they need more work. I mostly play against the bots rated 2400 and above that don't use the NNs, just short searches and modified params. From what I have seen of the lower rated bots, they do make gross tactical errors similar to weak human players, though perhaps not quite of the same frequency or type. I don't think that a low rated engine using normal eval (not NN) will be a good simulation of weak human play. As for the ratings, I think they are mostly based on testing vs. the Komodo Skill levels, trusting our own rating estimates for those levels, which are based on a combination of chess.com play, test matches, and engine vs engine play. Since the human play is untimed, the ratings don't have a clearly defined meaning, but at one point we were told to base our ratings on the assumption of a game/10' (or 5' +5") time control, and I think they are at least in the right ballpark for that time limit.

[maksimKorzh]

Hi guys

I've been researching several general questions related to chess.com personality bots.
1. Does engine run on front or back end?
2. What engine are they built on top of?
3. What settings were used to tune personalities?
4. How interesting to play with them?

Answering first question was probably the most trivial - switching of internet connection
didn't prevented engine to response hence it's running on the front end, but this
didn't yet satisfy me, so I've started inspecting requests within browser's developer tools network tab. There were lot's of
requests... I was expecting some sort of the REST API being available (imagine how cool it
could be to play with bots via API using HTTP requests!) but didn't find any.

Instead I saw this like URL endpoints being requested:
https://www.chess.com/bundles/app/js/ve ... 70d7b6.bin
So... we have some "komodo-lite" located in "jschessengine" folder. I'm not aware of a used
technology here, maybe it's a web assembly maybe not so if someone knows what it is please tell it to me)

The rest of experiment was to actually try to play with available bots (you need to pay to play with any of them).
The very first thing that is clear that bots are using different opening books with a different quality and length of opening lines.
When it comes to actual search I could distinguish between defensive/attacking style - I might be horribly wrong but
it feels like more/less aggressively tuned LMR - again like if someone knows it for sure please tell it to me)
On the other hand I guess either material weights/PST are tuned differently or maybe some other additional eval params.
Also from time to time especially with weak bots it seems like eval is randomized from time to time, I might be wrong though.

Finally - how interesting to play with them. Well, it's hard to say probably because all the time I was enjoying the work
of developers working on this project and tried to imagine how it could be done technically.

QUESTIONS:

To chess players:
Now due to the lack of understanding regarding the last point I'd like to ask chess players
whether you enjoy playing versus bots in general and chess.com bots in particular or not.
The reason behind why I ask that is simple - I'm now thinking about possible adding of
personality bots into my javascript engine WukongJS but before that I'd like to figure out
how requested this bot feature could be.

To engine developers:
I know that we're more likely about to be spending time on improving playing strength
and don't really care much about this like bot features but maybe someone has tried
something similar? Or may be these bots are just a commercial trick to attract new
players for potentially paid subscription based accounts?

So simply saying - does the work on features intended to potentially satisfy chess players make sense
if we're talking about free open source projects? Or all these "user pleasing" modes/settings are just
commercial tricks.

I would appreciate any feedback because currently apart from obvious strength improvement for my
engine I'm also thinking about some additional features but not sure how requested they are.

Thanks in advance!

The bots are based on Komodo using the Skill levels to control strength and various parameter settings to modify playing style. However I believe that the bots rated below about 2400 are using tiny NNs (like Dragon, but aimed to emulate amateurs, not to play super-strong). I think that the ratings are reasonable if you are playing quickly, something like 5' + 5" speed, but if you take your time the ratings are unrealistically high.

Very interesting, thank you! I'm surprised to know that NNs are used for bot as well.
All of NNs are trained by Dietrich Kappe I suppose?
Did you run tests to rate bots, e.g. 300 games with different bots/skills versions in the tournament or it's just a rough estimate regarding
rating based on human testing?
Didn't know komodo team collaborates with chess.com, cool, or are you a single company now?
Sorry for my curiosity, I'm just researching how people making money on chess programming)
Before I thought that it's possible only if you've created 3000+ engine but these bots are very basic as you said...

Larry, I'm really curious to know your personal opinion regarding the following:
Does it make sense trying to create "bots" for my online JS engine which is pretty basic at the moment (no more than 2000 Elo)
As an alternative to monsters like chess.com and similar?
I guess not but still want to know what you think.
I mean I would never ever be even close the the level of Komodo or whatever other top-engine team,
so does mean that I'm doomed to non-commercial chess programming only?

KomodoChess and chess.com are partners, but not a single company. Dietrich Kappe only works on nets for Komodo Dragon; the nets used by chess.com for the bots are by their own personnel. KomodoChess isn't directly involved with the bots; we just give chess.com the right to use Komodo software and modify it for their own needs. The NNs in the bots are supposed to make tactical errors similar to humans; if they don't play like (weak) humans it's not due to something basic about A/B, it just means that they need more work. I mostly play against the bots rated 2400 and above that don't use the NNs, just short searches and modified params. From what I have seen of the lower rated bots, they do make gross tactical errors similar to weak human players, though perhaps not quite of the same frequency or type. I don't think that a low rated engine using normal eval (not NN) will be a good simulation of weak human play. As for the ratings, I think they are mostly based on testing vs. the Komodo Skill levels, trusting our own rating estimates for those levels, which are based on a combination of chess.com play, test matches, and engine vs engine play. Since the human play is untimed, the ratings don't have a clearly defined meaning, but at one point we were told to base our ratings on the assumption of a game/10' (or 5' +5") time control, and I think they are at least in the right ballpark for that time limit.

Thank you very much Larry!
This was very insightful!
I would deeply appreciate if you give m javascript chess engine a try via playing versus it in a browser:
https://maksimkorzh.github.io/wukongJS/wukong.html
Obviously It's too week for a GM but I'm really curious what kind of feedback a real PRO in both chess & chess programming can give)
So please be kind to dedicate 5-10 mins to it)

[BrianErdelyi]

Great thread. I too have wondered how the personalities are created.

How do the bots run in the browser?

Rodent has a robust personality profile. This is separate from elo settings.

So, Komodo has personalities? I could run this locally?

[Cornfed]

QUESTIONS:

To chess players:
Now due to the lack of understanding regarding the last point I'd like to ask chess players
whether you enjoy playing versus bots in general and chess.com bots in particular or not.
The reason behind why I ask that is simple - I'm now thinking about possible adding of
personality bots into my javascript engine WukongJS but before that I'd like to figure out
how requested this bot feature could be.

Chess player here!
Why on earth would I want to play a bot when there are countless number of humans there to play?? Think of the infinite number of 'styles' that is....

[M ANSARI]

QUESTIONS:

To chess players:
Now due to the lack of understanding regarding the last point I'd like to ask chess players
whether you enjoy playing versus bots in general and chess.com bots in particular or not.
The reason behind why I ask that is simple - I'm now thinking about possible adding of
personality bots into my javascript engine WukongJS but before that I'd like to figure out
how requested this bot feature could be.

Chess player here!
Why on earth would I want to play a bot when there are countless number of humans there to play?? Think of the infinite number of 'styles' that is....

Somehow losing to a bot doesn't feel as bad as losing to a human. Also ... beating a bot (especially one that is rated 2400 or higher) feels incredibly good ... much better than beating a 2400 ELO player!

[Cornfed]

QUESTIONS:

To chess players:
Now due to the lack of understanding regarding the last point I'd like to ask chess players
whether you enjoy playing versus bots in general and chess.com bots in particular or not.
The reason behind why I ask that is simple - I'm now thinking about possible adding of
personality bots into my javascript engine WukongJS but before that I'd like to figure out
how requested this bot feature could be.

Chess player here!
Why on earth would I want to play a bot when there are countless number of humans there to play?? Think of the infinite number of 'styles' that is....

Somehow losing to a bot doesn't feel as bad as losing to a human. Also ... beating a bot (especially one that is rated 2400 or higher) feels incredibly good ... much better than beating a 2400 ELO player!

Got to disagree with you there. Beating a fully functioning human (be it a normal one, or one you tricked them in) is much more satisfaction to me...is the same as outrunning a track star vs outrunning a Speedy car that has had a wheel removed, 2 others flattened and some spark pulgs missing....but to each his own.

[Ovyron]

I have yet to see any computer bot play anything like a human.

Have you tried the ProDeo personalities? I think this engine is capable of creating bots that can beat the turing test, I remember making one that had 1.b3 as its favorite choice.

The most human-like bots I've seen have been the Leela Networks back when they weren't super-strong. They were the most human-like blunders, so if a rated 1800 Leela lost a game, it wasn't lost in a computer-way fashion, the games were lost because it missed something despite doing its best, like humans do, and blunders weren't random like modern bots.

Too bad nobody did anything with those.

[dkappe]

I have yet to see any computer bot play anything like a human.

Have you tried the ProDeo personalities? I think this engine is capable of creating bots that can beat the turing test, I remember making one that had 1.b3 as its favorite choice.

The most human-like bots I've seen have been the Leela Networks back when they weren't super-strong. They were the most human-like blunders, so if a rated 1800 Leela lost a game, it wasn't lost in a computer-way fashion, the games were lost because it missed something despite doing its best, like humans do, and blunders weren't random like modern bots.

Too bad nobody did anything with those.

Check out my old distilled nets (as small as 16x2) and small nets trained on bad gyal data. You can really turn the knob for nps or total nodes much more than on a big net.

https://github.com/dkappe/leela-chess-w ... d-Networks
https://github.com/dkappe/leela-chess-w ... -style-net

[Ovyron]

Check out my old distilled nets (as small as 16x2) and small nets trained on bad gyal data. You can really turn the knob for nps or total nodes much more than on a big net.

https://github.com/dkappe/leela-chess-w ... d-Networks
https://github.com/dkappe/leela-chess-w ... -style-net

Thanks. They still seem to strong, though? At 2408 rating it's still going to beat most people.

The idea here is that to get this strong, in the process of the training, the engine was at various strengths (so at some point it was 1000 rated, 1200 rated, 1400 rated...) we'd take a "snapshot" of them (stop their training) and then we'd have many bots at different strengths that people interested could play with (for free, because chess.com's bots are commercial.) I don't know if decreasing to, say, 1 node would be enough to play at some 1000 rated human player level.

[kinderchocolate]

Didn't Microsoft already publish their LC0 benchmark results on scaling down rating points? But it still doesn't have anything to do with "personality". To me, the only way is to define a new reliable numerical scheme to measure "personality", but not sure how to approach it.