MadChess With Odds Versus Elite Engines

[emadsen]

I was curious how MadChess would perform against elite engines if given odds (move and/or piece handicap for the elite engine), so I ran a series of matches.

MadChess scored between 7 and 9.5 against various opponents, starting from 17 different odds positions. Details on my blog.

Have any authors of modest-strength engines done the same? I'm curious to see your results. My engine is about 2200 ELO.

[Ajedrecista]

Hello Erik:

FWIW, I have Aquarium Demo and there is a table with different material handicaps and their corresponding estimated rating differences. Here I go:

Code: Select all

Human plays with the strong side; uppercase is for white and lowercase is for black when naming pieces.
(The number of possibilities is between brackets).
'&xm' denotes 'and x moves'.

====================================

Very easy (8):
--------------

   -Q-R        (2) >>> rating - 2400
   -Q-R-N      (4) >>> rating - 2500
   -Q-R-R      (1) >>> rating - 2600
   -q-r-r &1m  (1) >>> rating - 2700

====================================

Easy (4):
---------

   -Q          (1) >>> rating - 2200
   -R-R        (1) >>> rating - 2100
   -r-r &1m    (1) >>> rating - 2150
   -q &1m      (1) >>> rating - 2250

====================================

Average (18):
-------------

   -Q-b        (2) >>> rating - 1650
   -Q-n        (2) >>> rating - 1700
   -R-P-P     (10) >>> rating - 1750  // -Ra1-a2-{b2,c2,d2,e2,f2} or -Rh1-h2-{c2,d2,e2,f2,g2}
   -R-N        (4) >>> rating - 1900

====================================

Hard (12):
----------

   -R          (2) >>> rating - 1250
   -r &1m      (2) >>> rating - 1300
   -R-P        (2) >>> rating - 1500  //  -Ra1-a2 or -Rh1-h2
   -2*MINOR    (6) >>> rating - 1600

====================================

Very hard (12):
---------------

   -N          (2) >>> rating -  800
   -B          (2) >>> rating -  850
   -n &1m      (2) >>> rating -  900
   -b &1m      (2) >>> rating -  950
   -N-P        (2) >>> rating - 1050  //  -Nb1-b2 or -Ng1-g2
   -B-P        (2) >>> rating - 1100  //  -Bc1-b2 or -Bf1-g2

====================================

Master-level (6):
-----------------

   -p &3m      (1) >>> rating -  550  //  -g7 and white pawns on d4 and e4
   -R-n        (1) >>> rating -  400  //  -Ra1-Nb8
   -p &2m      (1) >>> rating -  450  //  -f7 and white pawn on e4
   -r-N &1m    (1) >>> rating -  500  //  -Ra8-Nb1
   -N-p        (1) >>> rating -  500  //  -Nb1-f7
   -P-P        (1) >>> rating -  500  //  -c2-f2

====================================

Grandmaster-level (12):
-----------------------

   -Q-r-n      (1) >>> rating -  300  //  -Qd1-Ra8-Nb8
   -p &1m      (8) >>> rating -  250
   -kq rights  (1) >>> rating -  275  //  Not allowed O-O and O-O-O for black
   -p &1m      (1) >>> rating -  325  //  -f7
   &3m         (1) >>> rating -  350  //  Whitepawns on d4 and e4

====================================

Top Grandmaster-level (10):
---------------------------

   -P          (8) >>> rating -  200
   &2m         (1) >>> rating -  200  //  White pawn on e4
   -KQ rights  (1) >>> rating -  200  //  Not allowed O-O and O-O-O for white

I hope no typos. I see some inconsistencies like in Grandmaster level with pawn and move odds: without f7, it says rating - 325; but without random pawn (including f7), it says rating - 250. I hope everyone will find it useful.

Regards from Spain.

Ajedrecista.

[fern]

As I play right now your madchess 1.4 I wonder when you will deliver another version. MC likes me a lot as much it gives option to get a good result, not being far superior to me.

a hug
Fern

[syzygy]

I hope no typos. I see some inconsistencies like in Grandmaster level with pawn and move odds: without f7, it says rating - 325; but without random pawn (including f7), it says rating - 250. I hope everyone will find it useful.

I don't see the inconsistency. f7 might be worth 325 and a random pawn on average might be worth 250. It's intuitively clear that missing f7 is worse for black than missing, say, a7.

[chetday]

As I play right now your madchess 1.4 I wonder when you will deliver another version. MC likes me a lot as much it gives option to get a good result, not being far superior to me.

a hug
Fern

Erik, just an echo here... like Fernando, I have a good time playing against the various levels you've made available with MadChess. It's one of my favorite engines, and one of these days I'm going to break the 1500 ELO level and from... the sky's the limit. Thank you for your hard work with this project.

[Ajedrecista]

Hello Ronald:

I hope no typos. I see some inconsistencies like in Grandmaster level with pawn and move odds: without f7, it says rating - 325; but without random pawn (including f7), it says rating - 250. I hope everyone will find it useful.

I don't see the inconsistency. f7 might be worth 325 and a random pawn on average might be worth 250. It's intuitively clear that missing f7 is worse for black than missing, say, a7.

If I must understand that these values are averages, then I agree that f7 odd is the most difficult handicap for black while a7 odd could be the easiest one. Please read this post by myself:

Re: Nakamura vs Stockfish, public match 8/23.

I misunderstood that rating - 250 was a constant value independent on the missing pawn. Thanks for your input.

Regards from Spain.

Ajedrecista.

[fern]

Wait wait Chess, I do NOT play one of the low level of madchess, but his best.
What do you think?

My lawyers already prepare the sue against you for defamation...

Fern

[emadsen]

I have Aquarium Demo and there is a table with different material handicaps and their corresponding estimated rating differences.

Very interesting Jesús. Seems to me the table underestimates an engine's playing strength even when handicapped. But I suppose against a strong human player the handicap ELO estimates are reasonable.

As I play right now your madchess 1.4 I wonder when you will deliver another version.

I started a new version with a different board representation, and wrote some of the core code. But I'm not even at the point of testing perft counts. I haven't had much time lately for chess programming. In the free time I've had, I've been playing chess instead of programming it. But I imagine in the winter I'll have more time.

I have a good time playing against the various levels you've made available with MadChess. It's one of my favorite engines.

Thanks Chet, I appreciate the compliment. I'm glad to know people get enjoyment out of my project. Devising an algorithm to weaken the engine was an important feature for a patzer like me!

Wait wait Chess, I do NOT play one of the low level of madchess, but his best.
What do you think?

My lawyers already prepare the sue against you for defamation...

Ha ha! Funny as always, Fernando. I am impressed you can hang with MadChess at full strength.

[fern]

I can hang some time, not all the time. Many times I am hanged instead.

Thanks for your effort.
there is something in Mad style It please me a lot.

Fern