Making dumb dumber

[abulmo2]

As many threads here discuss about weak and new engines, I take my minimalist chess engine Dumb 1.8 and strip all "sophisticated" heuristics, like history, killers, null move, lmr, lmp, check extension, razoring, SEE, etc. to just keep a plain alpha-beta, a simple quiescence search and MVVLVA move ordering to get Dumber 1.0. This Dumber engine is available on github here: https://github.com/abulmo/Dumb/releases/tag/v1.0
My purpose Is to compare Dumb to this new comers. For example, although simpler than Rustic Alpha 1, Dumber is about 100 Elo stronger, maybe because of a better evaluation function (material + pst +tempo, but with tuned weights), but also thanks to a faster search. On my computer Dumber usually analyzes around 10M nodes/s, but Rustic less than 5 M nodes/s, despite a 10% slower move generator.
Compared to dumb 1.8, Dumber is 1000 Elo weaker. All the removed codes just count for 200 lines, so they are worth 5 Elo per lines and are not that complicated.

[mvanthoor]

Interesting experiment.

Rustic doesn't have tuned weights yet; that will be in the cards for Rustic 4 or 5 (the first version after the Alpha's). Rustic Alpha 1 basically has no evaluation: it just counts material + PST, and that's it. I often see that tuning PST's and material alone can add 60-100 Elo, so it could be the reason for Dumb being stronger.

It's almost impossible to compare node counts. When I compare Rustic Alpha 1 to other engines on my computer (older i7-6700K) it manages 3-5 million nps. What is your search doing that mine doesn't? If you have plain A/B, I doubt that it can be twice as fast. I'll have to take a look at it. On my computer, I can almost test against any 1600-1800 engine I want, and Rustic is often _much_ faster.

My NPS node count is actually low compared to what could be done. My NPS counts nodes that are actually searched. So if the engine reports 3.2 Mnps, it has generated moves for 3.2 M nodes in that second, and searched them. If the engine enters a node and then immediately exists because (for example) the TT cuts off the search, or is_draw() reports true, the node is NOT counted. (Same for QSearch, btw.)

If I would count a node immediately at the beginning of the search and qsearch functions, I'd not be counting nodes searched, but nodes visited (and left before actually doing anything). That would skyrocket the node count though.

[hgm]

Dumber has a transposition table?

[abulmo2]

Dumber has a transposition table?

No.

[abulmo2]

My NPS node count is actually low compared to what could be done. My NPS counts nodes that are actually searched. So if the engine reports 3.2 Mnps, it has generated moves for 3.2 M nodes in that second, and searched them. If the engine enters a node and then immediately exists because (for example) the TT cuts off the search, or is_draw() reports true, the node is NOT counted. (Same for QSearch, btw.)

If I would count a node immediately at the beginning of the search and qsearch functions, I'd not be counting nodes searched, but nodes visited (and left before actually doing anything). That would skyrocket the node count though.

So you do not count terminal nodes?

[hgm]

If I understood Marcel correctly, then in any case he doesn't count nodes where a stand-pat cutoff occurs. I am not sure that this completely coincides with the concept of 'terminal node'. There must be QS nodes that do not have eval >= beta, so that you would start searching captures, but where you discover (after move generation) that there actually are none. (Or, when you prune bad captures, you discover all captures are bad.) I would also qualify these as 'terminal nodes', bu I am not sure whether Rustic counts these. Counting those would be tantamount to counting move generations.

I suspect that the speed difference between Rustic and Dumber is largely due to this.

The advantage of Rustic's way of counting is that the speedup you get by implementing futility pruning ('delta pruning') shows up in the node count. When you also count stand-pat nodes, futility pruning would reduce the node count, because it prevents that such nodes will be reached.

When comparing such things between engines, it would be a good idea to make a more differentiated 'node count': counts move generations and evaluations separately, and even distinguis capture-only move generation from that in the full-width search.

[mvanthoor]

If I understood Marcel correctly, then in any case he doesn't count nodes where a stand-pat cutoff occurs. I am not sure that this completely coincides with the concept of 'terminal node'. There must be QS nodes that do not have eval >= beta, so that you would start searching captures, but where you discover (after move generation) that there actually are none. (Or, when you prune bad captures, you discover all captures are bad.) I would also qualify these as 'terminal nodes', bu I am not sure whether Rustic counts these. Counting those would be tantamount to counting move generations.

In Rustic it's simple: it counts a node, as soon as it generates moves in it. If it doesn't generate moves due to an early exit, it doesn't count the node. Depth == 0 is counted by qsearch of course, IF the node generates moves... if it doesn't, it's not counted. I could count both at the point where it generates moves (because it's doing work there), and at the point where it calls evaluation (also doing work there). I don't like just incrementing the node count at the beginning of search/qsearch, because it will massively inflate the speed when nodes are cut off with early exits.

Kiwipete position:

Code: Select all

Counting directly after move generation (so does not count a node when evaluating).

info score cp 25 depth 1 seldepth 16 time 0 nodes 1598 nps 0 pv e2a6 b4c3 b2c3 e6d5
info score cp 25 depth 2 seldepth 16 time 1 nodes 3196 nps 3196000 pv e2a6 b4c3 b2c3 e6d5
info score cp 20 depth 3 seldepth 20 time 2 nodes 8550 nps 4275000 pv e2a6 e6d5 a6b7 b4c3 b7a8 c3d2
info score cp 20 depth 4 seldepth 20 time 6 nodes 18819 nps 3136500 pv e2a6 e6d5 a6b7 b4c3 b7a8 c3d2
info score cp 5 depth 5 seldepth 20 time 25 nodes 69957 nps 2798280 hashfull 2 pv e2a6 b4c3 d2c3 e6d5 e4d5 f6d5
info score cp 15 depth 6 seldepth 22 time 96 nodes 212014 nps 2208479 hashfull 19 pv e2a6 e6d5 c3d5 b6d5 a6b7 a8d8 b7d5 f6d5
info score cp 5 depth 7 seldepth 24 time 396 nodes 891736 nps 2251859 hashfull 69 pv e2a6 e6d5 c3d5 f6d5 e5c4 f7f5 c4b6 a7b6
info score cp -15 depth 8 seldepth 26 time 1722 nodes 3348919 nps 1944785 hashfull 401 pv e2a6 e6d5 c3d5 f6d5 e5d3 f7f5 g2h3 f5e4
info score cp -40 depth 9 seldepth 27 time 8717 nodes 16334347 nps 1873850 hashfull 978 pv d5e6 e7e6 e2a6 e6e5 g2h3 b4c3 d2c3 e5e6 c3f6 g7f6

2M nps, +/- 200K.

Code: Select all

Counting directly at the top of search / qsearch. Counts everything, even early exists where no work is done.

info score cp 25 depth 1 seldepth 16 time 0 nodes 4319 nps 0 pv e2a6 b4c3 b2c3 e6d5
info score cp 25 depth 2 seldepth 16 time 0 nodes 8716 nps 0 pv e2a6 b4c3 b2c3 e6d5
info score cp 20 depth 3 seldepth 20 time 2 nodes 25626 nps 12813000 pv e2a6 e6d5 a6b7 b4c3 b7a8 c3d2
info score cp 20 depth 4 seldepth 20 time 6 nodes 63015 nps 10502500 pv e2a6 e6d5 a6b7 b4c3 b7a8 c3d2
info score cp 5 depth 5 seldepth 20 time 24 nodes 283598 nps 11816583 hashfull 2 pv e2a6 b4c3 d2c3 e6d5 e4d5 f6d5
info score cp 15 depth 6 seldepth 22 time 96 nodes 935911 nps 9749073 hashfull 19 pv e2a6 e6d5 c3d5 b6d5 a6b7 a8d8 b7d5 f6d5
info score cp 5 depth 7 seldepth 24 time 397 nodes 4342636 nps 10938630 hashfull 69 pv e2a6 e6d5 c3d5 f6d5 e5c4 f7f5 c4b6 a7b6
info score cp -15 depth 8 seldepth 26 time 1731 nodes 16376305 nps 9460604 hashfull 401 pv e2a6 e6d5 c3d5 f6d5 e5d3 f7f5 g2h3 f5e4
info score cp -40 depth 9 seldepth 27 time 8744 nodes 87974242 nps 10061098 hashfull 978 pv d5e6 e7e6 e2a6 e6e5 g2h3 b4c3 d2c3 e5e6 c3f6 g7f6

10M nodes/second, +/- 500K.

There's your 10M nodes/second, on an older i7-6700K

Counting directly after move generation (so it's doing/going to do work), and where a node is evaluated (not generating moves, but also work) This should count "work done nodes" (either move generation/searching or evaluation), but not early exits.

Strange. Hardly any difference with only counting nodes that do move generation.

[hgm]

What "early exits" do you still have when you count evaluation as "work done"? Is this the version with TT?

[lithander]

Much appreciated! Even though, if it's 100 ELO stronger than rustic, it's not quite in reach for my engine just yet!

So you do not count terminal nodes?

It may be a stupid question but do you count every node evaluated in QSearch or stop counting as soon as you reach depth 0 and start QSearch?

[lithander]

Hmm... maybe you shouldn't have dumbed down the time control code, too!

Dumber-1.0 lost it's first game against MinimalChess on time. Played in CuteChess GUI. TC: 5seconds + 1sec increment. Still have it open if you need anything...

[pgn][Event "?"]
[Site "?"]
[Date "2021.03.04"]
[Round "?"]
[White "MinimalChess 0.2.6 SimpleMod"]
[Black "dumber-1.0"]
[Result "1-0"]
[ECO "B01"]
[GameDuration "00:01:26"]
[GameEndTime "2021-03-04T18:37:31.667 Mitteleuropäische Zeit"]
[GameStartTime "2021-03-04T18:36:05.343 Mitteleuropäische Zeit"]
[Opening "Scandinavian (center counter) defense"]
[PlyCount "77"]
[Termination "time forfeit"]
[TimeControl "40/5+1"]

1. e4 {+0.35/7 0.73s} d5 {+0.02/7 1.1s} 2. exd5 {+0.35/6 0.34s}
Qxd5 {0.00/7 1.1s} 3. Nc3 {+0.25/6 0.53s} Qe6+ {-0.35/6 1.1s}
4. Nge2 {+0.50/7 0.97s} Nh6 {0.00/7 1.1s} 5. d4 {+0.85/6 0.31s}
Nf5 {0.00/7 1.1s} 6. Bf4 {+0.75/6 1.00s} Qc6 {-0.43/6 1.1s} 7. d5 {+0.90/5 1.2s}
Qb6 {-0.46/6 0.86s} 8. Qd3 {+0.65/5 0.37s} Qxb2 {-0.42/6 1.1s}
9. Rb1 {+0.85/6 0.99s} Qa3 {-0.43/6 1.1s} 10. Bxc7 {+0.95/5 0.43s}
Na6 {0.00/6 1.1s} 11. Qb5+ {+1.55/5 1.3s} Bd7 {0.00/7 1.1s}
12. Qxb7 {+1.45/6 0.79s} Nxc7 {-0.61/5 0.96s} 13. Qxc7 {+1.45/5 0.28s}
Qd6 {-0.73/5 0.89s} 14. Qb7 {+1.90/6 0.60s} Rc8 {0.00/6 1.2s}
15. Qxa7 {+2.05/5 1.4s} Qe5 {0.00/6 1.2s} 16. f4 {+2.40/5 1.4s}
Qe3 {-0.84/6 1.2s} 17. Qxe3 {+2.15/6 0.73s} Nxe3 {0.00/7 1.2s}
18. Kd2 {+2.20/6 1.4s} Nc4+ {-0.69/6 1.2s} 19. Ke1 {+2.00/7 0.95s}
Na3 {-0.64/6 1.1s} 20. Rc1 {+2.00/6 1.4s} e6 {-0.35/7 1.2s}
21. dxe6 {+1.65/6 1.4s} Bxe6 {-0.05/7 0.83s} 22. Kf2 {+0.70/6 1.4s}
Rc4 {-0.15/6 0.84s} 23. Kf3 {+0.70/6 0.63s} Bb4 {0.00/7 1.2s}
24. Ne4 {+0.50/6 0.40s} f5 {+0.38/7 1.2s} 25. Ng5 {+0.65/7 1.0s}
Bd5+ {+0.20/7 1.2s} 26. Kf2 {+0.50/6 1.5s} Bc5+ {+0.23/7 1.2s}
27. Kg3 {+0.50/7 1.5s} Nxc2 {0.00/7 1.2s} 28. Nf3 {+0.30/6 0.81s}
Bxf3 {+0.17/6 1.2s} 29. gxf3 {+0.40/7 1.2s} Be3 {+0.22/7 1.2s}
30. Rb1 {+0.25/6 1.6s} O-O {+0.45/7 1.2s} 31. Rb3 {+0.25/6 0.69s}
Ra4 {+0.37/6 1.1s} 32. Rb7 {-0.35/6 0.36s} Bd2 {+0.41/6 1.2s}
33. Rb2 {-0.05/7 1.9s} Be1+ {+0.48/7 1.2s} 34. Kh3 {-0.55/7 0.48s}
Ne3 {+0.55/7 1.2s} 35. Bg2 {-0.65/6 2.2s} Bb4 {+0.50/6 0.99s}
36. Kg3 {-0.30/6 0.61s} Bd6 {+0.54/6 1.1s} 37. Rd2 {-0.35/6 2.6s}
Bb4 {+0.50/6 1.4s} 38. Rb2 {-0.60/7 1.2s} Bd6 {+0.54/6 1.1s}
39. Kf2 {-0.40/6 3.2s, Black loses on time} 1-0
[/pgn]