Festival of Sciences at Cagliari

[Rodolfo Leoni]

Few days ago, some Italian Chess Federation managers in Sardinia asked me to write an essay about chess engines, their evolution and their utility to chess players. I just sent it to them and I hope it'll serve for next FSI Chess Teachers Courses and for the Festival of Sciences at Cagliari. Thanks to Marco Zerbinati for his technical supervision!

It's long and everybody here knows how engines were, how they work now, and so on. But I want to share some extracts of it, as an acknowledgement to some special people.

About GM contributions

"Currently, the Chess Grand Master who continues to contribute the most with his knowledge is Larry Kaufman, co-author of Komodo, which is the current World Champion among the chess engines."

About the strongest engines

"The best three engines:

Stockfish by Tord Romstad, Marco Costala, Joona Kiiski, Gary Linscott (free and open source)
Komodo (World Champion in charge) by Don Dailey. Larry Kaufman, Mark Lefler (commercial)
Houdini by Robert Houdart (commercial)"


About learning engines

"As a first example and a milestone in the process of "learning" of an engine, I like to mention my friend Michael Sherwin and his unique Romichess engine. This engine, created by pure passion by Michael, was able to beat the world-famous Rybka several times, which had a considerably higher playing power. All this thanks to the special learning based reinforcement technique. The technique, devised in 2006, remains the most effective and the fastest, as learning is reflected, through a special algorithm, from the final position up to the initial one."

[Michael Sherwin]

Hi Rodolfo, Thanks for the kind words. And thanks for understanding the power in Romi's learning algorithm.

Did you see my subject "Understanding the power of reinforcement learning" and the post by Truls Edvard Stokke? Here are two quotes of what Mr. Stokke posted.

"Hey Michael, very interesting stuff, this seems like a table-based monte carlo policy evaluation. Impressive that you would independently discover such a thing on your own."

"However this is indeed a first step towards the policy evaluation used in A0."

Then Mr Stokke makes this post, "An AlphaZero inspired project".

"For those interested, I've managed to hobble together a piece of code that's based on the AG0/A0 approach of guiding an MCTS with neural nets."

"The goal of this project is to see if it's viable to get something like this working on regular hardware."

Mr. Stokke obviously knows what he is talking about. So obviously Romi's learning algorithm is indeed similar to that of A0.

And of course congrats on the teaching work that you have been asked to do.

[Rodolfo Leoni]

Hi Rodolfo, Thanks for the kind words. And thanks for understanding the power in Romi's learning algorithm.

Did you see my subject "Understanding the power of reinforcement learning" and the post by Truls Edvard Stokke? Here are two quotes of what Mr. Stokke posted.

"Hey Michael, very interesting stuff, this seems like a table-based monte carlo policy evaluation. Impressive that you would independently discover such a thing on your own."

"However this is indeed a first step towards the policy evaluation used in A0."

Then Mr Stokke makes this post, "An AlphaZero inspired project".

"For those interested, I've managed to hobble together a piece of code that's based on the AG0/A0 approach of guiding an MCTS with neural nets."

"The goal of this project is to see if it's viable to get something like this working on regular hardware."

Mr. Stokke obviously knows what he is talking about. So obviously Romi's learning algorithm is indeed similar to that of A0.

And of course congrats on the teaching work that you have been asked to do.

Hi Mike,

I remember the power of Romi learning, since I tested it alot. And I read your post about reinforcement learning. An absolute truth.

We must take into account some limits, tough. First one is software strenght: Romi can't learn as fast as a top engine which uses this system. It could happen that a winning line gets a malus because opponent was much stronger with middlegame and endings. But it's clear a single man cannot do a miracle. Real life is often a penalty even if idea is excellent. I could say the most relevant laws of engine learning are:

- The stronger the engine, the faster the learning;
- The stronger the opponents, the better the learning.

So the best SF could learn fast in self-play mode and with a group of very strong opponents, as Komodo, Houdini, other SF branches, Shredder, Fire, Andscacs... If SF (or any top engine) combines Cerebellum best line (second best move 0%) with Romi learning, playing some million games it could reach the AlphaZ performance, even with some dual xeon cpu. Chess games could become a bridge between opening and TBs.

Second limit is hardware performance. We see that system works great on Deep Mind just because the hardware was much better than conventional ones. AlphaZ had the opportunity to learn in few hours what a learning SF or Konodo would learn in months, IMO. In 2006, with single core slow CPUs, it was possible to learn such an amount of things within decades.

About my article... they asked me because they knew I'm inside computer chess and I was graduating for chess teacher. I just got an e-mail and they hope I'll give some more contributions in future. I'll be available but I'll always need some supervision on what I'll write...