Futility pruning, Ext futility pruning and Limited Razoring.

[bob]

The change to null-move suggested by Cleveland was another 5-10.

Which change?

Not doing null-move when the depth remaining is 1.

Just one more example of something that I have posted about years ago, that has been in RomiChess for years and that has been migrated to Crafty and other engins. And everyone else gets the credit for it.

I posted that history tables were useless for move ordering shortly after RomiChess was first released. Later they were dropped from Crafty.

History for _ordering_ has gone from Crafty since a 20.x version which is 3-4 years old at least. It was _not_ bad or useless. The effect was minimal and in cleaning up the code, we slightly modified killers and hashing and found we could do without it.

*****************************
RomiChess is older than 3-4 years.
*****************************

I posted that IID is of no benefit. Later it was removed from Crafty.

Again, it is not of "no benefit". It just adds some quirks to hashing that I wanted to get rid of, rather than working around them. There are circimstances where it is pretty good.

*************************************************************
There are circumstances when generating a random move to play may be pretty good. But, until you work around the problems of IID and the hash, and then cluster test it, there is no acceptable proof that it is of any use.
*************************************************************

Any idea I mention here has been cluster tested.

I posted way before Bob that LMR as done in Fruit did not add to Fruits strength. I was pooh poohed.

There you are wrong. LMR in fruit _does_ work. But the history part of it is completely ineffective.

*************************************************************
I removed LMR from Fruit 2.1 and it did not loose any strength in my test.
*************************************************************

Then you didn't do it right, because I did the same thing last year and disabled it in Fruit, Crafty and Glaurung, and discovered it is worth about 30-40 Elo if you remove it without disabling null move. Without null-move, disabling LMR will cost about 80 in any of those. From testing on the cluster...

There were a few other things that made it into Crafty or was dropped from Crafty sometimes years after I had posted about them. If I had the time I would go back through all my post and catalog them.

I posted many times that various ideas were left in engines just because they did not seem to hurt. Much later Bob post the same conclusion.

Yet Bob posted once at the old winboard site that he did not believe that there was any possibility that there was anything of value that he could learn from me concerning chess programming.

Please try again, and provide the _exact_ quote. I believe I commented about RomiChess, where you were going on and on about how it would _eventually_ use its learning to beat crafty. It never did, because that type of learning is simply flawed.

*************************************************************
Well I am not going to bother, but we were not talking about learning at that time. My comment to you was something like, 'no wonder Crafty is falling out from among the top engines since you are not willing to explore
the ideas from new open source engines. So, there is not possibly anything you could learn from me or any other new guy on the block.
************************************************************
************************************************************
For what my learning was intended for--human/engine sparring--it is not flawed. It works perfectly!
***********************************************************

In which case you should be able to play 1,000 games against crafty 23.1, and then win the majority of the games after that. if it works "perfectly"???

But that is just how the world works!

Somewhat "ego-centric" today, are we???

************************************************************
Not really. Just frustrated and wondering why I bother when all my ideas are crap.
***********************************************************

Going on vacation now.

[Michael Sherwin]

Well, I did say that the learning was perfect for what it was designed for--engine/human sparring. However, no one seems interested in that.

So, just to humor the masses!

RomiChessX64P3k - Crafty-23.1-win64 : 21.5/221 8-186-27 (00000=0000=000=0000000000000=00010000010000000=000000000000000000000000=0=01==00000000000010000000000000001000000=0000000000000000000=00000=00=0=000010000=0=0000000000=0000=00000000=00000==0=000000000=01000==00000=00100=0) 10% -382

I was a bit too generous and allowed Crafty to use two threads on a Q6600. Crafty being a lot stronger now and using two threads makes it hard for Romi to gain points. I am using the book from the ftp site. I assume that that is okay. Crafty is varying its openings and not repeating openings in which it lost. Or at least it seems that way so I assume that the Crafty default is set up that way. Romi's percentage is climbing but, with only 221 games in it may just be noise. We shall see.

[Michael Sherwin]

I just happened to check the progress when Romi got to 11%. It seems that either Crafty is trying some inferior lines from its book or it is already running out of new lines to try. Which ever way it is Romi is taking Crafty's opening lines away from it one at a time as Romi learns to defeat each and every one of them. But, there is a long way to go and I may decide to run this match past the 2,000 games that Bob dictated.

RomiChessX64P3k - Crafty-23.1-win64 : 58.0/545 29-458-58 (00000=0000=000=0000000000000=00010000010000000=000000000000000000000000=0=01==00000000000010000000000000001000000=0000000000000000000=00000=00=0=000010000=0=0000000000=0000=00000000=00000==0=000000000=01000==00000=00100=0=1000=0000000000000=0100000000000000000000000=000000000=00000000==0000=0=000=0=000000=00=10000000100=00000000=0000=000001000000=001000000000000000000000=00000000000000100010=0000000=000000000000000000000000001000=1001000000=0000000=00=00=0=000=010000000100=00000000000001000000000000000000000000000000000100010=0=1=001010101) 11% -363

[bob]

The book from my ftp site is reasonably small to keep download times low. It is made from a fairly reasonable PGN collection, and then very restricted in how many times a move has to be played before it is included...

[Michael Sherwin]

Romi is a bit weak now to give Crafty a fair fight. If the gap in elo was 100

points or less then in my opinion Romi would be ahead in the match. So,

even if the learning is flawed against Crafty's methodology that does not

mean that it is with out value. As it stands now Romi is of benefit to your

team (and others) because, out of about every thousand games played Romi can show

you ~50 losses that you could use to tune your eval and opening book to

eliminate weaknesses in both. Said another way--Crafty with two threads

is far out searching Romi and yet there are weaknesses in Crafty that

allow Romi to win some games after a few tries. These wins by Romi could

be useful in finding those weaknesses!

[michiguel]

Romi is a bit weak now to give Crafty a fair fight. If the gap in elo was 100

points or less then in my opinion Romi would be ahead in the match. So,

even if the learning is flawed against Crafty's methodology that does not

mean that it is with out value. As it stands now Romi is of benefit to your

team (and others) because, out of about every thousand games played Romi can show

you ~50 losses that you could use to tune your eval and opening book to

eliminate weaknesses in both. Said another way--Crafty with two threads

is far out searching Romi and yet there are weaknesses in Crafty that

allow Romi to win some games after a few tries. These wins by Romi could

be useful in finding those weaknesses!

Try Gaviota. They are of similar strength with Romichess and Gaviota comes with books of two different sizes (or you can make one). In addition, you can turn book and positional learning on and off in the ini file (or in Engine settings in Winboard )

Miguel

[Michael Sherwin]

Romi is a bit weak now to give Crafty a fair fight. If the gap in elo was 100

points or less then in my opinion Romi would be ahead in the match. So,

even if the learning is flawed against Crafty's methodology that does not

mean that it is with out value. As it stands now Romi is of benefit to your

team (and others) because, out of about every thousand games played Romi can show

you ~50 losses that you could use to tune your eval and opening book to

eliminate weaknesses in both. Said another way--Crafty with two threads

is far out searching Romi and yet there are weaknesses in Crafty that

allow Romi to win some games after a few tries. These wins by Romi could

be useful in finding those weaknesses!

Try Gaviota. They are of similar strength with Romichess and Gaviota comes with books of two different sizes (or you can make one). In addition, you can turn book and positional learning on and off in the ini file (or in Engine settings in Winboard )

Miguel

When this match is finished I will test against Gaviota with all its learning turned on. Unless someone were to beat me to it.

[bob]

Romi is a bit weak now to give Crafty a fair fight. If the gap in elo was 100

points or less then in my opinion Romi would be ahead in the match. So,

even if the learning is flawed against Crafty's methodology that does not

mean that it is with out value. As it stands now Romi is of benefit to your

team (and others) because, out of about every thousand games played Romi can show

you ~50 losses that you could use to tune your eval and opening book to

eliminate weaknesses in both. Said another way--Crafty with two threads

is far out searching Romi and yet there are weaknesses in Crafty that

allow Romi to win some games after a few tries. These wins by Romi could

be useful in finding those weaknesses!

That kind of tuning is worthless. All it would do is to make crafty avoid playing a move that is actually not necessarily bad, without any understanding of why, other than a program played it so many times that it eventually learned a path to reach a victory. Several have asked me to add position learning back into crafty, which would likely just about eliminate this kind of learning result because Crafty would also be learning what to not repeat the next time around... But that's a different issue.

I don't believe Romi is finding "weaknesses". It is essentially doing random probing thru a large search space to find paths that are more promising, without having a clue about why they are promising.

There as another famous learning program, I think from the University of California (maybe Irvine campus, not sure). It suffered from the same flaw. And once the flaw is understood, a human will have no problems with the program, because he will not continue to play the same opening moves game after game waiting on the program to discover a way to bust it. And he isn't going to play it thousands (or tens of thousands) of games to give it that opportunity either...

That's why I said the idea is highly impractical. It is just as impractical as "position learning" unless you pair two programs that do this against each other. And then they are not learning how to play better chess. They are stumbling around until they fall into a win and then try to repeat that without any understanding of why. That's not what I call "learning".

[Michael Sherwin]

The search for alternatives that win is not random at all. The bonus/penalty is

very small and will lead the engine to try the second best move and then

back to the first move again with a try of a second best at another ply before

going to a third best move elsewhere. It therefore does a very systematic

exploration of the search space using very high quality alternative moves

before exploring lesser moves.

By the way, Romi has just achieved 12%!

RomiChessX64P3k - Crafty-23.1-win64 : 87.0/743 40-609-94

(00000=0000=000=0000000000000=00010000010000000=000000000000

000000000000=0=01==00000000000010000000000000001000000=0000

000000000000000=00000=00=0=000010000=0=0000000000=0000=0000

0000=00000==0=000000000=01000==00000=00100=0=1000=00000000

00000=0100000000000000000000000=000000000=00000000==0000=0=

000=0=000000=00=10000000100=00000000=0000=000001000000=0010

00000000000000000000=00000000000000100010=0000000=0000000000

00000000000000001000=1001000000=0000000=00=00=0=000=0100000

00100=00000000000001000000000000000000000000000000000100010=

0=1=0010101010=0=0000=0000000=0000000=0=0000000=000==0=00

=0000=0000=0=0=0=00=00=0=0=000000000=1100000000001000=000

0000=0100000000000100=00=0000100100000000000000000=0===0000

=000=0=0=000000000000100=0=0001=000101) 12% -346

[bob]

The search for alternatives that win is not random at all. The bonus/penalty is

very small and will lead the engine to try the second best move and then

back to the first move again with a try of a second best at another ply before

going to a third best move elsewhere. It therefore does a very systematic

exploration of the search space using very high quality alternative moves

before exploring lesser moves.

"systematic" in this case is essentially "random". That was my point. You don't try to figure out why a move didn't work, and try to choose other moves that appear to be better, you actually try moves that initially appeared to be worse, only because the first one failed.

To see the flaw in this, create a new book from Crafty from something like the "enormous.pgn" file that is available all over the place. Use this command:

book create enormous.pgn 60 1

Then you will see the flaw in this type of learning. If Crafty has a book of 20M + games to choose from, it is going to take tens of thousands of games to get back to something previously played...

By the way, Romi has just achieved 12%!

RomiChessX64P3k - Crafty-23.1-win64 : 87.0/743 40-609-94

(00000=0000=000=0000000000000=00010000010000000=000000000000

000000000000=0=01==00000000000010000000000000001000000=0000

000000000000000=00000=00=0=000010000=0=0000000000=0000=0000

0000=00000==0=000000000=01000==00000=00100=0=1000=00000000

00000=0100000000000000000000000=000000000=00000000==0000=0=

000=0=000000=00=10000000100=00000000=0000=000001000000=0010

00000000000000000000=00000000000000100010=0000000=0000000000

00000000000000001000=1001000000=0000000=00=00=0=000=0100000

00100=00000000000001000000000000000000000000000000000100010=

0=1=0010101010=0=0000=0000000=0000000=0=0000000=000==0=00

=0000=0000=0=0=0=00=00=0=0=000000000=1100000000001000=000

0000=0100000000000100=00=0000100100000000000000000=0===0000

=000=0=0=000000000000100=0=0001=000101) 12% -346