Kieren Pearson wrote: ↑Sun Aug 30, 2020 4:03 pm
Do you use an opening book for your tournaments?
Of course. It's a must to use an opening book to get variety.
I am using 3moves books with thousands of openings. I made some tests with other books and found out, that actually it's mostly luck playing just 100 games. That's why the error is so high.
Before I rented this 32-core computer, I never run more than 100 games. (needed too much time). Now I know that I need as least 2000 games, better 5000 games, too get a safe conclusion.
I have seen +100 Elo in the early part of a contest (200-300 games) for a change with a net Elo change of +5.
I think when we make a change and see a big jump, we get very excited. That makes these starts stand out and we remember them more than the ones that do not start that way. So kind of a psychological factor.
When I first made smooth scaling null move pruning for Stockfish, after 100 games it as some incredible thing like +150 Elo, but it settled down to a lot less than that.
Naturally, I jumped on the internet to tell everyone of this incredible advance after I saw the early results. It turned out to be a lot more ho-hum than I had hoped for.
So when I see a change that looks tremendous early, I always try to temper my expectations.
I tried an eval hash for Hakkapeliitta which was even worse. It started at +50 Elo, and ended at zero. That was disappointing
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Dann Corbit wrote: ↑Mon Aug 31, 2020 3:38 am
I have seen +100 Elo in the early part of a contest (200-300 games) for a change with a net Elo change of +5.
I think when we make a change and see a big jump, we get very excited.
...
Naturally, I jumped on the internet to tell everyone of this incredible advance after I saw the early results. It turned out to be a lot more ho-hum than I had hoped for.
Very true. Look at this thread!
"OliThink 5.4.0... big leap...blabla"... Actually now I know that the ELO went 9 down with that version. (see table below)
I finished the Intra-OliThink-tourneys for Version 5 and you can see that this happened quite often.. But not again since 5.6.4. That's when I got the test server.
When I initially started testing changes in games for elo gain, I was just playing around 1000 or 2000 games, and if the elo difference was small sometimes another couple thousand to verify, and that was enough to accumulate elo gaining changes and make good progress. Possibly a lot of the changes weren't actually winners, but it was more good then bad, so progress was quick.
Now though it's a different story... if I get +8 elo on a change after 2000 games, then +4 elo after 4000 games, feels like it's likely an even or negative change, and I would have to play even more games (and maybe try different time controls like the standard LTC test, or different opening set) to have confidence in it. I think a big part of the reason for this originally there was nothing but intuition or a couple of test positions behind why the code was the way it was, now that there's in game testing behind a lot of it, getting lucky is not so likely... It tested well the way it is at least once maybe more, so something different is more likely to be luck and less likely to be actually good.
On the subject of opening books, I do find that even though there can be some randomness in the moves programs choose, going through the same section of the test book often gives similar results. Sometimes the first parts of the test book can start becoming the strongest because I'm not always running full tests with the same amount of games for all changes while I'm working. (Possibly from being actually better at those openings, possibly just from arranging values things so that what amounts to random luck in move choice is leading to better results.)
OliverBr wrote: ↑Sun Aug 30, 2020 11:52 am5.6.9 has been released. It gains about 25 ELO to 5.6.8, thus OliThink finally has more than ELO 2600.
Gauntlet running in TLCV.
If you install TLCV (Tom's Live Chess Viewer) on your computer, you can watch the games live move by move.
download/file.php?id=42959
Host - GrahamCCRL.dyndns.org Port - 16091
Unfortunately I had to remove 5.7.0 because of a mini-bug. It only affected pinned-pawn-under-promotions, which are extremely rare.
Still, a bug is a bug, so I have released 5.7.1 together with some change in time management.
OliverBr wrote: ↑Thu Sep 03, 2020 9:53 pm
I included the header and released OliThink 5.7.2. Everything is smaller now, code size (45k, 1615 lines), executable size (90kb), memory/cache usage, time usage.
Here is a mini tourney against friends.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Ok, it was a fast game, and Leela didn't have a GeForce 1080Ti this time, but still a decent AMD GPU with RX 560.
One position was her downfall on move 51 for white:
[d]2Q1b3/6pk/5p1p/P7/8/3N2PP/4qP1K/8 w - - 1 51
In post game analysis , Leela insists for a long time on white playing a6, while the clear (and very obvious) winner is Qf5+. She needs up to 1 minute thinking time to see this.
Is this Leela trolling again? Give away the knight without any need?!
