lithander wrote: ↑
Sat Feb 27, 2021 11:26 am
mvanthoor wrote: ↑
Sat Feb 27, 2021 10:37 am
I'm not going to fix Alpha 1 and do a complete new release for this, because in the beginning, the releases will probably be fairly regular; about once a month or so, maybe 6 weeks. In the future, if a release can take months because of testing, I'll probably start doing version x.1 hotfixes.
It's not like Rustic Alpha is becoming obsolete when you release the next version with transposition tables etc because the new version will play much stronger. Some of us
may like Rustic Alpha because it's current playing strength around 1600 is making it nice sparing partner whereas the next version would be just way too strong.
Though, personally I'm not concerned by the issue and just use incremental time controls. I've just reviewed my logs and never had an issue with Rustic losing because of timeout in that mode.
Yeah, you're right. There isn't really a reason not to backport the fix to Alpha 1 and release Alpha 1.1 for people who want to use that version of the engine. I can imagine that I just haven't used any GUI overhead protection, as it normally isn't an issue with incremental time controls (except if your increment is lower than 0.1 seconds or thereabout).
ydebilloez wrote: ↑
Sat Feb 27, 2021 9:49 pm
I like Rustic Alpha as it is way stronger as my engine, one some time controls at least. Why do I raise the issue?
Because I noticed that my upcoming version under certain time controls is about 400 elo stronger, while weaker on other time controls. So even if we only consider time controls as used on CCRL and others, it find it strange and an indicator of errors. (Pentium and Xeon scores are different as well !!!)
To give some examles Belofte 2.1.1 compared to 2.1.0 (partially tested on some levels)
# cutechess timecontrol (ref https://sourceforge.net/p/belofte/gitre ... version.sh
"tc=40/25" -> 400 elo stronger
"tc=inf/60+0.6" -> 300 elo stronger
"tc=40/120+2" -> 100 elo lower
"st=5" -> 100 elo lower
"tc=inf depth=4" -> 150 elo lower
I don't understand this post. Is it Belofte 2.1.1 against 2.1.0 that you are now comparing, so 2.1.1 is 400 Elo stronger than 2.1.0 on 40/25", or are you comparing against Rustic?
Belofte 2.1.1 isn't yet in your files download. I'm going to download version 2.1.0 and run some tests on my normal incremental time controls of 1m+0.6s (for quick testing if a change actually improves the engine), and 2m+1s (CCRL Blitz controls). For later versions, if Rustic gets into the 40/15 list, I may add tests at a comparable time control. (I'll have to ask how they determine the correct time control using Stockfish's benchmark.)
If Rustic is 400 Elo stronger than your engine on some time controls, and 100 Elo weaker at others, it is probably losing lots of games on time. If you have a better evaluation (more terms, probably) than Rustic, it can be the case that on fixed depth, your engine is actually stronger. When running on fixed depth, you're not comparing the complete engines; you're comparing the evaluations. I never use this for testing, because it's useless; Rustic only has material counting and PST's for now, so when tested against whatever engine that has a more extensive evaluation, it will lose; even if that engine is hundreds of points weaker in timed play.