Just found a 55 elo bug in SlowChess 2.1

[Alayan]

History is worth hundreds of elo, a bug there costing 50 elo is nothing outlandish.

[Gary Internet]

Brilliant news. Well done. You just need to ensure SlowChess can run on Linux on many threads and then you can start competing at TCEC and CCC. As you say, you should be competing with Xiphos 0.6 on a single thread, which is very nearly the same level as Fire 7.1 on a single thread.

[mar]

History is worth hundreds of elo, a bug there costing 50 elo is nothing outlandish.

That's a bold claim. I very much doubt that. Even if your "hundreds" would be 200, I still doubt that.
Even if you use history for other decisions but move ordering.
LMR alone is worth what, 100-200 max perhaps, still not "hundreds". I very much doubt that history is worth more than LMR.

[mvanthoor]

I tried Rust, briefly, after looking at some Rust evangelist video on Youtube. I just wanted to have enum for color, piece, square, file, rank, square:

• you can't even enumerate like in C. You have to write the values of each square one by one. What's the point of an enum that can't even enumerate ?
• then I trued to operate on them. You can't. It's unsafe, of course, what if the result of the operation is not a legal enum value. Ok, so I tried to cast them, can't do that either. Right, so I tried to enable operators on them. Still not.

That's where I gave up on Rust.

You can enumerate like in C. You can also cast an enum to an int and back again. (At least, in version 1.0 and up. I don't know before 1.0). I didn't say it can't be done: it's just _incredibly_ annoying in this particular case. An enum isn't an int. Therefore you'll be casting back and forth all the time (because you need the enum as an integer index, and the index needs to represent one of the enums variants), which makes code very verbose.

It this point it's the only drawback I've found for writing a chess engine. I just use constants now. In C, the enums basically are often used as "automatically initialized constants." Rust's enums are more powerful, but they're types in their own right, and thus need to be treated as such.

It's not impossible; it's just annoying in the case of writing a chess engine, where it would be convenient to have an enum that is interchangeable with an integer. It's the flip side of the safe type system; Rust doesn't implicitly convert between ANY types.

So Rust is safe, because it doesn't allow you to do anything, so that you can't do anything dangerous. No data races, if you can't share memory across threads, no risk of going out of bouds of an enum if you can't do any operations on it, etc.

At least C++ allows you to escape that type safety non-sense when you need it. You can use (unsafe) C-style enum, or type-safe C++ enum, but you can still define operators on them (and assert there to check bounds).

If you use "unsafe { }", Rust disables its internal borrow checker and static code analyzer, and lets you do anything C can do. If you don't want to do it yourself, there are crates such as crossbeam and others, that hide all of the unsafe complexity and provide a safe API to use. The language also has things such as messages and channels to share data between threads in a safe way.

Time wasted fighting against the language to do anything vs. time avoided debugging: I'm not convinved.

Then I tried Go, which is much nicer. But when I saw the size of statically linked Go executable, I gave up on go, and went back to C.

Apart from trying to use enums in a way they're not intended to be used (in Rust), and using a single line of "unsafe { }" to force the language to not unnecessarily initialize a list that I'm going to initialize myself on the next line, I haven't been fighting the language at all. I'm just writing code in idiomatic Rust, and it 'just works', with the added benefit of the compiler checking if I'm not doing anything stupid like use after free.

If you try to write code as you would do in C, then yes... you'll probably fight the language. The concept of "claim, use, free" memory as you would do in C doesn't exist in Rust. The owner/borrow concept is a different way of thinking about memory management. You either like it... (as I do), or you don't. They way Rust uses memory is the way I've been trying to use it myself in C since forever: one owner; many readers; one writer and no readers. It served me well in C, but now that concept is automatic as it's part of the language.

Go is just "C for the Internet" IMHO. It omits WAY to many features in an effort to keep the language as simple as possible. It has a garbage collector, but isn't as safe as other garbage collected languages like C# (and it's not faster as far as I've seen), and it can't do what C can. I'd rather write my code in C. If Rust hadn't existed, I would.

[mvanthoor]

Time wasted fighting against the language to do anything vs. time avoided debugging: I'm not convinved.

Absolutely, why bother with safety at language level when we have ASAN (and other sanitizers)?
There's always a price to pay (performance and/or friction).

Because the type safety and the code analyzer are built into the language as integral parts, the compiler can tell you exactly what is wrong, where it is wrong, why it's wrong, AND how to fix it, along with a suggestion or code snippet on how to accomplish the fix. If you have never seen a Rust error message, you should. They're awesome. The new IDE plugin rust_analyzer already uses parts of this system in addition to some of its own tricks; most of the time it can already tell you code isn't going to compile and why, even before you try.

From what I understand, Rust compiles even slower than C++, so I'm not sold at all. Slow compile times are #1 reason to look for an alternative,
at least for me.

Naturally for a chess program, compile speed doesn't matter much and C is an excellent choice I think.
Actually anything that gives you native performance.

True; in the beginning, Rust did compile very slowly because of the borrow checker and static code analyzer. Since some time, much work has been done: a new borrow checker that is now used by the static code analyzer, and the analyzer itself has been highly optimized in the last two years or so. I can't compare directly with C++, but Rust has become MUCH faster than it was in the past, and now it doesn't feel slow; at least not compared to, for example, C# or a Typescript build, which I routinely do at work.

As for 50 elo bugs: never heard of anything like that related to strong engines (we measure improvements, right?).
Can happen at lower levels for sure, but I don't believe in fairy tales.
From my experience, typically any bugfix was no measurable elo gain or barely anything significant at all. 50 elo is huge. (nullmove itself is like what, 70?)

People on this forum have repeatedly stated that engines that have a lot of features, but are still weak, probably contain a lot of bugs in those features too. Therefore, a buggy implementation of a feature could, theoretically, still gain you 20 ELO... and when you fix the bugs, it would gain 70 ELO. Then you have a 50 ELO bugfix.

[mar]

Because the type safety and the code analyzer are built into the language as integral parts, the compiler can tell you exactly what is wrong, where it is wrong, why it's wrong, AND how to fix it, along with a suggestion or code snippet on how to accomplish the fix. If you have never seen a Rust error message, you should. They're awesome. The new IDE plugin rust_analyzer already uses parts of this system in addition to some of its own tricks; most of the time it can already tell you code isn't going to compile and why, even before you try.

Well, C++ has very good type safety as well as type safe enums, clang produces very good and verbose output if something goes wrong.
Type safety is not the problem, memory safety is (this is why I mentioned ASAN).
(sure, you can still cast to void *, but nobody does that in C++; in Rust you can use unsafe blocks if I'm not mistaken, the same goes for C# and orther languages AFAIK, I see no real difference in being unsafe by default [C++] vs using unsafe blocks in otherwise safe languages)

True; in the beginning, Rust did compile very slowly because of the borrow checker and static code analyzer. Since some time, much work has been done: a new borrow checker that is now used by the static code analyzer, and the analyzer itself has been highly optimized in the last two years or so. I can't compare directly with C++, but Rust has become MUCH faster than it was in the past, and now it doesn't feel slow; at least not compared to, for example, C# or a Typescript build, which I routinely do at work.

One would have to compare equivalent large programs to say for sure, this is what I read some time ago (Rust being slower/on par with C++, which is still slow).
I don't know about typescript, but C# certainly compiles much faster than C++.

People on this forum have repeatedly stated that engines that have a lot of features, but are still weak, probably contain a lot of bugs in those features too. Therefore, a buggy implementation of a feature could, theoretically, still gain you 20 ELO... and when you fix the bugs, it would gain 70 ELO. Then you have a 50 ELO bugfix.

Right, but SlowChess is a very strong engine. I was talking lucky 50 elo bugfixes in 3k+ engines. Not many silver bullets at this level that add 100+ elo to offset a 50 elo loss.

[Alayan]

History is worth hundreds of elo, a bug there costing 50 elo is nothing outlandish.

That's a bold claim. I very much doubt that. Even if your "hundreds" would be 200, I still doubt that.
Even if you use history for other decisions but move ordering.
LMR alone is worth what, 100-200 max perhaps, still not "hundreds". I very much doubt that history is worth more than LMR.

Removing history updates from Ethereal lost 760 elo at STC : http://chess.grantnet.us/viewTest/4369/

Obviously, Ethereal uses history counters for some pruning decisions, so the result is worse than having an engine that doesn't depend on history anywhere. I stand by "hundreds" in any case. Move ordering is critically important. for AB, for LMR, for QMP...

The loss from removing LMR was measured at 248 elo +-10 (95%), QMP at 175 +-7 and NMP at 93 +-5.

[mvanthoor]

Well, C++ has very good type safety as well as type safe enums, clang produces very good and verbose output if something goes wrong.
Type safety is not the problem, memory safety is (this is why I mentioned ASAN).
(sure, you can still cast to void *, but nobody does that in C++; in Rust you can use unsafe blocks if I'm not mistaken, the same goes for C# and orther languages AFAIK, I see no real difference in being unsafe by default [C++] vs using unsafe blocks in otherwise safe languages)

The one difference is that an unsafe, but "working most of the time" C++ program can probably still compile and run. In Rust, it is just not possible. That isn't to say that there will never be any problems: it's VERY possible, for example, to index an array out of bounds, but the program will abort immediately. It will not continue if determined that operation has become unsafe.

The point of my post was is that there is no longer a reason you have to fight Rust as a language. Before 2015, you had to do some weird stuff and jump through hoops to accommodate Rust's borrow checker and static code analyzer. Since then, this has become better, and after the replacement of the original borrow checker and updates to the code analyzer, you can *almost* just write code and know it's safe and will do exactly what you intend, barring external circumstances. (You shouldn't go and pull a network plug while the program is running; you'd definitely need to handle that to avoid a panic. Obviously.)

At this point, Rust is a capable alternative for either C or C++, and some of it's philosophy is different; you either like it, or you don't. In the first case you use it, in the second, you don't

[mar]

Removing history updates from Ethereal lost 760 elo at STC : http://chess.grantnet.us/viewTest/4369/

Obviously, Ethereal uses history counters for some pruning decisions, so the result is worse than having an engine that doesn't depend on history anywhere. I stand by "hundreds" in any case. Move ordering is critically important. for AB, for LMR, for QMP...

The loss from removing LMR was measured at 248 elo +-10 (95%), QMP at 175 +-7 and NMP at 93 +-5.

Let's play along and let's pretend that large elo gaps are reliable and that self-play doesn't inflate:
this would imply that without history, Ethreal would be a 2500 elo engine (assuming CCRL scale). that's completely ridiculous of course.
So you've proven only one thing: that you managed to completely break Ethereal

[Alayan]

It doesn't matter if the gap really is 700 elo or 800 elo. It doesn't matter if it would turn to 600 elo at LTC, or to 500 elo against RofChade. That doesn't change my claims about the importance of history heuristics.

LMR completely falls apart if your move ordering is terrible, as LMR is all about reducing moves that aren't don't get searched early. If you don't have history, but only ttmove/killers/good captures/bad captures for ordering, your ordering will be awful. In many positions, your move generator would put good quiet moves towards the end where they'll get reduced to death. I wouldn't be surprised if LMR becomes elo-losing without any history.

The stronger an engine, the more removing history hurts the search. History alone might not give all the elo, but without history you can't get it.

When jonkr releases SlowChess 2.2, you'll be able to see for yourself if the elo gain is real or not...