Re: Progress on Rustic
Posted: Thu Sep 24, 2020 9:43 pm
On sept 19 I posted that I implemented message passing between threads.
Now I have an active communication module (in its own thread) that reads incoming stuff from stdin. There are three modules:
- console (this is the only one implemented now)
- uci
- xboard
The incoming stuff is transformed into something the engine understands. It means that the consoel, uci and xboard modules all translate incoming data to the same format. The engine sends data into the modules, and each translates it for its own use, in either the console, uci, and xboard protocol, and outputs it to STDOUT
It's more work than just interweaving uci throughout the entire engine and be done with it, but this way, the engine can support any current and future protocol by just writing a new module for it.
So far so good.
I also wanted to have a Search that acts as a main controller in its own thread. For now, the search will run directly in that thread, but later, this concept can be extended by having worker threads attached to the controller. (So the engine can send "Start" and "Stop" to the controller, and it starts and stops the search accordingly. Or it quits all workers and itself completely when it receives "Quit".)
Again, so far so good. Working fine...
But now I end up with two message receivers in my engine:
comm_rx receives messages from the communication module.
search_rx receives (intermediate) results from the search controller.
So now I have the problem that I have TWO blocking receivers in the engine's main thread, and the thread won't continue until BOTH are unblocked. I don't want non-blocking reception, because that entails polling, and thus the main engine thread will be active all the time.
Enter "select!"
Rust has a construct that is able to unblock a thread, as soon as one of the waiting receivers gets a message. The thread continues, so the message in the receiver can be handled, and then the thread blocks again. Great. That's what I need.
And then they go and deprecate this functionality because it's written in unsafe code a few years ago, and they don't have a replacement yet.
QUE?!
So since Rust 1.32, the language has no "official" way of handling more than one receiver in a thread, and there are multiple competing implementations by several people and project groups. It was actually suggested to just rip all the channels out of the standard library, in the ongoing effort to keep it as small as possible. (Then why not dispense with it entirely?! Sjeess...)
The solution, apparently, is to use "crossbeam-channel", part of the "crossbeam" crate.
Now some people want to throw out Rust's entire threading API out of the standard library, and replace it with crossbeam. Then there's also mio, Tokio, std-async, Futures (both a version in the standard library, and a crate called Futures), chan, and new-channel and probably others.
Why doesn't Rust learn from Javascript and C++, in heaven's sake?
C++ had a fairly small standard library, but since some time, Boost is basically required as an addition. Therefore at some point, C++ acquired the same functionality as Boost in the standard library, and if I remember correctly, it even just takes parts of Boost INTO the standard library.
Same with Javascript. jQuery became so omnipresent, that Javascript ES6 is basically a re-implementation of jQuery's functionality directly into the language itself. jQuery, by all intents and purposes, is now subsumed and outdated.
Why doesn't Rust do the same thing? Implement a small, working feature in the standard library that people can use. If an implementation comes along that is used by MILLIONS, to the extent that it actually becomes the de facto default, it SHOULD replace the functionality in the standard library, officially. But no, Rust still wants to have a tiny standard library, and on crates.io, people and entire project groups keep fighting to make their implementation the dominant one. So in time, we'll have 75 ways of doing one thing in Rust.
And yes, I'm now basically forced to pull another huge dependency into my engine to get one tiny threading feature, that was available but isn't anymore (or isn't going to be, in the near future).
Maybe Rust needs to become a lot older (It's 10 years old, but only at version 1.0 for about 5), and much more heavily used in companies. At that point, you can't get away with stuff like that; somehow they WILL have to give their mandate on what crates are "officially maintained" by the Rust foundation (and thus can be considered 'standard'), because multi-billion dollar companies are not going to build their programs on top of a crate created by John Doe down the street.
Now I have an active communication module (in its own thread) that reads incoming stuff from stdin. There are three modules:
- console (this is the only one implemented now)
- uci
- xboard
The incoming stuff is transformed into something the engine understands. It means that the consoel, uci and xboard modules all translate incoming data to the same format. The engine sends data into the modules, and each translates it for its own use, in either the console, uci, and xboard protocol, and outputs it to STDOUT
It's more work than just interweaving uci throughout the entire engine and be done with it, but this way, the engine can support any current and future protocol by just writing a new module for it.
So far so good.
I also wanted to have a Search that acts as a main controller in its own thread. For now, the search will run directly in that thread, but later, this concept can be extended by having worker threads attached to the controller. (So the engine can send "Start" and "Stop" to the controller, and it starts and stops the search accordingly. Or it quits all workers and itself completely when it receives "Quit".)
Again, so far so good. Working fine...
But now I end up with two message receivers in my engine:
comm_rx receives messages from the communication module.
search_rx receives (intermediate) results from the search controller.
So now I have the problem that I have TWO blocking receivers in the engine's main thread, and the thread won't continue until BOTH are unblocked. I don't want non-blocking reception, because that entails polling, and thus the main engine thread will be active all the time.
Enter "select!"
Rust has a construct that is able to unblock a thread, as soon as one of the waiting receivers gets a message. The thread continues, so the message in the receiver can be handled, and then the thread blocks again. Great. That's what I need.
And then they go and deprecate this functionality because it's written in unsafe code a few years ago, and they don't have a replacement yet.
QUE?!
So since Rust 1.32, the language has no "official" way of handling more than one receiver in a thread, and there are multiple competing implementations by several people and project groups. It was actually suggested to just rip all the channels out of the standard library, in the ongoing effort to keep it as small as possible. (Then why not dispense with it entirely?! Sjeess...)
The solution, apparently, is to use "crossbeam-channel", part of the "crossbeam" crate.
Now some people want to throw out Rust's entire threading API out of the standard library, and replace it with crossbeam. Then there's also mio, Tokio, std-async, Futures (both a version in the standard library, and a crate called Futures), chan, and new-channel and probably others.
Why doesn't Rust learn from Javascript and C++, in heaven's sake?
C++ had a fairly small standard library, but since some time, Boost is basically required as an addition. Therefore at some point, C++ acquired the same functionality as Boost in the standard library, and if I remember correctly, it even just takes parts of Boost INTO the standard library.
Same with Javascript. jQuery became so omnipresent, that Javascript ES6 is basically a re-implementation of jQuery's functionality directly into the language itself. jQuery, by all intents and purposes, is now subsumed and outdated.
Why doesn't Rust do the same thing? Implement a small, working feature in the standard library that people can use. If an implementation comes along that is used by MILLIONS, to the extent that it actually becomes the de facto default, it SHOULD replace the functionality in the standard library, officially. But no, Rust still wants to have a tiny standard library, and on crates.io, people and entire project groups keep fighting to make their implementation the dominant one. So in time, we'll have 75 ways of doing one thing in Rust.
And yes, I'm now basically forced to pull another huge dependency into my engine to get one tiny threading feature, that was available but isn't anymore (or isn't going to be, in the near future).
Maybe Rust needs to become a lot older (It's 10 years old, but only at version 1.0 for about 5), and much more heavily used in companies. At that point, you can't get away with stuff like that; somehow they WILL have to give their mandate on what crates are "officially maintained" by the Rust foundation (and thus can be considered 'standard'), because multi-billion dollar companies are not going to build their programs on top of a crate created by John Doe down the street.