Compile of Sashimi engine

[mvanthoor]

Cargo (or the package managers in general) aren't the problem.

In a way, they are - because they make dependencies "easy". The consequence is that people use them nilly-willy so that package managers create dependencies, and lots of them recursively. That's the same in every such ecosystem. You install some small package under Linux, and before you look, 120 packages are installed. You type NPM for "hello world", and 95 GB of JS trash get downloaded, then compiled for 7 days, and you end up with "only" 2 MB of JS code by nobody knows whom from all over the world.

Well... if you look at it like that, package managers help in the misuse of dependencies, but aren't the cause of the problems. I like package managers in general because they indeed make installing software and using dependencies easy. The one thing I DON'T like is that your software is installed or built from stuff gathered from over the entire internet, often without the option to save those downloads and reuse them, and if something changes, everything breaks. (The "stub installers" that are so popular nowadays are in the same boat. You start an installer, and it then downloads 4.5 GB of software when you just tick "C++ build tools". I'm looking at you, Microsoft.)

Search for the npm "leftpad fiasco" if you want to read an "entertaining" story. In short, some dude wrote a 10 line function to pad strings and numbers with some character, from the left and put it on the npm site. It got used in some HUGE world-encompassing projects; not used immediately probably, but through the dependency tree. At some point the guy got into an argument with the company maintaining NPM, and then removed his function from the registry. Half the internet and javascript/typescript world was broken for a few hours...

I have to look into Cargo's new "Vendor" option. It allows you download all the dependencies your program needs, and then stash them somewhere, so you don't need the internet to compile your program.

Now with websites, nobody cares whether the project will even build three weeks from now because by then, the whole website tech stack will have been replaced by one of the 200 new JS frameworks that will have appeared by then. Mozilla is part of this "move fast and break things" culture because nobody needs to build a three years old browser full of security holes. Even Mozilla's ESR cycle is just one year. It's the culture of an ecosystem that has neither use for nor appreciation of longevity.

I think that this is a real problem for a systems programming language. Dependencies in C and C++ are a hell; if you download some source code, you may never now what you're going to find because often, the dependencies are not in the repository. You'll have to scour the internet for just THAT version. Something like Cargo makes that much easier IF crates depend on specific versions. If they are configured like "Just use minimum version such-so to satisfy my dependencies", then it WILL break at some point if not maintained.

I've tried several chess engines written in Rust, and ALL of them have compilation problems because they depend on obscure unmaintained crates, or on just THIS experimental feature that just existed in THOSE three versions of Rust nightly and then got dropped.

The one reason I chose to do embedded software (for which there isn't a huge market anymore in the Netherlands compared to front-end and back-end internet work...) is because I can write a piece of software, and then have it exist for 20 years.

=== Some musings about longevity of software and the future. Skip if you're not interested. ===

I'm completely convinced that if there is ONE language in the world right now that COULD displace C at some point, it's Rust. The only reason why it probably won't, is because other languages such as Go are backed by huge corporations. And Microsoft has been experimenting with Rust, and they like it... instead of helping to improve it, they have started a similar language called "Project Verona". If that one ever lands, and MS positions it as a successor to C# and Visual C++, then THAT will be the new systems programming language for computers... for Android it'll be Go (because of Google), and Swift for iPhones (because of Apple). And C. Because some hardware is just too small to be able to run anything else but C.

I've decided to port my engine from Rust to C after it is finished. The programming paradigm used in Rust ("functions that work on structs", or "pseudo-object oriented programming") is very common in the C-world, and was what I've used often in the past. Therefore Rust was a natural fit for my coding style, and porting my engine to C will probably as easy as just typing the code again, 1-to-1 with some tiny changes to accommodate C. Then I'll maintain the engines next to one another.

The only reason why I'm doing that is just to have two sister engines in different languages. Maybe I'll even port it to FreePascal or Ada somewhere in the future... but it will NEVER be ported to Go. (Or any version of C#.) If it has a garbage collector, I won't be using it for personal projects, ever. These languages are too slow.

All of the 90's and 2000's things are on a decline, or being changed to what came before. What I mean is that, in the 90's and 2000's, we went to dynamic languages, almost without types (PHP, Javascript, Python...) and now, after software in those languages gets bigger and bigger, it is discovered that it's not maintainable or extendable. PHP now has type hints. Javascript is evolving into Typescript. Python has type hints. And so on... Even though C "won" in the 70's and the 80's because of raw power, and dynamic/loose-type languages "won" in the 90's and 2000's because they were easy to get started with, the language that actually gotten it right were Ada and Pascal: super strong typing (even stronger than Rust in some places), and fully static.

This is the reason why Ada is used in the automotive and aerospace industry.
Structured Text is used in PLC and factory/industrial programming; it basically is a standardized derivative of Pascal.

If you could add memory and thus thread safety to these languages, they'd be on par with Rust.

Raw power + memory safety + thread safety + no garbage collector (in whatever language) is the only way forward, IMHO. Some companies are porting their Node.js/Typescript and Python code bases over to Rust, and seeing multi-million dollars of savings per year, because the CPU- and memory requirements they had when running on cloud platforms dropped by like 80+%.

Why Discord is switching from Go to Rust

So Discord started out with a modern language... and it still switched to Rust, because Go is not fast enough and not safe enough. Imagine if you're coming from C#, or even Javascript or Python. As I said... a language such as Rust (but it doesn't necessarily have to be Rust itself...) is the only correct way forward.

[Ras]

like package managers in general because they indeed make installing software and using dependencies easy.

Which is why they at least exacerbate the problem they are trying to solve. However, most of the problems should vanish by themselves because Rust direly needs to stabilise both as language and with standard libraries, or else it will break itself out of existence. If your idea of an extended standard library collection gets implemented, that would at least solve most of the dependency issues.

The one reason I chose to do embedded software

My CT800 chess computer is actually an embedded system running on a microcontroller without OS. The UCI version is mainly the mockup for testing.

language that actually gotten it right were Ada and Pascal

Pascal was good as learning language, but the original standard Pascal was almost useless for real usage. Among the numerous defects was the rigid type system with no override, misunderstanding structured programming as dogma instead of guideline (hence no loop breaks or returns), misdesigned IO system, lack of bit operators, undefined order of evaluation in logical expressions and then some. Since the ISO standard specified a useless toy language, every implementation dealt with it by adding its own extensions. That lead to balcanisation of the language, which was one of the main reasons for its demise. If anything, Pascal was a cautionary example how not to do it.

And Ada isn't much used anymore even in the aircraft industry. Too much of a puzzle language, and the language crippled itself because back then, there was no free compiler available. It's mostly used in governmental projects when and because it's prescribed.

[mvanthoor]

Which is why they at least exacerbate the problem they are trying to solve. However, most of the problems should vanish by themselves because Rust direly needs to stabilise both as language and with standard libraries, or else it will break itself out of existence. If your idea of an extended standard library collection gets implemented, that would at least solve most of the dependency issues.

I hope Mozilla stops with that "we want a tiny standard library" crap. Now we have 74 crates competing to do the same thing. I think it's ridiculous I have to load Rand as a dependency (a crate which in itself is split into 10 crates or so) to get a random number generator.

What you want, is the hugest possible official standard library so people DON'T need to go and load external frameworks and crates to do things that any other language can do on its own. If you want to maintain a tiny core std, fine with me; but then also curate Crates.io and create a downloadable std extension package. Now I'm afraid to install any crate that doesn't have "The Rust Development Team" or "The ... Development Team" (being one of Rust's official sub-teams) as a developer. I'm not going to make any of my programs depend on RandomDude X's crate if I can't strip everything it uses from the program in a few hours if I ever need to.

Pascal was good as learning language, but the original standard Pascal was almost useless for real usage....

Now that you mention it, yes... you're right. The only Pascal I know is Borland Pascal. The later versions could do everything C did. The Pascal version that is "Pascal" for me, are Borland Pascal 6 and 7. They definitely did have bitwise operations, pointers, and even a complete graphics subsystem.

And Ada isn't much used anymore even in the aircraft industry. Too much of a puzzle language, and the language crippled itself because back then, there was no free compiler available. It's mostly used in governmental projects when and because it's prescribed.

Has it declined that much in current years? What I remember about it is that it was basically mandatory for any project where software needed to be mathematically proven correct, and Ada was the only language that was certified for this.

[Ras]

I think it's ridiculous I have to load Rand as a dependency

Maybe it's security geeks taking over. They often want to outright ban simple, but insecure PRNGs out of fear that someone just might accidentally use them for cryptographic purpose. In the domain Mozilla is working, that would be plausible, though still misguided. There should be something simple, but fast e.g. for games, and something secure right in the standard library.

What you want, is the hugest possible official standard library so people DON'T need to go and load external frameworks and crates to do things that any other language can do on its own.

I think C++ had that problem for a while, and it was first solved with Boost, and then most of that went into the STL.

The only Pascal I know is Borland Pascal.

Yeah I started with TP 3, and also worked up to 6 or 7, and the graphics was really cool. However, it was DOS only, and e.g. ST-Pascal or whatever Apple was using at the time was not compatible. TP never really made it post-Windows somehow.

What I remember about it is that it was basically mandatory for any project where software needed to be mathematically proven correct, and Ada was the only language that was certified for this.

When it's prescribed, it's used, especially on systems that are vital for aircraft safety (and trains, and weapons). But for anything that isn't vital, C and C++ have taken over, albeit with sophisticated tooling and testing to prove correctness. Basically anytime you can get away with not using Ada.

[mvanthoor]

I think C++ had that problem for a while, and it was first solved with Boost, and then most of that went into the STL.

Yes. C++ didn't have things such as threading, many types weren't standardized (because it needed to be compatible with C), so you could never be sure an int was actually the same size on different systems. Boost started to create abstractions that worked everywhere, and later either the Boost implementation itself or a similar one ended up in STL.

Yeah I started with TP 3, and also worked up to 6 or 7, and the graphics was really cool. However, it was DOS only, and e.g. ST-Pascal or whatever Apple was using at the time was not compatible. TP never really made it post-Windows somehow.

I forgot what the exact differences between Borland Pascal and Turbo Pascal (also from Borland) are. I don't even remember for sure if Borland C++ and Turbo C++ both existed. In version 5 (or 6?) Pascal changed into Object Pascal. That is the language later used for Borland Delphi. The same Delphi IDE, but with C++ as a language was Borland C++ Builder. I've worked -very- extensively with Borland C++ as an IDE while in university and traineeships. It ran circles around the Visual C++ versions of that time with regard to GUI programming and user friendlyness. Even to this day, I don't like Visual Studio.

When it's prescribed, it's used, especially on systems that are vital for aircraft safety (and trains, and weapons). But for anything that isn't vital, C and C++ have taken over, albeit with sophisticated tooling and testing to prove correctness. Basically anytime you can get away with not using Ada.

In many places where performance isn't absolutely critical, C and C++ have been displaced by C#, Java, or sometimes even Python. I'm fine with C# for everything that doesn't need the utmost in speed. Python is a language I will never voluntarily write software in. Slow, breaks compatibility to a HUGE existing code base without qualms, and irritating syntax.

Any language that runs in a VM or uses a garbage collector doesn't have my interest for things such as chess engines (or any other task that crunches numbers at top speed for a long time). I've done research between Go and Rust because I just didn't want to write YANEIC (Yet Another Engine In C), ans when I discovered Go had a garbage collector, it got dropped from memory. (Pun intended...)

[JohnWoe]

It requires cargo to build and lots of dependencies. No wonder it doesn't work. Never understood this cargo system. Only way to fix it would be pull all that dependency crap and build a static package.

I personally stay away from all libraries, frameworks (Especially JavaScript frontend frameworks ) and complex build systems where you pull crap over internet. They are all full of bugs and slow. A major pain in the ass to maintain manifests, installers, passwords, usernames ... to packaging websites etc.

To some extent, you're correct, but Cargo (or the package managers in general) aren't the problem. Cargo is just a package manager, just like apt on Debian, npm in node, pip for python, composer for PHP, and so on. It also works the same. It looks in cargo.toml, and sees that a program needs THIS, THAT, and SUCH. And then it downloads those dependencies and evaluates them, and if necessary, downloads more dependencies (recursively).

When everything it downloaded, cargo kicks off rustc, which is the compiler, which is also configured by cargo.toml.

So in essence, cargo is "make", with a built-in dependency downloader.

The problem is the fact that you can't just download the packages, and then put them somewhere for safekeeping. (PS: there is a way to download the ENTIRE crates.io website, but who wants THAT, apart from maybe a company?)

=== PACKAGE MANAGER RANT COMING UP. STOP HERE IF NOT INTERESTED. ===

What you should NEVER do, is create a dependency A that relies on dependency B, and then state "B needs to be at least version 1.2.0" or something like that. If you do so, Cargo will happily download version 1.2.375 of B if it exists, but if something has changed that breaks A, everything depending on A will break.

A dependency maintainer should take care that nothing breaks as long as he doesn't upgrade X in the x.y.z semantic version... but many programmers don't really care. (And libc is infamous for that. Search for libc/Linus Torvalds, where he has a HUGE rant about exactly this issue.)

This is not a problem with Rust. This is a problem with ANY system that uses package managers: dependency hell.

It is the one thing I don't like about Rust. Mozilla should just go and merge the top crates (RAND, Serde, Tokio, async, crossbeam, hyper, and so on) and everything they depend on, in an additional library download. Just download and install Rust version X, and if you then install the SLEC (Standard Library Extension Crates) package, you have like... 10, 20, 30...(or how many they'll put in) of the most well maintained and important libraries. In that case, many programs could be written without EVER having to download something from Crates.io.

It has been done before. C++ has a huge standard library, but before C++11, it was basically the default to install the BOOST library because it added so many things C++ was missing as compared to newer languages. Many people programming in C basically also installed BOOST, and then used bits of pieces from it. Technically, they were writing "C++" programs, but only using the C-subset of C++, and BOOST. After C++11 and C++14, is basically becoming obsolete.

I'd love to see the same thing happen to crates.io with regard to the big important crates. Keep Crates.io for people to share their stuff if they want, but if something appears that is huge, important or great, take it off and maintain it in an extension package to the standard library.

One example would be "if_chain". Rust can't chain some types of if-statements (if let...) because they're not expressions. They've been arguing for three years on how to fix that (or if they should, even). Not being able to chain "if let" with other if-statements makes the code ugly and unreadable. if_chain solves it:

Code: Select all

if_chain! {
	if x == 5;
	if y > 10;
	if let Some(a) == stuff
	if let Some(b) != whatever
	then {
		do_something()
	}
	else {
		burn()
	}
}

if_chain is beautiful if you need a statement that checks lots and lots of different types of "if" that need to be true. Instead of having this in a downloadable crate, Rust should just incorporate it into the language, in the standard library, or at least in a standard library extension.

Uh... rant over.

Cargo also rubygems they all feel like frameworks. You have to structure your code in certain way. Upload packages to some servers..
In Rust libc is the most downloaded package. Where you need C functions in Rust anyway?
I personally like all languages. For example about Java/Modern C++. Too verbose syntax. And you need globals, otherwise too much shitcode.
Python is great. All my games are written in Python.

I programmed a 3D renderer in 1 sloc. Yes, in 1 line of code. Just to show you don't need all this bloat. No libraries. Rotations, translations, physics engine, interactive and rendered in 3D. In JavaScript, my most beloved language: https://samuraidangyo.github.io/Cubes3D/

In Windows I remembered that you needed 500 lines of code to just put a pixel on screen.

I didn't find anything about Torvalds and glibc. I think even Linux need to forget plain C and go for C++. There is a big need of templates, namespaces, function overloading etc. The good parts of the C++. I think std+boost are too slow for kernel usage.

I just rewrote my compiler in C++. The old code was written in C. Memory management and string handling sucks in C. This is exactly what my language is all about == dynamic memory and strings. But man, in C++ that was easy. My rewritten language is about 20% of the original size. And much better. For example. Rust is implemented in C++ too. CPython is implemented in C. AST sucks tho.

[mvanthoor]

Rust isn't written in C++. The very first versions were written in OCaml, until it was possible to actually write a Rust compiler in Rust itself.

(This is called bootstrapping: the rustc compiler, written in Ocaml, compiles some Rust code... Which happens to be a rust compiler, that can compile that same Rust code. Then you gave the first compiler written in its own language.)

[JohnWoe]

Rust isn't written in C++. The very first versions were written in OCaml, until it was possible to actually write a Rust compiler in Rust itself.

(This is called bootstrapping: the rustc compiler, written in Ocaml, compiles some Rust code... Which happens to be a rust compiler, that can compile that same Rust code. Then you gave the first compiler written in its own language.)

Technically yes. LLVM is written in C++. Rust interfaces LLVM. So to compile Rust compiler you need the LLVM stuff aka binary.

For example. Start with empty computer. The you can't just grab the Rust source code. And boom. Magically it compiles the rust compiler from the Rust source code. You run it through the (rustc binary or what the fuck is the correct term anyway) and it produces the same binary. And 100x in a row succesfully.

My language is interpreted. To make it compile itself I would need to produce a static binary and convert the compiler source code to my language. Which is not interpreted language anyway. Maybe even irreducible complexity

[Pi4Chess]

So tried to compile Sashimi installing Cargo on my Rpi4 and did not managed to because i am getting the following errors :

Sashimi : error E0554 related to feature(integer_atomics2) in mcts/src/lib.cs