Compiler Speed Nowadays

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

mar
Posts: 2554
Joined: Fri Nov 26, 2010 2:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Compiler Speed Nowadays

Post by mar »

jshriver wrote: Wed Dec 30, 2020 7:14 am Eclipse is pretty good and cross environment. I started using it for Java but they have a C/C++ version as well. Personally I say use whatever is most comfortable for you and effective. Goal is to fight the task at hand, not the tools to get there. Cheers.
Thanks, I'm aware of Eclipse, but when I tried it some time ago it seemed quite sluggish to boot up, no idea if something has changed since then, it's been years, but those Java based IDEs are typically like this (and also memory hogs).

I've just tried CodeLite (again) after some years and it seems like what I've been looking for. Could use a couple more features and better project management,
but I've finally managed to (well, manually) specify correct library linking order. Had some issues with this in the past because GNU linker needs a specific linking order (reverse order of dependencies).
Martin Sedlak
Joost Buijs
Posts: 1563
Joined: Thu Jul 16, 2009 10:47 am
Location: Almere, The Netherlands

Re: Compiler Speed Nowadays

Post by Joost Buijs »

flok wrote: Tue Dec 29, 2020 9:31 pm
Joost Buijs wrote: Tue Dec 29, 2020 5:43 pm To be honest I have the feeling that development tools under Linux are so cumbersome compared to Visual Studio that I don't want to step away from it. When I really like a piece of software I'm always willing to pay for it, I don't want everything for free.
I have that feeling with visual studio: "you can do everything with it!" "oh, no you can't do that, maybe if you buy x" "oh and you can't do that, oh and you can't script around it". "haha and of course you can't use vi/emacs as your editor, the build-in notepad is perfect!".
You clearly have no idea what you are talking about. Despite having bugs nothing comes close for development.

You can even use it to develop software for Linux by using a SSH connection to a remote Linux box or by installing WSL 2 on your local Windows box that will run most flavors of Linux in a Hyper-V VM, and most of it's functions keep working as it is.

And who knows, maybe one day Microsoft will bring VS to Linux, like they did with VS code.
Sesse
Posts: 300
Joined: Mon Apr 30, 2018 11:51 pm

Re: Compiler Speed Nowadays

Post by Sesse »

mar wrote: Wed Dec 30, 2020 4:30 am The only thing I'm missing on Linux is a good C/C++ IDE.
VS (despite the insane amount of bugs) is so far the best IDE out there. Tried CodeBlocks, CLion, QtCreator, XCode on Mac, nothing comes even close.
Did you ever try VSCode? (I know it's the same only in name, but it seems to be popular on both Windows and Linux alike.)
The most important part (for me) is actually debugging, this is where VS is exceptional. natvis (pretty printing custom types) + conditional breakpoints, watches that actually work and so on.
I find it interesting how much Visual Studio users depend on online debugging; perhaps because actually outputting and grepping through a log is so painful in that environment. It's a circle (I won't say whether it's vicious or virtuous); VS users tend to use the debugger a lot, which makes them more dependent on the VS debugger. (Even back when I used Visual Studio, I didn't use single-stepping and the likes too much, so this wasn't hard for me to give up.)
Sesse
Posts: 300
Joined: Mon Apr 30, 2018 11:51 pm

Re: Compiler Speed Nowadays

Post by Sesse »

mar wrote: Wed Dec 30, 2020 7:31 am Had some issues with this in the past because GNU linker needs a specific linking order (reverse order of dependencies).
If you don't want to sort your dependencies topologically, you can use linker groups. Or the hackish way of specifying everything twice :-)

Of course, this will be slower, but LLD is already easily ten times as fast as the MSVC linker, so it's possible you won't care much…
User avatar
flok
Posts: 481
Joined: Tue Jul 03, 2018 10:19 am
Full name: Folkert van Heusden

Re: Compiler Speed Nowadays

Post by flok »

Sesse wrote: Wed Dec 30, 2020 12:10 pm I find it interesting how much Visual Studio users depend on online debugging; perhaps because actually outputting and grepping through a log is so painful in that environment. It's a circle (I won't say whether it's vicious or virtuous); VS users tend to use the debugger a lot, which makes them more dependent on the VS debugger. (Even back when I used Visual Studio, I didn't use single-stepping and the likes too much, so this wasn't hard for me to give up.)
It's something I don't understand: you have the source-code in front of you, you can READ what's going on right? You know how an instruction behaves, why use stepping? Maybe for really specific special cases, but not for something like a chess program.
In my time, we used printf and grep as you also mentioned. And we liked it that way.
Also for timing-related things it sometimes preferred over single stepping for obvious reasons.
mar
Posts: 2554
Joined: Fri Nov 26, 2010 2:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Compiler Speed Nowadays

Post by mar »

Sesse wrote: Wed Dec 30, 2020 12:10 pm Did you ever try VSCode? (I know it's the same only in name, but it seems to be popular on both Windows and Linux alike.)
nope, but I'm aware it exists. I thought it was a code editor more than a full-blown IDE.
I find it interesting how much Visual Studio users depend on online debugging; perhaps because actually outputting and grepping through a log is so painful in that environment. It's a circle (I won't say whether it's vicious or virtuous); VS users tend to use the debugger a lot, which makes them more dependent on the VS debugger. (Even back when I used Visual Studio, I didn't use single-stepping and the likes too much, so this wasn't hard for me to give up.)
because it saves a ton of time and pain, of course :)

I'm not a fan of invasive log/printf spamming => you have to modify the code, build and clean it up afterwards (ok, trivial with git, but still completely unnecessary).
I don't see how it's superior to non-invasive debugging. it's not just about stepping, like I said - data access breakpoints/conditional breakpoints or even break into the debugger in the case of a crash/assert, where you can immediately examine the context (threads,callstacks, variables,...), this is powerful, fast and very useful, actually.

also - how do you log a certain container, for example? let's say something happens 60 times per second under some specific conditions and you want to examine say a hashmap somewhere, "printf debugging" implies adding extra code to look something up (if you know the value of the key), if you want the whole hashmap you printf the whole thing? well good luck digging through that :)

and I haven't even mentioned cases where you want to debug heavily multithreaded code.

I still use printf debugging sometimes (like scripts), but my scripting language has a full RTTI support so I can print the whole type (say a 4d vector) in one go, this is one of the many reasons why we need a proper RTTI in C++ - have high hopes for metaclasses :)
If you don't want to sort your dependencies topologically, you can use linker groups. Or the hackish way of specifying everything twice :-)
Thanks for the tips (specifying everything twice looks like a very nice hack to me - didn't know that :)
on the other hand manually specifying 10 libraries is not much of a problem if the order is sort of obvious

oh and as for lld - in my experience microsoft linker does a much better job at removing unused code and doesn't actually require any specific order (assuming static linking)
Martin Sedlak
Ras
Posts: 2487
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: Compiler Speed Nowadays

Post by Ras »

flok wrote: Wed Dec 30, 2020 12:39 pmIt's something I don't understand: you have the source-code in front of you, you can READ what's going on right? You know how an instruction behaves, why use stepping?
I also find debuggers mostly useless, except for rare cases such as a buffer overflow that overwrites something else which crashes later in an unrelated part of the code. But then I just use GDB on the command line because I find that easier than graphical IDEs. In general, I find it more productive to read the code and think about it. That helps also with sporadic bugs that don't show up in the dev environment and where you don't have access to the prod environment - which is common in embedded systems.
Rasmus Althoff
https://www.ct800.net
benb
Posts: 16
Joined: Sat Dec 12, 2020 5:29 am
Full name: Ben Bradley

Re: Compiler Speed Nowadays

Post by benb »

I'm new here, poking around, and I saw this:
mar wrote: Wed Dec 30, 2020 4:30 am as for the OP's question: try and see, msc, gcc, clang, whatever works best for you. compilers gotten really amazing these days.
and, if you don't already know, try godbolt.org, where you can examine the assembly generated by various compilers
It may be hard to get a good feel for all the things compilers can do just by poking around with that site. Here's a video by the site's creator where he describes many amazing and wonderful things modern compilers do. I remember decades ago writing code to second-guess what the compiler will do (and the real test was "looking under the hood" at the generated assembly code), but nowadays, with full optimization on, compilers are way too good to be second-guessed. Part of what I get about this is compilers also "know" the target instruction set as well as expert assembly programmers.
The first 14 minutes or so is an intro to Intel x86/64 asm, the good part starts around 14:20.

Speaking of the popcount instruction, it's covered in this video, which very briefly mentions chess engines and bitboards at 5:18:
Sesse
Posts: 300
Joined: Mon Apr 30, 2018 11:51 pm

Re: Compiler Speed Nowadays

Post by Sesse »

mar wrote: Wed Dec 30, 2020 5:29 pm I'm not a fan of invasive log/printf spamming => you have to modify the code, build and clean it up afterwards (ok, trivial with git, but still completely unnecessary).

I don't see how it's superior to non-invasive debugging. it's not just about stepping, like I said - data access breakpoints/conditional breakpoints or even break into the debugger in the case of a crash/assert, where you can immediately examine the context (threads,callstacks, variables,...), this is powerful, fast and very useful, actually.
I don't think it's a matter of one being superior to the other. You can never really get to the bottom of the arsenal of debugging techniques, so yes, I do use a debugger from time to time. But one's debugging style does indeed tend to get colored by whatever tools are the most readily available.
also - how do you log a certain container, for example?
std::format?
and I haven't even mentioned cases where you want to debug heavily multithreaded code.
Urm, that's one of the cases that's a mighty pain to do with anything but logging.

(I've spent a good chunk of my professional life debugging distributed systems :-) Good luck attaching a debugger when your process is spread over 500 different machines in a datacenter at the other side of the world.)
Thanks for the tips (specifying everything twice looks like a very nice hack to me - didn't know that :)
on the other hand manually specifying 10 libraries is not much of a problem if the order is sort of obvious
Normally, your build system should take care of any required library ordering. (May I mention that MSBuild is primitive and antiquated?)
oh and as for lld - in my experience microsoft linker does a much better job at removing unused code and doesn't actually require any specific order (assuming static linking)
I haven't measured the size, but in the choice between size of binaries and link speed, I'll happy take the latter! (Well, except when sizecoding, which is a different art entirely.) For the stuff I do at work, we're easily talking 20–30 seconds per target with link.exe and sub-second with lld-link.exe… and then there's multiple targets to link.
Sesse
Posts: 300
Joined: Mon Apr 30, 2018 11:51 pm

Re: Compiler Speed Nowadays

Post by Sesse »

Ras wrote: Wed Dec 30, 2020 5:37 pm I also find debuggers mostly useless, except for rare cases such as a buffer overflow that overwrites something else which crashes later in an unrelated part of the code.
For buffer overflows, ASan is extremely useful. (It's mostly replaced Valgrind for my uses.)