Using Link-Time-Optimization in mingw/gcc: Any experience?

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

RubiChess
Posts: 584
Joined: Fri Mar 30, 2018 7:20 am
Full name: Andreas Matthies

Using Link-Time-Optimization in mingw/gcc: Any experience?

Post by RubiChess »

Hi.

I started to reorganize some of my code seperating one big module into several smaller modules (cpp files) which resulted in a slowdown (msys2/mingw64/gcc/g++ build), probably because the optimization used so far only works inside one module.

Then I discovered the -flto switch and gave it a try. Sadly the resulting exe just crashed at start with a segmentation violation error.
But... if I remove the -static switch and compile again, the resulting exe (needing some of the mingw64 dlls now) seems to work fine and gave a great speed improvement.
I don't like builds that rely on some dlls so I tried everything to make it work with -static but failed so far. I read several things about -flto in Internet from "Broken in windows, don't use it!" to some advises how to make it work using different linker/bin utils/parameters but nothing worked for me so far.

Has anybody of you experience in this field? Any hint is welcome.

.Andreas
User avatar
xr_a_y
Posts: 1871
Joined: Sat Nov 25, 2017 2:28 pm
Location: France

Re: Using Link-Time-Optimization in mingw/gcc: Any experience?

Post by xr_a_y »

Here is Minic mingw compile line :

Code: Select all

x86_64-w64-mingw32-g++ -Wall -Wno-char-subscripts -Wno-reorder -DNDEBUG -O3 -flto -march=native --std=c++14 minic.cc -static -static-libgcc -static-libstdc++ -o minic -Wl,-Bstatic -lpthread
Don't know if that helps ...
RubiChess
Posts: 584
Joined: Fri Mar 30, 2018 7:20 am
Full name: Andreas Matthies

Re: Using Link-Time-Optimization in mingw/gcc: Any experience?

Post by RubiChess »

xr_a_y wrote: Thu Aug 01, 2019 11:15 am Here is Minic mingw compile line :

Code: Select all

x86_64-w64-mingw32-g++ -Wall -Wno-char-subscripts -Wno-reorder -DNDEBUG -O3 -flto -march=native --std=c++14 minic.cc -static -static-libgcc -static-libstdc++ -o minic -Wl,-Bstatic -lpthread
Don't know if that helps ...
I will try. Thanks.
jdart
Posts: 4366
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: Using Link-Time-Optimization in mingw/gcc: Any experience?

Post by jdart »

I don't use MingW. -flto works fine with gcc. You need to use the gold linker on Linux (-fuse-ld=gold). How effective it is may vary. They have put quite a few improvements into LTO in gcc versions 8 and 9.

--Jon
RubiChess
Posts: 584
Joined: Fri Mar 30, 2018 7:20 am
Full name: Andreas Matthies

Re: Using Link-Time-Optimization in mingw/gcc: Any experience?

Post by RubiChess »

No luck so far.
It turns out that the call to getline(cin, inputsting) in the UCI input loop causes the segmentation fault when I compile with -static and -flto. Strange. Any bell ringing here?
I could rewrite it using cin >> inputstring or even fgets() which seems to work but I would like to find the real reason.

.Andreas
Ras
Posts: 2487
Joined: Tue Aug 30, 2016 8:19 pm
Full name: Rasmus Althoff

Re: Using Link-Time-Optimization in mingw/gcc: Any experience?

Post by Ras »

RubiChess wrote: Fri Aug 02, 2019 6:15 amIt turns out that the call to getline(cin, inputsting) in the UCI input loop causes the segmentation fault
How does that even compile? According to http://man7.org/linux/man-pages/man3/getline.3.html , the prototype looks like this:

Code: Select all

ssize_t getline(char **lineptr, size_t *n, FILE *stream);
Rasmus Althoff
https://www.ct800.net
mar
Posts: 2554
Joined: Fri Nov 26, 2010 2:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Using Link-Time-Optimization in mingw/gcc: Any experience?

Post by mar »

Ras wrote: Sat Aug 03, 2019 11:29 am How does that even compile? According to http://man7.org/linux/man-pages/man3/getline.3.html , the prototype looks like this:

Code: Select all

ssize_t getline(char **lineptr, size_t *n, FILE *stream);
Because he's using C++ https://en.cppreference.com/w/cpp/strin ... ng/getline
Martin Sedlak
RubiChess
Posts: 584
Joined: Fri Mar 30, 2018 7:20 am
Full name: Andreas Matthies

Re: Using Link-Time-Optimization in mingw/gcc: Any experience?

Post by RubiChess »

End of story: I switch to clang which builds static windows binaries combining lto and profile instrumented optimization. This binary is smaller and even slightly faster than the gcc built binary.
Build script is a little tricky using MSVC nmake and some llvm profiling tool that is not included in their windows distribution. But it works somehow.

- Andreas