Different performance of equal executables

[OliverBr]

Hello together,

investigating a performance worse than expected I fell over the following phenomena:

I have two Linux 64 executables named "olithink589" on a new Debian Linux with an idle AMD EPYC 7502P 32-Core Processor. One is in the current folded and another, older one is in "bin". The identity is checked by:

Code: Select all

diff ./olithink589 bin/olithink589

Now, I am running a performance check first on the underperforming executable "bin/olithink589":

Code: Select all

knps: 2090

Then testing the newer one "./olithink589":

Code: Select all

knps: 2221

This result is always reproducible and quite surprising: One executable is more than 6% faster than the other one. There are other executables in "bin" which are not affected by such slowdown.
Is there any explanation?

[abulmo2]

The environment of the executable, including the directory it runs from, its name, etc. can affect performance.
Videos about this subject:
https://www.youtube.com/watch?v=r-TLSBdHe1A
https://www.youtube.com/watch?v=koTf7u0v41o&t=1011s
and a paper:
https://www.researchgate.net/publicatio ... usly_Wrong

[syzygy]

Type the following as root:
# echo 1 > /proc/sys/vm/drop_caches

Now try again.

[RubiChess]

I noticed the same. Look here: http://www.talkchess.com/forum3/viewtop ... =7&t=73299 and here: https://github.com/AndyGrant/OpenBench/issues/50

I remember one case when syzygys drop_cache definitely helped so I'm using this before running tests on Linux and an "EmptyStandyList.exe" tool I found for Windows that should do similar.

Regards, Andreas

[flok]

I would not use diff, but use e.g. md5sum.
Because I'm not sure what happens if the difference is a \cr or a 0x00.

Hello together,

investigating a performance worse than expected I fell over the following phenomena:

I have two Linux 64 executables named "olithink589" on a new Debian Linux with an idle AMD EPYC 7502P 32-Core Processor. One is in the current folded and another, older one is in "bin". The identity is checked by:

Code: Select all

diff ./olithink589 bin/olithink589

Now, I am running a performance check first on the underperforming executable "bin/olithink589":

Code: Select all

knps: 2090

Then testing the newer one "./olithink589":

Code: Select all

knps: 2221

This result is always reproducible and quite surprising: One executable is more than 6% faster than the other one. There are other executables in "bin" which are not affected by such slowdown.
Is there any explanation?

[Sven]

"diff" would say "binary files differ" if they were different. But you don't need something like md5sum which requires to type several commands ... To determine whether two arbritrary files A and B are different at all, just type:

Code: Select all

cmp -s A B || echo "A and B are different"

If that produces empty output then A and B are binary identical. "cmp" should be slightly faster than "diff" because it does not try to present differences it finds but simply stops at the first difference.

[flok]

"diff" would say "binary files differ" if they were different. But you don't need something like md5sum which requires to type several commands ... To determine whether two arbritrary files A and B are different at all, just type:

Code: Select all

cmp -s A B || echo "A and B are different"

If that produces empty output then A and B are binary identical. "cmp" should be slightly faster than "diff" because it does not try to present differences it finds but simply stops at the first difference.

several commands? for md5sum?

folkert@msi:~$ echo hallo > a
folkert@msi:~$ echo Hallo > b
folkert@msi:~$ md5sum a b
aee97cb3ad288ef0add6c6b5b5fae48a a
3290ec3c19a8a39362f7d70043f15627 b

1 command!

[OliverBr]

I would not use diff, but use e.g. md5sum.

It doesn't matter.
Both files are absolutely identical. I also copied olithink589 from "bin" to another directory and got again the better (expected) performance of about 2220 knps. Only in the directory "bin" it's slower. Other executables in "bin" aren't slower.

[OliverBr]

Type the following as root:
# echo 1 > /proc/sys/vm/drop_caches

Yes, this solved it. Afterwards "bin/olithink589 performs exactly equal as "./olithink589" with 2220 knps.

Now this should explain the issue: What cache went probably wrong and how can this be avoided in the future?

[OliverBr]

I noticed the same. Look here: http://www.talkchess.com/forum3/viewtop ... =7&t=73299 and here: https://github.com/AndyGrant/OpenBench/issues/50

I remember one case when syzygys drop_cache definitely helped so I'm using this before running tests on Linux and an "EmptyStandyList.exe" tool I found for Windows that should do similar.

Regards, Andreas

Yes, thank you, drop_cache helped!
PS: Up to 20 ELO drop is about the same I encountered in my case. I never would have thought that 6% performance loss yields to such a big difference in ELO.