Elo/source-code meter

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Kotlov
Posts: 215
Joined: Fri Jul 10, 2015 7:23 pm
Location: Russia

Elo/source-code meter

Post by Kotlov » Tue May 01, 2018 12:37 pm

hgm wrote:
duncan wrote:using the metric elo/byte ?
Indeed, but at source-code level. So Elo/character.
Image
Eugene Kotlov
Hedgehog 2.0 64-bit coming soon...

Uri Blass
Posts: 8654
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Elo/source-code meter

Post by Uri Blass » Tue May 01, 2018 2:10 pm

Your metric is not elo/byte but playing strength/byte
elo is not proportional to playing strength.


stockfish source 57447
stockfish elo 3561
stockfish number 3561/57447

Micromax source 3730
Micromax elo 1947
Micromax number 1947/3730

Milos
Posts: 3389
Joined: Wed Nov 25, 2009 12:47 am

Re: Elo/source-code meter

Post by Milos » Tue May 01, 2018 2:15 pm

Uri Blass wrote:Your metric is not elo/byte but playing strength/byte
elo is not proportional to playing strength.


stockfish source 57447
stockfish elo 3561
stockfish number 3561/57447

Micromax source 3730
Micromax elo 1947
Micromax number 1947/3730
Seriously are you that slow?
Elo is an logaritmic (logistic actually) relative metric of playing strength. You can't just make simple proportion to perform calculation. HGM ofc knows this but is just arrogant and his keyboard is often faster than his brain.
You on the other hand, I really don't get.
OP is absolutely correct in his calculation.

Uri Blass
Posts: 8654
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Elo/source-code meter

Post by Uri Blass » Tue May 01, 2018 2:26 pm

Milos wrote:
Uri Blass wrote:Your metric is not elo/byte but playing strength/byte
elo is not proportional to playing strength.


stockfish source 57447
stockfish elo 3561
stockfish number 3561/57447

Micromax source 3730
Micromax elo 1947
Micromax number 1947/3730
Seriously are you that slow?
Elo is an logaritmic (logistic actually) relative metric of playing strength. You can't just make simple proportion to perform calculation. HGM ofc knows this but is just arrogant and his keyboard is often faster than his brain.
You on the other hand, I really don't get.
OP is absolutely correct in his calculation.
The metric that HGM used is elo/byte
It is possible to compare engines using this metric and I do not see what is the problem.

Nobody claim that elo is proportional to playing strength and I can make a simple proportion to calculate the metric that HGM is using.

You can claim that it is not important to have the best number for elo/byte
and it is more important to have the best number in different metric
but it does not contradict the fact that micromax is better than stockfish in this comparison.

Of course with different comparison stockfish is better.

Milos
Posts: 3389
Joined: Wed Nov 25, 2009 12:47 am

Re: Elo/source-code meter

Post by Milos » Tue May 01, 2018 2:32 pm

Uri Blass wrote:
Milos wrote:
Uri Blass wrote:Your metric is not elo/byte but playing strength/byte
elo is not proportional to playing strength.


stockfish source 57447
stockfish elo 3561
stockfish number 3561/57447

Micromax source 3730
Micromax elo 1947
Micromax number 1947/3730
Seriously are you that slow?
Elo is an logaritmic (logistic actually) relative metric of playing strength. You can't just make simple proportion to perform calculation. HGM ofc knows this but is just arrogant and his keyboard is often faster than his brain.
You on the other hand, I really don't get.
OP is absolutely correct in his calculation.
The metric that HGM used is elo/byte
It is possible to compare engines using this metric and I do not see what is the problem.

Nobody claim that elo is proportional to playing strength and I can make a simple proportion to calculate the metric that HGM is using.

You can claim that it is not important to have the best number for elo/byte
and it is more important to have the best number in different metric
but it does not contradict the fact that micromax is better than stockfish in this comparison.

Of course with different comparison stockfish is better.
You are just hopeless. You clearly cannot comprehend that mathematical operator used for comparison depends on the type of quantity you are trying to compare. Therefore Elo/byte is not Elo divided with byte of size and that simple division makes absolutely no sense. Just a simple illustration, maybe you can get that one, if not I give up. Because Elo is a relative metric reference engine Elo doesn't matter so you can say Micromax is 0 Elo, or if you wish 0.00000001 Elo and SF is 1614 Elo. Now try to use your proportion and see how much stronger Micromax is in Elo/byte :lol: :lol: :lol:.

User avatar
hgm
Posts: 23883
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Elo/source-code meter

Post by hgm » Tue May 01, 2018 2:56 pm

The mentioned sizes do not appear to be correct correct; micro-Max is just under 2K, not counting unnecessary white space and comments. This is even a more generous counting rule than for the Obfuscated C-Code Contest, where they also do not count semi-colons.

But apart from that, I don't see the point. You use the metric exp(Elo)/size, and you find Stockfish is better. You could also have used Elo/exp(size). This would highly amplify micro-Max' advantage, compared to Elo/size.

By picking the metric you can make the outcome anything you like. Which was exactly my point, and the reason I mentioned it in the other thread.

Actually the Elo/size measure is very generous as well: the Elo scale has the natural zero point of the random mover. And we now know that this does not ly near 0 Elo, but rather near -3000 (from the AlphaZero experiment). So if you correct for that and use 'absolute Elo', micro-Max is ~5000, and Stockfish ~6300, so only 1.26 times as strong. Defining micro-Max as 0 Elo makes just as much sense as defining the freezing point of ice as temperature zero.

Uri Blass
Posts: 8654
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Elo/source-code meter

Post by Uri Blass » Tue May 01, 2018 4:05 pm

Milos wrote:
Uri Blass wrote:
Milos wrote:
Uri Blass wrote:Your metric is not elo/byte but playing strength/byte
elo is not proportional to playing strength.


stockfish source 57447
stockfish elo 3561
stockfish number 3561/57447

Micromax source 3730
Micromax elo 1947
Micromax number 1947/3730
Seriously are you that slow?
Elo is an logaritmic (logistic actually) relative metric of playing strength. You can't just make simple proportion to perform calculation. HGM ofc knows this but is just arrogant and his keyboard is often faster than his brain.
You on the other hand, I really don't get.
OP is absolutely correct in his calculation.
The metric that HGM used is elo/byte
It is possible to compare engines using this metric and I do not see what is the problem.

Nobody claim that elo is proportional to playing strength and I can make a simple proportion to calculate the metric that HGM is using.

You can claim that it is not important to have the best number for elo/byte
and it is more important to have the best number in different metric
but it does not contradict the fact that micromax is better than stockfish in this comparison.

Of course with different comparison stockfish is better.
You are just hopeless. You clearly cannot comprehend that mathematical operator used for comparison depends on the type of quantity you are trying to compare. Therefore Elo/byte is not Elo divided with byte of size and that simple division makes absolutely no sense. Just a simple illustration, maybe you can get that one, if not I give up. Because Elo is a relative metric reference engine Elo doesn't matter so you can say Micromax is 0 Elo, or if you wish 0.00000001 Elo and SF is 1614 Elo. Now try to use your proportion and see how much stronger Micromax is in Elo/byte :lol: :lol: :lol:.
I have no problem to understand mathematics and I understand what you say.

Stockfish is clearly relatively stronger with some calculation and Micromax is relatively stronger with a different calculation.

It is not that the calculation that Stockfish is stronger is less impotant but that calculation is not elo/source-code metric that is the subject of the thread(when elo is elo in the same way that we normally calculate it in rating lists and not something near 0 for micromax that I agree can also be elo but not what I mean when I talk about elo).

User avatar
Kotlov
Posts: 215
Joined: Fri Jul 10, 2015 7:23 pm
Location: Russia

Re: Elo/source-code meter

Post by Kotlov » Tue May 01, 2018 8:14 pm

HGM knows that I'm right ))
Eugene Kotlov
Hedgehog 2.0 64-bit coming soon...

MonteCarlo
Posts: 62
Joined: Sun Dec 25, 2016 3:59 pm

Re: Elo/source-code meter

Post by MonteCarlo » Tue May 01, 2018 8:30 pm

Should probably slightly change the original calculation. It's taking the expected score of Engine A vs Engine B, then doing 1/expected score to show much better Engine B is.

When Engine A and Engine B have the same rating, that would output that each engine is twice as good as the other :)

If you measure strength this way, you would want to do (1-E_micromax)/E_micromax.

It wouldn't change the result much here, of course, but as long as we're all being picky :)

User avatar
Kotlov
Posts: 215
Joined: Fri Jul 10, 2015 7:23 pm
Location: Russia

Re: Elo/source-code meter

Post by Kotlov » Tue May 01, 2018 8:53 pm

MonteCarlo wrote:Should probably slightly change the original calculation. It's taking the expected score of Engine A vs Engine B, then doing 1/expected score to show much better Engine B is.

When Engine A and Engine B have the same rating, that would output that each engine is twice as good as the other :)

If you measure strength this way, you would want to do (1-E_micromax)/E_micromax.

It wouldn't change the result much here, of course, but as long as we're all being picky :)
Yes, my fault.
Thx!
Eugene Kotlov
Hedgehog 2.0 64-bit coming soon...

Post Reply