Remove MSB

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Remove MSB

Post by bob »

Henk wrote:
Rein Halbersma wrote:
hgm wrote:Oops, you are right. I ignored that N bits are done in parallel. And without that it would of course just be O(N), as you would simply scan.
In fact, the log base 2 is equal to the msb :)
Yes I used:

Code: Select all

    public class BitBoardBackwardIterator: IBitBoardIterator
    {
        UInt64 bits;


        public void Reset(UInt64 bits)
        {
            this.bits = bits;
        }

        public UInt64 Next()
        {
            while (bits != 0)
            {
                int index = (int)Math.Log(bits, 2);
                UInt64 bit = (&#40;UInt64&#41;1 << index&#41;;

                bits &= ~bit;
                return bit;
            &#125;
            return 0;
       &#125;
    &#125;
You REALLY want to use a math library function when it is called that many times? This is trivial and is FAR faster:

bits &= ~(1 << MSB(bits));

Or even better

bits ^= 1 << MSB(bits);

MSB is one instruction, shift is 1 instruction, complement is one instruction, and AND is one instruction. You will blow more than that just doing the procedure call and return.

There are fast ways to do these things. There are slow ways to do 'em. And then there are UNTHINKABLE ways of doing 'em. Little point in moving to bit boards if you are going to make everything 100x slower than it needs to be. BSF/BSR have been around since the original Intel pentium processor.
mar
Posts: 2552
Joined: Fri Nov 26, 2010 2:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Remove MSB

Post by mar »

bob wrote: You REALLY want to use a math library function when it is called that many times? This is trivial and is FAR faster:

bits &= ~(1 << MSB(bits));

Or even better

bits ^= 1 << MSB(bits);

MSB is one instruction, shift is 1 instruction, complement is one instruction, and AND is one instruction. You will blow more than that just doing the procedure call and return.

There are fast ways to do these things. There are slow ways to do 'em. And then there are UNTHINKABLE ways of doing 'em. Little point in moving to bit boards if you are going to make everything 100x slower than it needs to be. BSF/BSR have been around since the original Intel pentium processor.
And how would you do BSR in C#?
Actually the idea is clever, even though 64-bit integer doesn't fit in double, since it only extracts msb => potential loss of precision at lsbits is fine.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Remove MSB

Post by bob »

mar wrote:
bob wrote: You REALLY want to use a math library function when it is called that many times? This is trivial and is FAR faster:

bits &= ~(1 << MSB(bits));

Or even better

bits ^= 1 << MSB(bits);

MSB is one instruction, shift is 1 instruction, complement is one instruction, and AND is one instruction. You will blow more than that just doing the procedure call and return.

There are fast ways to do these things. There are slow ways to do 'em. And then there are UNTHINKABLE ways of doing 'em. Little point in moving to bit boards if you are going to make everything 100x slower than it needs to be. BSF/BSR have been around since the original Intel pentium processor.
And how would you do BSR in C#?
Actually the idea is clever, even though 64-bit integer doesn't fit in double, since it only extracts msb => potential loss of precision at lsbits is fine.
A. I would not write a chess program in C#.

B. I'm not a C# programmer, but do they not have either intrinsics or a way to get to external ASM? I certainly would not be using a math library Log2 function.
kbhearn
Posts: 411
Joined: Thu Dec 30, 2010 4:48 am

Re: Remove MSB

Post by kbhearn »

c# indeed does not have intrinsics. it's built as a high level language that is supposed to be agnostic to the hardware, not to compete with C and C++

the main alternative would be a binary search to narrow it down to a 1-2 byte region and then a lookup table. How that would compare to conversion to a floating point value and taking a log i have no idea - it might even be comparable.
mar
Posts: 2552
Joined: Fri Nov 26, 2010 2:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Remove MSB

Post by mar »

bob wrote:A. I would not write a chess program in C#.

B. I'm not a C# programmer, but do they not have either intrinsics or a way to get to external ASM? I certainly would not be using a math library Log2 function.
A. Agreed.
B. I'm not sure there's such thing

Actually I've investigated a bit further and Math.Log doesn't work as expected:

Code: Select all

&#40;int&#41;Math.Log&#40;~&#40;ulong&#41;0, 2&#41;
gives 64, not 63
Similarly for ((ulong)1 << 63)-1 you get 63, not 62.
So I should correct my statement and loss of precision is not fine; this approach is broken.
Whether it's practical for chess where bitboards are usually sparse is another question... Still I would probably look for something else.
Dann Corbit
Posts: 12482
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Remove MSB

Post by Dann Corbit »

Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: Remove MSB

Post by Sven »

kbhearn wrote:c# indeed does not have intrinsics. it's built as a high level language that is supposed to be agnostic to the hardware, not to compete with C and C++

the main alternative would be a binary search to narrow it down to a 1-2 byte region and then a lookup table. How that would compare to conversion to a floating point value and taking a log i have no idea - it might even be comparable.
At least it should be possible to put intrinsic wrapper functions into a C++/CLI library, compile them with "#pragma unmanaged", and call the real intrinsics from there. The overhead is certainly the function call overhead plus switching between managed and unmanaged code but that should still be better than all handmade C# software solutions, including those that do not work ...
mar
Posts: 2552
Joined: Fri Nov 26, 2010 2:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Remove MSB

Post by mar »

mar wrote:this approach is broken.
I think I found a remedy :) It makes it even slower and I've no idea why assuming 53 mantissa bits didn't work, but seems to work with 48: (the idea is to mask out least significant bits if we'd suffer a precision loss if input is too large)

Code: Select all

const int mbits = 48;
int bit = &#40;int&#41;&#40;Math.Log&#40;l & ~(((&#40;ulong&#41;1 << &#40;64 - mbits&#41;) - 1&#41; * Convert.ToUInt32&#40;l >= &#40;ulong&#41;1 << mbits&#41;), 2.0&#41;);
Very nice and readable :) One more reason to look for a different approach.
Also love the fact that you can't cast bool to integral types in C#
petero2
Posts: 680
Joined: Mon Apr 19, 2010 7:07 pm
Location: Sweden
Full name: Peter Osterlund

Re: Remove MSB

Post by petero2 »

mar wrote:
bob wrote: You REALLY want to use a math library function when it is called that many times? This is trivial and is FAR faster:

bits &= ~(1 << MSB(bits));

Or even better

bits ^= 1 << MSB(bits);

MSB is one instruction, shift is 1 instruction, complement is one instruction, and AND is one instruction. You will blow more than that just doing the procedure call and return.

There are fast ways to do these things. There are slow ways to do 'em. And then there are UNTHINKABLE ways of doing 'em. Little point in moving to bit boards if you are going to make everything 100x slower than it needs to be. BSF/BSR have been around since the original Intel pentium processor.
And how would you do BSR in C#?
Using De Bruijn multiplication is not that bad actually. On my computer the C# version is about 6 times slower than the C++ version that is using BSR (borrowed from crafty). This could be considered acceptable for a managed language that only supports the most basic bit manipulation instructions. It is at least a lot faster than using Math.log().

Code: Select all

class Test &#123;
    private static int&#91;&#93; firstBitTable = new int&#91;&#93;&#123;
        63,  0, 47,  1, 56, 48, 27,  2,
        60, 57, 49, 41, 37, 28, 16,  3,
        61, 54, 58, 35, 52, 50, 42, 21,
        44, 38, 32, 29, 23, 17, 11,  4,
        62, 46, 55, 26, 59, 40, 36, 15,
        53, 34, 51, 20, 43, 31, 22, 10,
        45, 25, 39, 14, 33, 19, 30,  9,
        24, 13, 18,  8, 12,  7,  6,  5
    &#125;;

    private static int log2&#40;ulong x&#41; &#123;
        x |= x >> 1;
        x |= x >> 2;
        x |= x >> 4;
        x |= x >> 8;
        x |= x >> 16;
        x |= x >> 32;
        x += 1;
        ulong debruijn64 = 0x03f79d71b4cb0a89ul;
        return firstBitTable&#91;&#40;x * debruijn64&#41; >> 58&#93;;
    &#125;

    public static void Main&#40;string&#91;&#93; args&#41; &#123;
        int max = System.Convert.ToInt32&#40;args&#91;0&#93;);
        long sum = 0;
        for &#40;uint i = 1; i <= max; i++) &#123;
            sum += log2&#40;i&#41;;
        &#125;
        System.Console.WriteLine&#40;"sum&#58;" + sum&#41;;
    &#125;
&#125;

Code: Select all

#include <iostream>
#include <cstdint>

static __inline__ int log2&#40;uint64_t word&#41; &#123;
  uint64_t dummy, dummy2;
  asm&#40;" 	 bsrq	 %1, %0     " "\n\t"
      &#58;   "=&r"&#40;dummy&#41;, "=&r" &#40;dummy2&#41;
      &#58;   "1"(&#40;uint64_t&#41; &#40;word&#41;)
      &#58;   "cc");
  return &#40;dummy&#41;;
&#125;

int main&#40;int argc, const char* argv&#91;&#93;) &#123;
    int max = atoi&#40;argv&#91;1&#93;);
    uint64_t sum = 0;
    for &#40;int i = 1; i <= max; i++)
        sum += log2&#40;i&#41;;
    std&#58;&#58;cout << "sum&#58;" << sum << std&#58;&#58;endl;
&#125;
User avatar
hgm
Posts: 27701
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Remove MSB

Post by hgm »

Conversion to double, and extracting the exponent still seems the way to go to me.