bitboards like mailbox

Discussion of chess software programming and technical issues.

Moderator: Ras

tcusr
Posts: 325
Joined: Tue Aug 31, 2021 10:32 pm
Full name: tcusr

Re: bitboards like mailbox

Post by tcusr »

dangi12012 wrote: Sun Nov 07, 2021 1:29 pm
tcusr wrote: Sun Nov 07, 2021 1:26 pm before writing a move generator with this method i still have some doubts
to generate captures should i use another algorithm better suited for bitboards instead of the technique 'loop (for me shift) toward a direction until an enemy piece is met' used by mailbox engines? a rook/bishop can at most capture 4 pieces so maybe it's better
You can hide the implementation behind an interface and just replace it later and start with what you feel more confortable.
When you find something faster just replace the implementation and nothin else changes.

for sliders I have: Lookup::Rook(uint64_t occupy, in square);

where Lookup is a namespace.
the thing is, i have no function to generate attacks on the fly
this is what i intend to do for sliders

Code: Select all

template<Direction D, Direction...Ds, typename F>
void generateSlider(int from, uint64_t not_us, uint64_t them, F&& f)
{
	if constexpr (sizeof(Ds...) > 0)
		generateSliders<Ds...>(from, not_us, them, f);

	auto to{ 1ull << from };
	while ((to = shift<D>(to)) & not_us) {
		f(from, bsf(to));

		if (to & them)
			break;
	}
}
to generate rook attacks from rookSquare i call it like this generateSlider<8, 1, -1, -8>(rookSquare, ~Us, Them, /* lambda that adds move to movelist */)
tcusr
Posts: 325
Joined: Tue Aug 31, 2021 10:32 pm
Full name: tcusr

Re: bitboards like mailbox

Post by tcusr »

i meant

Code: Select all

	if constexpr (sizeof...(Ds) > 0)
dangi12012
Posts: 1062
Joined: Tue Apr 28, 2020 10:03 pm
Full name: Daniel Infuehr

Re: bitboards like mailbox

Post by dangi12012 »

tcusr wrote: Sun Nov 07, 2021 1:38 pm
dangi12012 wrote: Sun Nov 07, 2021 1:29 pm
tcusr wrote: Sun Nov 07, 2021 1:26 pm before writing a move generator with this method i still have some doubts
to generate captures should i use another algorithm better suited for bitboards instead of the technique 'loop (for me shift) toward a direction until an enemy piece is met' used by mailbox engines? a rook/bishop can at most capture 4 pieces so maybe it's better
You can hide the implementation behind an interface and just replace it later and start with what you feel more confortable.
When you find something faster just replace the implementation and nothin else changes.

for sliders I have: Lookup::Rook(uint64_t occupy, in square);

where Lookup is a namespace.
the thing is, i have no function to generate attacks on the fly
this is what i intend to do for sliders

Code: Select all

template<Direction D, Direction...Ds, typename F>
void generateSlider(int from, uint64_t not_us, uint64_t them, F&& f)
{
	if constexpr (sizeof(Ds...) > 0)
		generateSliders<Ds...>(from, not_us, them, f);

	auto to{ 1ull << from };
	while ((to = shift<D>(to)) & not_us) {
		f(from, bsf(to));

		if (to & them)
			break;
	}
}
to generate rook attacks from rookSquare i call it like this generateSlider<8, 1, -1, -8>(rookSquare, ~Us, Them, /* lambda that adds move to movelist */)
You dont need to differentiate between them and not_us. Its blockers who stop your ray. - just AND the bitboard with EnemyOrEmpty to get the correct target squares.
Also I dont see you stopping when wrapping around the edges around the board
Its easiest to read some implementations there:
https://www.chessprogramming.org/Classical_Approach
Worlds-fastest-Bitboard-Chess-Movegenerator
Daniel Inführ - Software Developer
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: bitboards like mailbox

Post by hgm »

I don't understand any of the C++ stuff, but step-by-step generation of slider moves becomes indeed an inefficient method when you are only interested in captures. And unfortunately that is exacty what you are, in the overwhelming majority of the tree nodes.

In the 'mailbox trials' I solved this problem by maintaining a 'neighbor table', which for all occupied squares holds the square numbers of the nearest neigbors. There then never is the need to scan the board to see what a certain move direction will bump into; you can just directly look it up in the table.

I guess you could make a bitboard equivalent of this: tabulate the attack sets (given the current board occupancy) for every occupied square in an array of 64 bitboards. (The elements for the unoccupied squares would be unused.) It would probaby be best to have two such tables:

Code: Select all

Bitboard orth[64], diag[64];
To get the attacks for a given Rook or Bishop, you could then read those directly from these small tables.

But how to update these tables? Captures (about 95% of all moves in the tree) only evacuate a single square. This affects the attacks boards that were hitting this evacuated square. But since Rook and Bishop moves are symmetric, those are the attacks that are coming from the (occupied) squares that were attacked from the evacuated one.

So when square s gets evacuated, you take orth[s], intersect it with 'occupied' (orth[s] & occupied), and extract the thus hit squares t. For each of those, you add the attacks from s to the old attacks from t (orth[s] | orth[t]). This also adds the moves from s perpendicular to the t-s ray, which would not be reachable from t without turning a corner. To get rid of those, you also keep (completely static) bitboard tables orth0[64] and diag0[64] which contain (i.e. were initialized with) the attacks from each square on a completely empty board. And intersect with those ((orth[s] | orth[t]) & orth0[t]). So:

Code: Select all

Bitboard orth[64], orth0[64], diag[64], diag[64];

void Evacuate(int s)
{
  Bitboard todo = orth[s] & occupied; // all orthogonal neighbors
  while(todo) {
    int t = BSF(todo);
    todo -= 1LL<<t;
    orth[t] = (orth[t] | orth[s]) & orth0[t]; // add discovered ray to neighbor set
  }
  // same for diag
}
With a modest amount of work this would make all Rook and Bishop attacks sets for occupied squares available at all times in small tables (4 x 64 x 8 = 2048 bytes).

Problem is that you would also have to undo it when UnMaking a move. (The curse of incremental update...) One way to do that would be to put the original value of every modified orth[t] and diag[t] in the while loops on a stack (stack[i++] = orth[t]). And for the UnMake then use similar while loops, and copy them back (orth[t] = stack[i++]).
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: bitboards like mailbox

Post by hgm »

Actually, tabulating the conventional attacks sets might not be optimal, as it seems these sets are mainly used for intersecting them with occupied squares of one or both colors. Especially when we are only interested in generating captures. So a simplification might be to already store the intersections of the attacks set and the occupied board. That makes restoring the old values during UnMake easier (no extra storage needed):

Code: Select all

void Evacuate(int s)
{
  Bitboard todo = orth[s];
  while(todo) {
    int t = BSF(todo);
    todo -= 1LL<<t;
    orth[t] = (orth[t] | orth[s]) & orth0[t];
  }
  // same for diag
}

void Reoccupy(int s)
{
  Bitboard todo = orth[s];
  while(todo) {
    int t = BSF(todo);
    todo -= 1LL<<t;
    orth[t] = orth[t] & ~orth[s] | 1LL<<s;
  }
  // same for diag
}
tcusr
Posts: 325
Joined: Tue Aug 31, 2021 10:32 pm
Full name: tcusr

Re: bitboards like mailbox

Post by tcusr »

dangi12012 wrote: Sun Nov 07, 2021 2:55 pm
tcusr wrote: Sun Nov 07, 2021 1:38 pm
dangi12012 wrote: Sun Nov 07, 2021 1:29 pm
tcusr wrote: Sun Nov 07, 2021 1:26 pm before writing a move generator with this method i still have some doubts
to generate captures should i use another algorithm better suited for bitboards instead of the technique 'loop (for me shift) toward a direction until an enemy piece is met' used by mailbox engines? a rook/bishop can at most capture 4 pieces so maybe it's better
You can hide the implementation behind an interface and just replace it later and start with what you feel more confortable.
When you find something faster just replace the implementation and nothin else changes.

for sliders I have: Lookup::Rook(uint64_t occupy, in square);

where Lookup is a namespace.
the thing is, i have no function to generate attacks on the fly
this is what i intend to do for sliders

Code: Select all

template<Direction D, Direction...Ds, typename F>
void generateSlider(int from, uint64_t not_us, uint64_t them, F&& f)
{
	if constexpr (sizeof(Ds...) > 0)
		generateSliders<Ds...>(from, not_us, them, f);

	auto to{ 1ull << from };
	while ((to = shift<D>(to)) & not_us) {
		f(from, bsf(to));

		if (to & them)
			break;
	}
}
to generate rook attacks from rookSquare i call it like this generateSlider<8, 1, -1, -8>(rookSquare, ~Us, Them, /* lambda that adds move to movelist */)
You dont need to differentiate between them and not_us. Its blockers who stop your ray. - just AND the bitboard with EnemyOrEmpty to get the correct target squares.
Also I dont see you stopping when wrapping around the edges around the board
Its easiest to read some implementations there:
https://www.chessprogramming.org/Classical_Approach
the shift function takes care of wraps. since i'm shifting a single bit when it is on the edge and tries to pass it the functions returns 0
btw thanks for the tip, if i use a EnemyOrEmpty variable i don't have the branch if (to & them) in the loop
tcusr
Posts: 325
Joined: Tue Aug 31, 2021 10:32 pm
Full name: tcusr

Re: bitboards like mailbox

Post by tcusr »

as for hgm, i've already read about your neighbor tables idea and i found it very interesting but also complicated (like all your other ideas)
but this time i really want to understand, thank you very much for the explanation!
Karlo Bala
Posts: 373
Joined: Wed Mar 22, 2006 10:17 am
Location: Novi Sad, Serbia
Full name: Karlo Balla

Re: bitboards like mailbox

Post by Karlo Bala »

The true power of bitboards is in the eval. For example, you want different mobility patterns, normal, through pieces but not pawns, exclude knights, exclude bishops, etc. Then future mobility through safe squares, 2 or 3 steps in advance. King paths in pawn endgames, a lot of patterns for trapped pieces. All you need to do is to set or clear some bits in bitboard and feed the attacks routine. There are countless uses. It would be almost impossible to do it with pure mailbox representation (efficiently). If you want PST + some simple patterns eval then bitboards are most probably an overkill.
Best Regards,
Karlo Balla Jr.
tcusr
Posts: 325
Joined: Tue Aug 31, 2021 10:32 pm
Full name: tcusr

Re: bitboards like mailbox

Post by tcusr »

Karlo Bala wrote: Sun Nov 07, 2021 8:08 pm The true power of bitboards is in the eval. For example, you want different mobility patterns, normal, through pieces but not pawns, exclude knights, exclude bishops, etc. Then future mobility through safe squares, 2 or 3 steps in advance. King paths in pawn endgames, a lot of patterns for trapped pieces. All you need to do is to set or clear some bits in bitboard and feed the attacks routine. There are countless uses. It would be almost impossible to do it with pure mailbox representation (efficiently). If you want PST + some simple patterns eval then bitboards are most probably an overkill.
that's not what i'm going to do, i want to write an hand crafted eval from scratch. with bitboards it's really easy to go from the idea to the implementation
User avatar
hgm
Posts: 28353
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: bitboards like mailbox

Post by hgm »

Karlo Bala wrote: Sun Nov 07, 2021 8:08 pm The true power of bitboards is in the eval. For example, you want different mobility patterns, normal, through pieces but not pawns, exclude knights, exclude bishops, etc. Then future mobility through safe squares, 2 or 3 steps in advance. King paths in pawn endgames, a lot of patterns for trapped pieces. All you need to do is to set or clear some bits in bitboard and feed the attacks routine. There are countless uses. It would be almost impossible to do it with pure mailbox representation (efficiently). If you want PST + some simple patterns eval then bitboards are most probably an overkill.
Well, I understood that bitboard engines also use attack getters for Rooks and Bishops quite a lot, in evaluation. So I just tried to provide an alternative (less memory intensive) method for obtaining those. Just reading them from a board-size table, such as orth[s] for the Rook attacks from square s, seems a speed improvement over the usual magics, which would need several small-table lookups (for the mask and the multiplier), a multiply, a shift and a lookup in a large (non-L1-fitting) table.

The issue is just how much overhead it would take to maintain the table, as its contents should obviously be adapted to the board occupancy. Up to 8 elements could have to be changed because of a capture, and on a crowded board it is likely that most pieces indeed have neighbors in all 8 directions. (On a near-empty board this will of course sharply drop.) The time to make these updates can only be earned back if the attack getter is used often enough.