Hello and my engine plans!

[catugocatugocatugo]

Almost every comment, HGM, you remind me that the NN does all the work and uses most of the time. That is totally true at training time and probably less true when the network is used. By not going bitboards though the input of the NN would be totally different so probably ResNets won't work that well. The reason I have asked so many questions (some maybe silly), is that I want to have some clear idea what to do. I have started this so many times and got stuck thinking I'm to slow, like when searching for pieces in bitboards.

Now let me get at what I think is the crux of this post.

So far, the structure I understood would be good is a 20x16 structure that contains (why not in the middle?) piece IDs of all pieces initially on the board in the actual board. The rest is used to mark offboard jumps. I'm not sure as of now why the equal distance help, but I'm sure I will when the time comes. There have to be also variables for en-passant and last piece move (I have a joker).
The piece IDs are indexes in a vector of pieces.
What I think should be put in the vectors.
3 bits for the piece type (there are 11 piece types).
1 bit to store If the piece was not captured.
1 bit to store if the piece has moved.

Thanks a lot for your help!
I think I'll move forward with this.

[JoAnnP38]

I use magic bitboards for move generation, but to make all of my logic work efficiently I also keep an 8x8 board of Squares that maintain the content (i.e. piece or empty) and the color of any piece located there. I NEVER iterate through this board, but when I capture a piece by noting an intersection with the enemy's occupied square I can easily lookup what kind of piece it is. I find this works pretty well. It is certainly convenient. It doesn't cost me much other than the updating of the board through either Add/Remove piece methods on my board class. In fact, you don't really even need this extra board to maintain color since your occupation bitboard will already give you that information. For some reason I added color for some reason that I cannot remember. This board also makes it a little more straight forward when converting your board (or position) to a FEN string. So far, I really like this approach.

[hgm]

I use magic bitboards for move generation, but to make all of my logic work efficiently I also keep an 8x8 board of Squares that maintain the content (i.e. piece or empty) and the color of any piece located there. I NEVER iterate through this board, but when I capture a piece by noting an intersection with the enemy's occupied square I can easily lookup what kind of piece it is. I find this works pretty well. It is certainly convenient. It doesn't cost me much other than the updating of the board through either Add/Remove piece methods on my board class. In fact, you don't really even need this extra board to maintain color since your occupation bitboard will already give you that information. For some reason I added color for some reason that I cannot remember. This board also makes it a little more straight forward when converting your board (or position) to a FEN string. So far, I really like this approach.

Indeed, this is the logical approach. One should keep in mind that the bitboard stack is a piece-list replacement. Not a board replacement. It allows you to efficiently obtain (and update) the location of pieces of a given type. The main difference between a bitboard stack and a conventional piece list as used in mailbox engines is that a single entry can indicate the location of all pieces of a given type in a single scalar variable. So there is no need to index it by piece number; indexing by type is good enough. If the locations were represented as an array of square numbers, one would have to search through that list for updating the proper element, unless that element was already indicated by using piece numbers for indexing rather than type numbers.

[hgm]

Almost every comment, HGM, you remind me that the NN does all the work and uses most of the time. That is totally true at training time and probably less true when the network is used.

Even if training would be running the NN more intensively, (which I doubt, as most of the training effort is having the engine play games against itself, and thus not different from playing), it would still be true that during play almost all calculational effort goes into the NN.

By not going bitboards though the input of the NN would be totally different so probably ResNets won't work that well.

Not sure why you say that. You want to use some library function for running the NN? If your program it yourself the input format is what you say it is. You are talking here only about the input layer, a single layer out of perhaps 40, so whatever you do there should not have much impact on the speed of the NN as a whole. So you can always cast the board state in a number of bitboards, by running through the piece list and setting the bits for the pieces you encounter there in the bitboard they should go to. This seems a pretty simple operation compared to running a single layer of the NN.

Now let me get at what I think is the crux of this post.

So far, the structure I understood would be good is a 20x16 structure that contains (why not in the middle?) piece IDs of all pieces initially on the board in the actual board. The rest is used to mark offboard jumps. I'm not sure as of now why the equal distance help, but I'm sure I will when the time comes. There have to be also variables for en-passant and last piece move (I have a joker).
The piece IDs are indexes in a vector of pieces.
What I think should be put in the vectors.
3 bits for the piece type (there are 11 piece types).
1 bit to store If the piece was not captured.
1 bit to store if the piece has moved.

Thanks a lot for your help!
I think I'll move forward with this.

The middle is fine. As long as none of the leapers can jump out of the board from the 'physical area'.

The most important info that goes into the piece list is the location of the piece (i.e. the number of the square it is on, to be used as index in the 20x16 board).

You can store the piece type in the list, and then use that as index in a table that provides other information about that type. Such as piece value(s), a list of its possible moves or move directions, pointers to board-size tables (piece-square table, Zobrist keys). But I often store that detail information directly in the piece list. That leads to duplication of info, as the piece values etc. are then present in memory for all pieces of that type. But it provides faster access, as you don't have to get to the info in two steps, by getting the type first. Then it is often not needed to store the type at all; it is unambiguously determined by the list of properties of that type.

You could use a separate bit amongst the piece its info for indicating it is captured. But it is often easier to use an easily recognizable invalid value for its location in that case, such as -1. If you make a doubly linked list of the pieces, you would also have 'previous' and 'next' info in the piece list, and by following these 'pointers' to loop through the list, you would only encounter non-captured pieces, and there would be no reason to separately record that information anywhere.

To keep track of whether a piece has moved, there are better methods than recording it per piece. In most variants you don't care whether a piece has moved for most piece types, so it would be a waste of time to keep track of that. Usually it only matters for castling, i.e. for King and Rooks. You can use one bit in the game state for indicating whether a castling is spoiled by moving one of the pieces that could participate in it. In orthodox Chess that requires only 4 bits, which you could put all in the same integer variable. You can then use a 'spoiler' board array (i.e. indexed by square number), which indicates which castlings are spoiled by 'touching' each square. (E.g. a1 would spoil white Q-side castling, e1 would spoil both white castlings.) By doing

rights |= spoiler[fromSqr] | spoiler[toSqr];

you would keep track of which castling possibilities have been spoiled.

[catugocatugocatugo]

HGM,

Thanks, for your thoughtful and patient answer. Starting tomorrow I'll get to code writing. I think I got the basics. Then come back to more questions!

[catugocatugocatugo]

I have restarted this engine because in August 2023 I have got chatGPT 4-o. The reality is that due to a mental illness I can focus much and I am not that organized. ChatGPT help a lot with organization.
Now I'm restarting again as I face at least 2 problems.

First I have not made the fen to internal and vice versa conversion functions. It is hard to do debugging in this way. But that is easily solvable.

Second I am not happy with the isInCheck function when checking for check by bent riders. The way I have approached this is that not only the kPos is remembered but also the bent riders positions are remembered. When certain conditions are met (basically the king is possible on a bent rider ray), I search for the ray to see if it is empty. Anyway this is a bit tricky to test. Any suggestion here would be appreciated.

Last but not least: Happy new year!

[hgm]

Happy new year.

If you maintain a piece list you should know the location of any piece. The usual way of testing for checks (or other attacks) in mailbox is indeed to do a lookup indexed by relative location of the pieces, to see if they are 'aligned' such that attack is possible. For leapers alignment always implies attack. But sliders could be blocked, so you would have to scan the path. (A table lookup can provide the direction the attack can come from, for a given relative location.)

Bent sliders are a pain. For Griffon, Manticore and Osprey (which make a single leap before sliding in another direction) you could tabulate the square where they turn as a function of relative location. And then test if there is a free path between that and the target.

[jefk]

It is hard to do debugging in this way.

rather than going for the hard (and complicated) way of a full fledged
engine (with NN), why not first build a prototype with zillions og
https://www.chessprogramming.org/Zillions_of_Games
https://www.zillions-of-games.com/variants.html
https://www.chessvariants.org/tag/ZillionsOTB
http://cheonhyeong.com/English/Chess.html

So many chess variants are made for zillions (aka Zog), it's relatively easy to
program a board game in such a way , and possibly modify the game rules (of
your apothecary games) if in practice game play doesn't work out that well.

All up to you of course,
but anyway, maybe a useful (practical) suggestion

[catugocatugocatugo]

It is hard to do debugging in this way.

rather than going for the hard (and complicated) way of a full fledged
engine (with NN), why not first build a prototype with zillions og
https://www.chessprogramming.org/Zillions_of_Games
https://www.zillions-of-games.com/variants.html
https://www.chessvariants.org/tag/ZillionsOTB
http://cheonhyeong.com/English/Chess.html

So many chess variants are made for zillions (aka Zog), it's relatively easy to
program a board game in such a way , and possibly modify the game rules (of
your apothecary games) if in practice game play doesn't work out that well.

All up to you of course,
but anyway, maybe a useful (practical) suggestion

Thanks for your kind thoughts!
I did not knew about these resources on ZoG. Thanks!
The GUI programming is mostly up to ChatGPT, so it is not that hard.
Doing it the hard way has the hidden perk of understanding things better.
I'm not sure about ZoG supporting all that I need (like an imitator).
Isn't the ZoG's AI rather weak?

[jefk]

Isn't the ZoG's AI rather weak?

relatively weak for known games (as chess) but not with current fast hardware i think
for rare variants (where humans don't know how optimal play should go anyway)