For C experts

[Zach Wegner]

I think these are key. The only thing I wish I had a better handle on is the exact version.

One question: Can you check both option.c and init.c in Crafty, where you will find code that initializes the pawn hash table. In version 19.0, init.c had the _b/_w bug we are talking about in init.c and option.c (in option.c where the hashp command is handled as I have to do a malloc() for the new size, then initialize everything to be clean). So 19.0 screwed this up in both places. In 19.1, the init.c code was fixed. But the bug remained in option.c for many versions. If you find identical initialization code in both places in the rybka 1.6.1 binary, we would then know it is 19.0 or earlier. If it is in only one place, we would know it is 19.1 or later. Which narrows it down a bit. If I had some idea of when 1.6.1 was first released, I might could look thru a zillion games I have saved and find what crafty version was current around that time. However, I am not sure how complete the game coverage is since Crafty has played a _bunch_ of games on ICC and FICS over the years...

I'm not around the computer with the exe on it right now, but IIRC the same bug was in init.c (though the code we were discussing was indeed in 19.10).

I don't know the exact date, but I think they come from around the CCT6 time frame. But I guess we know that the Crafty version was indeed before 19.1.

[bob]

I think these are key. The only thing I wish I had a better handle on is the exact version.

One question: Can you check both option.c and init.c in Crafty, where you will find code that initializes the pawn hash table. In version 19.0, init.c had the _b/_w bug we are talking about in init.c and option.c (in option.c where the hashp command is handled as I have to do a malloc() for the new size, then initialize everything to be clean). So 19.0 screwed this up in both places. In 19.1, the init.c code was fixed. But the bug remained in option.c for many versions. If you find identical initialization code in both places in the rybka 1.6.1 binary, we would then know it is 19.0 or earlier. If it is in only one place, we would know it is 19.1 or later. Which narrows it down a bit. If I had some idea of when 1.6.1 was first released, I might could look thru a zillion games I have saved and find what crafty version was current around that time. However, I am not sure how complete the game coverage is since Crafty has played a _bunch_ of games on ICC and FICS over the years...

I'm not around the computer with the exe on it right now, but IIRC the same bug was in init.c (though the code we were discussing was indeed in 19.10).

I don't know the exact date, but I think they come from around the CCT6 time frame. But I guess we know that the Crafty version was indeed before 19.1.

in 19.0 and earlier, option.c and init.c both had that bug.

in 19.1, only option.c had the bug.

in 19.16 and later neither had the bug.

It was not in the Jakarta version (13.x) but was there by the 15's. I don't have all of those as those early versions were lost in a disk crash in 1990's. But they clearly pre-date Rybka

[sje]

I seem to recall that it was I who spotted Crafty's double-white/missing-black transposition table entry initialization bug; I notified Bob about it and he quickly fixed it. This must have been a long time ago in 1996 or earlier. But how much earlier I can't say.

[bob]

I seem to recall that it was I who spotted Crafty's double-white/missing-black transposition table entry initialization bug; I notified Bob about it and he quickly fixed it. This must have been a long time ago in 1996 or earlier. But how much earlier I can't say.

Only problem is, idiot programmer only fixed it in one place.

in 1996, we played in the WMCCC in Jakarta, with version 10.18.

The bug was fixed (partially) in version 18.0, and the other half was fixed in version 19.15. I suspect 18.0 was a couple of years after Jakarta because during that time, parallel search was done, which took time. Learning was added. Not to mention all the other usual idea changes including LMR and such. Crafty didn't exist until late December 1994, for reference. It was started right after the last ACM tournament, which was something like October or November of that year.

Stuart Cracraft helped me lash it up with xboard so that it would play on (at the time) FICS, later to become ICC.

[sje]

I seem to recall that it was I who spotted Crafty's double-white/missing-black transposition table entry initialization bug; I notified Bob about it and he quickly fixed it. This must have been a long time ago in 1996 or earlier. But how much earlier I can't say.

Only problem is, idiot programmer only fixed it in one place.

in 1996, we played in the WMCCC in Jakarta, with version 10.18.

The bug was fixed (partially) in version 18.0, and the other half was fixed in version 19.15. I suspect 18.0 was a couple of years after Jakarta because during that time, parallel search was done, which took time.

I am reminded of a similar duplicate-white/miss-black bug in my SAN Kit from about the same time; it wasn't in the hash table because there was no hash table. And it was found by some person more observant than I who emailed a correction. Chalk up another benefit of open source software.

In nearly all of my chess software over the past decade, I have tried to abstain from using explicit white/black scalar references in the code; two element vectors indexed by color are preferred. Also, nearly all recent code of mine refers to good/evil (color on-the-move/color not on-the-move) instead of white/black.

Here's a sample; it's the Lisp defstruct for a chess position taken from toay's version of the new CIL toolkit. It's called "pos" only because the name "position" is already taken by Common Lisp.

Code: Select all

;; ---------- Structure pos

(defstruct
  (pos
    (:include board)
    (:print-function meta-encode-pos))
  "A chess position with ancillary data"
  (good     cleared-good :type fixnum)                      ; The color (side) on the move
  (evil     cleared-evil :type fixnum)                      ; The color (side) not on the move (opposite of good)
  (cast     cleared-cast :type fixnum)                      ; Castling availability bits (see FEN specification)
  (epsq     cleared-epsq)                                   ; En passant target square, if any
  (hmvc     cleared-hmvc :type fixnum)                      ; Half move counter
  (fmvn     cleared-fmvn :type fixnum)                      ; Full move number
  (cens     (make-census) :type census)                     ; Board census
  (trac     (make-tracker) :type tracker)                   ; Target tracker
  (bbdb     (make-bbdb) :type bbdb)                         ; Bitboard database
  (ksqv     (make-array color-rlimit :initial-element nil)) ; Location squares of kings by color
  (rmtv     (make-array color-rlimit :initial-element 0))   ; Raw material total by color
  (pdsv     (make-array color-rlimit :initial-element 0))   ; Piece distribution signatures by color
  (inch     nil)                                            ; Good king in check flag (regenerated)
  (pmbb     (make-bb) :type bb)                             ; Pinned man bitboard (regenerated)
  (fmbb     (make-bb) :type bb)                             ; Frozen man bitboard (regenerated)
  (mdhs     (make-hash) :type hash)                         ; Merged data hash signature (board/cast/epsq)
  (pshs     (make-hash) :type hash)                         ; Pawn structure data hash signature
  (ltid     nil)                                            ; Last captured target ID (or nil)
  (cast-pdl nil)                                            ; Castling bits PDL
  (epsq-pdl nil)                                            ; En passant target PDL
  (hmvc-pdl nil)                                            ; Halfmove counter PDL
  (inch-pdl nil)                                            ; Good king in check flag PDL
  (pmbb-pdl nil)                                            ; Pinned man bitboard PDL
  (fmbb-pdl nil)                                            ; Frozen man bitboard PDL
  (ltid-pdl nil)                                            ; Last capture square/target ID PDL
  (hhss     nil)                                            ; History hashes (saved mdhs values)
  (hmvs     nil)                                            ; History moves
  (fmvs     nil)                                            ; Future moves
  (pops     0 :type fixnum)                                 ; Count of retraction pops available
  (ifen     (copy-seq cleared-fen) :type string))           ; Initial FEN string; set at position load

[bob]

I seem to recall that it was I who spotted Crafty's double-white/missing-black transposition table entry initialization bug; I notified Bob about it and he quickly fixed it. This must have been a long time ago in 1996 or earlier. But how much earlier I can't say.

Only problem is, idiot programmer only fixed it in one place.

in 1996, we played in the WMCCC in Jakarta, with version 10.18.

The bug was fixed (partially) in version 18.0, and the other half was fixed in version 19.15. I suspect 18.0 was a couple of years after Jakarta because during that time, parallel search was done, which took time.

I am reminded of a similar duplicate-white/miss-black bug in my SAN Kit from about the same time; it wasn't in the hash table because there was no hash table. And it was found by some person more observant than I who emailed a correction. Chalk up another benefit of open source software.

In nearly all of my chess software over the past decade, I have tried to abstain from using explicit white/black scalar references in the code; two element vectors indexed by color are preferred. Also, nearly all recent code of mine refers to good/evil (color on-the-move/color not on-the-move) instead of white/black.

Here's a sample; it's the Lisp defstruct for a chess position taken from toay's version of the new CIL toolkit. It's called "pos" only because the name "position" is already taken by Common Lisp.

Code: Select all

;; ---------- Structure pos

(defstruct
  (pos
    (:include board)
    (:print-function meta-encode-pos))
  "A chess position with ancillary data"
  (good     cleared-good :type fixnum)                      ; The color (side) on the move
  (evil     cleared-evil :type fixnum)                      ; The color (side) not on the move (opposite of good)
  (cast     cleared-cast :type fixnum)                      ; Castling availability bits (see FEN specification)
  (epsq     cleared-epsq)                                   ; En passant target square, if any
  (hmvc     cleared-hmvc :type fixnum)                      ; Half move counter
  (fmvn     cleared-fmvn :type fixnum)                      ; Full move number
  (cens     (make-census) :type census)                     ; Board census
  (trac     (make-tracker) :type tracker)                   ; Target tracker
  (bbdb     (make-bbdb) :type bbdb)                         ; Bitboard database
  (ksqv     (make-array color-rlimit :initial-element nil)) ; Location squares of kings by color
  (rmtv     (make-array color-rlimit :initial-element 0))   ; Raw material total by color
  (pdsv     (make-array color-rlimit :initial-element 0))   ; Piece distribution signatures by color
  (inch     nil)                                            ; Good king in check flag (regenerated)
  (pmbb     (make-bb) :type bb)                             ; Pinned man bitboard (regenerated)
  (fmbb     (make-bb) :type bb)                             ; Frozen man bitboard (regenerated)
  (mdhs     (make-hash) :type hash)                         ; Merged data hash signature (board/cast/epsq)
  (pshs     (make-hash) :type hash)                         ; Pawn structure data hash signature
  (ltid     nil)                                            ; Last captured target ID (or nil)
  (cast-pdl nil)                                            ; Castling bits PDL
  (epsq-pdl nil)                                            ; En passant target PDL
  (hmvc-pdl nil)                                            ; Halfmove counter PDL
  (inch-pdl nil)                                            ; Good king in check flag PDL
  (pmbb-pdl nil)                                            ; Pinned man bitboard PDL
  (fmbb-pdl nil)                                            ; Frozen man bitboard PDL
  (ltid-pdl nil)                                            ; Last capture square/target ID PDL
  (hhss     nil)                                            ; History hashes (saved mdhs values)
  (hmvs     nil)                                            ; History moves
  (fmvs     nil)                                            ; Future moves
  (pops     0 :type fixnum)                                 ; Count of retraction pops available
  (ifen     (copy-seq cleared-fen) :type string))           ; Initial FEN string; set at position load

All the black/white stuff in Crafty is long gone... now everything is indexed by color.

[micron]

In nearly all of my chess software over the past decade, I have tried to abstain from using explicit white/black scalar references in the code; two element vectors indexed by color are preferred. Also, nearly all recent code of mine refers to good/evil (color on-the-move/color not on-the-move) instead of white/black.

It's one of the great divides in chess programming. But surely nobody uses explicit white/black coding any more (other than a few well-known older programs)? Instead of tedious and seemingly interminable sequences of

Code: Select all

switch ( stm ) {...}

we want to write expressions resembling

Code: Select all

theBoard->my.pawns

and

Code: Select all

theBoard->opp.kingSquare

There are amusingly many ways to achieve this goal.

[sje]

In nearly all of my chess software over the past decade, I have tried to abstain from using explicit white/black scalar references in the code; two element vectors indexed by color are preferred. Also, nearly all recent code of mine refers to good/evil (color on-the-move/color not on-the-move) instead of white/black.

It's one of the great divides in chess programming. But surely nobody uses explicit white/black coding any more (other than a few well-known older programs)?

But I can think of one case where explicit color coding may be justified: a speed-above-all program, perhaps with critical routines written in assembly language. Even if color indexed vectors are used, constant propagation (either manually in assembler or automatically via compilation) could still produce faster code for move generation and attack calculation. While a few routines would be nearly duplicated, it might not adversely affect the CPU code cache hit rate.

My first assembly language program written some thirty years ago for the Zilog Z80 used such a technique.

[UncombedCoconut]

Using C++ templates or C macros, you can avoid explicit color coding but force the compiler to emit two copies of the function (which will be differently optimized). Stockfish, for instance, uses this trick a lot.