Telepath: 2008 ACCA Pan American Computer Chess Championship

[CRoberson]

I tried a different strategy in this tournament.
I noticed that Telepath plays better without a book than
with the one I created years ago. So, I created a shallower
book and autotuned it. The idea was to get other programs
out of book early. Based on the games with Symbolic and
ZCT, I think I went too far with the idea.

I used ChessPartner 5.4 in this tournament based on a
recommendation from GCP. I've used it several times for
online play and think it has some nice features for a program
that will connect to ICC. What I didn't know is that it has a
bug in its implementation of the UCI protocol. This became
grossly obvious during the 3rd round against Dirty. This bug
hurt Telepath in the first, thrid and fourth matches. More on this
as I go through the games.

Round 1: Telepath vs SlowChess
SlowChess has a CCRL rating around 2700. Based on some of
my testing Telepath has a chance. This match could set the tone
for the rest of the tournament. Tournament strategy hits on move
9 - both programs are out of book. SlowChess gets a pair of central
pawns controlling the center which gives it a slight edge and it is a
classic fight for the center. SlowChess keeps center occupation via
its pawns and Telepath uses its pieces to restrain their advancement.
On move 38, Telelpath attacks the base of the pawn phalanx and starts
to destroy blacks center control. In a classic response, black gets a
wing attack against the king. Sort of the opposite of the old
axiom - "the best way to counter a wing attack is with a counter attack
in the center".

By move 41, blacks center pawn control is gone and white has to
deal with an attack on its kingside. Also, Telepath is up a pawn for a
brief period. At move 55, several of us in the room are discussing this.
Ted, Bob and I think the game is a draw, but Rybka (with tablebases)
claims it is a win for black. We looked at Rybka's PV and we still don't
see the win. Yes, we were using several programs to real-time analyze
the games. Well, the game ends in a draw. A draw against a strong
opponent. My hopes for this tournament are up.

The ChessPartner 5.4 bug hurts Telepath a little in this game but
analysis reveals not too much. Turns out that a planned ponder
response move can be good (sometimes) even after a ponder miss.

Round 2: Romi vs Telepath
Telepath takes Romi out of book on move 8 and Romi resigns.
I wasn't even there for the game. We had gone to eat lunch at
the UAB cafeteria and I planned to buy my kids something from
the UAB student store. I left lunch early and was held up crossing
the street by a motorcycle parade and they all had Teddy bears.
Turns out it was a fund raiser or something for the local children's
hospital. After navigating the motorcycles (yes, I became impatient
after about 8 minutes), I ran into the student store and had two ladies
help me find stuff then ran back to see the game. Too late, it was
over.

There was some discussion about letting Romi start back up and
the TD said no. When I returned, several of us discussed it and we
decided to agree with the TD. The event is a programming contest and
as such these things happen. Bob even said that the same bug was
once in a very early version of Crafty.

Round 3: Telepath vs Dirty
This is the game where I rudely find out about a problem somewhere.
After the game, I analyzed the log files to find out that ChessPartner 5.4
doesn't properly handle ponder misses under the UCI protocol.

Telepath is out of book after move 5. Dirty is out of book at move 5.
Telepath goes back into book for a few moves and is back out at
move 11. I am happy with the opening, I prefer this type of setup for
white. On move 14, it happens. Dirty plays 13... BxN on c3. I expect
Telepath to make the obvious bc with the b2 pawn. No, I am stunned.
e4e5 is played. Dirty goes on a 1 minute 23 second think which seems
like an hour and runs its bishop away from the recapture. Now, Dirty
is up a pawn. I am amazed. I look at the CP 5.4 readout and it looks
like it may have dropped back into a bad book line. Pradu and Ted
were watching the game on Playchess.com (they were relaying the
event) and Andres tells Pradu that it was a bad ponder move.

After then game, I look at Telepath's logs. The log file clearly
shows that Telepath was pondering on Dirty playing Nc6d4 and would
respond with e4e5. The logs show Telepath receiving the stop command
after the pondermiss and then Telepath receives the new position.
After 1.3 seconds of search, Telepath is planning b2xc3, but CP 5.4
sends e4e5 to ICC. Telepath receives a setposition and "go ponder"
command for the bogus position while it is thinking on the real position.
I don't have time to code a work around before the next round.
Hopefully, Telepath will not have a big blunder due to it.

Round 4: Telepath vs Arasan
I've had several ICC matches against Arasan and every time I improve
Telepath enough to equalize with Arasan Jon improves Arasan. The last
time he bought new hardware. So, I am expecting a tough match.
Despite my book strategy, Telepath stays in book for 16 moves and Arasan
seems out of book at move 12. Well, that is still early than move 27.

By move 22, it is another classic battle. Telepath is trying to control
the center and Arasan is countering on the wing and trying to get
a king attack. So, whose king safety code is better?

Arasan achieves an attack at move 28. Telepath is out searched.
Telepath on one CPU of a Q6600 didn't look as deep as Arasan on
4 CPUs of a Qx6750.

That night
Ted, Bob, Brian, Pradu and I go to Ruby Tuesday's for dinner.
We spend hours talking about computer chess, optimization algorithms,
previous tournament experiences and politics. The food was great
and the company was better.

The next morning
Brian and I are discussing the Game 3 lose over breakfast. I explain
the relevant parts of the UCI protocol and what the logs say happened.
I thought up a quick work around for the problem and Brian seems to think
that I shouldn't make code changes just before the round starts.

We get to the room and I code and test the work around in 50 minutes.
This shouldn't work to any GUI that properly handles the UCI protocol
for ponder miss, but it does work for ChessPartner 5.4.

So, here is the issue.
The UCI protocol states that all "go" commands must be followed
by a "bestmove" command. The "go" command is used to tell the
engine to think and to ponder. So, the GUI sends "go ponder...."
to the engine. On a ponder miss, the GUI sends a "stop" command
and the engine sends a "bestmove" command which is bogus due
to the ponder miss. Now, the GUI sends the new position and a
"go" command to search. After the search completes the engine
sends a valid "bestmove" command. So, after each ponder miss
two best move commands are sent by the engine and the first is
bogus.

My work around was to not send the bogus "bestmove" command.
For any GUI that implements the UCI protocol as specified in the
document by RH and SMK, this should cause the GUI to hang because
it should expect two "bestmove" commands. Under Arena and Fritz 8
GUI's, it does hang on a ponder miss. In ChessPartner 5.4, things work
fine. So, I start up CP 5.4 and the patched Telepath to finish out the
tournament.

Round 5: Symbolic vs Telepath
This is an Alekhines defense and Telepath follows a variation that
I prefer it didn't (1. e4, Nf6 2. e5, Nf8 instead of Nd5). On the bright
side, we are out of book in less than 5 moves. Telepath gains an
edge and Symbolic traps one of its rooks with its king. I am dissapointed
that Telepath never gains a winning advantage due to this.
The game is long and Telepath is in time trouble. I wrote and rewrote
new code the 5 days leading up to the tournament for the timing
algorithm. Specifically, when Telepath has less than 10 seconds left
and when it has less than 5 seconds left. I am hoping it doesn't lose
on time. Bob asks about the timing algorithm in Telepath and states
that it is burning 5 secs per move. Brian says so what there is a
10 sec increment. Then we get into a debate on the definition of
burn. Telepath was using 15 seconds per move and thus burning
5 seconds of clock time per move and it only had 3 minute left while
Symbolic had more than 10 minutes left at move 72.
We surmise that it is a drawn game but can't call it. There is plenty
of room for either side to goof up not to mention Telepath running
out of time.

We are all hungry and are debating where, when and how to eat
giving the game is still going and it is the last game of the round to
complete. After discussing several options, we order pizza delivery.

The game is still going past 100 moves. At move 103, Symbolic
has 5 min 38 seconds left and Telepath has 37 seconds left. It looks
like a draw but will the new timing algorithm hold and still give Telepath
enough time to make good enough moves. Finally, Telepath gets the
draw. The pizza arrives 5 minutes before the start of the next round.

Round 6: ZCT vs Telepath
Telepath stays in book until move 8 and about the same for ZCT.
So, the tournament book strategy is in effect this game. It looks like
ZCT goes back into book and stays there until move 18 while Telepath
stays out of book and racks up a time disadvantage.

At move 20, Telepath has an edge due to ZCT blocking its rook with
its king, but ZCT has 39+ minutes to Telepath's 24 minutes. As I understand
it, ZCT is using a multiprocessor search to Telepath's single processor
search. At move 32, Telepath has 4 active pieces to ZCT's 2 active
pieces. ZCT's rooks are passive at this point, but Telepath never
comes up with a decisive attack. I think the issue is the time. Telepath
has only 17 minutes left. I think I'll go back to deeper books for
tournaments.

At move 42, Telepath has an extra pawn and it is an outside passer.
I am rooting for Telepath to advance the passer with the rook behind
for support. If it can do this, ZCT may be forced to sac a piece to
keep the pawn from promoting. At that point, the game will be
won for Telepath. That doesn't happen. The game goes into a long
sequence similar to the match with Symbolic ending with insufficient
material.

The tournament was great fun and informational. Lots of ideas shared
and much talk about eval tuning techniques. It was great to meet
Ted and Pradu as well as seeing Brian and Bob again.

[swami]

Thanks for the wonderful Tourney, Charles! I enjoyed talking to others and watching the games etc, I hope there will be more Online ACCA tourneys in the future. Official tourneys in comp chess is such a rarity.

[Andres Valverde]

Thx Charles for the interesting report !, and sorry about the trouble in the game vs Dirty, I expected a tough fight in that game.

Can you please fix the link for the games PGN file ? I had a wrong email address set up at ICC and couldn't receive any.

[CRoberson]

Thanks for the wonderful Tourney, Charles! I enjoyed talking to others and watching the games etc, I hope there will be more Online ACCA tourneys in the future. Official tourneys in comp chess is such a rarity.

The ACCA has 4 to 5 events per year. This has been going on for the
last 3 years. The 5th event is difficult to organize, so we've had it only
once. That is the Western Hemisphere vs Eastern Hemisphere team
match.

The other events are as follows:

Code: Select all

      o Annual ACCA President's Tournament:
            Meant to be a fun event and the format changes each year.
            Last year, it was a one day blitz event.
            Typically held around April. This year I'll try for March depending
             on the date of the CCT.
      o  Annual ACCA World Computer Rapid Chess Championships
             This is a G/25 +4 event. It is held in June or July depending
             on the date of the ICGA event. Open to all world wide.
      o Annual ACCA Pan American Championships.
           Typically in late October or early November. Open only to authors
            that are citizens or residents of the Pan American region.
      o The fourth event is an ongoing one. We started an ACCA Computer
         Chess Ladder limited to the Pan American region. It had activity at
         first then died down. We decided to open it up to all in the world.
         I hope we see more activity on it. Some have come up with excuses
         like their computer is not fast enough. Since this is just a one on one
         four game match. One can get loaner hardware. The author can 
         be online for discussion but an operator with fast hardware can 
         run the program. The TC is G/14 +1.

[CRoberson]

Thx Charles for the interesting report !, and sorry about the trouble in the game vs Dirty, I expected a tough fight in that game.

Can you please fix the link for the games PGN file ? I had a wrong email address set up at ICC and couldn't receive any.

Yes, I was expecting a tough fight in that game as well.

I have collected all the games except two: Tinker vs Amateur
and Amateur vs Slowbo. I will post all games when I get those.

Can somebody post those two games in this thread?

[CRoberson]

Given that I discussed an issue with ChessPartner, I must state the
resolution.

I ran several experiments and contacted Lex. He thought that
there shouldn't be a problem on his end but was open to the thought.
During our discussion, I suggested there might be a race condition
which would explain all my tests results. He responded with ChessPartner
only gives 3 seconds to send the bogus bestmove command.
Well, that is a race condition, but one would think that is plenty of
time. Lex told me how to turn on ChessPartner's logging utility.

The logs were excellently detailed and had timestamps down to
the millisecond level. They revealed that Telepath had 2 inefficiencies.
The first was delay in the noticing of a search interrupt while in the Qsearch.
The second was that ChessPartner delays by 2 seconds after every
"uciok" message and for some reason Telepath was sending extra
"uciok" messages after every move. So, that alone caused a 2 second
delay from ChessPartner then add that to the other delay that varied
in length and you have a problem.

So, I modified the Q search issue. It still exists but is much better
than before and I removed the extra "uciok" message. Testing
revealed Telepath gaining 2+ seconds per search. Also, the errant
incorrect ponder miss move is gone. I ran it on ICC for a night and
woke up to see over 100 extra ELO. The bad thing is that this seems
to have caught the attention of much stronger engines running on
ICC and they commenced to taking points.

Overall, my experience dealing with Lex was quite excellent. It took
very little time (3 days which is fast via email in different time zones).
I am not sure I believe he needs a timer in his code, but maybe so
because he is supporting Xboard and UCI protocols. All server software
I've seen that directly support both protocols typically use timers.

[BubbaTough]

I ran it on ICC for a night and
woke up to see over 100 extra ELO. The bad thing is that this seems
to have caught the attention of much stronger engines running on
ICC and they commenced to taking points.

I find my rating on ICC is less related to the strength of my engine in general and more related to my results against Rybka 3. The only defense against those monsters is making sure your engine can achieve occasional draws, 1 in 10 or even 1 in 20 will protect you from those beasts .

-Sam

[CRoberson]

I ran it on ICC for a night and
woke up to see over 100 extra ELO. The bad thing is that this seems
to have caught the attention of much stronger engines running on
ICC and they commenced to taking points.

I find my rating on ICC is less related to the strength of my engine in general and more related to my results against Rybka 3. The only defense against those monsters is making sure your engine can achieve occasional draws, 1 in 10 or even 1 in 20 will protect you from those beasts .

-Sam

Yes, that can be a problem. My previous solution was to put them
on my noplay list. With the new strength, Telepath is getting 2 draws
out of 10 against Q9550 (Rybka 3 on a Q9550).