Pedantic Developer's Log Stardate...

[JoAnnP38]

I tentatively made a major milestone today as I am finally getting reasonable (and if not reasonable at least understandable) results back from my implementation of search. I am currently using iterative deepening with a standard negamax search (no PVS, no null moves) with very crude windowing of alpha/beta. I understand that I may not know exactly what I am doing at the moment, but I felt like I had to get something outputting results just so I can place a stake in the ground. This is what my iterative deepening with windowing looks like:

Code: Select all

public short Search()
{
	short score = Evaluation.Compute(board);
	short depth = 0;
	while (depth++ < maxSearchDepth)
	{
		short alpha = (short)(score - Constants.ALPHA_BETA_WINDOW);
		short beta = (short)(score + Constants.ALPHA_BETA_WINDOW);
		score = Search(alpha, beta, depth);

		if (score <= alpha)
		{
			alpha = -Constants.INFINITE_WINDOW;
			score = Search(alpha, beta, depth);
		}
		else if (score >= beta)
		{
			beta = Constants.INFINITE_WINDOW;
			score = Search(alpha, beta, depth);
		}
		Uci.Info(depth, score, nodes, PV.Moves[0]);
	}

	return score;
}

At first it was returning nonsensical moves until I figured out that I had the sign reversed on my evaluation function. And now, while the moves may not be perfect, and given I am just now starting serious testing I am still happy that they moves don't look completely unreasonable for a chess duffer like myself.

Here is the output I captured from one of my tests:

Code: Select all

info depth 1 score cp 137 nodes 21 nps 3000 time 7 pv e2e4
info depth 2 score cp 0 nodes 174 nps 17400 time 10 pv d2d4 d7d5
info depth 3 score cp 116 nodes 2579 nps 117227 time 22 pv d2d4 d7d5 g1f3
info depth 4 score cp 15 nodes 18890 nps 178207 time 106 pv e2e4 e7e5 d2d4 e5d4 d1d4
info depth 5 score cp 106 nodes 159321 nps 398302 time 400 pv d2d4 d7d5 g1f3 g8f6 c1f4
info depth 6 score cp 4 nodes 1104873 nps 764089 time 1446 pv d2d4 d7d5 e2e4 d5e4 f1b5 c7c6 b5c4

The performance is not currently great, but I don't have much of the infrastructure in place to improve pruning (i.e. no killers, no history, no PV sorting). I actually had to comment out my code for reductions for now and that should help with pruning in the future. I just noticed that the node count looks to be off from what I would expect. I definitely need to look into that.

If anyone has any suggestions about how to test a new engine, I would love to hear them. I still need to get UCI complete wired up and implement time controls. I also don't have checkmate or stalemate checking complete correct yet so I know I can't play a game yet, but it's a start.

[Mike Sherwin]

At first you will need a small set of positions to test with. It needs to be small because the turnover rate for new things to try and for tuning will be quite fast. Here is one that you can use.
https://www.mediafire.com/file/mlqei71i ... 0.pgn/file

If you want to make your own set here is a pgn file with all the columns from Chess Openings the Easy Way by Nick DeFirmmin (spelling?). Just truncate each line where you want to.
https://www.mediafire.com/file/6m9qn4hx ... y.zip/file

[JoAnnP38]

At first you will need a small set of positions to test with. It needs to be small because the turnover rate for new things to try and for tuning will be quite fast. Here is one that you can use.
https://www.mediafire.com/file/mlqei71i ... 0.pgn/file

If you want to make your own set here is a pgn file with all the columns from Chess Openings the Easy Way by Nick DeFirmmin (spelling?). Just truncate each line where you want to.
https://www.mediafire.com/file/6m9qn4hx ... y.zip/file

Thanks Mike. It seems like tomorrow I'm going to be busy!

[JoAnnP38]

I'm still making good progress at this point. I've played several games against Spike 1.4 (lost every single game), but it is very instructive all the same. I definitely need to take a hard look at my Quiescence search as it always seems to be terminating too early thus ignoring some important capture. Of course, Pedantic only is able to get to depth 8 or 9 in the time constraints while Spike is zipping along to 16-18 ply deep. Correctness first and performance later, right? Also, it is very careless with its pawns, but I think that is just my evaluation that needs to be optimized. I also have my opening book working. I created the Polyglot openings file months ago so it's nice to see that part working. However, it seems to have some illegal moves in it. I guess it could be from an incorrectly parsed PGN or perhaps something else. Arena of course calls the game as soon as Pedantic makes an illegal move. Here is one where the black king is slapping down his own rook .


[fen]r2qk2r/5pbp/p1npb3/3Np2Q/2B1Pp2/N7/PP3PPP/R4RK1 b kq - 0 15[/fen]
bestmove e8h8???

I guess I need to write a utility to go through every position in my opening book file to verify all of the responses are legal. I love it!

[JacquesRW]

[fen]r2qk2r/5pbp/p1npb3/3Np2Q/2B1Pp2/N7/PP3PPP/R4RK1 b kq - 0 15[/fen]
bestmove e8h8???

This could be an alternate way of writing kingside castling, which is a legal move in this position.

Also is the source code available?

[JoAnnP38]

[fen]r2qk2r/5pbp/p1npb3/3Np2Q/2B1Pp2/N7/PP3PPP/R4RK1 b kq - 0 15[/fen]
bestmove e8h8???

This could be an alternate way of writing kingside castling, which is a legal move in this position.

Yes, you are correct. I'll need to update my code to reflect that.

Also is the source code available?

As soon as I make the update above I will share my repo.

[JoAnnP38]

Pedantic has now played many games against the engines included in Arena; however, these engines are too strong at the moment. I have managed to sneak out a draw here or there, but I have yet to win a game against one of these engines. Typically, during the opening (at least after I've exhausted my opening book) the engine can search to a depth of 13 or 14 which I thought was really good, but it doesn't stack up against Arena's bundled engines. I really need to spend some time in a serious optimization phase to see if more NPS can be achieved. Three of the biggest improvements I made to performance are:

1. Changed my move generation so that it incrementally generates moves as one is needed and in the correct order so that sorting isn't required (i.e. PV move, captures, killers and quiet moves). This also saves time if the any moves end up getting pruned by futility/delta pruning. No use spending time generating something that isn't used.
2. Removed my attempt at aspiration windows. My PV search was too unstable when using narrow windows which seems to reduce the effectiveness of that search.
3. There are a couple of places where I have implemented pruning. I'm using futility margins in the main search and delta pruning in the quiescence search. My conditions for pruning were too expensive so I had to settle on simpler ones.

Originally, I had written a specialized zero-window search, but I have opted to just reuse my main search with appropriate alpha/beta. I may go back to the ZwSearch implementation when I have more time to really dig into why it doesn't perform as expected.

Question to the group -- does anyone know of any UCI engines rated between 1800 and 2200? I'm guessing this is the around the ELO strength of Pedantic, but I really need to see it win some games before I can be sure.

One final thing, I've made the Pedantic repository public if anyone wants to look at the code. Several pieces are highly derivative of lithander's MinimalChess. I had no clue how chess time controls worked or how to implement them, so I pretty much swiped his TimeControl class and I modeled the interface between the Engine and UCI after it as well. When I have time to comment my code I'll be sure to pay appropriate homage. MinimalChess is still quite a bit faster than Pedantic and it has won every match between the two except for one draw based on 3-fold repetition. Anyone interested in C# chess engines would be well advised to look over that engine. It is simple, fast, elegant and to the point. I've already told him that I believe his coding is quite beautiful and it's amazing how well it plays with such small and elegant code. Many other engines I've looked over are in terrible need of refactoring. Pedantic needs that as well. I have a lot of dead code or code that I thought I would need but no longer do.

[JacquesRW]

I really need to spend some time in a serious optimization phase to see if more NPS can be achieved.

I would recommend not worrying about it too much, beyond a few million NPS the benefits of increasing speed diminish quickly, and pruning is far more beneficial.

Question to the group -- does anyone know of any UCI engines rated between 1800 and 2200? I'm guessing this is the around the ELO strength of Pedantic, but I really need to see it win some games before I can be sure.

I’d recommend checking out the CCRL Blitz list:
http://ccrl.chessdom.com/ccrl/404/
In your elo range Rustic is always reliable:
https://github.com/mvanthoor/rustic

[hgm]

Question to the group -- does anyone know of any UCI engines rated between 1800 and 2200? I'm guessing this is the around the ELO strength of Pedantic, but I really need to see it win some games before I can be sure.

Fairy-Max/micro-Max is always a good benchmark, because it is stripped of almost all knowledge. It should be around 1950 CCRL. But it is a WB engine, not UCI.

[lithander]

One final thing, I've made the Pedantic repository public if anyone wants to look at the code. Several pieces are highly derivative of lithander's MinimalChess. I had no clue how chess time controls worked or how to implement them, so I pretty much swiped his TimeControl class and I modeled the interface between the Engine and UCI after it as well. When I have time to comment my code I'll be sure to pay appropriate homage. MinimalChess is still quite a bit faster than Pedantic and it has won every match between the two except for one draw based on 3-fold repetition. Anyone interested in C# chess engines would be well advised to look over that engine. It is simple, fast, elegant and to the point. I've already told him that I believe his coding is quite beautiful and it's amazing how well it plays with such small and elegant code.

Thanks for your kind words about MinimalChess!

Question to the group -- does anyone know of any UCI engines rated between 1800 and 2200? I'm guessing this is the around the ELO strength of Pedantic, but I really need to see it win some games before I can be sure.

You could play against earlier version like 0.4 (1890 Elo) or version 0.5 (2270 Elo) of MinimalChess of course. And you may want to try Leorik 1.0 which is also in the desired Elo range (2150) but has the interesting property that it does not implement any unsafe pruning techniques, reductions, extensions etc. Not even null-move pruning. This causes a high branching factor and the search remains quite shallow even at higher time controls. But it solves all mate puzzle with the shortest path for example because it never overlooks a move. This could expose some weaknesses in your engine if you overdo it with the pruning and reductions!