Tunguska 1.1 STS test results - improve eval

[Necromancer]

Hi there,
So here are the results for Tunguska 1.1:

movetime 50 ms
id "STS(v1.0) Undermine 37/100
id "STS(v10.0) Simplification 67/100
id "STS(v11.0) King Activity 28/100
id "STS(v12.0) Center Control 44/100
id "STS(v13.0) Pawn Play in the Center 51/100
id "STS(v14.0) 7th Rank 45/100
id "STS(v15.0) AT (avoid trades) 23/100
id "STS(v2.2) Open Files and Diagonals 29/100
id "STS(v3.0) Knight Outposts/Repositioning/Centralization 43/100
id "STS(v4.0) Square Vacancy 34/100
id "STS(v5.0) Bishop vs Knight 52/100
id "STS(v6.0) Recapturing 45/100
id "STS(v7.0) Simplification 33/100
id "STS(v8.0) AKPC 24/100
id "STS(v9.0) Advancement of a/b/c pawns 29/100
Found 584 of 1500 (38.9%)

movetime 1000ms
id "STS(v1.0) Undermine 49/100
id "STS(v10.0) Simplification 64/100
id "STS(v11.0) King Activity 42/100
id "STS(v12.0) Center Control 57/100
id "STS(v13.0) Pawn Play in the Center 60/100
id "STS(v14.0) 7th Rank 58/100
id "STS(v15.0) AT (avoid trades) 24/100
id "STS(v2.2) Open Files and Diagonals 37/100
id "STS(v3.0) Knight Outposts/Repositioning/Centralization 51/100
id "STS(v4.0) Square Vacancy 50/100
id "STS(v5.0) Bishop vs Knight 59/100
id "STS(v6.0) Recapturing 55/100
id "STS(v7.0) Simplification 48/100
id "STS(v8.0) AKPC 39/100
id "STS(v9.0) Advancement of a/b/c pawns 31/100
Found 724 of 1500 (48.3%)

My search routine still needs a rewrite from scratch, but given the results above, what kind of evaluation code would bring more benefits?
Currently it has just the basics: material, piece squares, pawns (isolated, passed, connected), open files, king safety and a very basic outposts eval.

In particular, the engine keeps accepting pointless trades. Any strategy to improve on that??

https://github.com/fernandotenorio/Tunguska/

[Dann Corbit]

In particular, the engine keeps accepting pointless trades. Any strategy to improve on that??

If you are behind in material, never trade unless there are no alternative moves. Bravely run away!
If you are even and well behind in development, never trade unless there are no alternative moves. Bravely run away!
Don't forget that a pair of bishops gives added punch. That will keep you from trading a bishop for a knight if you have the other bishop (unless there is a driving need to do it).

Have a look at material imbalance. You can calculate it heuristically from actual game data, or you can calculate it on first principles.

[Necromancer]

Cool, I'll look into material imbalances.
Regarding the "run away", can I use SSE to help?

Something like

if (isCapture(move)) {
if (see() == 0 and relativ_material >= 0){
move.score = bad capture score
}
}

[Dann Corbit]

I think move score = bad capture score is too severe.
A small penalty as a function of the amount of material difference should be sufficient.

A small logistic curve to give perhaps +50 centipawns when you are way ahead to trade and -50 centipawns when you are behind.
(Those are not tuned values, just conceptually small to show you don't want to put searching the trade way at the end of the list. It might turn out well).
Do you have a bishop pair bonus?
You should also have a bonus for knights if there is a long blocking pawn chain because bishops are not useful for that and knights are.
You should also devalue a bad bishop.
If you have an outpost next to the enemy king, a knight with a pawn guarding it can be the value of a rook.

Larry Kaufman wrote the first article I ever saw on imbalances in chess.
I think that Ed Schroeder has a tool that will calculate an imbalance table from a large pool of PGN.

Something that might be worth examination is to read things that other people have written on evaluation. I seem to recall that Ed has a nice long discussion about it at his site somewhere.

[Necromancer]

Yep, I'll try these things out. Actually, I have disabled many eval terms, either because they are not tuned or because they lose ELO.
Mobility is a example that loses ELO.

Bishop pair bonus, piece values and attack weights were tuned using texel, which gave me a good ELO boost (especially piece values).
What I need to to is rewrite the eval, so that it is easier to tune using keras.

One thing that I never saw in other linear-eval engines is parameter interaction. This requires some data munching experiments, so maybe I'll try that later. Thanks!

[jd1]

Yep, I'll try these things out. Actually, I have disabled many eval terms, either because they are not tuned or because they lose ELO.
Mobility is a example that loses ELO.

That is surprising as mobility typically adds a very significant amount of Elo to most alpha-beat chess programs.

I wonder if there is a bug, very poorly tuned values, or perhaps your mobility routine is simply far too slow. It is surely worth investigating this further before tuning the rest of the evaluation and trying to add new terms, as mobility evaluation is quite fundamental and alters the playing style of the engine considerably.

[Dann Corbit]

I will second that. Look at how much mileage the engine Olithink gets out of mobility.

[Necromancer]

That's very good to hear, because Tunguska needs some ELO

I follow more or less the idea of a mobility area (don't count squares where enemy pawns are attacking). I use magic bitboards for attack generation, so I don't think it's slow (it is of course relatively slow, compared to other parts of the eval function).

I also remember reading somewhere that mobility is tricky, some engines benefit from it others don't. My eval is so simple, there is not much stuff that could be a proxy for mobility, apart from piece squares and open files for rooks/queen. I'm reviewing it right now =)

[Necromancer]

After 5350 games (100 ms fixed per move, hash table 256MB):

score of Tunguska mob vs. Tunguska dev = 50.69% +/- 1.09%

So almost no improvement for a ~2450 engine. Here is my mobility code along with the bonus which I copied from some other engine. I know that the endgames bonuses should be different, but I leave that for tuning. Also AttackCache is just a structure with the attacked squares cached.

I don't see any bug, do you?

Code: Select all

static const int MOB_N[2][9] = {{-15, -8, -6, -4, 0, 6, 8, 12, 15}, {-15, -8, -6, -4, 0, 6, 8, 12, 15}};
static const int MOB_B[2][14] = {{-15, -9, -8, -7, -5, -2, 0, 2, 4, 6, 8, 10, 12, 15}, {-15, -9, -8, -7, -5, -2, 0, 2, 4, 6, 8, 10, 12, 15}};
static const int MOB_R[2][15] = {{-15, -9, -8, -7, -6, -5, -4, 0, 2, 4, 6, 8, 10, 12, 12}, {-15, -9, -8, -7, -6, -5, -4, 0, 2, 4, 6, 8, 10, 12, 12}};
static const int MOB_Q[2][28] = {{-15, -10, -9, -9, -8, -8, -7, -7, -6, -6, -5, -5, -4, -2,0, 2, 2, 4, 4, 6, 6, 8, 8, 10, 10, 12, 12, 12}, {-15, -10, -9, -9, -8, -8, -7, -7, -6, -6, -5, -5, -4, -2,0, 2, 2, 4, 4, 6, 6, 8, 8, 10, 10, 12, 12, 12}};		

void Evaluation::mobility(const Board& board, int& mg, int& eg, AttackCache *attCache){
	
	U64 occup = board.bitboards[Board::WHITE] | board.bitboards[Board::BLACK];	
	int s = 1;
	int n = 0;

	for (int side = 0; side < 2; side++){
		
		int opp = side^1;
		U64 oppPawnBB = board.bitboards[Board::PAWN | opp];
		U64 oppPawnAttacks = 0;
		
		// TODO avoid if else, maybe templatize
		if (opp == Board::WHITE){
			oppPawnAttacks = ((oppPawnBB << 9) & ~BitBoardGen::BITBOARD_FILES[0]) | ((oppPawnBB << 7) & ~BitBoardGen::BITBOARD_FILES[7]);
		} else{
			oppPawnAttacks = ((oppPawnBB >> 9) & ~BitBoardGen::BITBOARD_FILES[7]) | ((oppPawnBB >> 7) & ~BitBoardGen::BITBOARD_FILES[0]);
		}
		
		//enemy or empty squares, also exclude opp king square and ranks 1 and 2
		U64 mobilityArea = ~(board.bitboards[side] | board.bitboards[Board::KING | opp] | oppPawnAttacks | exclude_ranks[side]); 
		
		//knights
		if (board.bitboards[Board::KNIGHT | side]){
			n = BitBoardGen::popCount(attCache->knights[side] & mobilityArea);
			mg+= s * MOB_N[0][n];
			eg+= s * MOB_N[1][n];
		}

		//bishops
		if (board.bitboards[Board::BISHOP | side]){
			n = BitBoardGen::popCount(attCache->bishops[side] & mobilityArea);
			mg+= s * MOB_B[0][n];
			eg+= s * MOB_B[1][n];
		}
		
		//rooks
		if (board.bitboards[Board::ROOK | side]){
			n = BitBoardGen::popCount(attCache->rooks[side] & mobilityArea);
			mg+= s * MOB_R[0][n];
			eg+= s * MOB_R[1][n];
		}

		//queen
		if (board.bitboards[Board::QUEEN | side]){
			n = BitBoardGen::popCount(attCache->queens[side] & mobilityArea);
			mg+= s * MOB_Q[0][n];
			eg+= s * MOB_Q[1][n];
		}
		
		s = -1;
	}
}

[jdart]

mobility area being "enemy or empty squares, also exclude opp king square and ranks 1 and 2" does not make much sense to me. I'd just include empty squares and exclude squares attacked by enemy pawns, at least for starters. Commonly pawns that are attacked are part of a chain and actually restrict mobility, so being able to move to an enemy pawn is maybe not a thing you want a bonus for. The other comment I have is that the values need to be tuned. If a pawn value is 100 or something like that, then -15 for zero mobility is probably not enough penalty.

--Jon