Can a chess engine pray?

[Dann Corbit]

Sounds silly, but wait a minute...
An endgame tablebase is perfect information. Infallible.
Stockfish tends to get in trouble against LC0, but but then gives LC0 a beating in the endgame.
Stockfish often has 1000 times as many TB hits in the endgame compared to LC0 on my machine.
So, as a model, we can say that SF spends a great deal of time talking to God.
LC0, on the other hand, spends a great deal of time using nothing but his own wits.
Imagine all those idle threads that LC0 is not using. Suppose that all of those threads were searching the tablebase files like madmen...
I suspect that LC0 would do a lot better if he spent more time "in prayer."

[Mike Sherwin]

Sounds silly, but wait a minute...
An endgame tablebase is perfect information. Infallible.
Stockfish tends to get in trouble against LC0, but but then gives LC0 a beating in the endgame.
Stockfish often has 1000 times as many TB hits in the endgame compared to LC0 on my machine.
So, as a model, we can say that SF spends a great deal of time talking to God.
LC0, on the other hand, spends a great deal of time using nothing but his own wits.
Imagine all those idle threads that LC0 is not using. Suppose that all of those threads were searching the tablebase files like madmen...
I suspect that LC0 would do a lot better if he spent more time "in prayer."

As an analogy sure, why not. But why not take it a step further. If a chess engine all of a sudden sees it is going to lose a pawn (give or take) then why not start praying by doubling the importance of the positional evaluation. Maybe then give up a second pawn on faith that "God" will show the way.

[lithander]

Stockfish tends to get in trouble against LC0, but but then gives LC0 a beating in the endgame.

Does that mean a hypothetical hybrid engine that uses LC0 during the midgame and switches to Stockfish in the endgame would be the world's strongest engine?

[Dann Corbit]

Stockfish tends to get in trouble against LC0, but but then gives LC0 a beating in the endgame.

Does that mean a hypothetical hybrid engine that uses LC0 during the midgame and switches to Stockfish in the endgame would be the world's strongest engine?

It has been tried and was fairly successful. There were some posts on that topic in the General board a while back.
But I think that approach is a mistake.
It would be better to have an engine that can do a GPU and Alpha Beta search (not sure if it is necessary to do it simultaneously) all using the same hash table. When a GPU search is running, typically the engine will only use 3 threads. That leaves 61 free on my machine. Why not use them?

I think the new infinity fabric approach might make it feasible to do so with no copy operations or other horrid lags.

I guess it would be possible to write an engine stronger than either SF or LC0 because it would contain both strengths and neither weaknesses.

[smatovic]

Stockfish tends to get in trouble against LC0, but but then gives LC0 a beating in the endgame.

Does that mean a hypothetical hybrid engine that uses LC0 during the midgame and switches to Stockfish in the endgame would be the world's strongest engine?

It has been tried and was fairly successful. There were some posts on that topic in the General board a while back.
But I think that approach is a mistake.
It would be better to have an engine that can do a GPU and Alpha Beta search (not sure if it is necessary to do it simultaneously) all using the same hash table. When a GPU search is running, typically the engine will only use 3 threads. That leaves 61 free on my machine. Why not use them?

I think the new infinity fabric approach might make it feasible to do so with no copy operations or other horrid lags.

I guess it would be possible to write an engine stronger than either SF or LC0 because it would contain both strengths and neither weaknesses.

Interesting take, run different search algorithms with own eval on the same game tree, communication via game tree in memory or hash table, shared across CPU/GPU/etc.

---
Srdja

[Joerg Oster]

Stockfish tends to get in trouble against LC0, but but then gives LC0 a beating in the endgame.

Does that mean a hypothetical hybrid engine that uses LC0 during the midgame and switches to Stockfish in the endgame would be the world's strongest engine?

It has been tried and was fairly successful. There were some posts on that topic in the General board a while back.
But I think that approach is a mistake.
It would be better to have an engine that can do a GPU and Alpha Beta search (not sure if it is necessary to do it simultaneously) all using the same hash table. When a GPU search is running, typically the engine will only use 3 threads. That leaves 61 free on my machine. Why not use them?

I think the new infinity fabric approach might make it feasible to do so with no copy operations or other horrid lags.

I guess it would be possible to write an engine stronger than either SF or LC0 because it would contain both strengths and neither weaknesses.

You have an engine which runs a GPU search?

[Ronald]

It's very difficult to let different searchtypes communicate via the hashtable. One mayor problem seems to be the role of the depth parameter in using the hashtable. If searches have too much differing depths, the searches with much higher depth will probably hardly use any information of the other searches. I did experiments with excluding some prunings in certain threads, but I never got it to work, probably due to the lesser depth of those threads.

[mvanthoor]

Does that mean a hypothetical hybrid engine that uses LC0 during the midgame and switches to Stockfish in the endgame would be the world's strongest engine?

Not per se.

Engines have a distinct playing style due to how the evaluation function is built. Therefore, if you give one engine a position that another engine finds very favorable, the first engine may not like it at all, because it contains elements that it either doesn't evaluate, or evaluates differently. It will play a move according to that evaluation, but it may not be the best move for _that_ particular position, because the engine doesn't understand it.

It is the same for humans. Karpov, in the 70's and early 80's was known for a very slow positional play. Some players completely hated playing against him, because he was willing to just sit there for 8 hours and squeeze them to death one pawn move at a time like a steamroller driving at two miles/hour. Attackers hated to play against Petrosian. Many players crack and make mistakes when under heavy attack, but Petrosian was just fine with defending a position for 80 moves in a row, waiting for his opponent to make a mistake so he could clinch the draw.

If you give a position in which one player is completely comfortable to another player with a vastly different playing style, then it won't work. Engines are often the same.

[Sopel]

Stockfish tends to get in trouble against LC0, but but then gives LC0 a beating in the endgame.
Stockfish often has 1000 times as many TB hits in the endgame compared to LC0 on my machine.

It's amazing what A0 propaganda did. Not only is Stockfish better than Lc0 in every element of play but also TBs have negligible impact on playing strength.

[Dann Corbit]

Stockfish tends to get in trouble against LC0, but but then gives LC0 a beating in the endgame.
Stockfish often has 1000 times as many TB hits in the endgame compared to LC0 on my machine.

It's amazing what A0 propaganda did. Not only is Stockfish better than Lc0 in every element of play but also TBs have negligible impact on playing strength.

You are correct about SF having the same strength difference regardless of the number of chessmen on the board. I did some statistical calculations using my SQL database, and I think you are correct. I suppose that I had heard that said, and when I watched games where LC0 got ahead early, I simply assumed that was the reason. But SF gets ahead early as often as LC0.

As to Syzygy tablebase files, experiments have show a +50 Elo gain on several occasions.
However, we have no data for high speed SSD based files at long time control, so projections Elo gain would be conjecture.
It could even be negative {with super-powerful compute resources, it is possible that the lookup slows down the search enough to hurt} or it might be 200 Elo. Nobody knows. I did find that SF's search slows down tremendously at long time control if I put the SSD files on mechanical disk drives rather than my m.2 PCIe4.0 drives.