CPU NN question

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Leo
Posts: 996
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

CPU NN question

Post by Leo » Fri Nov 13, 2020 4:46 pm

I dont remeber people saying NNs on CPUs was going to happen. I dont know for sure. Does anyone have comments on NNs being used on CPUs ? It seems like its a big surprise. It seemed that only GPUs could do that. FYI I am an amateur hack on these questions.
Advanced Micro Devices fan.

smatovic
Posts: 1648
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Re: CPU NN question

Post by smatovic » Fri Nov 13, 2020 5:17 pm

Leo wrote:
Fri Nov 13, 2020 4:46 pm
I dont remeber people saying NNs on CPUs was going to happen. I dont know for sure. Does anyone have comments on NNs being used on CPUs ? It seems like its a big surprise. It seemed that only GPUs could do that. FYI I am an amateur hack on these questions.
IMO Peter Osterlund's 2017 Texel/Giraffe experiment showed the potential of NN in AB on CPU...

http://talkchess.com/forum3/viewtopic.p ... 10#p719539
...
From the results it can be seen that the giraffe evaluation function makes texel around 250-350 elo weaker depending on time control. This is caused by the giraffe evaluation function being very slow. If it was somehow possible to make the giraffe evaluation function run as fast as the texel evaluation function, the giraffe eval version would actually be around 100-120 elo stronger than the texel eval version.

Whether future hardware and software improvements will make it possible to run an ANN evaluator as quickly as a traditional evaluator remains to be seen.
--
Srdja

Leo
Posts: 996
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: CPU NN question

Post by Leo » Fri Nov 13, 2020 6:13 pm

smatovic wrote:
Fri Nov 13, 2020 5:17 pm
Leo wrote:
Fri Nov 13, 2020 4:46 pm
I dont remeber people saying NNs on CPUs was going to happen. I dont know for sure. Does anyone have comments on NNs being used on CPUs ? It seems like its a big surprise. It seemed that only GPUs could do that. FYI I am an amateur hack on these questions.
IMO Peter Osterlund's 2017 Texel/Giraffe experiment showed the potential of NN in AB on CPU...

http://talkchess.com/forum3/viewtopic.p ... 10#p719539
...
From the results it can be seen that the giraffe evaluation function makes texel around 250-350 elo weaker depending on time control. This is caused by the giraffe evaluation function being very slow. If it was somehow possible to make the giraffe evaluation function run as fast as the texel evaluation function, the giraffe eval version would actually be around 100-120 elo stronger than the texel eval version.

Whether future hardware and software improvements will make it possible to run an ANN evaluator as quickly as a traditional evaluator remains to be seen.
--
Srdja
Nice. Thanks.
Advanced Micro Devices fan.

brianr
Posts: 506
Joined: Thu Mar 09, 2006 2:01 pm

Re: CPU NN question

Post by brianr » Fri Nov 13, 2020 6:46 pm

There are many different NN sizes and architectures.

The larger ones like Lc0 uses are only really practical on GPUs.

The smaller ones like SF-NNUE run on CPUs (and are very cleverly incrementally updated to make even faster).

Leo
Posts: 996
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: CPU NN question

Post by Leo » Fri Nov 13, 2020 6:50 pm

brianr wrote:
Fri Nov 13, 2020 6:46 pm
There are many different NN sizes and architectures.

The larger ones like Lc0 uses are only really practical on GPUs.

The smaller ones like SF-NNUE run on CPUs (and are very cleverly incrementally updated to make even faster).
Was it a mater of programing it to be clever?
Advanced Micro Devices fan.

User avatar
yurikvelo
Posts: 608
Joined: Sat Dec 06, 2014 12:53 pm

Re: CPU NN question

Post by yurikvelo » Fri Nov 13, 2020 11:22 pm

Leo wrote:
Fri Nov 13, 2020 4:46 pm
I dont remeber people saying NNs on CPUs was going to happen.
maybe they were talking about MCTS, not about static evaluation
GPU run MCTS, CPU NN run alpha-beta minimax, but static evaluation is not handcrafted

Leo
Posts: 996
Joined: Fri Sep 16, 2016 4:55 pm
Location: USA/Minnesota
Full name: Leo Anger

Re: CPU NN question

Post by Leo » Sat Nov 14, 2020 2:25 am

yurikvelo wrote:
Fri Nov 13, 2020 11:22 pm
Leo wrote:
Fri Nov 13, 2020 4:46 pm
I dont remeber people saying NNs on CPUs was going to happen.
maybe they were talking about MCTS, not about static evaluation
GPU run MCTS, CPU NN run alpha-beta minimax, but static evaluation is not handcrafted
OK. Interesting.
Advanced Micro Devices fan.

Madeleine Birchfield
Posts: 297
Joined: Tue Sep 29, 2020 2:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

Re: CPU NN question

Post by Madeleine Birchfield » Sat Nov 14, 2020 5:32 am

Leo wrote:
Fri Nov 13, 2020 4:46 pm
I dont remeber people saying NNs on CPUs was going to happen.
yurikvelo wrote:
Fri Nov 13, 2020 11:22 pm
maybe they were talking about MCTS, not about static evaluation
GPU run MCTS, CPU NN run alpha-beta minimax, but static evaluation is not handcrafted
Leo wrote:
Sat Nov 14, 2020 2:25 am
OK. Interesting.
Leo's original post was correct; people didn't say that neural networks on CPUs were going to happen because at the time the neural networks (like in Leela, Allie) were too slow to be calculated on CPU. The development of NNUE from computer shogi happened without much attention from the computer chess community, and when Hisayori Noda ported NNUE to his Stockfish fork in autumn 2019, it went unnoticed and was left untouched for almost an entire year until people like Henk Drost, Mark Jordan, Sergio Vieri, and so on started experimenting with it around the beginning of June 2020. When it succeeded it shocked the computer chess community as many previously believed that neural networks were too slow to work on CPUs.

MCTS search is completely unrelated to neural network evaluation, as shown by Komodo and Shashchess, both engines that (have an option to) use MCTS with their handcrafted eval on CPU.

smatovic
Posts: 1648
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Re: CPU NN question

Post by smatovic » Sat Nov 14, 2020 7:10 am

Another one, according to this TC poll from 2018, most programmers simply underestimated NNs for chess:

http://talkchess.com/forum3/viewtopic.php?f=7&t=67121

but at least one voted for "take Giraffe and tune it", hence I guess a minority was aware of the potential, or alike.

--
Srdja

Madeleine Birchfield
Posts: 297
Joined: Tue Sep 29, 2020 2:29 pm
Location: Dublin, Ireland
Full name: Madeleine Birchfield

Re: CPU NN question

Post by Madeleine Birchfield » Sat Nov 14, 2020 7:40 am

smatovic wrote:
Sat Nov 14, 2020 7:10 am
Another one, according to this TC poll from 2018, most programmers simply underestimated NNs for chess:

http://talkchess.com/forum3/viewtopic.php?f=7&t=67121

but at least one voted for "take Giraffe and tune it", hence I guess a minority was aware of the potential, or alike.

--
Srdja
I was a big Leela fan back in 2018 and believed it was going to become the strongest chess engine on the scene and it was superior to traditional engines due to its strong positional evaluation that handcrafted evaluations simply cannot replicate, but I largely saw the neural network thing as something that was largely incompatible with the traditional alpha-beta search paradigm, due to the fact that no traditional engine supports the use of GPUs. So for me, it was more a fact of how many people were willing to shift away from the traditional paradigm over to the Leela-style paradigm using GPUs, and nobody, aside from Daniel Shawul with Scorpio, seemed interested in moving to the new Leela-style paradigm, so I concluded that it would be a very long time until traditional engines adopted neural networks.

Post Reply