Fabian is a true master of the assert();jp wrote: ↑Thu May 31, 2018 7:23 pmThe question is how Fruit got rid of bugs. What Leela needs is clever ideas for bug hunting.Albert Silver wrote: ↑Tue May 29, 2018 12:40 am Well, I think it was Dann who commented that the secret to Fruit's success was the sheer lack of bugs, proving it was more important than having clever ideas for algorithms. As such, in Discord today a rather glaring issue came up that is now the source of a serious hunt. Somehow, something happened precisely during the transition to 15x192.
20x256 Leela seemed quite strong
Moderators: hgm, Rebel, chrisw
-
- Posts: 12568
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: 20x256 Leela seemed quite strong
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 4190
- Joined: Wed Nov 25, 2009 1:47 am
Re: 20x256 Leela seemed quite strong
You can't assert a ResNet. But keep dreaming on.Dann Corbit wrote: ↑Thu May 31, 2018 8:23 pmFabian is a true master of the assert();jp wrote: ↑Thu May 31, 2018 7:23 pmThe question is how Fruit got rid of bugs. What Leela needs is clever ideas for bug hunting.Albert Silver wrote: ↑Tue May 29, 2018 12:40 am Well, I think it was Dann who commented that the secret to Fruit's success was the sheer lack of bugs, proving it was more important than having clever ideas for algorithms. As such, in Discord today a rather glaring issue came up that is now the source of a serious hunt. Somehow, something happened precisely during the transition to 15x192.
-
- Posts: 3019
- Joined: Wed Mar 08, 2006 9:57 pm
- Location: Rio de Janeiro, Brazil
Re: 20x256 Leela seemed quite strong
He was answering the question on how Fruit got rid of bugs, or avoided them.Milos wrote: ↑Thu May 31, 2018 8:26 pmYou can't assert a ResNet. But keep dreaming on.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
-
- Posts: 4190
- Joined: Wed Nov 25, 2009 1:47 am
Re: 20x256 Leela seemed quite strong
That is true, but implication was (though not directly by Dann) that the similar approach could be used with LC0.Albert Silver wrote: ↑Thu May 31, 2018 9:08 pmHe was answering the question on how Fruit got rid of bugs, or avoided them.
-
- Posts: 4052
- Joined: Thu May 15, 2008 9:57 pm
- Location: Berlin, Germany
- Full name: Sven Schüle
Re: 20x256 Leela seemed quite strong
Using assert() is always possible for the engine code itself (as far as I know they do not use it). Regarding the network I am not sure how this could be done, simply due to lack of sufficient NN knowledge. So let me ask:Milos wrote: ↑Thu May 31, 2018 11:08 pmThat is true, but implication was (though not directly by Dann) that the similar approach could be used with LC0.Albert Silver wrote: ↑Thu May 31, 2018 9:08 pmHe was answering the question on how Fruit got rid of bugs, or avoided them.
1) Do you think that it would be doable and realistic to write some consistency checks for a network, in order to detect "illegal states"?
2) Do you think that it would be doable and realistic to formulate some "quality criteria" for a network that help to identify valid but undesired states, and write functions that calculate such quality scores?
If so, one could write a "network checker" that looks like this:
Code: Select all
assert(consistencyCheck01());
assert(consistencyCheck02());
...
cout << "quality score 01: " << qualityScore01() << endl;
cout << "quality score 02: " << qualityScore02() << endl;
Sven Schüle (engine author: Jumbo, KnockOut, Surprise)
-
- Posts: 3019
- Joined: Wed Mar 08, 2006 9:57 pm
- Location: Rio de Janeiro, Brazil
Re: 20x256 Leela seemed quite strong
The entire project is open source so anyone can try their hand at modifications, and if you think it works in tests, submit it as a patch.Sven wrote: ↑Sat Jun 02, 2018 3:11 pmUsing assert() is always possible for the engine code itself (as far as I know they do not use it). Regarding the network I am not sure how this could be done, simply due to lack of sufficient NN knowledge. So let me ask:Milos wrote: ↑Thu May 31, 2018 11:08 pmThat is true, but implication was (though not directly by Dann) that the similar approach could be used with LC0.Albert Silver wrote: ↑Thu May 31, 2018 9:08 pm
He was answering the question on how Fruit got rid of bugs, or avoided them.
1) Do you think that it would be doable and realistic to write some consistency checks for a network, in order to detect "illegal states"?
2) Do you think that it would be doable and realistic to formulate some "quality criteria" for a network that help to identify valid but undesired states, and write functions that calculate such quality scores?
If so, one could write a "network checker" that looks like this:Code: Select all
assert(consistencyCheck01()); assert(consistencyCheck02()); ... cout << "quality score 01: " << qualityScore01() << endl; cout << "quality score 02: " << qualityScore02() << endl;
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
-
- Posts: 4052
- Joined: Thu May 15, 2008 9:57 pm
- Location: Berlin, Germany
- Full name: Sven Schüle
Re: 20x256 Leela seemed quite strong
You are certainly right. But my question to Milos was not meant as an attempt to modify the source but as a thought about something new that (to my knowledge) does not exist yet (a "network checker" as a separate tool). It was also mainly a reaction on the statement that using "assert" were not applicable in this project.Albert Silver wrote: ↑Sat Jun 02, 2018 5:44 pmThe entire project is open source so anyone can try their hand at modifications, and if you think it works in tests, submit it as a patch.Sven wrote: ↑Sat Jun 02, 2018 3:11 pmUsing assert() is always possible for the engine code itself (as far as I know they do not use it). Regarding the network I am not sure how this could be done, simply due to lack of sufficient NN knowledge. So let me ask:
1) Do you think that it would be doable and realistic to write some consistency checks for a network, in order to detect "illegal states"?
2) Do you think that it would be doable and realistic to formulate some "quality criteria" for a network that help to identify valid but undesired states, and write functions that calculate such quality scores?
If so, one could write a "network checker" that looks like this:Code: Select all
assert(consistencyCheck01()); assert(consistencyCheck02()); ... cout << "quality score 01: " << qualityScore01() << endl; cout << "quality score 02: " << qualityScore02() << endl;
Of course that could be added in the usual way via Github. Before starting to investigate into such a project (for which I am not even sure I would have time for) I thought it would be best to ask someone who knows a lot more about NN than I do. Asking should be allowed
Sven Schüle (engine author: Jumbo, KnockOut, Surprise)
-
- Posts: 4190
- Joined: Wed Nov 25, 2009 1:47 am
Re: 20x256 Leela seemed quite strong
To my knowledge the only type of checks you could do is at output layer in the fully connected layer. You can just forget about convolutional layers. I don't think anyone knows you to check anything there even for far simpler CNNs and simple classification problems such as recognizing digits or even binary classification.Sven wrote: ↑Sat Jun 02, 2018 3:11 pmUsing assert() is always possible for the engine code itself (as far as I know they do not use it). Regarding the network I am not sure how this could be done, simply due to lack of sufficient NN knowledge. So let me ask:Milos wrote: ↑Thu May 31, 2018 11:08 pmThat is true, but implication was (though not directly by Dann) that the similar approach could be used with LC0.Albert Silver wrote: ↑Thu May 31, 2018 9:08 pm
He was answering the question on how Fruit got rid of bugs, or avoided them.
1) Do you think that it would be doable and realistic to write some consistency checks for a network, in order to detect "illegal states"?
2) Do you think that it would be doable and realistic to formulate some "quality criteria" for a network that help to identify valid but undesired states, and write functions that calculate such quality scores?
If so, one could write a "network checker" that looks like this:Code: Select all
assert(consistencyCheck01()); assert(consistencyCheck02()); ... cout << "quality score 01: " << qualityScore01() << endl; cout << "quality score 02: " << qualityScore02() << endl;