MP Stockfish

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

hcyrano

MP Stockfish

Post by hcyrano »

hi,

in void ThreadsManager::idle_loop(int threadID, SplitPoint* sp);

can you explain this lines:


// Because sp->slaves[] is reset under lock protection,
// be sure sp->lock has been released before to return.
lock_grab(&(sp->lock));
lock_release(&(sp->lock));

i don't see real utility :-(

thx a lot
hcyrano

Re: MP Stockfish

Post by hcyrano »

ok i see :D

return a object Splitpoint clean.

@+
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: MP Stockfish

Post by mcostalba »

hcyrano wrote:ok i see :D

return a object Splitpoint clean.

@+
Sorry for late reply, I have seen your question, but I wanted to answer during the week end because is not a trivial question and I want to have a bit of time....more later !
zamar
Posts: 613
Joined: Sun Jan 18, 2009 7:03 am

Re: MP Stockfish

Post by zamar »

hcyrano wrote:hi,

in void ThreadsManager::idle_loop(int threadID, SplitPoint* sp);

can you explain this lines:


// Because sp->slaves[] is reset under lock protection,
// be sure sp->lock has been released before to return.
lock_grab(&(sp->lock));
lock_release(&(sp->lock));

i don't see real utility :-(

thx a lot
If I remember correctly, I came to conclusion that this could be safely removed. For some reason the change never entered the codebase...

The point is of course to have clean splitPoint object, but I think I came to conclusion that this isn't absolute necessity.
Joona Kiiski
hcyrano

Re: MP Stockfish

Post by hcyrano »

.
the point is of course to have clean splitPoint object, but I think I came to conclusion that this isn't absolute necessity.
I agree, but I think it is useful to have an object splitPoint clean after using.

this is more satisfying, keep this code.

for info: i prefere "old" code this nCpus :D, because i can use std::vector<int> slaves and not a fix table volatile int slaves[].
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: MP Stockfish

Post by mcostalba »

hcyrano wrote:.
the point is of course to have clean splitPoint object, but I think I came to conclusion that this isn't absolute necessity.
I agree, but I think it is useful to have an object splitPoint clean after using.

this is more satisfying, keep this code.

for info: i prefere "old" code this nCpus :D, because i can use std::vector<int> slaves and not a fix table volatile int slaves[].
Yes but you have an additional shared variable. Here the aim is to reduce the shared vairables to the bare minimum to keep things easier.

A shared variable is a source of possible races especially ncpu that was used to sync the threads to leave the split point.

Anyhow please feel free to send a patch according to your idea and if it is better then current code we will apply.
hcyrano

Re: MP Stockfish

Post by hcyrano »

hi,

i think you can reduce the protected region in ThreadsManager::split at this simple code:


pseudo code :wink:

Code: Select all

 
    // Pick the next available split point object from the split point stack
    SplitPoint& splitPoint = masterThread.splitPoints&#91;masterThread.activeSplitPoints&#93;;

    int workersCnt = 1; // At least the master is included

    lock_grab&#40;&MPLock&#41;;


    // Allocate available threads setting state to THREAD_BOOKED
    for &#40;unsigned int i = 0; !Fake && i < ActiveThreads; i++) &#123;
    
    	splitPoint.slaves&#91;i&#93; = 0;
    	
        if &#40;workersCnt < MaxThreadsPerSplitPoint && thread_is_available&#40;i, master&#41;)
        &#123;
            threads&#91;i&#93;.state = THREAD_BOOKED;
            threads&#91;i&#93;.splitPoint = &splitPoint; //initialisation later
            splitPoint.slaves&#91;i&#93; = 1;
            workersCnt++;
        &#125;
   &#125;
       
    // We can release the lock because slave threads are already booked and master is not available
    lock_release&#40;&MPLock&#41;;

    if &#40;workerCnt == 1&#41;
    	return;
    	
    masterThread.activeSplitPoints++;

point 2 :

you don't love my "helpful master" concept recursif ?


Code: Select all

	for&#40;RXSplitPoint* sp = threads&#91;master&#93;.splitPoint; sp != NULL; sp = sp->parent&#41; &#123;
		if&#40;sp == &&#40;threads&#91;slave&#93;.splitPointStack&#91;localActiveSplitPoints-1&#93;))
			return true;
	&#125;
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: MP Stockfish

Post by mcostalba »

Thanks, this is a possible patch (only slightly tested)

Code: Select all

--- a/src/search.cpp
+++ b/src/search.cpp
@@ -2578,16 +2578,12 @@ namespace &#123;
     int i, master = p.thread&#40;);
     Thread& masterThread = threads&#91;master&#93;;
 
-    lock_grab&#40;&MPLock&#41;;
+    // If we are here it means we are not available
+    assert&#40;masterThread.state != THREAD_AVAILABLE&#41;;
 
-    // If no other thread is available to help us, or if we have too many
-    // active split points, don't split.
-    if (   !available_thread_exists&#40;master&#41;
-        || masterThread.activeSplitPoints >= MAX_ACTIVE_SPLIT_POINTS&#41;
-    &#123;
-        lock_release&#40;&MPLock&#41;;
+    // If we have too many active split points, don't split
+    if &#40;masterThread.activeSplitPoints >= MAX_ACTIVE_SPLIT_POINTS&#41;
         return;
-    &#125;
 
     // Pick the next available split point object from the split point stack
     SplitPoint& splitPoint = masterThread.splitPoints&#91;masterThread.activeSplitPoints++&#93;;
@@ -2612,12 +2608,11 @@ namespace &#123;
 
     masterThread.splitPoint = &splitPoint;
 
-    // If we are here it means we are not available
-    assert&#40;masterThread.state != THREAD_AVAILABLE&#41;;
-
     int workersCnt = 1; // At least the master is included
 
-    // Allocate available threads setting state to THREAD_BOOKED
+    lock_grab&#40;&MPLock&#41;;
+
+    // Try to allocate available threads setting state to THREAD_BOOKED
     for &#40;i = 0; !Fake && i < ActiveThreads && workersCnt < MaxThreadsPerSplitPoint; i++)
         if &#40;thread_is_available&#40;i, master&#41;)
         &#123;
@@ -2627,11 +2622,16 @@ namespace &#123;
             workersCnt++;
         &#125;
 
-    assert&#40;Fake || workersCnt > 1&#41;;
-
-    // We can release the lock because slave threads are already booked and master is not available
+    // We can release the lock because slave threads are already booked
     lock_release&#40;&MPLock&#41;;
 
+    // If we fail to book any slave release the split point and return
+    if (!Fake && workersCnt == 1&#41;
+    &#123;
+        masterThread.activeSplitPoints--;
+        return;
+    &#125;
+
     // Tell the threads that they have work to do. This will make them leave
     // their idle loop. But before copy search stack tail for each thread.
     for &#40;i = 0; i < ActiveThreads; i++)
While there I have also done this one to remove an useless lock

Code: Select all

--- a/src/search.cpp
+++ b/src/search.cpp
@@ -2658,14 +2658,10 @@ namespace &#123;
 
     // We have returned from the idle loop, which means that all threads are
     // finished. Update alpha and bestValue, and return.
-    lock_grab&#40;&MPLock&#41;;
-
     *alpha = splitPoint.alpha;
     *bestValue = splitPoint.bestValue;
     masterThread.activeSplitPoints--;
     masterThread.splitPoint = splitPoint.parent;
-
-    lock_release&#40;&MPLock&#41;;
   &#125;
and especially this that seems important because fixes a possible race in case someone books us while we are waiting for other threads to finish. I don't know how was possible to miss that for so long, or perhaps I am missing something obvious. But I would like to show you so to have your comment on this:

Code: Select all

--- a/src/search.cpp
+++ b/src/search.cpp
@@ -2384,10 +2384,15 @@ namespace &#123;
             lock_grab&#40;&&#40;sp->lock&#41;);
             lock_release&#40;&&#40;sp->lock&#41;);
 
-            assert&#40;threads&#91;threadID&#93;.state == THREAD_AVAILABLE&#41;;
+            lock_grab&#40;&MPLock&#41;;
 
-            threads&#91;threadID&#93;.state = THREAD_SEARCHING;
-            return;
+            if &#40;threads&#91;threadID&#93;.state == THREAD_AVAILABLE&#41;
+            &#123;
+                threads&#91;threadID&#93;.state = THREAD_SEARCHING;
+                lock_release&#40;&MPLock&#41;;
+                return;
+            &#125;
+            lock_release&#40;&MPLock&#41;;
         &#125;
     &#125;
   &#125;

Regarding your proposal is not that we don't love, simply it doesn't seems to improve anything with a QUAD, perhas with an OCTAL we could see something better, but we don't have an OCTAL :-(
hcyrano

Re: MP Stockfish

Post by hcyrano »

and especially this that seems important because fixes a possible race in case someone books us while we are waiting for other threads to finish. I don't know how was possible to miss that for so long, or perhaps I am missing something obvious. But I would like to show you so to have your comment on this:
helpful master concept : can help only a sub-tree, or here all is finish, so it is not possible taht this thread are booked.
hcyrano

Re: MP Stockfish

Post by hcyrano »

mcostalba wrote:Thanks, this is a possible patch (only slightly tested)


Regarding your proposal is not that we don't love, simply it doesn't seems to improve anything with a QUAD, perhas with an OCTAL we could see something better, but we don't have an OCTAL :-(
New macpro can have 24 threads (12 + 12HT) :D