volatile?

[mcostalba]

Other questions?

They seem most related to I/O access, like gbe.h.

I suggest to (at least slightly) understand what you are doing before doing it...also after it would suffice.

[syzygy]

Ok, I will not check whether they all fall under one of the exceptions listed in https://www.kernel.org/doc/Documentatio ... armful.txt (prima facie most seem to do). And your "offer" related so spinlocks (and spinlock_types_up.h smells like uniprocessor), but never mind.

Fact is, most shared data in the kernel is not volatile.

Fact is, MOST shared data in Crafty is not volatile. Big surprise? Volatile has a specific purpose.

No, I am not surprised at all. Nothing wrong with Crafty. The problem is only that you, as always, butt into threads without knowing what it is about and start blabbering "all wrong". So then I think:

Oh boy, there we go again...

Your use of volatile in Crafty falls under:

If you don't want to be restricted by a thread library, but (in addition) want to rely on how the compiler will map your code to the machine, then volatile is useful. Your program will be full of undefined behavior, but if that is a conscious choice that is not necessarily bad. It will just not be "standard C/C++ plus pthreads" or "standard C11/C++11".

In C11 you can do this in a standard-compliant way using atomics.

Senpai's use falls under:

For multithreaded programs using volatile is not necessary if you are properly using the synchronisation primitives of a thread library, e.g. pthreads or C++11 threads.

Senpai properly uses C++11 locking primitives, so does not need volatile (nor atomic) for its shared variables. (I'm ignoring the tt table.)

Those locks were not amateuristically written by Mr Hyatt, but include the necessary compiler and hardware memory barriers.

Your ignorance is showing. My lock was NOT "amateurishly written by me.

I referred to those you fabricated by copying and pasting from the pthreads source.

Since X86 is always in-order store architecture, barriers are not needed.

They include a compiler barrier... Otherwise memory accesses would be moved across lock operations by the compiler.

Just because someone doesn't like the volatile approach does NOT mean it does not work correctly AND efficiently.

Can you read this? It is only the fourth or maybe fifth time I repeat, it's from the start of this thread:

If you don't want to be restricted by a thread library, but (in addition) want to rely on how the compiler will map your code to the machine, then volatile is useful. Your program will be full of undefined behavior, but if that is a conscious choice that is not necessarily bad. It will just not be "standard C/C++ plus pthreads" or "standard C11/C++11".

I have nothing against an educated use of volatile for multithreaded programming (when C11 is not available and the consequences are well-understood).

But you blabbered "all wrong" against something which is simply "all right":

For multithreaded programs using volatile is not necessary if you are properly using the synchronisation primitives of a thread library, e.g. pthreads or C++11 threads.

[bob]

Other questions?

They seem most related to I/O access, like gbe.h.

I suggest to (at least slightly) understand what you are doing before doing it...also after it would suffice.

Most of the kernel is related to doing I/O. I only included a few of the 1000+ that I found...

Volatile has its place. It also can be abused and/or broken and can kill performance. I'm VERY selective in Crafty as to what is volatile. Less than a dozen variables total. It was NOT done thoughtlessly/carelessly.

[syzygy]

It seems you amazingly completely misread my simple:

For multithreaded programs using volatile is not necessary if you are properly using the synchronisation primitives of a thread library, e.g. pthreads or C++11 threads.

as "it is always wrong to use volatile in a multithreaded program".

English comprehension...

[bob]

Ok, I will not check whether they all fall under one of the exceptions listed in https://www.kernel.org/doc/Documentatio ... armful.txt (prima facie most seem to do). And your "offer" related so spinlocks (and spinlock_types_up.h smells like uniprocessor), but never mind.

Fact is, most shared data in the kernel is not volatile.

Fact is, MOST shared data in Crafty is not volatile. Big surprise? Volatile has a specific purpose.

No, I am not surprised at all. Nothing wrong with Crafty. The problem is only that you, as always, butt into threads without knowing what it is about and start blabbering "all wrong". So then I think:

Oh boy, there we go again...

Your use of volatile in Crafty falls under:

If you don't want to be restricted by a thread library, but (in addition) want to rely on how the compiler will map your code to the machine, then volatile is useful. Your program will be full of undefined behavior, but if that is a conscious choice that is not necessarily bad. It will just not be "standard C/C++ plus pthreads" or "standard C11/C++11".

In C11 you can do this in a standard-compliant way using atomics.

Senpai's use falls under:

For multithreaded programs using volatile is not necessary if you are properly using the synchronisation primitives of a thread library, e.g. pthreads or C++11 threads.

Senpai properly uses C++11 locking primitives, so does not need volatile (nor atomic) for its shared variables. (I'm ignoring the tt table.)

Those locks were not amateuristically written by Mr Hyatt, but include the necessary compiler and hardware memory barriers.

Your ignorance is showing. My lock was NOT "amateurishly written by me.

I referred to those you fabricated by copying and pasting from the pthreads source.

Let me get this straight. You REALLY want to claim that the ONLY way one can use the pthread library is to use it in its pre-compiled form? Can you cite a source? (hint: such a source does NOT exist, because that is NOT a requirement for using the pthreads library. In fact, if it were a requirement, it would break way too much because library behavior is defined, specifically, to behave IDENTICALLY with behavior produced when the library source code AND the application are compiled together. That is the very DEFINITION of a library, it eliminates the NEED to compile the library source each time it is used. It does NOT preclude doing so.

A few years ago I was running on a lightweight kernel where there were NO shared libraries at all. I simply compiled with the pthread source included and all worked just fine. As it should. Absolutely NOTHING in any POSIX standard says you can't replace a library by including the source. Such a concept is simply broken in the extreme.

Since X86 is always in-order store architecture, barriers are not needed.

They include a compiler barrier... Otherwise memory accesses would be moved across lock operations by the compiler.

Just because someone doesn't like the volatile approach does NOT mean it does not work correctly AND efficiently.

Can you read this? It is only the fourth or maybe fifth time I repeat, it's from the start of this thread:

If you don't want to be restricted by a thread library, but (in addition) want to rely on how the compiler will map your code to the machine, then volatile is useful. Your program will be full of undefined behavior, but if that is a conscious choice that is not necessarily bad. It will just not be "standard C/C++ plus pthreads" or "standard C11/C++11".

I have nothing against an educated use of volatile for multithreaded programming (when C11 is not available and the consequences are well-understood).

But you blabbered "all wrong" against something which is simply "all right":

For multithreaded programs using volatile is not necessary if you are properly using the synchronisation primitives of a thread library, e.g. pthreads or C++11 threads.

For "some places" volatile is necessary, because max performance is an absolute requirement. Using locks to avoid volatile takes a big performance hit. And I am not talking about pthreads locks which take a MUCH bigger performance hit over a standard xchg() type x86 lock. But that xchg() lock is a big hit itself.

[bob]

It seems you amazingly completely misread my simple:

For multithreaded programs using volatile is not necessary if you are properly using the synchronisation primitives of a thread library, e.g. pthreads or C++11 threads.

as "it is always wrong to use volatile in a multithreaded program".

English comprehension...

Did you not cite a blog post where that was THE TOPIC?

Or do you cite references but really don't mean that they be looked at?

[syzygy]

The lock used in Crafty came from older kernel sources. Still works flawlessly.

You can safely remove the "volatile" from "volatile int *lock" in this code:

Code: Select all

static void __inline__ LockX86(volatile int *lock) {
  int dummy;
  asm __volatile__(
      "1:          movl    $1, %0"   "\n\t"
      "            xchgl   (%1), %0" "\n\t"
      "            testl   %0, %0"   "\n\t"
      "            jz      3f"       "\n\t"
      "2:          pause"            "\n\t"
      "            movl    (%1), %0" "\n\t"
      "            testl   %0, %0"   "\n\t"
      "            jnz     2b"       "\n\t"
      "            jmp     1b"       "\n\t"
      "3:"                           "\n\t"
      :"=&q"(dummy)
      :"q"(lock)
      :"cc", "memory");
}
static void __inline__ UnlockX86(volatile int *lock) {
  int dummy;
  asm __volatile__(
      "movl    $0, (%1)"
      :"=&q"(dummy)
      :"q"(lock));
}

The __volatile__ in "asm __volatile__" is important, though.

Can you explain why the shared variable gets reloaded:

You DO realize this program runs on other than X86? The alpha compiler from DEC had a software lock function that did what mine does above, but in C, using the interlocked exchange() mechanism their compiler used. Remove the volatile and the alpha would hang there.

Duh? This is precisely your example from the very beginning that was supposed to show why volatile was needed to avoid the reordering.

On alpha one would need to include an appropriate memory fence in the LOCK() macro, obviously.

YOUR point was that the compiler could reorder around the LOCK(). But this lock includes a compiler barrier, as it should, so there won't be any reordering.

No need to make "variable" volatile.

The above will fail because variable was not changed in this code between the first reference and the second. Another thread COULD have acquired the lock, modified variable, and then released the lock. When you access variable a second time, the compiler is free to use the original value (now stored in i) rather than re-loading it again, even though the other thread might have modified it inside the locks.

Wrong! Properly written locks include memory barriers! Get it now?!

I didn't feel up to looking at what you are doing. But first things first. The volatile declaration on the inline asm is important. The compiler can't reorder across that, ever, nor can it change the location of the inline assembler by moving it up or down, before or after existing code.

If that is what you are talking about, that's the reason. Done quite intentionally, obviously.

Yeah! That IS what I was talking about. All... the... <explicit>... time... Wow, we.. have... gotten.. somewhere... finally...

[bob]

The lock used in Crafty came from older kernel sources. Still works flawlessly.

You can safely remove the "volatile" from "volatile int *lock" in this code:

Code: Select all

static void __inline__ LockX86(volatile int *lock) {
  int dummy;
  asm __volatile__(
      "1:          movl    $1, %0"   "\n\t"
      "            xchgl   (%1), %0" "\n\t"
      "            testl   %0, %0"   "\n\t"
      "            jz      3f"       "\n\t"
      "2:          pause"            "\n\t"
      "            movl    (%1), %0" "\n\t"
      "            testl   %0, %0"   "\n\t"
      "            jnz     2b"       "\n\t"
      "            jmp     1b"       "\n\t"
      "3:"                           "\n\t"
      :"=&q"(dummy)
      :"q"(lock)
      :"cc", "memory");
}
static void __inline__ UnlockX86(volatile int *lock) {
  int dummy;
  asm __volatile__(
      "movl    $0, (%1)"
      :"=&q"(dummy)
      :"q"(lock));
}

The __volatile__ in "asm __volatile__" is important, though.

Can you explain why the shared variable gets reloaded:

You DO realize this program runs on other than X86? The alpha compiler from DEC had a software lock function that did what mine does above, but in C, using the interlocked exchange() mechanism their compiler used. Remove the volatile and the alpha would hang there.

Duh? This is precisely your example from the very beginning that was supposed to show why volatile was needed to avoid the reordering.

You raised the issue about volatile int lockxx; in Crafty and the volatile not being needed.

MY example has been the spin loop waiting on work.

while(!tree);

as a crude example. That's all in C. Volatile is critical.

On alpha one would need to include an appropriate memory fence in the LOCK() macro, obviously.

And the required volatile lock type declaration (lock_t includes that on the alpha C compiler I last used). Otherwise the compiler will produce a lock that you never get through once you reach it and the lock is already set.

YOUR point was that the compiler could reorder around the LOCK(). But this lock includes a compiler barrier, as it should, so there won't be any reordering.

No need to make "variable" volatile.

The above will fail because variable was not changed in this code between the first reference and the second. Another thread COULD have acquired the lock, modified variable, and then released the lock. When you access variable a second time, the compiler is free to use the original value (now stored in i) rather than re-loading it again, even though the other thread might have modified it inside the locks.

Wrong! Properly written locks include memory barriers! Get it now?!

I didn't feel up to looking at what you are doing. But first things first. The volatile declaration on the inline asm is important. The compiler can't reorder across that, ever, nor can it change the location of the inline assembler by moving it up or down, before or after existing code.

If that is what you are talking about, that's the reason. Done quite intentionally, obviously.

Yeah! That IS what I was talking about. All... the... <explicit>... time... Wow, we.. have... gotten.. somewhere... finally...

OK, volatile is required there. Didn't I say that already?

[syzygy]

It seems you amazingly completely misread my simple:

For multithreaded programs using volatile is not necessary if you are properly using the synchronisation primitives of a thread library, e.g. pthreads or C++11 threads.

as "it is always wrong to use volatile in a multithreaded program".

English comprehension...

Did you not cite a blog post where that was THE TOPIC?

Or do you cite references but really don't mean that they be looked at?

Well, the above quote of mine is what made you start whining. Not something else.

And you whined not only about what I wrote:
http://talkchess.com/forum/viewtopic.ph ... 696#562696

[syzygy]

Those locks were not amateuristically written by Mr Hyatt, but include the necessary compiler and hardware memory barriers.

Your ignorance is showing. My lock was NOT "amateurishly written by me.

I referred to those you fabricated by copying and pasting from the pthreads source.

Let me get this straight. You REALLY want to claim that the ONLY way one can use the pthread library is to use it in its pre-compiled form?

I am saying that POSIX does not guarantee that if you randomly copy and paste stuff into your program you end up with something that works.

Can you cite a source? (hint: such a source does NOT exist, because that is NOT a requirement for using the pthreads library. In fact, if it were a requirement, it would break way too much because library behavior is defined, specifically, to behave IDENTICALLY with behavior produced when the library source code AND the application are compiled together. That is the very DEFINITION of a library, it eliminates the NEED to compile the library source each time it is used. It does NOT preclude doing so.

OK, why don't you file a bug report to the pthreads maintainers so that they can have as big a laugh at you as we are having.

From the start of this thread, posted by Alvaro:
http://pubs.opengroup.org/onlinepubs/96 ... #tag_04_11

The following functions synchronize memory with respect to other threads:
...
pthread_mutex_lock()
...

If your POSIX system doesn't do this, it is buggy. You believe you found a bug, so report it. Woohoo.