Writing to a Text File (Thread Safe)

[mar]

If you use threads and you only want to use your program at Windows 7 and above, you can also use slim reader/writer locks SRWLOCK, this is a little bit more efficient than critical sections.

That's interesting, thanks for mentioning. I wonder how it's implemented. Looks like a simple spinlock to me. So probably only ok if you lock for a very short period of time, otherwise you end up wasting a core (even if you use pause instruction - btw. has anyone some experience with it? - I always thought it only helps when hyperthreading is on but I may be wrong).
A naive spinlock can be implemented using InterlockedCmpXchg or in assembly. There are also cases where you simply prefer atomic increment/decrement (InterlockedInc/Decrement).
What might be interesting would be a hybrid spinlock, which would go to sleep after n unsuccessful attempts to acquire the lock. It may be tricky to choose a proper n however...
I would probably stick with what the OS offers, I wouldn't be surprised if it already does something like that.

[bob]

If you use threads and you only want to use your program at Windows 7 and above, you can also use slim reader/writer locks SRWLOCK, this is a little bit more efficient than critical sections.

That's interesting, thanks for mentioning. I wonder how it's implemented. Looks like a simple spinlock to me. So probably only ok if you lock for a very short period of time, otherwise you end up wasting a core (even if you use pause instruction - btw. has anyone some experience with it? - I always thought it only helps when hyperthreading is on but I may be wrong).
A naive spinlock can be implemented using InterlockedCmpXchg or in assembly. There are also cases where you simply prefer atomic increment/decrement (InterlockedInc/Decrement).
What might be interesting would be a hybrid spinlock, which would go to sleep after n unsuccessful attempts to acquire the lock. It may be tricky to choose a proper n however...
I would probably stick with what the OS offers, I wouldn't be surprised if it already does something like that.

Pause is for hyperthreading only. It simply tells the CPU switch to the other logical processor context RIGHT NOW. So that you are not spinning while the other is trying to clear the lock but can't execute.

[Sven]

So.... can you explain why this code does what one would expect it to do:

Code: Select all

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>

static pthread_attr_t thread_attr;
static pthread_t thread;

FILE *F;
int num;

void *worker(void *arg)
{
  int i;
  int n = num;

  for (i = 1; i <= n; i++)
    fprintf(F, "%d\n", i);
}

int main(int argc, char **argv)
{
  F = fopen("bla.txt", "w");
  if (argc == 2)
    num = atoi(argv[1]);
  else
    num = 10000;
  pthread_attr_init(&thread_attr);
  pthread_attr_setdetachstate(&thread_attr, PTHREAD_CREATE_JOINABLE);
  pthread_create(&thread, NULL, worker, NULL);
  worker(NULL);
  pthread_join(thread, NULL);
  fclose(F);
  return 0;
}

Code: Select all

$ gcc -pthread -O3 atomictest.c -o atomictest
$ ./atomictest 10000
$ sort -n bla.txt | md5sum
3d7fe033a73a69382345fbe46907194c  -

The resulting bla.txt has interleaved lines, but none of the lines is corrupted.

Insert a random sleep of few milliseconds into the worker loop and see what happens

Sven

[syzygy]

Insert a random sleep of few milliseconds into the worker loop and see what happens

Nothing happened.

[sje]

Code: Select all

  pthread_attr_init(&thread_attr);
  pthread_attr_setdetachstate(&thread_attr, PTHREAD_CREATE_JOINABLE);
  pthread_create(&thread, NULL, worker, NULL);

Why is the thread attribute initialized but then not passed to pthread_create() (second parameter slot)?

[syzygy]

Code: Select all

  pthread_attr_init(&thread_attr);
  pthread_attr_setdetachstate(&thread_attr, PTHREAD_CREATE_JOINABLE);
  pthread_create(&thread, NULL, worker, NULL);

Why is the thread attribute initialized but then not passed to pthread_create() (second parameter slot)?

Because I quickly hacked this code together. It does not matter, since threads are created as joinable by default. So the thread attribute lines can be safely deleted.

[bob]

Code: Select all

  pthread_attr_init(&thread_attr);
  pthread_attr_setdetachstate(&thread_attr, PTHREAD_CREATE_JOINABLE);
  pthread_create(&thread, NULL, worker, NULL);

Why is the thread attribute initialized but then not passed to pthread_create() (second parameter slot)?

It is not needed. That is the default (joinable).

[Sven]

Insert a random sleep of few milliseconds into the worker loop and see what happens

Nothing happened.

I tested with 4 threads and num=100 on Windows 7, after inserting a random sleep of 1..10 msec into the loop body. No single line of the resulting log file looks suspicious but obviously many lines are overwritten, e.g. the line containing the number 100 appears more than 4 times while others appear less than 4 times, etc.

How did you verify that everything worked correctly in your case, other than with the "sort -n bla.txt | md5sum" command?

Sven

[mar]

It is possible that stdio is thread-safe on Linux but not on Windows (the standard doesn't mandate it).
Plus you don't know what fancy stuff stdio functions do behind the curtains,
so if you want to actually measure anything, you should be using low level functions for writing stuff instead of testing thread safety of CRT.

[syzygy]

Insert a random sleep of few milliseconds into the worker loop and see what happens

Nothing happened.

I tested with 4 threads and num=100 on Windows 7, after inserting a random sleep of 1..10 msec into the loop body. No single line of the resulting log file looks suspicious but obviously many lines are overwritten, e.g. the line containing the number 100 appears more than 4 times while others appear less than 4 times, etc.

Windows 7 might not be POSIX compliant...

How did you verify that everything worked correctly in your case, other than with the "sort -n bla.txt | md5sum" command?

Do a few runs and check that the md5sums stay constant. On a POSIX-compliant platform.

It is possible that stdio is thread-safe on Linux but not on Windows (the standard doesn't mandate it).

As I understand it, POSIX does mandate it (and not just thread-safety of fprintf() but also atomicity). Of course Windows does not care about POSIX.

Plus you don't know what fancy stuff stdio functions do behind the curtains,
so if you want to actually measure anything, you should be using low level functions for writing stuff instead of testing thread safety of CRT.

We want to know whether fprintf() is atomic, so we should test fprintf() and not something lower level such as write(). This is the main point of the discussion: (non-)atomicity of lower level functions says nothing about (non-)atomicity of higher level functions.