Debugging: GCC/Clang __attribute__((format)) Specifier

[Tom Likens]

Happy Boxing Day Everyone,

I know we've had a lot of nonsense about this lately, but this was really an eye-opening discovery for me and I thought I'd pass it on. Basically, I discovered this while debugging why the 32-bit version of Djinn was crashing.

A bit of background first, I have a number of functions that take a variable number of arguments, similar to how the "printf()" class of functions work. I like this because I can simultaneously print to a pipe, screen and/or log file if I so choose. What I didn't realize was that neither gcc or clang check that the format string matches the arguments unless the function prototype is declared with the "__attribute__" specifier. This is what my prototypes originally looked like:

Code: Select all

void print(   const char *format, ...);
void printL(  const char *format, ...);
void printX(  const char *format, ...);
void printT(  const char *format, ...);
void printE(  const char *format, ...);
void ics_tell(const char *format, ...);

And here they are afterwards:

Code: Select all

void print(   const char *format, ...) __attribute__((format (printf, 1, 2)));
void printL(  const char *format, ...) __attribute__((format (printf, 1, 2)));
void printX(  const char *format, ...) __attribute__((format (printf, 1, 2)));
void printT(  const char *format, ...) __attribute__((format (printf, 1, 2)));
void printE(  const char *format, ...) __attribute__((format (printf, 1, 2)));
void ics_tell(const char *format, ...) __attribute__((format (printf, 1, 2)));

With this in place, the compiler flagged a number of violations that have been in the code for years! Again, how this thing *ever* makes a legal move, I'll never know. Especially, eye-opening was the difference in the 64-bit vs. 32-bit MingW compiles.

And yes, before you start throwing stones I know that C++'s "cout << blah blah" syntax would have prevented all this, but frankly I viserally hate that syntax, so to each their own.

Anyway, I'm cleaning this up and making sure it all works cleanly. But, first I thought I'd pass this along since it may be useful to others (hence the post).

regards,
--tom

[lucasart]

This is what my prototypes originally looked like:

Code: Select all

void print(   const char *format, ...);
void printL(  const char *format, ...);
void printX(  const char *format, ...);
void printT(  const char *format, ...);
void printE(  const char *format, ...);
void ics_tell(const char *format, ...);

When you implement these functions, I'm guessing you don't reinvent the parsing wheel. So you must use something like fprintf() or vsptrintf(), right ?

The compiler can catch error where the format string is a compile time constant. But what if it's variable? I'm guessing that never happens in practice though.

Aren't there any "safe" versions of fprintf(), vsprintf(), etc. ? If it cost a little bit of performance, that's not a problem because the real bottleneck is I/O.

[bob]

This is what my prototypes originally looked like:

Code: Select all

void print(   const char *format, ...);
void printL(  const char *format, ...);
void printX(  const char *format, ...);
void printT(  const char *format, ...);
void printE(  const char *format, ...);
void ics_tell(const char *format, ...);

When you implement these functions, I'm guessing you don't reinvent the parsing wheel. So you must use something like fprintf() or vsptrintf(), right ?

The compiler can catch error where the format string is a compile time constant. But what if it's variable? I'm guessing that never happens in practice though.

Aren't there any "safe" versions of fprintf(), vsprintf(), etc. ? If it cost a little bit of performance, that's not a problem because the real bottleneck is I/O.

An alternative is the standard va_ stuff that lets you write procedures with a variable number of arguments quite simply. My "Print()" function in Crafty is an example, I use it to replace printf(), it splits the output into stdout using printf() and into the log file using fprintf(), but I pass it an argument that can filter some stdout output while still sending everything to the log file...

[Tom Likens]

This is what my prototypes originally looked like:

Code: Select all

void print(   const char *format, ...);
void printL(  const char *format, ...);
void printX(  const char *format, ...);
void printT(  const char *format, ...);
void printE(  const char *format, ...);
void ics_tell(const char *format, ...);

When you implement these functions, I'm guessing you don't reinvent the parsing wheel. So you must use something like fprintf() or vsptrintf(), right ?

The compiler can catch error where the format string is a compile time constant. But what if it's variable? I'm guessing that never happens in practice though.

Aren't there any "safe" versions of fprintf(), vsprintf(), etc. ? If it cost a little bit of performance, that's not a problem because the real bottleneck is I/O.

Hello Lucas,

I've implemented all these using C's "va_xxx" constructs. The "vprintf" and "vfprintf" routines do all the heavy lifting. As an example, here's how my custom "print()" routine is implemented.

Code: Select all

void print(const char *format, ...)
{
    va_list args;

    // First, print to stdout.
    if (!Xboard) {
        va_start(args, format);
        vprintf(format, args);
        fflush(stdout);
        va_end(args);
    }
    // Then set things up again and print to the log file.
    if (Logging && Logfp) {
        va_start(args, format);
        vfprintf(Logfp, format, args);
        fflush(Logfp);
        va_end(args);
    }
}

Now that I've added the attribute stuff both Clang and gcc catch format vs. argument errors inside all these custom functions. The immediate downside though is that this ties you to those compilers. That's not too tremendous a handicap though, because the prototypes could be #ifdef in and out depending on the compiler.

--tom

[Tom Likens]

An alternative is the standard va_ stuff that lets you write procedures with a variable number of arguments quite simply. My "Print()" function in Crafty is an example, I use it to replace printf(), it splits the output into stdout using printf() and into the log file using fprintf(), but I pass it an argument that can filter some stdout output while still sending everything to the log file...

Bob,

I haven't looked at Crafty's source in a very long time. Do you use this attribute feature of gcc on the "Print()" prototype?

--tom

[mvk]

An alternative is the standard va_ stuff that lets you write procedures with a variable number of arguments quite simply. My "Print()" function in Crafty is an example, I use it to replace printf(), it splits the output into stdout using printf() and into the log file using fprintf(), but I pass it an argument that can filter some stdout output while still sending everything to the log file...

Bob,

I haven't looked at Crafty's source in a very long time. Do you use this attribute feature of gcc on the "Print()" prototype?

--tom

It doesn't use the format specifier for it's Print() declaration. Consequently, there are several places where the compiler detects an error with the arguments being passed into it when enabled.

Code: Select all

book.c:939:7: warning: unknown conversion type character ']' in format [-Wformat]
option.c:394:13: warning: too many arguments for format [-Wformat-extra-args]
option.c:3446:5: warning: format '%d' expects argument of type 'int', but argument 3 has type 'uint64_t' [-Wformat]
test.c:178:9: warning: format '%d' expects argument of type 'int', but argument 3 has type 'uint64_t' [-Wformat]
test.c:357:9: warning: format '%d' expects argument of type 'int', but argument 3 has type 'uint64_t' [-Wformat]

[Tom Likens]

An alternative is the standard va_ stuff that lets you write procedures with a variable number of arguments quite simply. My "Print()" function in Crafty is an example, I use it to replace printf(), it splits the output into stdout using printf() and into the log file using fprintf(), but I pass it an argument that can filter some stdout output while still sending everything to the log file...

Bob,

I haven't looked at Crafty's source in a very long time. Do you use this attribute feature of gcc on the "Print()" prototype?

--tom

It doesn't use the format specifier for it's Print() declaration. Consequently, there are several places where the compiler detects an error with the arguments being passed into it when enabled.

Code: Select all

book.c:939:7: warning: unknown conversion type character ']' in format [-Wformat]
option.c:394:13: warning: too many arguments for format [-Wformat-extra-args]
option.c:3446:5: warning: format '%d' expects argument of type 'int', but argument 3 has type 'uint64_t' [-Wformat]
test.c:178:9: warning: format '%d' expects argument of type 'int', but argument 3 has type 'uint64_t' [-Wformat]
test.c:357:9: warning: format '%d' expects argument of type 'int', but argument 3 has type 'uint64_t' [-Wformat]

Yes, it's amazing how many of these creep in--especially, when compiling for 32-bit vs. 64-bit. I even saw differences between 64-bit Linux and 64-bit Windows. It's was rather humbling. And they can be damaging, when I cleaned this up the 32-bit Windows compile was no longer unstable (i.e. no more crashing).

--tom

[bob]

An alternative is the standard va_ stuff that lets you write procedures with a variable number of arguments quite simply. My "Print()" function in Crafty is an example, I use it to replace printf(), it splits the output into stdout using printf() and into the log file using fprintf(), but I pass it an argument that can filter some stdout output while still sending everything to the log file...

Bob,

I haven't looked at Crafty's source in a very long time. Do you use this attribute feature of gcc on the "Print()" prototype?

--tom

No. The va_args stuff gives you the ability to write a procedure that accepts a variable number of arguments. The prototype definition in chess.h looks like this:

void Print(int, char *, ...);

you have a set of procedures you can use to extract each argument, one at a time.

[Evert]

I have the impression there's some misunderstanding. With printf() (and its siblings) the compiler can generate warnings if the format string and the argument list don't match up (obviously the format string needs to be a constant for that to work). If you use the GCC printf attribute for a function of your own, the compiler will perform the same type of compile-time check as it does for the standard library functions.

Nothing major, but it is indeed very convenient.

[Tom Likens]

I have the impression there's some misunderstanding. With printf() (and its siblings) the compiler can generate warnings if the format string and the argument list don't match up (obviously the format string needs to be a constant for that to work). If you use the GCC printf attribute for a function of your own, the compiler will perform the same type of compile-time check as it does for the standard library functions.

Nothing major, but it is indeed very convenient.

Exactly, I couldn't have stated it better. That's exactly what happens. I'm a strong proponent of letting the compiler catch the lion's share of the bugs at compile time.

I passed this along because I've been programming C/C++ a long time and didn't realize that

1) custom functions weren't automatically checked [which seems obvious in retrospect]
2) That this gcc feature was available (both Clang and the Linux Intel compiler support it)

It seems to be a feature many people aren't aware of, and it's useful. It's also trivial to implement and turn on, both desirable features.

--tom