Crafty 22.5 leaking memory ?

[kiroje]

First of all a bug thanks to Robert Hyatt for Crafty 22.4

But there may be a memory leak of some sort

Im using SCID and Peter Skinners Win32 port of crafty for analysing my own games, but it seems that it steadily consumes more and more memory until Windows runs out

Im going back and forth in the games while crafty analyses, so maybe someone could do the same and see if they get the same problem ?

[kiroje]

And of course big thanks, and not bug thanks

[bob]

First of all a bug thanks to Robert Hyatt for Crafty 22.4

But there may be a memory leak of some sort

Im using SCID and Peter Skinners Win32 port of crafty for analysing my own games, but it seems that it steadily consumes more and more memory until Windows runs out

Im going back and forth in the games while crafty analyses, so maybe someone could do the same and see if they get the same problem ?

You will have to explain how to reproduce this. I have been running lots of games and Crafty is not getting bigger as it plays more and more games...

[kiroje]

I have SCID installed with Peter Skinners Win32 compile of crafty plus his crafty.rc(unedited)

After analysing one of my games for some hours(yes im quite thorough ) windows ran out of memory.

Restarting the engine again I kept an eye on the size of ram crafty used and it started with the allocated amount (384Mb hash and 128Mb phash) but very slowly rose to 650Mb before I stopped(this was over maybe 1½-2 hours)

The way I use the engine is in SCID where I step back and forth in my game while looking at variations(crafty is only analysing 1 variation at a time, not several)

On the good side though the new crafty seems very good at endgame analysis(although there are some positions where it doesnt have a clue, but every engine have that)

[Jim Ablett]

I have SCID installed with Peter Skinners Win32 compile of crafty plus his crafty.rc(unedited)

After analysing one of my games for some hours(yes im quite thorough ) windows ran out of memory.

Restarting the engine again I kept an eye on the size of ram crafty used and it started with the allocated amount (384Mb hash and 128Mb phash) but very slowly rose to 650Mb before I stopped(this was over maybe 1½-2 hours)

The way I use the engine is in SCID where I step back and forth in my game while looking at variations(crafty is only analysing 1 variation at a time, not several)

On the good side though the new crafty seems very good at endgame analysis(although there are some positions where it doesnt have a clue, but every engine have that)

I've compiled some builds which use Crafty's Unix/Linux threading code
and the 3rd party Windows pthread libraries.
These builds seems to work ok.

Download contains both Intel compiler 10 win32 & x64 Crafty 22.4 test p.g.o builds.

http://www.mediafire.com/?zkbdonhqzi2

Jim.

[CThinker]

First of all a bug thanks to Robert Hyatt for Crafty 22.4

But there may be a memory leak of some sort

Im using SCID and Peter Skinners Win32 port of crafty for analysing my own games, but it seems that it steadily consumes more and more memory until Windows runs out

Im going back and forth in the games while crafty analyses, so maybe someone could do the same and see if they get the same problem ?

You will have to explain how to reproduce this. I have been running lots of games and Crafty is not getting bigger as it plays more and more games...

This is very easy to repro. Try the following and watch the memory usage of crafty climb. Note that this binary is from Peter.

Code: Select all

C:\Chess\Etc\crafty-22.4-win32>crafty-22.4-win32.exe
EPD Kit revision date: 1996.04.21
unable to open book file [./book.bin].
book is disabled
unable to open book file [./books.bin].

Initializing multiple threads.
System is SMP, not NUMA.

Crafty v22.4 (1 cpus)

White(1): xboard

tellicsnoalias set 1 Crafty v22.4 (1 cpus)
tellicsnoalias kibitz Hello from Crafty v22.4! (1 cpus)

smpmt=2
new
tellicsnoalias set 1 Crafty v22.4 (2 cpus)
time 1000
easy
go
move Nf3

smpmt=2
new
tellicsnoalias set 1 Crafty v22.4 (2 cpus)
time 1000
easy
go
move Nc3

smpmt=2
new
tellicsnoalias set 1 Crafty v22.4 (2 cpus)
time 1000
easy
go
move Nc3

[kiroje]

First of all a bug thanks to Robert Hyatt for Crafty 22.4

But there may be a memory leak of some sort

Im using SCID and Peter Skinners Win32 port of crafty for analysing my own games, but it seems that it steadily consumes more and more memory until Windows runs out

Im going back and forth in the games while crafty analyses, so maybe someone could do the same and see if they get the same problem ?

You will have to explain how to reproduce this. I have been running lots of games and Crafty is not getting bigger as it plays more and more games...

This is very easy to repro. Try the following and watch the memory usage of crafty climb. Note that this binary is from Peter.

Code: Select all

C:\Chess\Etc\crafty-22.4-win32>crafty-22.4-win32.exe
EPD Kit revision date: 1996.04.21
unable to open book file [./book.bin].
book is disabled
unable to open book file [./books.bin].

Initializing multiple threads.
System is SMP, not NUMA.

Crafty v22.4 (1 cpus)

White(1): xboard

tellicsnoalias set 1 Crafty v22.4 (1 cpus)
tellicsnoalias kibitz Hello from Crafty v22.4! (1 cpus)

smpmt=2
new
tellicsnoalias set 1 Crafty v22.4 (2 cpus)
time 1000
easy
go
move Nf3

smpmt=2
new
tellicsnoalias set 1 Crafty v22.4 (2 cpus)
time 1000
easy
go
move Nc3

smpmt=2
new
tellicsnoalias set 1 Crafty v22.4 (2 cpus)
time 1000
easy
go
move Nc3

Ahh at least im not the only one to experience it ... bot that im happy about it

[bob]

First of all a bug thanks to Robert Hyatt for Crafty 22.4

But there may be a memory leak of some sort

Im using SCID and Peter Skinners Win32 port of crafty for analysing my own games, but it seems that it steadily consumes more and more memory until Windows runs out

Im going back and forth in the games while crafty analyses, so maybe someone could do the same and see if they get the same problem ?

OK. I have now released the source for version 22.5.

The problem was not actually in Crafty, although I originally thought it might be. the new posix threads now defaults to "joinable" for the threads, where Crafty does not every join. Last time I used posix threads (at least a couple of years ago) the default was "detached" which means when a thread terminates the stack is not kept until the join is completed. This resulted in N "stacks" being kept for each search done, and the pthreads stacks are quite large. I have fixed this although as far as I am aware, this only applies to UNIX systems. This version sets the thread attribute correctly to solve this...

[kiroje]

Ahh that is great!

Looking forward to getting the new version when the compiles from Jim or Peter is ready.

Thanks

[Guetti]

First of all a bug thanks to Robert Hyatt for Crafty 22.4

But there may be a memory leak of some sort

Im using SCID and Peter Skinners Win32 port of crafty for analysing my own games, but it seems that it steadily consumes more and more memory until Windows runs out

Im going back and forth in the games while crafty analyses, so maybe someone could do the same and see if they get the same problem ?

OK. I have now released the source for version 22.5.

The problem was not actually in Crafty, although I originally thought it might be. the new posix threads now defaults to "joinable" for the threads, where Crafty does not every join. Last time I used posix threads (at least a couple of years ago) the default was "detached" which means when a thread terminates the stack is not kept until the join is completed. This resulted in N "stacks" being kept for each search done, and the pthreads stacks are quite large. I have fixed this although as far as I am aware, this only applies to UNIX systems. This version sets the thread attribute correctly to solve this...

22.5 seems hopelessly broken for me. You shortened rook_open_file from a three dimensional array to a one dimensional array which contradicts data.c. Could you please update data.c?

In file included from crafty.c:13:
evaluate.c: In function ‘EvaluateRooks’:
evaluate.c warning: unused variable ‘open_files’
In file included from crafty.c:27:
data.c: At top level:
data.c:905: error: conflicting types for ‘rook_open_file’
data.h:398: error: previous declaration of ‘rook_open_file’ was here
In file included from crafty.c:33:
init.c: In function ‘InitializePawnMasks’:
init.c:877: warning: unused variable ‘file’
init.c:877: warning: unused variable ‘k’
In file included from crafty.c:39:
option.c: In function ‘Option’:
option.c warning: initialization makes integer from pointer without a cast
option.c error: invalid operands to binary *
option.c error: invalid operands to binary -
make[2]: *** [crafty.o] Error 1