File size for files > 4Gb

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

Cardoso
Posts: 362
Joined: Thu Mar 16, 2006 7:39 pm
Location: Portugal
Full name: Alvaro Cardoso

File size for files > 4Gb

Post by Cardoso »

Hi,
I'm generating the 8man checkers EGTB and for the first time one slice is greater than 4GB.
I'm using MSVS 2010, and c++, a x64 project and windows 7 x64, fwrite and fread don't work so I made my own versions of these functions to write and read files by chunks.
But another problem occured, I can't determine the file size of files bigger than 4gb.
I use _fseeki64 and _ftelli64, but these don't work.
The size is allways zero. I open files with _fsopen().
I came across the _filelength64() function but this func don't work with (FILE *) structure.
Can some please help me getting the file size of files bigger than 4GB?

best regards,
Alvaro
F. Bluemers
Posts: 868
Joined: Thu Mar 09, 2006 11:21 pm
Location: Nederland

Re: File size for files > 4Gb

Post by F. Bluemers »

Cardoso wrote:Hi,
I'm generating the 8man checkers EGTB and for the first time one slice is greater than 4GB.
I'm using MSVS 2010, and c++, a x64 project and windows 7 x64, fwrite and fread don't work so I made my own versions of these functions to write and read files by chunks.
But another problem occured, I can't determine the file size of files bigger than 4gb.
I use _fseeki64 and _ftelli64, but these don't work.
The size is allways zero. I open files with _fsopen().
I came across the _filelength64() function but this func don't work with (FILE *) structure.
Can some please help me getting the file size of files bigger than 4GB?

best regards,
Alvaro
What about the underlying I/O functions?
_open() and _tell(),they might work.
F. Bluemers
Posts: 868
Joined: Thu Mar 09, 2006 11:21 pm
Location: Nederland

Re: File size for files > 4Gb

Post by F. Bluemers »

Cardoso wrote:Hi,
I
I came across the _filelength64() function but this func don't work with (FILE *) structure.
Can some please help me getting the file size of files bigger than 4GB?
You can use the _fileno() function to get the file handle from FILE,
with that you should be able to use _tell64() or filelengthi64()

http://msdn.microsoft.com/en-US/library ... x(v=vs.80)

Best
Fonzy
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: File size for files > 4Gb

Post by diep »

Cardoso wrote:Hi,
I'm generating the 8man checkers EGTB and for the first time one slice is greater than 4GB.
I'm using MSVS 2010, and c++, a x64 project and windows 7 x64, fwrite and fread don't work so I made my own versions of these functions to write and read files by chunks.
But another problem occured, I can't determine the file size of files bigger than 4gb.
I use _fseeki64 and _ftelli64, but these don't work.
The size is allways zero. I open files with _fsopen().
I came across the _filelength64() function but this func don't work with (FILE *) structure.
Can some please help me getting the file size of files bigger than 4GB?

best regards,
Alvaro
do not use ansi-c here. that goes to 2GB max then is over.

What i use is in windows the windows file functions, since 1995 or so those can handle very huge sizes. And for linux there is a special define after which it works.

Note 7 men in Diep format can be very huge :)
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: File size for files > 4Gb

Post by diep »

F. Bluemers wrote:
Cardoso wrote:Hi,
I'm generating the 8man checkers EGTB and for the first time one slice is greater than 4GB.
I'm using MSVS 2010, and c++, a x64 project and windows 7 x64, fwrite and fread don't work so I made my own versions of these functions to write and read files by chunks.
But another problem occured, I can't determine the file size of files bigger than 4gb.
I use _fseeki64 and _ftelli64, but these don't work.
The size is allways zero. I open files with _fsopen().
I came across the _filelength64() function but this func don't work with (FILE *) structure.
Can some please help me getting the file size of files bigger than 4GB?

best regards,
Alvaro
What about the underlying I/O functions?
_open() and _tell(),they might work.
they will not
diep
Posts: 1822
Joined: Thu Mar 09, 2006 11:54 pm
Location: The Netherlands

Re: File size for files > 4Gb

Post by diep »

from some egtb code mine:

Code: Select all

 if( ed->fegtb[side] == INVALID_HANDLE_VALUE ) { // open file first

...

//   #else
    ed->fegtb[side] = CreateFile(buf,GENERIC_READ,FILE_SHARE_READ,NULL,OPEN_EXISTING,FILE_FLAG_RANDOM_ACCESS,NULL);
    if( ed->fegtb[side] == INVALID_HANDLE_VALUE ) {
   // #endif
      mprint("Skipping EGTB File %s. Failed to open. Automatic Disable!\n",buf);
      ed->exists ^= (1+side); // automatic disable; could happen in game
      return -1;
    }
some further...

Code: Select all

 li.QuadPart = (INT64)readfrombyte;
  seekresult = (int)SetFilePointerEx(ed->fegtb[side],li,NULL,FILE_BEGIN);
  if( seekresult == 0 ) {
    mprint("Error setting file pointer in EGTB\n");
    return -1;
  }
  errorcod = (int)ReadFile(ed->fegtb[side],(unsigned char *)(&egtb_ll.CACHE_BYTES[cb->cache_indexstart]),
   (DWORD)bytestoread,&bytesread,NULL);
  if( errorcod == 0 || bytesread != bytestoread ) {
    mprint("Error reading %i bytes from EGTB. read=%i bytes! errcode=%i\n",bytestoread,bytesread,errorcod);
    return -1;