Some endian code for C++ programmers

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Some endian code for C++ programmers

Post by sje »

My chess programs are designed to have portable coded opening books that have the same representation on both big endian (Intel/AMD) and little endian (PowerPC/Motorola) machines. Having to code for both flavors help make for a more reliable program. To support multi-byte integer I/O, my programs use a small and simple library to help.

The routines do I/O one byte at a time. This isn't slow as one might think as the underlying system buffering takes care of much of the work. Every single I/O operation is tested for success/failure.

Note: The streams must be opened in std::ios::binary mode to work on some systems (e.g., Windows).

To assist other C++ programmers, here's the source:

Code: Select all

#include <iostream>

// Integer types &#40;natural&#41;

typedef signed   int           si;
typedef unsigned int           ui;

// Integer types &#40;specific lengths required&#41;

typedef signed   char          si8;
typedef unsigned char          ui8;

typedef signed   short int     si16;
typedef unsigned short int     ui16;

typedef signed   int           si32;
typedef unsigned int           ui32;

typedef signed   long long int si64;
typedef unsigned long long int ui64;

#define ByteBitsLen 8

bool GetNetUi16&#40;ui16& value, std&#58;&#58;istream& istr&#41;
&#123;
  bool valid = true;
  ui index = 0;
  char ch;

  value = 0;
  while &#40;valid && &#40;index < sizeof&#40;ui16&#41;))
  &#123;
    istr.read&#40;&ch, 1&#41;;
    if &#40;istr.fail&#40;)) valid = false; else &#123;value = &#40;value << ByteBitsLen&#41; | &#40;ui8&#41; ch; index++;&#125;;
  &#125;;
  if (!valid&#41; value = 0;
  return valid;
&#125;

bool GetNetUi32&#40;ui32& value, std&#58;&#58;istream& istr&#41;
&#123;
  bool valid = true;
  ui index = 0;
  char ch;

  value = 0;
  while &#40;valid && &#40;index < sizeof&#40;ui32&#41;))
  &#123;
    istr.read&#40;&ch, 1&#41;;
    if &#40;istr.fail&#40;)) valid = false; else &#123;value = &#40;value << ByteBitsLen&#41; | &#40;ui8&#41; ch; index++;&#125;;
  &#125;;
  if (!valid&#41; value = 0;
  return valid;
&#125;

bool GetNetUi64&#40;ui64& value, std&#58;&#58;istream& istr&#41;
&#123;
  bool valid = true;
  ui index = 0;
  char ch;

  value = 0;
  while &#40;valid && &#40;index < sizeof&#40;ui64&#41;))
  &#123;
    istr.read&#40;&ch, 1&#41;;
    if &#40;istr.fail&#40;)) valid = false; else &#123;value = &#40;value << ByteBitsLen&#41; | &#40;ui8&#41; ch; index++;&#125;;
  &#125;;
  if (!valid&#41; value = 0;
  return valid;
&#125;

bool PutNetUi16&#40;const ui16 value, std&#58;&#58;ostream& ostr&#41;
&#123;
  bool valid = true;
  ui index = 0;
  char ch;

  while &#40;valid && &#40;index < sizeof&#40;ui16&#41;))
  &#123;
    ch = &#40;value >> &#40;ByteBitsLen * &#40;sizeof&#40;ui16&#41; - 1 - index&#41;)) & 0x00ff;
    ostr.write&#40;&ch, 1&#41;; if &#40;ostr.fail&#40;)) valid = false; else index++;
  &#125;;
  return valid;
&#125;

bool PutNetUi32&#40;const ui32 value, std&#58;&#58;ostream& ostr&#41;
&#123;
  bool valid = true;
  ui index = 0;
  char ch;

  while &#40;valid && &#40;index < sizeof&#40;ui32&#41;))
  &#123;
    ch = &#40;value >> &#40;ByteBitsLen * &#40;sizeof&#40;ui32&#41; - 1 - index&#41;)) & 0x00ff;
    ostr.write&#40;&ch, 1&#41;; if &#40;ostr.fail&#40;)) valid = false; else index++;
  &#125;;
  return valid;
&#125;

bool PutNetUi64&#40;const ui64 value, std&#58;&#58;ostream& ostr&#41;
&#123;
  bool valid = true;
  ui index = 0;
  char ch;

  while &#40;valid && &#40;index < sizeof&#40;ui64&#41;))
  &#123;
    ch = &#40;value >> &#40;ByteBitsLen * &#40;sizeof&#40;ui64&#41; - 1 - index&#41;)) & 0x00ff;
    ostr.write&#40;&ch, 1&#41;; if &#40;ostr.fail&#40;)) valid = false; else index++;
  &#125;;
  return valid;
&#125;

User avatar
WinPooh
Posts: 267
Joined: Fri Mar 17, 2006 8:01 am
Location: Russia
Full name: Vladimir Medvedev

Re: Some endian code for C++ programmers

Post by WinPooh »

typedef signed int si32;
typedef unsigned int ui32;
This code is 32/64 bit sensible. On 64-bit systems signed int is not 32-bit.
I use the following approach:

Code: Select all

#ifndef _MSC_VER
	#include <stdint.h>
	typedef int8_t            I8;
	typedef uint8_t           U8;
	typedef int32_t          I32;
	typedef uint32_t         U32;
	typedef uint64_t         U64;
	#define LL&#40;x&#41; x##LL
#else
	typedef signed char       I8;
	typedef unsigned char     U8;
	typedef int              I32;
	typedef unsigned int     U32;
	typedef unsigned __int64 U64;
	#define LL&#40;x&#41; x##L
#endif
Of course, on MSVC it has the same assumption - but I never compile on 64-bit Windows.
Aleks Peshkov
Posts: 892
Joined: Sun Nov 19, 2006 9:16 pm
Location: Russia

Re: Some endian code for C++ programmers

Post by Aleks Peshkov »

Example of code where C++ templates will rule. :wink:
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Re: Some endian code for C++ programmers

Post by sje »

It was too simple to use templates.
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Some endian code for C++ programmers

Post by mcostalba »

Code: Select all

bool GetNetUi16&#40;ui16& value, std&#58;&#58;istream& istr&#41; 
I would overload operator<<() and operator>>() instead

so to write

Code: Select all

myStream << v16 << v32 << v64;

myStream >> vxx;

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Re: Some endian code for C++ programmers

Post by sje »

The very first bit of code in the application class constructor:

Code: Select all

  if &#40;sizeof&#40;si8&#41;  != 1&#41; Die&#40;"Bad sizeof&#40;si8&#41;  != 1");
  if &#40;sizeof&#40;ui8&#41;  != 1&#41; Die&#40;"Bad sizeof&#40;ui8&#41;  != 1");

  if &#40;sizeof&#40;si16&#41; != 2&#41; Die&#40;"Bad sizeof&#40;si16&#41; != 2");
  if &#40;sizeof&#40;ui16&#41; != 2&#41; Die&#40;"Bad sizeof&#40;ui16&#41; != 2");

  if &#40;sizeof&#40;si32&#41; != 4&#41; Die&#40;"Bad sizeof&#40;si32&#41; != 4");
  if &#40;sizeof&#40;ui32&#41; != 4&#41; Die&#40;"Bad sizeof&#40;ui32&#41; != 4");

  if &#40;sizeof&#40;si64&#41; != 8&#41; Die&#40;"Bad sizeof&#40;si64&#41; != 8");
  if &#40;sizeof&#40;ui64&#41; != 8&#41; Die&#40;"Bad sizeof&#40;ui64&#41; != 8");
Where I don't need to know integer sizes, I just use either "ui" or "si".
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Re: Some endian code for C++ programmers

Post by sje »

mcostalba wrote:

Code: Select all

bool GetNetUi16&#40;ui16& value, std&#58;&#58;istream& istr&#41; 
I would overload operator<<() and operator>>() instead

so to write

Code: Select all

myStream << v16 << v32 << v64;

myStream >> vxx;

I've also done this before and it can work reasonably well. However, it once led to a hard-to-find bug where I had forgotten to include the header file with the override definition and got the defualt operator semantics by mistake. There is no compiler issued warning or error to help in that case.
mathmoi
Posts: 289
Joined: Mon Mar 13, 2006 5:23 pm
Location: Québec

Re: Some endian code for C++ programmers

Post by mathmoi »

Here is my solution with C++ templates.

Code: Select all

   enum EEndian
   &#123;
      LITTLE_ENDIAN_ORDER,
      BIG_ENDIAN_ORDER,
#if defined&#40;BOOST_LITTLE_ENDIAN&#41;
      HOST_ENDIAN_ORDER = LITTLE_ENDIAN_ORDER
#elif defined&#40;BOOST_BIG_ENDIAN&#41;
      HOST_ENDIAN_ORDER = BIG_ENDIAN_ORDER
#else
#error "Impossible de determiner l'indianness du systeme cible."
#endif
   &#125;;

   template <class T, unsigned int size>
   inline T SwapBytes&#40;T value&#41;
   &#123;
      union
      &#123;
         T value;
         char bytes&#91;size&#93;;
      &#125; in, out;

      in.value = value;

      for &#40;unsigned int i = 0; i < size / 2; ++i&#41;
      &#123;
         out.bytes&#91;i&#93; = in.bytes&#91;size - 1 - i&#93;;
         out.bytes&#91;size - 1 - i&#93; = in.bytes&#91;i&#93;;
      &#125;

      return out.value;
   &#125;

   template<EEndian from, EEndian to, class T>
   inline T EndianSwapBytes&#40;T value&#41;
   &#123;
      // A &#58; La donnée à swapper à une taille de 2, 4 ou 8 octets
      BOOST_STATIC_ASSERT&#40;sizeof&#40;T&#41; == 2 || sizeof&#40;T&#41; == 4 || sizeof&#40;T&#41; == 8&#41;;

      // A &#58; La donnée à swapper est d'un type arithmetic
      BOOST_STATIC_ASSERT&#40;boost&#58;&#58;is_arithmetic<T>&#58;&#58;value&#41;;

      // Si from et to sont du même type on ne swap pas.
      if &#40;from == to&#41;
         return value;

      return SwapBytes<T, sizeof&#40;T&#41;>&#40;value&#41;;
   &#125;
To use it you do something like this :

Code: Select all

litleEndian = EndianSwapBytes<HOST_ENDIAN_ORDER, LITTLE_ENDIAN_ORDER&#40;anyIntegerTypeVariable&#41;
Thisway littleEndian is the representation of anyIntegerVariable in little endian order whatever the host endian order is. If the host endian order is the same as the target order (in this case LITTLE_ENDIAN_ORDER) no code is generated at all.
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Some endian code for C++ programmers

Post by mcostalba »

sje wrote:The very first bit of code in the application class constructor:

Code: Select all

  if &#40;sizeof&#40;si8&#41;  != 1&#41; Die&#40;"Bad sizeof&#40;si8&#41;  != 1");
  if &#40;sizeof&#40;ui8&#41;  != 1&#41; Die&#40;"Bad sizeof&#40;ui8&#41;  != 1");

  if &#40;sizeof&#40;si16&#41; != 2&#41; Die&#40;"Bad sizeof&#40;si16&#41; != 2");
  if &#40;sizeof&#40;ui16&#41; != 2&#41; Die&#40;"Bad sizeof&#40;ui16&#41; != 2");

  if &#40;sizeof&#40;si32&#41; != 4&#41; Die&#40;"Bad sizeof&#40;si32&#41; != 4");
  if &#40;sizeof&#40;ui32&#41; != 4&#41; Die&#40;"Bad sizeof&#40;ui32&#41; != 4");

  if &#40;sizeof&#40;si64&#41; != 8&#41; Die&#40;"Bad sizeof&#40;si64&#41; != 8");
  if &#40;sizeof&#40;ui64&#41; != 8&#41; Die&#40;"Bad sizeof&#40;ui64&#41; != 8");
Where I don't need to know integer sizes, I just use either "ui" or "si".
I would catch this at compile time with a static assert.

For some info look at

http://www.boost.org/doc/libs/1_38_0/do ... ssert.html

Altough you can build one yourself much more simple and without requiring including third part libraries.
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Some endian code for C++ programmers

Post by mcostalba »

sje wrote: I've also done this before and it can work reasonably well. However, it once led to a hard-to-find bug where I had forgotten to include the header file with the override definition and got the defualt operator semantics by mistake. There is no compiler issued warning or error to help in that case.

Subclass std::istream to have type safety.

Code: Select all

class MyStream &#58; public std&#58;&#58;istream &#123;
public&#58;
  explicit MyStream &#40;streambuf * sb&#41;;
  virtual ~MyStream ();
&#125;