ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Compression of chess databases
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Daniel Shawul



Joined: 14 Mar 2006
Posts: 2186
Location: Ethiopia
PostPost subject: Re: Compression of chess databases    Posted: Wed Jul 18, 2012 9:35 am Reply to topic Reply with quote
Quote:

I divide compression a chess database into two phases:
1) compress by encode moves such as changing from 2 bytes to 1 byte or 6 bits or 4 bits per move
2) compress further by applying normal compressor such as zip, tar, bsz2... library / software on data files

Don't play dumb here. If you did that you will get far more than 10% but I guess we won't see data from you.
Quote:

I am usually disappointed about phase 2 because of poor rate.

Back to your work, if you save your moves by 4 bits only, you can save immediately half of your size without using any extra compressor. That job is also faster and using less code (you don't need to integrate some compression libs).

It doesn't matter if you use 8 bits/move or 4 bits/move you will probable get same result. That is as long as you use the same number of bits per move and not mess up the entropy.
_________________
https://sites.google.com/site/dshawul/
https://github.com/dshawul
Back to top
View user's profile Send private message Visit poster's website Yahoo Messenger
Display posts from previous:   
Subject Author Date/Time
Compression of chess databases Edmund Moshammer Mon Jul 09, 2012 8:28 am
      Re: Compression of chess databases Richard Vida Mon Jul 09, 2012 11:14 am
            Re: Compression of chess databases Richard Vida Mon Jul 09, 2012 11:36 am
                  Re: Compression of chess databases Edmund Moshammer Mon Jul 09, 2012 11:48 am
                        Re: Compression of chess databases Richard Vida Mon Jul 09, 2012 11:58 am
                              Re: Compression of chess databases Edmund Moshammer Mon Jul 09, 2012 1:03 pm
                  Re: Compression of chess databases Daniel Shawul Mon Jul 09, 2012 3:06 pm
                        Re: Compression of chess databases Daniel Shawul Mon Jul 09, 2012 3:27 pm
                        Re: Compression of chess databases Richard Vida Mon Jul 09, 2012 5:34 pm
                              Re: Compression of chess databases Daniel Shawul Mon Jul 09, 2012 6:23 pm
                                    Re: Compression of chess databases Gary Mon Jul 09, 2012 8:39 pm
                                          Re: Compression of chess databases Daniel Shawul Mon Jul 09, 2012 9:01 pm
                                                Re: Compression of chess databases Gary Mon Jul 09, 2012 9:17 pm
                                                      Re: Compression of chess databases Daniel Shawul Mon Jul 09, 2012 9:29 pm
                  Re: Compression of chess databases Nguyen Pham Sun Jul 15, 2012 8:05 am
                        Re: Compression of chess databases J. Wesley Cleveland Sun Jul 15, 2012 6:02 pm
                              Re: Compression of chess databases Nguyen Pham Sun Jul 15, 2012 11:15 pm
                                    Re: Compression of chess databases Ronald de Man Mon Jul 16, 2012 1:01 am
                        Re: Compression of chess databases Richard Vida Mon Jul 16, 2012 12:31 pm
                              Re: Compression of chess databases Nguyen Pham Mon Jul 16, 2012 2:57 pm
                                    Re: Compression of chess databases Álvaro Begué Mon Jul 16, 2012 6:01 pm
                                          Re: Compression of chess databases Nguyen Pham Tue Jul 17, 2012 12:59 am
                                                Re: Compression of chess databases Kevin Hearn Tue Jul 17, 2012 2:39 am
                                                Re: Compression of chess databases Ronald de Man Tue Jul 17, 2012 9:29 pm
                                                      Re: Compression of chess databases Daniel Shawul Tue Jul 17, 2012 10:12 pm
                                    Re: Compression of chess databases J. Wesley Cleveland Tue Jul 17, 2012 5:20 am
                                          Re: Compression of chess databases Nguyen Pham Tue Jul 17, 2012 7:19 am
                                    Re: Compression of chess databases Richard Vida Tue Jul 17, 2012 9:04 pm
                                          Re: Compression of chess databases Richard Vida Tue Jul 17, 2012 11:35 pm
                                                Re: Compression of chess databases Daniel Shawul Tue Jul 17, 2012 11:59 pm
                                                      Re: Compression of chess databases Ronald de Man Wed Jul 18, 2012 12:55 am
                                                            Re: Compression of chess databases Daniel Shawul Wed Jul 18, 2012 1:35 am
                                                Re: Compression of chess databases Nguyen Pham Wed Jul 18, 2012 5:23 am
                                                      Re: Compression of chess databases Daniel Shawul Wed Jul 18, 2012 9:35 am
                                                            Re: Compression of chess databases Nguyen Pham Wed Jul 18, 2012 10:07 am
                                                                  Re: Compression of chess databases Uri Blass Wed Jul 18, 2012 10:31 am
                                                                        Re: Compression of chess databases Nguyen Pham Wed Jul 18, 2012 10:48 am
                                                                              Re: Compression of chess databases Daniel Shawul Wed Jul 18, 2012 11:00 am
                                                                  Re: Compression of chess databases Daniel Shawul Wed Jul 18, 2012 11:07 am
                                                                        Re: Compression of chess databases Nguyen Pham Wed Jul 18, 2012 11:47 am
                                                                              Re: Compression of chess databases Daniel Shawul Wed Jul 18, 2012 12:41 pm
                                                                                    Re: Compression of chess databases Nguyen Pham Wed Jul 18, 2012 1:20 pm
                                                                                          Re: Compression of chess databases Daniel Shawul Wed Jul 18, 2012 3:01 pm
                                                                                          Re: Compression of chess databases Daniel Shawul Wed Jul 18, 2012 3:40 pm
                                                                                          Re: Compression of chess databases Nguyen Pham Thu Jul 19, 2012 1:45 am
                                                                                          Re: Compression of chess databases Daniel Shawul Thu Jul 19, 2012 2:32 am
                                                                                          Re: Compression of chess databases Nguyen Pham Thu Jul 19, 2012 3:19 am
                                                                                          Re: Compression of chess databases Daniel Shawul Thu Jul 19, 2012 4:21 am
                                                                                          Re: Compression of chess databases Nguyen Pham Thu Jul 19, 2012 5:33 am
                                                                                          Re: Compression of chess databases Daniel Shawul Thu Jul 19, 2012 6:53 am
                                                                                          Re: Compression of chess databases Daniel Shawul Thu Jul 19, 2012 4:49 am
                                    Re: Compression of chess databases Daniel Shawul Tue Jul 17, 2012 9:25 pm
      Re: Compression of chess databases Vincent Diepeveen Wed Jul 11, 2012 12:14 am
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads