I know I can download Lichess databases here: https://database.lichess.org/
But I'm looking for collections of high quality Lichess databases.
I already find a collection database of human vs human games between players of at least 2200 elo
https://rebel13.nl/download/data.html
I also find the lichess_elite_scid_2021-03 database
https://database.nikonoel.fr/
I'm looking for collections of high quality Lichess games. Like all standard human games played on Lichess.org, where one of the players have a minimum rating of 2000 Elo.
I want a database of at least 100,000,000 games
I know I could try to build this database myself from the databases I can find here: https://database.lichess.org/
But that will take a verry long time, and I don't know if it will even be possible on my PC, because I have limited hardware.
Looking for high quality selection of Lichess games.
Moderator: Ras
-
Jonathan003
- Posts: 243
- Joined: Fri Jul 06, 2018 4:23 pm
- Full name: Jonathan Cremers
-
dkappe
- Posts: 1632
- Joined: Tue Aug 21, 2018 7:52 pm
- Full name: Dietrich Kappe
Re: Looking for high quality selection of Lichess games.
I’ve made use of lichess pgn’s, but my requirements were quite different. For filtering and processing I used a very modest PC with pgn-extract and python-chess. Unless you’re using a 386 with floppies, you should be able to use those tools to get what you want.
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
-
jshriver
- Posts: 1360
- Joined: Wed Mar 08, 2006 9:41 pm
- Location: Morgantown, WV, USA
Re: Looking for high quality selection of Lichess games.
This could be done by using pgn-extract to dump only games of a certain ELO or higher. The only issue is that it will also include computer matches since there is no tag specifying human or computer.
-
Volker Pittlik
- Posts: 628
- Joined: Wed Mar 08, 2006 9:10 pm
- Location: Murten / Morat, Switzerland
- Full name: Volker Pittlik
Re: Looking for high quality selection of Lichess games.
100,000,000? My private collections doesn't come even close to that number. And there s a lot commercial stuff inside. I guess such a number of high-quality games simply does not exits. The necessary size of the database depends on the purpose of it.Jonathan003 wrote: ↑Wed Feb 02, 2022 1:28 am ...
I'm looking for collections of high quality Lichess games. Like all standard human games played on Lichess.org, where one of the players have a minimum rating of 2000 Elo.
I want a database of at least 100,000,000 games
...
vp
-
Jonathan003
- Posts: 243
- Joined: Fri Jul 06, 2018 4:23 pm
- Full name: Jonathan Cremers
Re: Looking for high quality selection of Lichess games.
With high quality games I mean games that are useful for me to study and to make tactics from. I'm rated about 1600 elo on chess.comVolker Pittlik wrote: ↑Wed Feb 02, 2022 7:06 am100,000,000? My private collections doesn't come even close to that number. And there s a lot commercial stuff inside. I guess such a number of high-quality games simply does not exits. The necessary size of the database depends on the purpose of it.Jonathan003 wrote: ↑Wed Feb 02, 2022 1:28 am ...
I'm looking for collections of high quality Lichess games. Like all standard human games played on Lichess.org, where one of the players have a minimum rating of 2000 Elo.
I want a database of at least 100,000,000 games
...
vp
So games where one player has a rating of at least 2000 elo will do it for me.
-
Jonathan003
- Posts: 243
- Joined: Fri Jul 06, 2018 4:23 pm
- Full name: Jonathan Cremers
Re: Looking for high quality selection of Lichess games.
I like PgnTools to remove duplicates and merge pgn databases the same time.dkappe wrote: ↑Wed Feb 02, 2022 1:57 am I’ve made use of lichess pgn’s, but my requirements were quite different. For filtering and processing I used a very modest PC with pgn-extract and python-chess. Unless you’re using a 386 with floppies, you should be able to use those tools to get what you want.
https://eman.zohosites.com/
I think its much faster than pgn-extract and you can see the progress, so you have some idea how long it is gone take.
-
whelanh
- Posts: 6
- Joined: Mon Jun 28, 2021 4:04 pm
- Full name: Hugh Whelan
Re: Looking for high quality selection of Lichess games.
In terms of high quality games, I feel time control is quite important. A large number of blitz and bullet games with many blunders won't help you much. Also lichess has a lot of games with very few moves (i.e. abandoned games). For my own filtering of lichess games I use pgn-extract with a time and move number (and ratings) filters to only retain games with at least 15 moves, time control of at least 1000 seconds, (and ratings of both players >= 2050). The pgn extract command to do that is:
.\pgn-extract.exe -t.\filterFile.txt -#10000 -bl15 .\lichessMonthlyDownload.pgn
where filterFile.txt is just a simple text file with three lines:
WhiteElo >= "2050"
BlackElo >= "2050"
TimeControl >= "1000"
The #10000 argument just separates the output into separate files of 10,000 games (I find this speeds up processing). This typically reduces the massive number of lichess games down to something like only 20,000 - 30,000 games per month.
.\pgn-extract.exe -t.\filterFile.txt -#10000 -bl15 .\lichessMonthlyDownload.pgn
where filterFile.txt is just a simple text file with three lines:
WhiteElo >= "2050"
BlackElo >= "2050"
TimeControl >= "1000"
The #10000 argument just separates the output into separate files of 10,000 games (I find this speeds up processing). This typically reduces the massive number of lichess games down to something like only 20,000 - 30,000 games per month.
-
Cornfed
- Posts: 511
- Joined: Sun Apr 26, 2020 11:40 pm
- Full name: Brian D. Smith
Re: Looking for high quality selection of Lichess games.
We all have our excentricities, but with that being your intention, you are truly wasting your time in trying to 'reinvent the wheel' here. Plenty of tactics trainers out there already exist to 'study'.Jonathan003 wrote: ↑Wed Feb 02, 2022 12:44 pmWith high quality games I mean games that are useful for me to study and to make tactics from. I'm rated about 1600 elo on chess.comVolker Pittlik wrote: ↑Wed Feb 02, 2022 7:06 am100,000,000? My private collections doesn't come even close to that number. And there s a lot commercial stuff inside. I guess such a number of high-quality games simply does not exits. The necessary size of the database depends on the purpose of it.Jonathan003 wrote: ↑Wed Feb 02, 2022 1:28 am ...
I'm looking for collections of high quality Lichess games. Like all standard human games played on Lichess.org, where one of the players have a minimum rating of 2000 Elo.
I want a database of at least 100,000,000 games
...
vp
So games where one player has a rating of at least 2000 elo will do it for me.
-
Jonathan003
- Posts: 243
- Joined: Fri Jul 06, 2018 4:23 pm
- Full name: Jonathan Cremers
Re: Looking for high quality selection of Lichess games.
I think you don't understand the idea I try to explain. So you shouldn't judge.