Building Chess Opening Repertoire - Video Tutorial

This is a 3-part tutorial for building a chess database with all your openings. The process is also described in my book "How to Study Chess Openings".


If you intend to store your opening preparation in a computer database, this method will save you a lot of time with a well tested approach.

Searching for Patterns and Combinations using various Chess Software

I already wrote about search in a couple of posts, and these features of ChessBase and Chess Assistant are surprisingly still essentially unchanged and relevant:
Online tactics: Find Sacrifices feature in Chessbase Light
Chess Assistant feature - Search for current pawn structure and material

Since then I found two more interesting search features:

1) SCID has this pawn structure search that I was not aware of:

2) Also Chessbase has relatively recently added an ability to search for similar pawn structures. 

Still, you can’t directly search for IQP with Chessbase, and it seems that only Chessbase allows to search for sacrifices (if you are aware of other programs that can do that – please leave a comment!). That’s why we have more than one chess database software …

ChessBase - One File Format to Rule them All

As of Chessbase 11 and 12, Chessbase offers to put database files into one file with *.cbone extension. That seems very convenient compared to cbv files that cannot be viewed or modified without unpacking, and sure beats dozens of random extension (cbh, cbg, etc) files that were getting created/deleted behind the scenes in the old paradigm. For emailing games around, or backing up databases, cbone seems great. But I don't want to convert my databases unless I know that the new format is reliable. I will  experiment with cbone in the next while and see how this goes.
So far a couple of issues I can think of are:

• I noticed that read-only cbone files cause errors, even when opened up for viewing
• It is obvious that any older program such as Chessbase Light would not able to read cbone file format

Has anyone else noticed other problems?

Editing Pgn Files without ChessBase or Other Database Program – Syntax Highlighting

Six years ago I already once wrote about pgn file format, and the benefits of using it for storing and sharing your chess data. While working with many pgn files, I want to be able to load them into a text editor, and do various operations on it that can’t be done easily in a chess database, such as SCID, ChessBase, Chess Assistant, etc. Notepad++ , my favourite editor, lets me

• check spelling
• translate from one language to another
• do massive search and replacements across the entire file (or even many files)
• and many many other things you would normally do in a powerful text editor

The problem with text editors is that normally pgn like any text looks Black/White, and is not very pretty.

Instead I want it to look like this:

Here is the trick: once you loaded the pgn file, make sure to set language to “Pascal”, so that you get the proper syntax highlighting for:

• game headers
• different types of braces (for annotations and variations)
• move numbers

I never programmed (and probably never will) in Pascal, but this nice little similarity between syntaxes between PGN and Pascal comes really handy!

What is your Favourite Chess Software – poll?

Chess is what drove my interest to software in the first place, and in a way, the two have been fuelling my curiosity for each other. On this blog I write about chess software quite a bit, with the recent focus on free tools in particular. I added a new poll to the blog – I am curious to learn what chess tools are really the most popular among the readers of this blog. Please make your choice:

Chess Software – SCID vs. ChessBase – Cultural Differences

In his famous article titled Biculturalism, a famous software “expert” Joel Spolsky asks a rherorical question - “What are the cultural differences between Unix and Windows programmers?” His answer is simple: “for the most part it comes down to one thing: Unix culture values code which is useful to other programmers, while Windows culture values code which is useful to non-programmers.”

Looking at the chess software called SCID I realized that in comparison with other most popular software such as ChessBase, it clearly comes from a different culture. Where it becomes most noticeable is an ability to customize, extend and automate each program.

 Customizing ChessBase or ChessBase Light, or Fritz (and same applies to Chess Assistant and Convekta’s tools) – is mostly done via updating shortcuts and moving window layout with a mouse. Customizing SCID is achieved by updating text files on disk or adding sets of images in a different directory. Staying textual is what Joel says is typical for the Unix culture.

Automation is another component where the two differ – automating SCID can be achieved via writing TCL scripts that use a clearly defined interface (API) that has been a natural part of the software's architecture - http://scid.sourceforge.net/doc/progref.html. Nearly everything that is done by the SCID UI goes through the same interface. You can also run SCID without user interface at all, and just use the core system from the command line, as it was clearly intended to be used by a programmer.

Automating FRITZ is only possible by using whatever automation has been provided in the main user interface (so you can print several games at once – that’s the kind of “automation” it mostly provides). Aquarium (another Windows program) at some point added a scripting interface, as the developers realized that a lot of their users would like to extend a tool. However it clearly seemed like like a “slap on” effort and that the original system was not designed for this. Limitations and bugs seemed to be endless – at least that’s the impression I got based on user feedback online.

In the end of day, it is not surprising that Fritz is the most popular chess software while most chess fans have never heard about SCID unless they are using Linux as they are primary operating system. Marketing has a lot to do with it, and so does the slightly strange looking window layout that SCID uses. It is also important to remember that ChessBase’s founder Frederic Friedel was one of the pioneers of chess software from the mid-eighties, so it is only fair that his products succeed.

I do hope, however, that the community behind SCID will continue to grow as there are so many things it can do that many do not realize. Also, in the time of many new platforms emerging very quickly - having something written for a programmer makes it easy to port software, so is not surprising that Apps like SCID on the go become popular and appear for free even before larger software companies find the resources to port their windows oriented programs to mobile (and some of them never do because it is too complicated). Sometimes biculturalism is a good thing!

PS.

- “SCID on the Go” UI  - ever wonder why the free “SCID on the go” can read SCID’s native database format, and $5 ChessBase on Android (which is wonderful otherwise) – can’t do that for ChessBase format? Might have something to do with the “UI is the most important thing” approach in the original system).

History of Chess on The Internet

Just for fun, I decided to compile the record of how I experienced Internet chess, your mileage will of course vary!

1998 – free internet chess club (FICS)allows to play games on the internet any time
1999 – chess databases like on sites TWIC get updated on a regular basis and people can get access to them on a regular basis
2000 – major tournaments are broadcast on ICC with thousands of people following and commenting on games
2000 – chess portals like Kasparov Chess begin to publish regular articles
2001 – online 4 and 5 piece Endgame TableBases such as Nalimov are accessible
2001 – online chess lessons become as popular as the ones in person
2004 – instructional chess videos begin to get published by companies like ChessBase on a large scale
2005 – even non-major tournaments like Canadian Open get broadcast over the internet via DGT
2006 – YouTube allows regular chess amateurs like myself to share their analysis with others
2009 – commentators like Sergei Shipov switch to video format for sharing their analysis
2010 – playing online chess on a mobile device such as IPhone becomes a viable option
2010 – live chess ratings get updated on the nearly daily basis
2011 – websites like whychess.org broadcast major chess events with live engine analysis
2011 – Live, multi hour HD full game broadcasts of events like World Cup are streamed live with up to date commentary
2011 – Fritz 13 is released, allowing users to upload and share chess analysis via their online database
2012 – What is coming ahead??

Solving Simple Chess Tactics – Part 1

To improve your tactical vision, there is no need to buy expensive chess software, it can be done with free copy of ChessBase Light and database that can be downloaded from this page – the one under “Over 3500 tactical training positions from actual games”.

In the video I then follow these steps:

1. Load a game and change board to use “Training mode”
2. guess which move should be played, hit “forward” key
3. If my guess is correct, mark the game in the database as “deleted”
4. Load next game with “F10” shortcut

Here are few simple positions for you to solve. It is White to move in all of them.

     

     

The video shows the setup and solutions:

Fritz 13 Let’s Check Feature

Fritz 13 is adding a new feature that is called “Let’s check”. The feature allows to share engine analysis with other users and upload it to the “cloud”. Such analysis would be easy to look at in one of the Fritz panels:

The benefits of this kind of feature seem very exciting:

• it will allow fast access to all previously made engine analysis
• reduce the need to redundantly run engine on positions that someone else has analysed
• encourage sharing between chess players on an unprecedented scale
• it actually has a UI that is easy to understand. Convekta’s IDea still seems very complicated to me when I read explanations of how it works

The Video tutorials are brief and to the point:

But the scary aspects of the feature seem a lot more obvious:

• spying on each other – sounds like the option is on by default!
• in perspective, this takes us much closer to chess being completely solved
• ChessBase may control data contributed by many chess players, many engines and so on. While games are now being shared in databases produced by more than one vendor, ChessBase having billions of extra positions stored in their private databases will give them a monopoly over most of chess data, data contributed by their own users, who would now have to pay yearly membership fees to access that data.
• focus is on engine analysis, although I think this kind of system should have put emphasis on people’s verbal commentaries (Comments Network feature does seem to go in that direction though, although why not instead use existing GM comments that are spread out through their MegaBase already?)

Yes, this is just a tool to help players with what they do – use best engines to solve mysteries of various chess positions. Some would argue that this is inevitable anyway, but I find this tool more disturbing than anything else.

The biggest question I have though – for how long are they going to have enough storage space to maintain trillions of possible chess positions??

ChessBase turning 25

In his annotations to the game against Smyslov from the 1958 match, Botvinnik noted that a younger grandmaster ignored a transposition to the Botvinnik-Smyslov game, and as a result – made a faulty comment on some game played in the 1970s. Botvinnik’s conclusion was: “young grandmasters don’t study games of older generation, but it is also clear that chess openings should be stored using computer software”. Yes, this is so true, and Garry Kasparov was one of the first chess players who picked up on this idea, and therefore got involved in the development and promotion of what now became ChessBase software. Here is the story: http://www.chessbase.com/newsdetail.asp?newsid=7229. Time flies and ChessBase software is now the most popular one among chess players, and it has affected the development of the game quite profoundly.

The photo from the article comes with the caption “It was Kasparov who first recognized the power of the system and used it intensely”. Good point, but it feels like it was Kasparov’s teacher who was first to recognize the need for such software!

Years have passed, and Kasparov now is also inside the software!

Creating Opening Repertoire in ChessBase

I already wrote about how to better manager your repertoire as well as why you need to build your repertoire in the first place, but this video (posted by “Robofriven” on ChessVideos.TV) illustrates nicely the details of doing it in a particular tool – ChessBase 9 in this case. The author also mentions Chess Position Trainer, so you definitely don’t have to be restricted to one tool; more important is that your repertoire is stored somewhere at all, and you have simple ways of it updating it.

 
http://www.chessvideos.tv/forum/viewtopic.php?t=4477

Chess Assistant feature - Search for current pawn structure and material

Chess Assistant and Chessbase are both very powerful products, and it feels that the majority of chess players in the recent years have almost completely switched to Chessbase. I also have long ago decided that the convenience of the user interface even in various versions of Chessbase Light is more important than anything else in a chess package because it allows me to focus better on the 3 primary functions that I care most about:

1. Game entry
2. Viewing games
3. Engine analysis (and adding engine’s line to my analysis)

Yes, inability to easily resize the board has just turned me off from my good old copy of Chess Assistant 6, but every once in a while I still load it up, and I must say there are a few things that the Russian-based software company nailed back 10 years ago. For example:

1. Position Tree (integrated into the database)
2. Search

What is so fancy about “Search” in Chess Assistant? It is just more powerful! It

1. Remembers your previous search criteria (up to 10 or so)
2. Allows you to search multiple databases at once
3. Stores search results into datasets, and they are still around when you re-start the application
4. Is fast (again, goes back to positional tree integrated with the database, not stored in a separate lonely .ctg file Chessbase-style)

And here is the kicker, a very neat feature that allows you to better understand and study typical middlegames: “Search for Current pawn structure and material”:

That brings up a scary looking but otherwise very functional window like this:

It is very convenient to choose a database to search in, or multiple databases (something that is a traditional pain in Chessbase):

Searching hugebase then give a set of useful games with Black pawns on a5 and b5, and all 4 minor pieces still on the board:

It then remembers the dataset for future sessions

 

Sometimes it just feels that software makers don’t want to look at each other’s products! So 10 years later, some features are still better in one package, and resizing the board window is still problematic in its competitor. Or is it just that some things don’t fit into your original design, leaving you permanently screwed?

ChessBase in Silverlight

Chessbase finally has re-written their database software using one Microsoft’s latest technologies – Silverlight. You can “install” it by going to http://www.chesslive.de/. And it’s free to use! And it looks all pretty and shiny!

Ok, the first paragraph is full of overstatements:

1. this just gives the basics of functionality compared to their main products
2. It’s free to use if you run a non-free operating system, Windows. It seems to be using Silverlight 3, which is as far as I know has not been ported to Moonlight yet, so I suspect you won’t be able to use this interface on Linux today.
3. This just worked on my main PC, but on my laptop the install was a bit of a pain: it was first downloading “stuff” for a couple of minutes, and then informed me that I needed to upgrade Silverlight 2 to Silverlight 3. After I did that, my IE would begin to crash on startup, so I gave up on this whole Silverlight install thing in frustration. I suppose this complaint goes against Silverlight, not Chessbase.
4. “Paste position” did not work for me, I hope they fix it soon

But enough of whining, I think it’s great that more and more chess websites adopt Silverlight (The Week in Chess did that about a year ago). The Chessbase old Java-based online interface looked hideous and was barely functional. This one is pretty nice, and I can drag and drop UI panels around!

Chessbase Light 2009 – create custom shortcuts

I use Chessbase Light for creating diagrams for this blog. Certain operations, such as copying a diagram to later paste it into Windows Live Writer, need to be done a repeatedly. Just yesterday I learned how to create custom shortcuts for operations that currently don’t have any shortcuts assigned to them. Once you have a game window open, pick Tools->Customize menu item (not a very good label since shortcuts is the only thing it lets you customize, calling it “Custom Shortcuts” would make it more discoverable…).

The UI looks self explanatory, but you need to put the cursor into the “Input new shortcut” before it lets you assign a new shortcut – something that drove me a bit crazy before I realized it. The list of commands is a bit different if you do this in the “Database Window”.

Anyway, this is cool because this can save me time for creating blog entries. Also, there are some features in this long list (look at the scrollbar!)  that I never realized exist. Again, so much for the discoverability. Turns out you can “Generate Repertoire – Scan the database for games of your repertoire and generate the report”, a feature for which I have never seen a menu item in the UI! You do have an opening repertoire database, don’t you?

ChessBase Light 2009 – Prepare against opponent

ChessBase Light 2009, which came out recently, is not really fundamentally different from ChessBase Light 2007, but has a couple of nice features. One of such features is a helper menu to create a temporary tree and list of reference games – just for your opponent, and just for the colour that you care about. So if I was to play Kevin Spraggett in the next game as White, I would come to the players tab, pick Kevin’s name, and choose “Prepare against Black”.

That would give me a list of games (a short one, because I use a really small database), and show the likely options I have to face against 1.c4

I can see that he has not played 1…e5 for 25 years, so I should really focus on 1…Nf6. Similar functionality had existed in ChessBase Light 2007, but it was not really as convenient. This is a truly time saving feature if you are preparing against a specific opponent, their opening repertoire varies over time, and you’re trying to figure out what move they are most likely to play against you in the forthcoming game. The download is free, try it yourself!

The King Walk – Chigorin – Caro, 1898

Black to move

Try to predict the outcome of this chess game! Amazingly, 10 moves later Black got mated with a sacrifice that many of you probably have seen while studying tactics. After the following moves 25. …Qg2+ 26. Kc4 b5+ 27. Kd3 Qf3+ 28. Kc2 Qf2+ 29. Kb3 Rc8 30. Rc2 Qxf4 31. Kb2 Na5 32. Ka1 Qc4 33. e6 Nc6 34. Qd1 h5 35. Rg1 Rh7 the players got to this position:

 White to move

36. Rg7!! ends the game on a spot. Notice how the white king now is safely hidden.

Replay the full game here. Below is the raw pgn that you can copy and paste into a Chessbase Light board window.

[Event "Vienna"]
[Site "Vienna"]
[Date "1898.??.??"]
[EventDate "?"]
[Round "?"]
[Result "1-0"]
[White "Mikhail Chigorin"]
[Black "Horatio Caro"]
[ECO "C29"]
[WhiteElo "?"]
[BlackElo "?"]
[PlyCount "71"]

1. e4 e5 2. Nc3 Nf6 3. f4 d5 4. d3 Bb4 5. fxe5 Nxe4 6. dxe4
Qh4+ 7. Ke2 Bxc3 8. bxc3 Bg4+ 9. Nf3 dxe4 10. Qd4 Bh5 11. Ke3
Bxf3 12. Bb5+ c6 13. gxf3 Qh6+ 14. Kxe4 Qg6+ 15. Ke3 cxb5
16. Ba3 Nc6 17. Qd5 Qxc2 18. Rac1 Qf5 19. Rhe1 Rd8 20. Qxb5 a6
21. Qb1 Qg5+ 22. f4 Qg2 23. Bd6 Qh3+ 24. Ke4 f5+ 25. Kd5 Qg2+
26. Kc4 b5+ 27. Kd3 Qf3+ 28. Kc2 Qf2+ 29. Kb3 Rc8 30. Rc2 Qxf4
31. Kb2 Na5 32. Ka1 Qc4 33. e6 Nc6 34. Qd1 h5 35. Rg1 Rh7
36. Rxg7 1-0

Chess Statistics – results as White vs. Black

As I am addicted to statistics, I decided to count my losses in tournament games, and just as I expected – I lost a lot more games as Black than as White. Since the year 2000 – 31 losses with the Black pieces, compared to 13 games I lost as White. These kind of stats indicate that the opening, and the direction it gives to the game have a significant impact on your tournament results (surprise …). With better opening preparation, you can hope for better results, but this also indicates that having a solid repertoire as Black is generally more important, or else you will be losing a lot of games without any fight.

Following a similar pattern, my results as White are not that different – White wins more games. With ChessBase Light it is easy to generate such statistics, just pick all your games (or maybe first filter it by year to exclude your kindergarten games), and hit the `S` key.

Total breakdown as White:

And as Black:

This also shows that I played more games as Black (looking at the results – tournament pairings probably did cost me a few rating points here and there…). Of course, you need to keep a database of your own games to be able to do this kind of analysis.

Peshk@ – new chess training software from developers of Chess Assistant

I just downloaded this neat tool from http://chessok.com/?p=22212 and after playing with it for a few minutes – I must say it looks pretty neat. The interface shell itself is free, it comes with a few demo courses and you can download more for extra charge. This follows a model similar to what Chessbase is doing with ChessBase Light as a viewer for their media lessons, where the viewer is free, but you have to pay for extra courses. Here are a few screenshots of the demo endgame course I downloaded off their website also for free (based on a book by Alexander Panchenko, who sadly recently passed away). Even without buying anything – there is a fair bit of interesting content for you to look at, before deciding which course is most worth your hard earned $25.

Training mode:

Theory mode:

The course browser, as you can see – I got the demo one:

From what I understand, this is to replace all the old training software that Convekta had, where you had to install a separate shell for every course, which was obviously more clunky, and probably harder to sell.

Online tactics: Find Sacrifices feature in Chessbase Light

Chessbase Light has an ability to find simple sacrifices in an entire database of games. Since I keep track of all my online blitz games in one database, that’s a very handy feature. In the search dialog box, go to the “Manoeuvres” tab, and make sure that “Sacrifice” box is checked.

Leave crawling through all your games for a while, and out of every 5 positions it finds, 1 is usually a pretty decent tactical shot. Yet another example of extra options computers offer for learning about the game.

I was pleasantly surprised to see that in the same 2+1 bullet game 8 years ago, I played two decent sacrifices. Under the pressure of having to make a move every couple of seconds – a player can intuitively come up with quick shots and just trust himself that they do actually work.

moon-DDT3000, 2001. Black to move

14… Nc5!

The knight goes to e4 or b3, it cannot be taken as then Bc5 would win the queen.

moon-DDT3000, 2001. Black to move

A few moves later in the same game - White’s pieces are quite disorganized.

22… Rxf3! followed by Nxd4 and Bc5 was a good way to wrap up the game.

Opening preparation – how to make a tree

I am not a big fan of opening preparation, it’s never been very easy for me. I enjoy studying the endgame a lot more. At the same time, I must admit that without proper opening knowledge – every game can potentially turn into a lot of suffering. In some openings you may get away with just knowing general ideas, but generally you can’t get away without knowing some precise variations.
Around the year 1998, I lost a speed chess game to a master. I lost it in an embarrassing manner, by getting my rook trapped on move 10 in the opening. I forgot the move order in exchange variation of Slav defence, and there was no return. After the game, the opponent looked at me with understanding, and gave me a very valuable advice (ok, one of the many he gave me). He said – “For every opening you play – you must have a tree. Does not matter how you do it, on paper, on computer, you must have it recorded somewhere”. Indeed, if you don’t bother to record it once, how likely are you ever going to REMEMBER thing?
Computers make it really easy to do this – you can create a database of games that looks like this (you can create it in Chessbase Light 6, and then load in Chessbase light 2007 for viewing – both are free).

In the body of the game, you enter the variations, and then if you do File-Print-Print Repertoire, you can get a nice printout like the screenshot below. It looks just like your own NCO! Store it under your pillow and that way you won’t forget your moves after 23…Rc4 in the Panov attack…

I recently wrote another post on opening preparation that has more to do with the psychological aspects of studying the opening.