I'm creating the index for my upcoming book, Hans Urs von Balthasar and the Question of Tragedy in the Novels of Thomas Hardy (T&T Clark). I've never done an index before; Giles Waller was gracious enough to do the index on our co-edited book, Christian Theology and Tragedy.
This is what I have learned so far: preparing a good index is something of an art form. What to include, what to exclude? What sorts of things is the reader looking for? Some topics simply can't be covered, as they are too continuous throughout the book. An index entry needs good scope: not too much information as to be vertiginous and useless, but enough information to be genuinely useful and even interesting (the Chicago Manual of Style, 15th edition, recommends no more than 6 locators to an entry [18.130]). The Chicago Manual of Style rightly uses the word "decide" a lot. The indexer must decide what terms will be looked for by a reader, and when entries are too specific and too general. Like a good, clear roadmap, there is a line where a particular map is no longer useful; it's either too detailed for simple driving on a the highway, or it's too broad to help on the backroads.
As freshman indexers have been warned, you can't index everything, so the indexer must read with care, trying to discern what the book is about and what the reader might be interested in. This is an odd feeling, even when it's your own work. For me, I wrote this 5 years ago, so there is a feeling of distance to the work (which may actually be useful and more objective). Further, I've scoured this manuscript countless times for meaning, coherency, and typos; now I am reading it yet again, but looking for something different this time. The Chicago Manual of Style recommends an average of 5 citations a page for a decent sized index (18.120), which is enormously helpful in terms of how much to cite.
The process of indexing feels a bit like diagramming sentences for grammar, but here you are diagramming a page for useful content for the reader. Balancing this process is the fact that there is already a diagram/dissection to the manuscript in the table of contents. The index doesn't need to reference a titled section that is in the table of contents; rather, it stands as a cousin to the table of contents, indicating not the sections and flow of thought but the oddities that someone skimming, glancing, or returning to the text might require.
I have also found technology to be somewhat unhelpful here. A printout of the manuscript is easier to mark up and notate than a digital one, which requires clicking, clicking, typing, clicking, hitting save, etc. A printed manuscript has a record of marks, entries, and potential titles; can more quickly leave information on a printout, marked by hand, then I can in a computer file. So I am marking by hand while typing the index itself in OmniOutliner. OmniOutliner keeps it all nicely alphabetized and arranged, so that I don't have a bunch of scratches and changes on a bunch of papers or index cards, and the printed manuscript gives me a clear history of marks, entries, and ideas. If I want to check an entry, it's fairly easy to consult the marked manuscript. At first I was attempting everything digitally, creating the index while looking at the PDF of the manuscript, but I soon realized I had no history of what I was doing! So my current system feels both trustworthy and efficient.
In the end, the index is a strange summary of a book, an alphabetized listing of a book's oddities, emphases, references, subjects, and name-drops ("occasional vanity entries are not forbidden," the Chicago Manual jauntily intones [18.31]). It provides different sort map to a book, one that gestures to specifics while obfuscating the book's flow of ideas and larger themes. It's sort of … fun. So here's to indexing.