There are times when you wonder about your own sanity in starting something that is really hard.
The Po-Rating Pilot (currently underway) is a bit like that.
Sphinx only reviews chapbooks/pamphlets. That's because most magazines only review books, so it's an attempt to restore the balance in favour of the flimsies. However, it is a very feeble attempt because for many pamphlets, the Sphinx review is the only one they will get. There might be the odd mini-reference or two-line review here and there, but chapbooks don't get much attention generally.
That means the sole review can sound like the last word on the subject. Which, of course, it is not. It is just one reviewer's opinion. And even Sphinx reviews are relatively short (about 300 words usually). Who writes an in-depth consideration of a 32-page booklet?
The Poetry Book Society, however, does select a pamphlet choice every quarter. It seems likely that the Chosen One is
- more widely read
- more widely (and more fully) reviewed.
However, that selection - from all the pamphlets submitted in triplicate - is made by only two poets. Again, not many people's opinions weigh in the balance in this situation and, as we all know, the response to poetry is subjective. Even though nearly every single poetry editor says the sole criterion for selection is 'quality', nobody can agree (where poetry is concerned) how quality should be defined.
So I thought it would be interesting to see what happened if a pamphlet was run past a good number of readers, and in an attempt to make the judgement process a little less subjective and a little more transparent, those readers should have rating criteria -- just like ice skating. This would arrive at a rating result based on the feedback from all of them. I thought I'd try to get 10 because statistically speaking that makes it easy. I even thought a pamphlet might get a certain number of stripes, according to the readers' reactions - so you could be rather proud of having published a five-stripe pamphlet but a lot less pleased about a two-striper. A one-striper will probably send me hate-mail.
So far, so theoretically reasonable. I drew up 10 criteria and restricted the readers to a maximum of twenty minutes with the publication (because in that time, I reckon, most people have either decided whether they find the chapbook interesting and would recommend it, or they have dumped it in the recycling bin. Besides, longer than that is asking too much of people who have another nine pamphlets to consider).
For each of the ten criteria (e.g. Typography, Cost, Overall Design, Quality of Writing, Originality etc), the reader had to award a rating between 1 and 5, with 1 as Ugh! and 5 as Yeay!
Each rating was done anonymously, but I asked for personal feedback on the process, and already some fascinating comments have come back. I've had time to reflect, too, and already I can see several things are quite wrong.
Out of my ten criteria, for example, four related to production quality and only three to the poetry itself. That can't be right. The poems must be the most important thing - although in twenty minutes...?
So there should probably be five criteria, not ten (though definitely ten readers). And I wonder whether I should have suggested picking two poems and reading carefully - forming a judgement on that basis. The first and the last, maybe? Or simply the first two that caught the eye?
It's already clear too that some of my raters are very sparing with their fives, while others are much more generous. But hey - isn't that typical of any set of judges in any competitive set-up? It is the group assessment in this case that restores the overall balance. Maybe.
Think about it. How do you read a poetry publication - not as a reviewer, I mean, but as an ordinary reader. Don't you pick it up, weigh it in your hand, decide whether you like the look and feel of it, flick through, look at a couple of poems and think either -- 'Oh yes, quite like the look of this...' or 'Oh no, life is too short'? This is complicated by other factors like whether you know the author, whether the pamphlet was a gift or a purchase, whether you have bought the pamphlet after a reading, whether you're thinking of submitting your own poetry to the publisher and are comparing yours and theirs, whether you're having a Bad Day or a Good Day, whether you really need to be somewhere else etc.
Anyway, the Po-Rating Pilot is in progress. Already I see other problems (not listed above) and also some very interesting results. I haven't given up on this yet, despite the guddle and cost of posting things all over the place to people who post them back to me to post them, in haste, to someone else. The organisation is less than ideal because I am so busy. But eventually Sphinx 10 WILL be finished, and I'll write more about this there.
[Re. being busy: the attempt to change my teaching from full-time to part-time has not only not worked yet, but has been complicated by another educational job that I'm also engaged in and also haven't got time to do properly. So everything here is behind and I'm struggling more than somewhat. By April, things will be better. However, apologies if you are reading this and also waiting for a letter from me; sorry for being behind on Sphinx and behind on this year's publications in general. I am not a person who defaults on deadlines, so I'm finding this painful. But it can't be helped.]