Here's the wording the IPO used in their response to me:
The test of a series is NOT simply whether the marks in the series would be regarded as confusingly similar to each other if used by unrelated undertakings. Any variation in the non-distinctive features in the marks must leave the visual, aural and conceptual identity of each of the trade marks substantially the same.
I read that as they don't accept my argument, i.e. that if "confusingly similar" then they are a series. They are saying the judgement is stricter than that, they must be substantially the same (whatever that means). If someone did come along and register something (in the same class) that was "confusingly similar" it would be rejected.