Hmm. I chewed on that for a bit, and the following thought crystallised in my mind, which was sort of inspired by what you said and might be seen as consistent with it in the right light, but doesn't really have the same emphasis. How about this:

The diametric opposite of a point in a space is the point in the same space furthest away from it. The polar opposites of a space are the two points furthest away from each other. So there's nothing fundamentally special about two diametrically opposed positions: it's reasonable to talk about a space in which every point is one of a pair of diametric opposites. But polar opposites are a fundamental property of the space you're working in, and if they exist at all then they're unique.

So, if your space of possible opinions looks like a line segment, then there's only one pair of polar-opposite points of view, and everyone can agree on what they are: they're unquestionably the ones at the very ends of the line segment. Whereas if your space of possible opinions looks like a circle, then there are any number of pairs of diametrically opposite points of view, and as you suggest, which pair you consider the most important is up to you.

Hmm. Does that make sense?

Hmm, there's a thought. Yes, I think it does make sense.

Curiously, though, the poles of a planet are the two diametrically opposite points on its surface that are closest together.