The really polite option would be to invite them to a restaurant near where they lived, but I don't think many circumstances demand that one go that far.

Surely "would you like to do dinner some time?" would be a graceful path into negotiations about how much hassle the various logistic options might be?

I've dined very well in Leicestershire with a friend from the Manchester area before now, which is a very literal example of meeting him half-way.

You could wait until you both chance to be in the same area as one another for unrelated reasons and seize the opportunity to eat together?