I think if someone has bothered to go to the effort of making a joke and you say it isn't funny, it is automatically a criticism of their sense of humour, which people are very touchy about. (Which isn't to say that you shouldn't be saying it).