The reading for 味 is the on'yomi you learned with the kanji, but the reading for 嫌 is the kun'yomi, いや. Just imagine someone saying words so hurtful that you have to scream, "Iyaaaaa!" いや.
He’s kind of an idiot, really. Why didn’t he realize that it was sarcastic?