You didn't learn this reading for 由 with the kanji, so here's a mnemonic to help you:
If you had good lineage you'd never make a U-ey (ゆい) on the highway. That's dangerous, and people with good lineage and a long history would simply never do such a thing.
Kiyomizu Temple in Kyoto is a temple with a long history.
Just because he has a prestigious lineage, it doesn't make him a nice person.
It's the oldest and most venerable shrine in Japan.