Best way to be let down gently is to tell a white lie.

Start talking about another guy you've been interested in, assuming he's not a complete nutter you will probably survive the jealousy onslaught considering you've only had coffee.

Worse case scenario, you end up chopped into pieces and buried somewhere. Life is about risks.