You really need him to swing at you before you clean his clock so you can claim self defense.
During one of these marathon "bueno...bueno" sessions, just go over and tell him:
"Would you mind getting off the phone? Your Mom agreed to blow a bunch of us tonight for $10 each and she's supposed to call and let me know where and how many guys I can bring."