I've got the opposite problem, Trig. I'll be staring at the computer screen, and one of the kids will speak to me, and I'll go, "Yeah, yeah, sure, fine, whatever. Now get outta here," and then later I'll learn that I've agreed to, say, let half-a-dozen teenage boys sleep over for the weekend at our house. Maybe I should change my script? "Absolutely
not! Out of the question!" or maybe, "Go see what your dad has to say."
How NT is the rest of your family? Maybe they're not picking up on the non-verbal cues you are sending out. Perhaps you could establish a set of hand signals or gestures. My kids are teaching me new ones all the time.