Andrew welcomes him in, forgetting in the joy of rediscovering telepathic contact that he normally gets embarrassed about even verbally sharing things he's deemed private, even with James (and even though so many of the 'private' things in his life are things he's done with James). If the Master wanted to (and could) cause damage, he'd be able to do it before Andrew even realized anything was wrong.

He gives a little nod, still giving James a ridiculous grin. "I'd gone for a lie-down," he replies. "It was the end of our first day there -- I woke up again in Times Square." And he'd been very cold and in need of comfort and etc.