Ginsberg's joking accusation, and the corresponding implication that he is hiding some huge secret, some separate identity that Ginsberg has no knowledge about, catches him off guard, despite the topic of the conversation. He wasn't quite expecting it, so his initial reaction is one of stunned, and guilty, surprise. His expression is, in fact, so transparently shocked that it is quite possible Ginsberg will mistake it for hyperbolic acting.

Because the truth is, despite never questioning if he is making the right decision, Ned feels guilty about keeping secrets from Ginsberg. That's a new thing, for him. He's never really let himself get close enough to someone that he felt bad about shutting them out from that whole part of his life. Now, however, it's been starting to eat at him, more and more. If he can't trust Ginsberg, well, who can he trust? The answer is, of course, no one. Does he really want to be a man who faces the prospect of a life of lies, in which he cannot bring himself to trust a single other person?

"That's me," he admits, a beat too late for it to be quite funny, with a reluctant smile, "You caught me. I'm working for, uh, the agency of... for your rival agency. Deep cover. Can't see how you found me out."

Enough with the spy stuff, Ned decides. He turns back towards the crowd of people on the street, points at an older man on the other side of the road, asks, "What about that guy? What's his story?"