The "real" reason, according to screenwriter Steven E. De Souza, is that the terrorists were supposed to be shown wearing and synchronizing the same type of watch in the beginning of the movie. After killing a couple of terrorists, John McClane started to notice the watches, so when Hans handed John a cigarette, John saw the watch on Hans' wrist, which tipped him off. However, due to some script changes during the shooting of the movie, the scene of the terrorists synchronizing their watches was dropped, leaving this clue unaddressed to the viewer. That, of course, does not rule out that John still identified Hans as one of the terrorists by his watch. Another logical explanation is that John simply recognized Gruber's voice. John was hiding on the roof of the elevator in the beginning, where he could hear Gruber's distinct voice quite well, and later he had several conversations with him over the radio. And even if John did not recognize the voice, he may have just been overly careful, since giving a gun to a stranger in this sort of situation is after all very risky. Gruber's pseudonym of Bill Clay appearing on the staff board is a reason to trust him, although there are a few subtle reasons to raise suspicion. Even if Gruber was actually an escaped hostage, John may have expected him to hide somewhere, instead of wandering around. John may have been observing for a moment as Hans was checking on the explosives, which is certainly suspicious behavior, before approaching him. John probably knew from the beginning that Hans was one of the hostage takers but decided to humor him to see how Hans would try to trick him.