In Shemuel I (3:1) the Pasuk tells us that prophecy was a very rare phenomenon. How, then, did 'Eli realize that Shemuel was receiving prophecy and not merely dreaming?
Radak (to v. 8) explains simply: one or two times it might indeed have been Shmuel's imagination (thinking in his sleep that someone was calling him), but three times means that Shmuel really had heard something. Since no one else was there except Eli and Shmuel, Eli understood that it must be a Divine voice.
He also quotes Ibn Ezra, who says that it's because Eli himself hadn't heard anything. If it was a human voice, then once Eli had established (as above) that Shmuel wasn't just imagining things, he should have been able to hear it too. The only way, then, that Shmuel could hear a voice and Eli not, would be if it was a prophecy addressed directly to Shmuel.