I'm looking for the source in the Gemorah which says there was once a community that said Hallel on a day that wasn't Rosh Chodesh, and that even though it wasn't the correct minhag it was seen as something to keep. I think it might be in Maseches Taanis, or the Yerushalmi, but I can't find it.
You're probably thinking of the story of Rav's visit to Bavel, recorded on Taanit 28b, but you got the Rosh Chodesh part of it confused.
While most people nowadays are used to thinking of Rosh Chodesh as the best example of a Hallel day, really Rosh Chodesh is not a Hallel day at all. Hallel is recited on the entire holidays of Sukkot and Chanukkah as well as the first day[s] of Pesach and Shavuot [and the Seder night[s]]. That's it.
Rav visited Bavel and was upset to find them reciting Hallel on Rosh Chodesh which is not a traditional Hallel day. He soon calmed down when he noticed they were skipping parts to indicate it was just a fake-Hallel: some nice Psalms to help enliven a minor holiday. That's the custom you see nowadays to recite a fake-Hallel (generally known as "Half Hallel") on Rosh Chodesh [and the latter 6 days of Pesach].
רב איקלע לבבל חזינהו דקא קרו הלילא בריש ירחא סבר לאפסוקינהו כיון דחזא דקא מדלגי דלוגי אמר שמע מינה מנהג אבותיהם בידיהם
Rav came to Bavel. He saw that they read Hallel on Rosh Chodesh. He thought to stop them. Once he say that they were skipping around, he deduced that it was just a custom of their fathers.