Three possibilities in which the answer to your question is "neither":
1 - The Chazon Ish (O.C. 138:4) writes that this is an application of the general principle that "שיעורין הלכה למשה מסיני," meaning that halachic measurements are matters of Divine oral tradition. Thus, the verse (and the Talmud thereon) are not attempting to estimate pi, but rather to teach the halachic value of pi which should be used, as per the halachic tradition. (A novelty of this suggestion is that the הלכה למשה מסיני would be telling us to suspend the true mathematical calculation in favor of an inaccurate one, whereas normally שיעורין הלכה למשה מסיני applies to matters where we would have no other basis for determining the measurement, such as the volume of bone matter to cause impurity or the volume of food to constitute eating.)
2 - The Ein Eliyahu says that the "sea" was a hexagonal shape and therefore the calculation is precise. (With regards to only the diagonal and perimeter, this works out very neatly with a regular hexagon in which each side is 5 cubits, and the diagonal is therefore 10 cubits. However, the Ein Eliyahu seems to not be discussing a regular hexagon, as this does not resolve the issue of calculating the volume, which is what he is discussing. His assertion works out with certain non-regular hexagons. ואכמ"ל.)
3 - The Tiferes Tzvi (R' Tzaddok HaKohen) to Yoreh De'ah 30 says that pi is indeed exactly 3, as the verse and the gemara state, and shame on those who would accept the words of geometrists over the wisdom of our Sages!