From time to time I find myself listening to someone making a siyum, I am thinking of structuring my learning to pursue finishing enough to make a siyum. But before I commit myself I want to know if that is a noble goal.
So...
Why do we say the "hadran" and other things after completing a sizeable amount of learning (what are the traditional sizes and of what?) Is there a source for this custom? On a deeper, philosophical level what purpose does the siyum have in Jewish life?
Philosophically, the siyyum is a way to celebrate your accomplishment with your community. Especially since the siyyum requires (1) a minyan, so you can say the kaddish derabbanan, and (2) a celebratory meal, it's a way of sharing your personal study accomplishment.
The hadran is one's declaration of intent to return to this subject matter again someday. Just as when we finish reading the book of Devarim on Simchat Torah, we immediately begin again with Bereshit, the idea is that we are completing a phase of study but not, God forbid, completing our study of Talmud without the intent to return.
When I celebrated my fortieth birthday, I timed my completion of learning a masechet of Talmud to make my birthday party be more than a secular-style party; it was a religious celebration as well.
As others have said, the minimum amount of text that justifies a siyyum is dependent on the person's abilities and experience. In my community, the general standard is a masechet (tractate) of Gemara or an entire seder (order) of Mishna; but for those who are first coming to the study of these texts, learning in depth one's first masechet of Mishna is worthy of a siyyum.
This doesn't answer the questions you posted, like "what purpose does the siyum have in Jewish life", but I'll address your first point, whether "structuring my learning to pursue finishing enough to make a siyum" "is a noble goal". Yes, it is. I speak from experience (and relate advice I've heard, too) in saying that having a goal like that in mind will encourage, and thereby help, you to complete the set amount (as long as the goal is realistic). Then, after the siyum, you can set a new goal.
This webpage has a good list of the sources for making a siyum, starting with Abaya in the Gemara (Shabbos 118b):
Abaye said: If I saw a scholar who completed a tractate, then I would make a holiday for the Rabbis.
This document from the Kof-K website (I'm pretty sure it's one of the earlier editions of Halachically Speaking), breaks down the why, when and how we make siyumim.
R' Moshe Isserlis (aka Rema) codifies the celebration of finishing a mesechta in his glosses to Shulchan Aruch (YD 246:26):
כשמסיים מסכתא מצוה לשמוח ולעשות סעודה ונקראת סעודת מצוה (נ"י פ' יש נוחלין ומימרא דאביי פ' כל כתבי)
Upon completing a Mesecha, it is incumbent to be happy and to make a feast, which is a Seudas Mitzvah.
Rema sources this statement from a statement of Abaye (Shabbos 118b). Abaye said:
תיתי לי, דכי חזינא צורבא מרבנן דשלים מסכתיה עבידנא יומא טבא לרבנן
I will recieve reward, for when I see a young Torah scholar who finishes his tractate, I make a feast for the Rabbis.
Furthermore, he cites Nemukei Yosef in Baba Basra, who comments on the importance on celebrating upon the completing of a mitzvah.
The mitzvah to celebrate the siyum is codified by Rema (551:10) in relevance to the custom to abstain from meat and wine during the mourning period for the Destruction of the Beis Hamikdash, where he lists a siyum as an occasion worthy of eating meat and wine even while mourning.
ובסעודת מצווה, כגון מילה ופדיון הבן וסיום מסכתא וסעודת אירוסין, אוכלים בשר ושותים יין כל השייכים לסעודה.
The primary purpose of this feast is a celebration of completing a portion of Torah. This is simultaneously a celebration of the privilege to learn and complete Torah, and a thanksgiving to Hashem.
Maharshal (Yam Shel Shlomo BK 7:37) writes that this is the highest form of joy to Hashem.
ומשום זה היה נראה לברך שהשמחה במעונו בסיום מסכת. דאין לך שמחה יותר לפני הקב"ה, אלא שמחה ורינה של תורה. דאין להקב"ה בעולמו אלא ד' אמות של הלכה (ברכות ח, א). דלא גרע מפדיון הבן דלפרסום מצוה בעלמא הוא.
Maharshal seems to mention two purposes - thanksgiving to Hashem, and publicizing the simchah.
מכאן נהגו כל ישראל לסיים בשמעתא וליתן שבח והודי'ה למקום ולפרסם אותה שמחה שזכה לכך, שעושין סעודה בעת שמסיימין
The source of reciting the 'Hadran' (text found at the end of all gemaras) spans hundreds of years. It is discussed extensively in the responsa of the Rema, where he gives an extensive explanation to the specific references of the sons of R' Papa (he explains how each of them corresponds to one of the 10 commandments, and why that it relevant to a siyum). The earliest source I could find to the recital of the Hadran text (or at least part of the Hadran) is R' Hai Gaon (939-1038), quoted by Sefer Eshkol (Sefer Torah 19).
I heard from my Rebbe R' Yaacov Haber that a siyum is held in honor of a significant achievement in learning - which is measured in terms of the level of the mesayem. For some people, reading through a masechta is an exercise like reading through a newspaper and doesn't merit any special celebration; for others, completing one blatt of gemara is enough of an accomplishment to justify a siyum.
As @msh210 said, the main purpose is probably for goal-setting, to encourage you to make measurable and substantial progress in your learning.
Siyum is traditionally being said after you finish one "masechet" of the talmud or one "seder" of Mishnayot.
More details here: Siyum (Wikipedia).
In the Hebrew version it's much more detailed: סיום מסכת
