Where did the idea of paying a shadchan come from? Is it an obligation? Is it just like paying anyone for a service? If so, why do we only pay if the shidduch leads to a marriage? Additionally, who defines the salary of a particular shadchan and at what point during the process is the amount specified?
There are many questions here, but all quite interesting.
Where did the idea of paying a shadchan come from? Is it an obligation?
This depends on the community
The first shidduch in Jewish history (Eliezer finding Rivka for Yitzhak) doesn't mention a fee and dinonline writes that the Sephardi custom is that "the shadchan traditionally receives no remuneration for his services" (see also here: R Moshe Shternbuch writes (Teshuvos V’Hanhagos 1:736) that he has heard that in many sefardic communities the custom is to not pay the shadchan).
In the Ashkenazi world, the Rema in Shulchan Aruch codifies the payment of a shachan in at least two places (CM 185:10 and end of CM 264:7) so it was already established in the 1550s and is the norm today in the haredi world.
Eretz Hemda writes that "in the Dati Leumi / Modern Orthodox community, a shadchan does not usually get paid other than a present at the couple’s discretion".
Is it just like paying anyone for a service? If so, why do we only pay if the shidduch leads to a marriage?
R Yirmiyohu Kaganoff explains here that shachan fees are halachically classified as brokerage fees, therefore they work on a success basis
Although it sometimes sounds strange, shadchanus fees are halachically categorized as brokerage fees. Just as one pays a real estate agent for arranging a transaction, so too one pays a shadchan for making the arrangements necessary for the engagement and marriage to transpire [...] The Gemara (Bava Metzia 63b) mentions the responsibility to pay a broker’s fee to the person who arranges the sale of property or merchandise (Shulchan Aruch Choshen Mishpat 185:1; Rama 87:39). This is a standard business practice, similar to paying a commission to a stockbroker, real estate agent, or headhunter).
dinonline explains it somewhat differently but with the same outcome
The shadchan that is asked to act in bringing the parties together, is a worker known in halachic parlance as a kablan. Unlike a day‐laborer (po’el), the kablan can perform his labor whenever he wishes to (although he might be bound to a deadline), and he is paid for the product of his labor rather than the labor itself. Having been asked to perform his labor, the shadchan is paid as a kablan for the result he brings about: the shidduch between the two parties.
Additionally, who defines the salary of a particular shadchan and at what point during the process is the amount specified?
dinonline writes that "The sum of shadchanus gelt is determined by the accepted custom of the land." The fee of USD 2000 appears standard, split between the parents of the bride and groom. As such it might be that the amount is never formally specified as it is informally known.
Beyond the sources notes above, see also here.