What is the relationship between the letters samekh and sin? Did they ever have distinct sounds? Why do they exist as separate letters?
"A sin is just a samech with three branches."
-- A contemporary American ראש ישיבה
The idea here is that in modern usage they are both actually interchangeable with samech.
For an illustration of this interchangeability see the 15th line of the alphabetical acrostic א-ל אדון, in which a sin appears where we would expect a samech, or the common root א.ר.ס used by חז"ל in place of the Torah's synonymous root א.ר.ש.
The difference in sound between the samech, a voiceless alveolar fricative ([s]) and sin, a putative voiceless lateral fricative ([ɬ]), is hypothesized to have been lost as early as Biblical Hebrew, with its remnants still evident in transliterations like "Chaldean" for "כשדי" and "balsam" for "בושם". In each case, the sin that we would conventionally pronounce as [s] was ostensibly closer to [ɬ], which is, in a way, halfway between /s/ and /l/.
There is a complicated diachronic explanation of exactly how and why this shift took place, which is summarized very briefly on page 73 of this paper, by Biblical Hebraicist Gary Rendsburg.
I am not a speaker of modern Hebrew, but am a beginning level learner of Biblical Hebrew. However I did study about ancient Hebrew pronunciation, linguistics, and specifically phonetics quite a bit. I lean heavily in three areas, 1. "logical" phonetics, belief that original/Biblical Hebrew is designed and there should be logical rules to discover when pursuing such questions. 2. Phonotactics constrains languages and thus Semitic languages can be helpful with similar structures phonetically 3. Mizrahi Hebrew class of Hebrew pronunciation, as these are closest to the phonetic logic of the Hebrew Bible. Over the years of pursuing answers to Hebrew pronunciation questions, I came across an example of a variety of Yemenite Hebrew (lost at this time to myself) said to pronounce Schin with dot to the upper left as IPA ɕ, which is in the Polish language, for example, as ś. To me this is a logical progression of the tongue contact with the roof of the mouth front to back for each sibilant, instead of a "radical" departure for some sort of lateral. It sounds nice, is logical, flows easily in the mouth, but I sure would like to find examples in Mizrahi Hebrew/Semetic languages of IPA ɕ, beyond the reference I lost. For now I use it in my studying, and it makes a great distinction for disambiguating biblical/Hebrew words by sound.
Being able to read and speak Hebrew fluently,I can answer this question. Both Samekh and Sin make an 's' sound. You can tell what sound a Hebrew letter makes by the first sound you speak when pronouncing the it. For example, Bet makes a 'b' sound. However, this technique will not work for the Hebrew letters Alef, Ayin, Chet and chaf. Alef and Ayin being silent and there being no English equivalent for Chet and Chaf is the reason for this.
It was my understanding that Ugaritic had four "s" sounds, Modern Standard Arabic had only three, and Modern Hebrew only has 2 but originally Ancient Hebrew had three like Arabic.
One of these "s" sounds was the "sh" sound that appears in "shin" in both Arabic and Hebrew (Shin with the dot over the right of the letter).
Both Hebrew and Arabic lost one of the Ugaritic "S"'s by combining it with another "S" however Hebrew and Arabic did not combine the lost "s" with the same letter. Arabic combined it with Sin and Hebrew combined it with Shin (with the dot on the right). This is why peace is "salaam" in Arabic but "shalom" in Hebrew (as well as many other words that are common between the sister languages). Originally I read that this lost "s" was halfway between sin and shin in pronunciation with a slight "sh" sound but not as heavy as the sh in "sheep" in English.
Next, from what I've read on the subject, Arabic Saad (the heavy S sound similar to the "s" in "saw" in English) took the hard "S" sound from Ugaritic and the other Shin (with the dot on the left) in Hebrew adopted this pronunciation as well. The remaining light "s" sound as in English "see" was applied to Arabic's Sin and to Hebrew's Samech. However personally, this seems backwards to me as "Samech" seems to be better pronounced with the heavy "S "as in "saw" rather than the light "s" in "see".
Over time, long, long ago, the Hebrew language dropped the heavy "s" pronunciation and began pronouncing both letters the same (most probably due to the influence of foreigners who incorrectly pronounced the hard "s" sound which didn't exist in their languages and the Hebrew speakers were influenced by this change and ended up adopting it in practice (probably out of apathy from deciding it wasn't worth the hassle of correcting everyone all the time, much like English "often" is correctly pronounced "of-en" but has been mispronounced as "of-ten" so much that the incorrect "of-ten" pronunciation is now listed as a correct alternate pronunciation in Modern English dictionaries). However since written language is slower to change than verbal language, Hebrew still retains the archaic spellings from when the letters each had distinct sounds, even though today and even in latter Biblical times, the two letters represented the same sound in spoken language.
Other letters were adapted differently from Ugaritic and Akkadian, which is why 'ard in Arabic (spelled with the heavy D (daad) in Arabic at the end) is spelled with Tzaddi in Hebrew even though they were both descended from the same letter in Akkadian for the word meaning land/earth.