I am writing a program to do various computations with the Torah. To do this I need to remove all punctuation, spaces and other notations.
I have removed the (פ) breaks and all non-Hebrew letters (including nikodot) from the sefaria chumash using the regular expression:
"\\(.\\)|\\[.*?\\]|[^\u05D0-\u05EA]" (i.e. any single letter in parenteces, any non Hebrew letter, any hebrew letters with punctuation or any pattern within brackets - to account for kiris) . However I seem to still have 45 extra letters.
See http://www.filedropper.com/biblestats_2 for the output of the program. Each line has a before and after version of the text. At the end of each book I print out the stats thus far. At the end of the entire file, I print the stats of the entire Torah.
I am getting:
Total: Verses: 5846, Letters: 304850 Total Frequencies: א: 27060 ב: 16345 ג: 2109 ד: 7032 ה: 28055 ו: 30533 ז: 2198 ח: 7189 ט: 1804 י: 31556 ך: 3358 כ: 8610 ל: 21570 ם: 10624 מ: 14466 ן: 4259 נ: 9867 ס: 1833 ע: 11250 ף: 830 פ: 3975 ץ: 1035 צ: 2927 ק: 4695 ר: 18125 ש: 15595 ת: 17950
However according to http://www.aishdas.org/toratemet/en_pamphlet9.html and other sources there should be only 304,805 letters. Where are the extra 45 letters coming from?