I am writing a program to do various computations with the Torah. To do this I need to remove all punctuation, spaces and other notations.
I have removed the (פ) breaks and all non-Hebrew letters (including nikodot) from the sefaria chumash using the regular expression: "\\(.\\)|\\[.*?\\]|[^\u05D0-\u05EA]"
(i.e. any single letter in parenteces, any non Hebrew letter, any hebrew letters with punctuation or any pattern within brackets - to account for kiris) . However I seem to still have 45 extra letters.
See http://www.filedropper.com/biblestats_2 for the output of the program. Each line has a before and after version of the text. At the end of each book I print out the stats thus far. At the end of the entire file, I print the stats of the entire Torah.
I am getting:
Total: Verses: 5846, Letters: 304850
Total Frequencies:
א: 27060
ב: 16345
ג: 2109
ד: 7032
ה: 28055
ו: 30533
ז: 2198
ח: 7189
ט: 1804
י: 31556
ך: 3358
כ: 8610
ל: 21570
ם: 10624
מ: 14466
ן: 4259
נ: 9867
ס: 1833
ע: 11250
ף: 830
פ: 3975
ץ: 1035
צ: 2927
ק: 4695
ר: 18125
ש: 15595
ת: 17950
However according to http://www.aishdas.org/toratemet/en_pamphlet9.html and other sources there should be only 304,805 letters. Where are the extra 45 letters coming from?