I would like to fine-tune GPT-3 to produce a chatbot similar to ChatGPT based on all authoritative halachic question-answer pairs out there. There is a corpus on Sefaria and supposedly Bar-Ilan has a collection but I have not been able to find a publicly available version.

The Sefaria collection seems to be really modest. Surely, we should have at least a million recorded responsas including everything from the previous and from this century. It looks like I'm going to have to scrape sites like Chabad.org, Aish.com, Yeshiva.co etc. manually - recommendations of additional similar sites would be greatly appreciated too.

Please share any known resources with detailed answers to halachic questions. Any codex that indexes halachic rules would also work as long as the split between the "question" (something like "Laws of family purity", "List of kosher fish", "Morning blessings explained" would all be great questions for the purposes of training an AI model).

  • Do you want English or Hebrew?
    – Double AA
    Jan 11, 2023 at 22:14
  • The bar Ilan database was made to be thorough and contain basically all the mainstream shailos uteshuvos, but their work wasn't voluntary and cost a lot of money so they charge for it.
    – robev
    Jan 11, 2023 at 22:14
  • @robev do you consider Rav Kook outside the mainstream? Bar Ilan has big gaps due to "marketing" issues.
    – Double AA
    Jan 11, 2023 at 22:15
  • 1
    Related: judaism.stackexchange.com/q/6008
    – Fred
    Jan 12, 2023 at 0:27
  • So, if I'm not mistaken responsas are written by tier 1 scholars who address novel issues no one else dared to provide an answer to. For my purposes, I need a large bank of classic answers even more than I need these, not just some extreme corner cases. The more of these I get, the more accurate the model will be and the closer will its actual alignment to the Torah be also.
    – rudolfovic
    Jan 12, 2023 at 6:59


You must log in to answer this question.