admin/LLM-from-scratch
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[2025.06.10] adjust directory structure

Browse Source
This commit is contained in:
coffee
2025-06-10 14:04:57 +08:00
parent ef2f7a1bc5
commit c6e171e8ed
29 changed files with 0 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src_dev/src_tokenizer/README.md
Executable
+3
View File
@@ -0,0 +1,3 @@
# USAGE
使用sentencepiece和huggingface transformers分别进行tokenizer的训练
support BPE and BBPE