admin/LLM-from-scratch
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[2025.06.05] update tokenizer train code, use the style of Qwen

Browse Source
This commit is contained in:
coffee
2025-06-05 18:09:18 +08:00
parent db4c2658b1
commit 6a30309848
3 changed files with 149 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+1 -2
View File
@@ -5,5 +5,4 @@ dolly_env_local
llm_dolly.egg-info
src_llm/__pycache__
weights_MoE
weights_llm
weights_tokenizer/*
weights_llm