From b61572a8a1251309399e3881fe9825ba0687f51b Mon Sep 17 00:00:00 2001
From: coffee <hanluzhi0429@gmail.com>
Date: Wed, 4 Jun 2025 17:17:19 +0800
Subject: [PATCH] [2025.06.04]update tokenizer train code and style

---
 README.md | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/README.md b/README.md
index 1b5c57b..6100072 100755
--- a/README.md
+++ b/README.md
@@ -4,6 +4,9 @@ LLM-Dolly is a custom LLM built from scratch
 
 ## 🗓️ 更新日志
 
+#### 2025-06-04
+- 📝 调整tokenizer的格式和训练方式，和qwen2的tokenizer风格保持一致
+
 #### 2025-05-26
 - 📝 增加LLM文档，涵盖LLM全周期技术点。