update code file

README_cn.md

@@ -1,6 +1,11 @@
+**其他语言版本: [English](README_en.md), [中文](README_zh.md).**
 # OpenVLA Lightweight Version
 
-OpenVLA的轻量化版本
+OpenVLA的轻量化版本，基于llava-next框架
+特点：
+1.主体模型仅0.5B，使用qwen2-0.5B
+2.不使用RLDS格式数据集，使用mllm格式进行微调
+3.不占用LLM模型固有tokens,使用新增的271个tokens表示动作
 
 ## Installation
 
@@ -26,7 +31,7 @@ pip install -e ".[train]"
 ```
 ## Acknowledgements
 
-I would like to express our gratitude to the following repositories:
+I would like to express my gratitude to the following repositories:
 
 - [OpenVLA](https://github.com/openvla/openvla) - For providing the core structure and functionality that inspired OpenVLA-Qwen.
 - [LLaVA_OpenVLA](https://github.com/Darren-greenhand/LLaVA-Next) - For his contribution to data preprocessing techniques.

README_en.md

@@ -0,0 +1,37 @@
+**Read this in other languages: [English](README_en.md), [中文](README_zh.md).**
+# OpenVLA Lightweight Version
+
+OpenVLA的轻量化版本，基于llava-next框架
+特点：
+1.主体模型仅0.5B，使用qwen2-0.5B
+2.不使用RLDS格式数据集，使用mllm格式进行微调
+3.不占用LLM模型固有tokens,使用新增的271个tokens表示动作
+
+## Installation
+
+Follow the steps below to install the OpenVLA-Qwen lightweight version.
+
+### 1. Clone the repository
+
+Clone the repository to your local machine:
+
+```bash
+git clone https://github.com/IP127000/openVLA-Qwen.git
+cd openVLA-Qwen
+```
+### 2. Create a new conda environment and activate it:
+```bash
+conda create -n vla-qwen python=3.10 -y
+conda activate vla-qwen
+```
+### 3. Install the required dependencies
+```bash
+pip install --upgrade pip
+pip install -e ".[train]"
+```
+## Acknowledgements
+
+I would like to express my gratitude to the following repositories:
+
+- [OpenVLA](https://github.com/openvla/openvla) - For providing the core structure and functionality that inspired OpenVLA-Qwen.
+- [LLaVA_OpenVLA](https://github.com/Darren-greenhand/LLaVA-Next) - For his contribution to data preprocessing techniques.

llava/actiontokenizer.py

@@ -9,71 +9,9 @@ from typing import List, Union
 import numpy as np
 from transformers import PreTrainedTokenizerBase
     
-
-# class ActionTokenizer:
-#     def __init__(
-#         self, tokenizer: PreTrainedTokenizerBase, bins: int = 256, min_action: int = -1, max_action: int = 1
-#     ) -> None:
-#         """
-#         Discretizes continuous robot actions into N bins per dimension and maps to the least used tokens.
-
-#         NOTE =>> by default, assumes a BPE-style tokenizer akin to the LlamaTokenizer, where *the least used tokens*
-#                  appear at the end of the vocabulary!
-
-#         :param tokenizer: Base LLM/VLM tokenizer to extend.
-#         :param bins: Number of bins for each continuous value; we'll adopt a uniform binning strategy.
-#         :param min_action: Minimum action value (for clipping, setting lower bound on bin interval).
-#         :param max_action: Maximum action value (for clipping, setting upper bound on bin interval).
-#         """
-#         self.tokenizer, self.n_bins, self.min_action, self.max_action = tokenizer, bins, min_action, max_action
-
-#         # Create Uniform Bins + Compute Bin Centers
-#         self.bins = np.linspace(min_action, max_action, self.n_bins)
-#         self.bin_centers = (self.bins[:-1] + self.bins[1:]) / 2.0
-
-#         # [Contract] Set "action_token_begin_idx" based on `self.tokenizer.vocab_size - (self.n_bins + 1)`
-#         #   =>> Assumes we're always overwriting the final `n_bins` tokens of the vocabulary!
-#         self.action_token_begin_idx: int = int(self.tokenizer.vocab_size - (self.n_bins + 1))
-
-#     def __call__(self, action: np.ndarray) -> Union[str, List[str]]:
-#         """Clip & bin actions to *the last `n_bins` tokens* of the vocabulary (e.g., tokenizer.vocab[-256:])."""
-#         action = np.clip(action, a_min=float(self.min_action), a_max=float(self.max_action))
-#         discretized_action = np.digitize(action, self.bins)
-
-#         # Handle single element vs. batch
-#         if len(discretized_action.shape) == 1:
-#             return self.tokenizer.decode(list(self.tokenizer.vocab_size - discretized_action))
-#         else:
-#             return self.tokenizer.batch_decode((self.tokenizer.vocab_size - discretized_action).tolist())
-
-#     def decode_token_ids_to_actions(self, action_token_ids: np.ndarray) -> np.ndarray:
-#         """
-#         Returns continuous actions for discrete action token IDs.
-
-#         NOTE =>> Because of the way the actions are discretized w.r.t. the bins (and not the bin centers), the
-#                  digitization returns bin indices between [1, # bins], inclusive, when there are actually only
-#                  (# bins - 1) bin intervals.
-
-#                  Therefore, if the digitization returns the last possible index, we map this to the last bin interval.
-
-#         EXAMPLE =>> Let's say self._bins has 256 values. Then self._bin_centers has 255 values. Digitization returns
-#                     indices between [1, 256]. We subtract 1 from all indices so that they are between [0, 255]. There
-#                     is still one index (i==255) that would cause an out-of-bounds error if used to index into
-#                     self._bin_centers. Therefore, if i==255, we subtract 1 from it so that it just becomes the index of
-#                     the last bin center. We implement this simply via clipping between [0, 255 - 1].
-#         """
-#         discretized_actions = self.tokenizer.vocab_size - action_token_ids
-#         discretized_actions = np.clip(discretized_actions - 1, a_min=0, a_max=self.bin_centers.shape[0] - 1)
-
-#         return self.bin_centers[discretized_actions]
-
-#     @property
-#     def vocab_size(self) -> int:
-#         return self.n_bins
-    
 class ActionTokenizer:
     def __init__(
-        self, tokenizer: PreTrainedTokenizerBase, bins: int = 285, min_action = -0.0025, max_action = 0.0025
+        self, tokenizer: PreTrainedTokenizerBase, bins: int = 271, min_action = -1.0, max_action = 1.0
         # self, tokenizer: PreTrainedTokenizerBase, bins: int = 256, min_action: int = -1, max_action: int = 1
     ) -> None:
         self.tokenizer, self.n_bins, self.min_action, self.max_action = tokenizer, bins, min_action, max_action
@@ -93,8 +31,6 @@ class ActionTokenizer:
         # print('initializing action tokenizer, the new len(tokenizer) is:', len(self.tokenizer))
 
     def __call__(self, action: np.ndarray) -> Union[str, List[str]]:
-
-
         action = np.clip(action, a_min=float(self.min_action), a_max=float(self.max_action))
         discretized_action = np.digitize(action, self.bins) - 1
         discretized_action = np.clip(discretized_action, a_min=0, a_max=self.n_bins - 1)
@@ -114,7 +50,36 @@ class ActionTokenizer:
         action_indices = vectorized_lookup(action_token_ids)
         return self.bin_centers[action_indices]
 
-    # 返回所有 tokens 的数量，包括动作 tokens
+    def actionValue_to_actionName(self,action):
+        action_token_ids=self.__call__(action)
+        actionName=self.tokenizer.decode(action_token_ids)
+        return actionName
+
+    def actionName_to_actionValue(self,actionName):
+        action_ids=self.tokenizer(actionName)
+        action_ids=action_ids["input_ids"]
+        actionValue=self.decode_token_ids_to_actions(action_ids)
+        return actionValue
+
+    def str_to_mllm(self, result_str):
+        new_result=result_str
+        for token in self.action_tokens:
+            value=self.actionName_to_actionValue(token)
+            new_result=new_result.replace(token, str(value[0]))
+        return new_result
+
+    def value_to_mllm(self, result_str):
+        new_result=result_str
+        data=[]
+        for token in self.action_tokens:
+            if new_result.startswith(token):
+                value=self.actionName_to_actionValue(token)
+                data.append(float(value))
+            elif token in new_result:
+                value=self.actionName_to_actionValue(token)
+                data.append(float(value))
+        return data
+
     @property
     def vocab_size(self) -> int:
         return len(self.tokenizer)

scripts/train/finetune_ov_vla.sh

@@ -3,22 +3,11 @@ export NCCL_IB_DISABLE=0
 export NCCL_IB_GID_INDEX=3
 export NCCL_SOCKET_IFNAME=eth0
 export NCCL_DEBUG=INFO
-VISION_MODEL_VERSION="google/siglip-so400m-patch14-384"
-VISION_MODEL_VERSION_CLEAN="${VISION_MODEL_VERSION//\//_}"
-
-############### Pretrain ################
-
-BASE_RUN_NAME="llavanext-google_siglip-so400m-patch14-384-Qwen_Qwen2-7B-Instruct-mlp2x_gelu-pretrain_blip558k_plain"
-echo "BASE_RUN_NAME: ${BASE_RUN_NAME}"
-
 ############### Finetune ################
 MIXTURE="qwen"
-
-# Stage 2
 PROMPT_VERSION="qwen_vla"
-RUN_NAME="llava-openvla-${MIXTURE}" 
+RUN_NAME="openvla-${MIXTURE}" 
-PREV_STAGE_CHECKPOINT="/mnt/d/temp/huggingface/hub/llava_qwen_vla" 
+PREV_STAGE_CHECKPOINT="models" 
-# PREV_STAGE_CHECKPOINT="/mnt/d/ckpt/llava-openvla-qwen"
 echo "PREV_STAGE_CHECKPOINT: ${PREV_STAGE_CHECKPOINT}"
 echo "MID_RUN_NAME: ${RUN_NAME}"
 
@@ -30,8 +19,8 @@ ACCELERATE_CPU_AFFINITY=1 torchrun --nproc_per_node="${NUM_GPUS}" --nnodes="${NN
     --deepspeed scripts/zero2.json \
     --model_name_or_path $PREV_STAGE_CHECKPOINT \
     --version $PROMPT_VERSION \
-    --data_path /root/code/openvla-llava/LLaVA-Next/scripts/train/vla.yaml \
+    --data_path scripts/train/vla.yaml \
-    --image_folder /root/code/openvla-llava/LLaVA-Next/images \
+    --image_folder /images \
     --mm_tunable_parts="mm_language_model" \
     --mm_vision_tower_lr=2e-6 \
     --vision_tower google/siglip-so400m-patch14-384 \
@@ -45,7 +34,7 @@ ACCELERATE_CPU_AFFINITY=1 torchrun --nproc_per_node="${NUM_GPUS}" --nnodes="${NN
     --mm_patch_merge_type spatial_unpad \
     --bf16 True \
     --run_name $RUN_NAME \
-    --output_dir /mnt/d/ckpt_3_28/$RUN_NAME \
+    --output_dir /$RUN_NAME \
     --num_train_epochs 5 \
     --per_device_train_batch_size 8 \
     --per_device_eval_batch_size 100 \
@@ -70,5 +59,3 @@ ACCELERATE_CPU_AFFINITY=1 torchrun --nproc_per_node="${NUM_GPUS}" --nnodes="${NN
     --dataloader_drop_last True \
     --frames_upbound 1
 exit 0;
-
-# You can delete the sdpa attn_implementation if you want to use flash attn