admin/openVLA-Qwen2-0.5B
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update code file

Browse Source
This commit is contained in:
hanluzhi0429@gmail.com
2025-04-13 11:19:40 +08:00
parent 53df4d14aa
commit 146e0d18c3
4 changed files with 81 additions and 87 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md → README_cn.md
+7 -2
View File
@@ -1,6 +1,11 @@
**其他语言版本: [English](README_en.md), [中文](README_zh.md).**
# OpenVLA Lightweight Version
OpenVLA的轻量化版本
OpenVLA的轻量化版本，基于llava-next框架
特点：
1.主体模型仅0.5B，使用qwen2-0.5B
2.不使用RLDS格式数据集，使用mllm格式进行微调
3.不占用LLM模型固有tokens,使用新增的271个tokens表示动作
## Installation
@@ -26,7 +31,7 @@ pip install -e ".[train]"
```
## Acknowledgements
I would like to express our gratitude to the following repositories:
I would like to express my gratitude to the following repositories:
- [OpenVLA](https://github.com/openvla/openvla) - For providing the core structure and functionality that inspired OpenVLA-Qwen.
- [LLaVA_OpenVLA](https://github.com/Darren-greenhand/LLaVA-Next) - For his contribution to data preprocessing techniques.
README_en.md
+37
View File
@@ -0,0 +1,37 @@
**Read this in other languages: [English](README_en.md), [中文](README_zh.md).**
# OpenVLA Lightweight Version
OpenVLA的轻量化版本，基于llava-next框架
特点：
1.主体模型仅0.5B，使用qwen2-0.5B
2.不使用RLDS格式数据集，使用mllm格式进行微调
3.不占用LLM模型固有tokens,使用新增的271个tokens表示动作
## Installation
Follow the steps below to install the OpenVLA-Qwen lightweight version.
### 1. Clone the repository
Clone the repository to your local machine:
```bash
git clone https://github.com/IP127000/openVLA-Qwen.git
cd openVLA-Qwen
```
### 2. Create a new conda environment and activate it:
```bash
conda create -n vla-qwen python=3.10 -y
conda activate vla-qwen
```
### 3. Install the required dependencies
```bash
pip install --upgrade pip
pip install -e ".[train]"
```
## Acknowledgements
I would like to express my gratitude to the following repositories:
- [OpenVLA](https://github.com/openvla/openvla) - For providing the core structure and functionality that inspired OpenVLA-Qwen.
- [LLaVA_OpenVLA](https://github.com/Darren-greenhand/LLaVA-Next) - For his contribution to data preprocessing techniques.
llava/actiontokenizer.py
+31 -66
View File
@@ -9,71 +9,9 @@ from typing import List, Union
import numpy as np
from transformers import PreTrainedTokenizerBase
# class ActionTokenizer:
# def __init__(
# self, tokenizer: PreTrainedTokenizerBase, bins: int = 256, min_action: int = -1, max_action: int = 1
# ) -> None:
# """
# Discretizes continuous robot actions into N bins per dimension and maps to the least used tokens.
# NOTE =>> by default, assumes a BPE-style tokenizer akin to the LlamaTokenizer, where *the least used tokens*
# appear at the end of the vocabulary!
# :param tokenizer: Base LLM/VLM tokenizer to extend.
# :param bins: Number of bins for each continuous value; we'll adopt a uniform binning strategy.
# :param min_action: Minimum action value (for clipping, setting lower bound on bin interval).
# :param max_action: Maximum action value (for clipping, setting upper bound on bin interval).
# """
# self.tokenizer, self.n_bins, self.min_action, self.max_action = tokenizer, bins, min_action, max_action
# # Create Uniform Bins + Compute Bin Centers
# self.bins = np.linspace(min_action, max_action, self.n_bins)
# self.bin_centers = (self.bins[:-1] + self.bins[1:]) / 2.0
# # [Contract] Set "action_token_begin_idx" based on `self.tokenizer.vocab_size - (self.n_bins + 1)`
# # =>> Assumes we're always overwriting the final `n_bins` tokens of the vocabulary!
# self.action_token_begin_idx: int = int(self.tokenizer.vocab_size - (self.n_bins + 1))
# def __call__(self, action: np.ndarray) -> Union[str, List[str]]:
# """Clip & bin actions to *the last `n_bins` tokens* of the vocabulary (e.g., tokenizer.vocab[-256:])."""
# action = np.clip(action, a_min=float(self.min_action), a_max=float(self.max_action))
# discretized_action = np.digitize(action, self.bins)
# # Handle single element vs. batch
# if len(discretized_action.shape) == 1:
# return self.tokenizer.decode(list(self.tokenizer.vocab_size - discretized_action))
# else:
# return self.tokenizer.batch_decode((self.tokenizer.vocab_size - discretized_action).tolist())
# def decode_token_ids_to_actions(self, action_token_ids: np.ndarray) -> np.ndarray:
# """
# Returns continuous actions for discrete action token IDs.
# NOTE =>> Because of the way the actions are discretized w.r.t. the bins (and not the bin centers), the
# digitization returns bin indices between [1, # bins], inclusive, when there are actually only
# (# bins - 1) bin intervals.
# Therefore, if the digitization returns the last possible index, we map this to the last bin interval.
# EXAMPLE =>> Let's say self._bins has 256 values. Then self._bin_centers has 255 values. Digitization returns
# indices between [1, 256]. We subtract 1 from all indices so that they are between [0, 255]. There
# is still one index (i==255) that would cause an out-of-bounds error if used to index into
# self._bin_centers. Therefore, if i==255, we subtract 1 from it so that it just becomes the index of
# the last bin center. We implement this simply via clipping between [0, 255 - 1].
# """
# discretized_actions = self.tokenizer.vocab_size - action_token_ids
# discretized_actions = np.clip(discretized_actions - 1, a_min=0, a_max=self.bin_centers.shape[0] - 1)
# return self.bin_centers[discretized_actions]
# @property
# def vocab_size(self) -> int:
# return self.n_bins
class ActionTokenizer:
def __init__(
self, tokenizer: PreTrainedTokenizerBase, bins: int = 285, min_action = -0.0025, max_action = 0.0025
self, tokenizer: PreTrainedTokenizerBase, bins: int = 271, min_action = -1.0, max_action = 1.0
# self, tokenizer: PreTrainedTokenizerBase, bins: int = 256, min_action: int = -1, max_action: int = 1
) -> None:
self.tokenizer, self.n_bins, self.min_action, self.max_action = tokenizer, bins, min_action, max_action
@@ -93,8 +31,6 @@ class ActionTokenizer:
# print('initializing action tokenizer, the new len(tokenizer) is:', len(self.tokenizer))
def __call__(self, action: np.ndarray) -> Union[str, List[str]]:
action = np.clip(action, a_min=float(self.min_action), a_max=float(self.max_action))
discretized_action = np.digitize(action, self.bins) - 1
discretized_action = np.clip(discretized_action, a_min=0, a_max=self.n_bins - 1)
@@ -114,7 +50,36 @@ class ActionTokenizer:
action_indices = vectorized_lookup(action_token_ids)
return self.bin_centers[action_indices]
# 返回所有 tokens 的数量，包括动作 tokens
def actionValue_to_actionName(self,action):
action_token_ids=self.__call__(action)
actionName=self.tokenizer.decode(action_token_ids)
return actionName
def actionName_to_actionValue(self,actionName):
action_ids=self.tokenizer(actionName)
action_ids=action_ids["input_ids"]
actionValue=self.decode_token_ids_to_actions(action_ids)
return actionValue
def str_to_mllm(self, result_str):
new_result=result_str
for token in self.action_tokens:
value=self.actionName_to_actionValue(token)
new_result=new_result.replace(token, str(value[0]))
return new_result
def value_to_mllm(self, result_str):
new_result=result_str
data=[]
for token in self.action_tokens:
if new_result.startswith(token):
value=self.actionName_to_actionValue(token)
data.append(float(value))
elif token in new_result:
value=self.actionName_to_actionValue(token)
data.append(float(value))
return data
@property
def vocab_size(self) -> int:
return len(self.tokenizer)
scripts/train/finetune_ov_vla.sh
+5 -18
View File
@@ -3,22 +3,11 @@ export NCCL_IB_DISABLE=0
export NCCL_IB_GID_INDEX=3
export NCCL_SOCKET_IFNAME=eth0
export NCCL_DEBUG=INFO
VISION_MODEL_VERSION="google/siglip-so400m-patch14-384"
VISION_MODEL_VERSION_CLEAN="${VISION_MODEL_VERSION//\//_}"
############### Pretrain ################
BASE_RUN_NAME="llavanext-google_siglip-so400m-patch14-384-Qwen_Qwen2-7B-Instruct-mlp2x_gelu-pretrain_blip558k_plain"
echo "BASE_RUN_NAME: ${BASE_RUN_NAME}"
############### Finetune ################
MIXTURE="qwen"
# Stage 2
PROMPT_VERSION="qwen_vla"
RUN_NAME="llava-openvla-${MIXTURE}"
PREV_STAGE_CHECKPOINT="/mnt/d/temp/huggingface/hub/llava_qwen_vla"
# PREV_STAGE_CHECKPOINT="/mnt/d/ckpt/llava-openvla-qwen"
RUN_NAME="openvla-${MIXTURE}"
PREV_STAGE_CHECKPOINT="models"
echo "PREV_STAGE_CHECKPOINT: ${PREV_STAGE_CHECKPOINT}"
echo "MID_RUN_NAME: ${RUN_NAME}"
@@ -30,8 +19,8 @@ ACCELERATE_CPU_AFFINITY=1 torchrun --nproc_per_node="${NUM_GPUS}" --nnodes="${NN
--deepspeed scripts/zero2.json \
--model_name_or_path $PREV_STAGE_CHECKPOINT \
--version $PROMPT_VERSION \
--data_path /root/code/openvla-llava/LLaVA-Next/scripts/train/vla.yaml \
--image_folder /root/code/openvla-llava/LLaVA-Next/images \
--data_path scripts/train/vla.yaml \
--image_folder /images \
--mm_tunable_parts="mm_language_model" \
--mm_vision_tower_lr=2e-6 \
--vision_tower google/siglip-so400m-patch14-384 \
@@ -45,7 +34,7 @@ ACCELERATE_CPU_AFFINITY=1 torchrun --nproc_per_node="${NUM_GPUS}" --nnodes="${NN
--mm_patch_merge_type spatial_unpad \
--bf16 True \
--run_name $RUN_NAME \
--output_dir /mnt/d/ckpt_3_28/$RUN_NAME \
--output_dir /$RUN_NAME \
--num_train_epochs 5 \
--per_device_train_batch_size 8 \
--per_device_eval_batch_size 100 \
@@ -70,5 +59,3 @@ ACCELERATE_CPU_AFFINITY=1 torchrun --nproc_per_node="${NUM_GPUS}" --nnodes="${NN
--dataloader_drop_last True \
--frames_upbound 1
exit 0;
# You can delete the sdpa attn_implementation if you want to use flash attn