update README.md file
This commit is contained in:
@@ -1,11 +1,17 @@
|
||||
**Read this in other languages: [English](README.md), [中文](README_zh.md).**
|
||||
|
||||
# OpenVLA Lightweight Version
|
||||
|
||||
OpenVLA的轻量化版本,基于llava-next框架
|
||||
特点:
|
||||
1.主体模型仅0.5B,使用qwen2-0.5B
|
||||
2.不使用RLDS格式数据集,使用mllm格式进行微调
|
||||
3.不占用LLM模型固有tokens,使用新增的271个tokens表示动作
|
||||
Lightweight version of OpenVLA, based on the llava-next framework
|
||||
|
||||
Features:
|
||||
|
||||
1.The main model is only 0.5B, using qwen2-0.5B.
|
||||
|
||||
2.Does not use RLDS format datasets, instead fine-tuned using mllm format.
|
||||
|
||||
3.Does not occupy inherent tokens of the LLM model, instead using an additional 271 tokens to represent actions.
|
||||
# How to use
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -29,9 +35,24 @@ conda activate vla-qwen
|
||||
pip install --upgrade pip
|
||||
pip install -e ".[train]"
|
||||
```
|
||||
### 4. Download the weights from huggingface
|
||||
```bash
|
||||
cd openVLA-Qwen/models
|
||||
git clone git clone https://huggingface.co/lmms-lab/llava-onevision-qwen2-0.5b-ov
|
||||
```
|
||||
### 5. Change the file path in scripts/train/finetune_ov_vla.sh
|
||||
```bash
|
||||
PREV_STAGE_CHECKPOINT="models/llava-onevision-qwen2-0.5b-ov"
|
||||
--data_path scripts/train/vla.yaml
|
||||
--image_folder /images \
|
||||
```
|
||||
### 6. Change the data paht in scripts/train/vla.yaml
|
||||
```bash
|
||||
json_path: /root/data/vla_llava.json
|
||||
```
|
||||
## Acknowledgements
|
||||
|
||||
I would like to express my gratitude to the following repositories:
|
||||
This project makes use of the following open-source projects, and I would like to express my gratitude to their creators:
|
||||
|
||||
- [OpenVLA](https://github.com/openvla/openvla) - For providing the core structure and functionality that inspired OpenVLA-Qwen.
|
||||
- [LLaVA_OpenVLA](https://github.com/Darren-greenhand/LLaVA-Next) - For his contribution to data preprocessing techniques.
|
||||
|
||||
Reference in New Issue
Block a user