seeker
143 lines
5.5 KiB
Markdown
143 lines
5.5 KiB
Markdown
# Minimal OpenVLA with Qwen2-0.5B VLM
|
||
|
||
A single-file, experimentally verified reproduction of the core OpenVLA training loop using the Hugging Face checkpoint [`lmms-lab/llava-onevision-qwen2-0.5b-ov`](https://huggingface.co/lmms-lab/llava-onevision-qwen2-0.5b-ov).
|
||
|
||
This is intentionally a **minimal OpenVLA-style experiment**, not a reproduction of the complete Stanford OpenVLA architecture. It demonstrates:
|
||
|
||
- image and language instruction input;
|
||
- a pretrained SigLIP visual encoder and multimodal projector;
|
||
- a Qwen2-0.5B language-model backbone;
|
||
- 256-bin discretization of normalized continuous actions;
|
||
- autoregressive prediction of seven robot action tokens;
|
||
- action-only cross-entropy supervision;
|
||
- overfitting and constrained action decoding on three synthetic examples.
|
||
|
||
|
||
|
||
## Verified result
|
||
|
||
The experiment was run on an Apple M5 GPU through PyTorch MPS for 100 optimizer steps.
|
||
|
||
| Metric | Before training | After 100 steps |
|
||
| --- | ---: | ---: |
|
||
| Mean action loss | 15.2092 | 0.2834 |
|
||
| Teacher-forced action-token accuracy | 0.0% | 90.5% |
|
||
|
||
Two of the three autoregressive action predictions closely matched their training targets:
|
||
|
||
```text
|
||
red target = [-0.750, -0.500, 0.200, 0, 0, 0, 1]
|
||
red pred = [-0.746, -0.496, 0.199, 0.004, 0.004, 0.004, 0.996]
|
||
|
||
blue target = [0.650, 0.600, -0.150, 0, 0, 0, 1]
|
||
blue pred = [0.652, 0.598, -0.152, 0.004, 0.004, 0.004, 0.996]
|
||
```
|
||
|
||
The green example was confused with the blue example. This limitation is reported deliberately: the run proves that the complete multimodal training and decoding path works and strongly overfits, but the three-example run does not reach 100% sequence memorization.
|
||
|
||
## Architecture
|
||
|
||
```text
|
||
synthetic image ─→ SigLIP vision tower ─→ LLaVA projector ─┐
|
||
├→ Qwen2-0.5B → 7 action tokens
|
||
language instruction ──────────────────────────────────────┘
|
||
|
||
7 action tokens → 256-bin decoder → normalized 7-D continuous action
|
||
```
|
||
|
||
Compared with the Stanford OpenVLA paper, this experiment replaces the fused DINOv2+SigLIP encoder and Llama-2 7B backbone with the smaller LLaVA-OneVision SigLIP + Qwen2-0.5B stack. It preserves the central VLA mechanism rather than the exact model architecture or Open-X training recipe.
|
||
|
||
## Repository layout
|
||
|
||
```text
|
||
.
|
||
├── minimal_openvla_qwen05b.py # complete training and evaluation experiment
|
||
├── requirements.txt # verified Python dependencies
|
||
└── assets/training-results.png # plots from the verified 100-step run
|
||
```
|
||
|
||
The script expects a local checkout of [`IP127000/openVLA-Qwen2-0.5B`](https://github.com/IP127000/openVLA-Qwen2-0.5B) next to it:
|
||
|
||
```text
|
||
parent-directory/
|
||
├── minimal-openvla-qwen05b/
|
||
└── qwen2-openvla/
|
||
```
|
||
|
||
## Installation
|
||
|
||
Python 3.10-3.12 is recommended.
|
||
|
||
```bash
|
||
git clone https://github.com/IP127000/openVLA-Qwen2-0.5B.git qwen2-openvla
|
||
git clone https://github.com/IP127000/minimal-openvla-qwen05b.git
|
||
|
||
python3 -m venv .venv-openvla
|
||
source .venv-openvla/bin/activate
|
||
pip install --upgrade pip
|
||
pip install -r minimal-openvla-qwen05b/requirements.txt
|
||
```
|
||
|
||
The Hugging Face model is public. A token is normally unnecessary. If your environment requires authentication, pass it through the environment rather than putting it in source code:
|
||
|
||
```bash
|
||
export HF_TOKEN="your-token"
|
||
```
|
||
|
||
## Run
|
||
|
||
From the parent directory containing both repositories:
|
||
|
||
```bash
|
||
PYTORCH_ENABLE_MPS_FALLBACK=1 \
|
||
python minimal-openvla-qwen05b/minimal_openvla_qwen05b.py \
|
||
--steps 100 \
|
||
--learning-rate 5e-5 \
|
||
--save-dir minimal-openvla-qwen05b/output/minimal-openvla-overfit
|
||
```
|
||
|
||
On CUDA, `PYTORCH_ENABLE_MPS_FALLBACK` can be omitted.
|
||
|
||
The first run downloads approximately 1.8GB of LLaVA-Qwen weights plus the separate SigLIP vision tower. Later runs use the Hugging Face cache.
|
||
|
||
## What is trained
|
||
|
||
To keep memory and runtime manageable, the script freezes most parameters and trains:
|
||
|
||
- the multimodal projector;
|
||
- input token embeddings;
|
||
- the language-model output head;
|
||
- the final Qwen transformer block.
|
||
|
||
On Apple Silicon the model is trained in FP32 because FP16 AdamW produced NaNs during real testing. CUDA uses BF16 when supported and otherwise FP16.
|
||
|
||
## Synthetic dataset
|
||
|
||
The script creates three deterministic 224×224 images containing red, green, or blue squares. Each image/instruction pair is assigned a normalized seven-dimensional action:
|
||
|
||
```text
|
||
[Δx, Δy, Δz, Rx, Ry, Rz, gripper]
|
||
```
|
||
|
||
This dataset is only intended to verify that the multimodal model can learn and decode action tokens. It does not measure robotics performance or generalization.
|
||
|
||
## Known limitations
|
||
|
||
- Uses SigLIP only, not OpenVLA's fused DINOv2+SigLIP visual encoder.
|
||
- Uses Qwen2-0.5B instead of Llama-2 7B.
|
||
- Uses synthetic images rather than Open-X/RLDS demonstrations.
|
||
- Assumes actions are already normalized to `[-1, 1]`.
|
||
- Does not implement per-dataset quantile normalization or robot-specific unnormalization.
|
||
- The verified three-sample run reached 90.5%, not 100%, action-token accuracy.
|
||
|
||
## References
|
||
|
||
- [OpenVLA paper](https://arxiv.org/abs/2406.09246)
|
||
- [Official OpenVLA repository](https://github.com/openvla/openvla)
|
||
- [LLaVA-OneVision Qwen2-0.5B checkpoint](https://huggingface.co/lmms-lab/llava-onevision-qwen2-0.5b-ov)
|
||
- [openVLA-Qwen2-0.5B source](https://github.com/IP127000/openVLA-Qwen2-0.5B)
|
||
|
||
## License
|
||
|
||
MIT. Model checkpoints and upstream components remain subject to their respective licenses.
|