# Minimal OpenVLA with Qwen2-0.5B VLM

A single-file, experimentally verified reproduction of the core OpenVLA training loop using the Hugging Face checkpoint [`lmms-lab/llava-onevision-qwen2-0.5b-ov`](https://huggingface.co/lmms-lab/llava-onevision-qwen2-0.5b-ov).

This is intentionally a **minimal OpenVLA-style experiment**, not a reproduction of the complete Stanford OpenVLA architecture. It demonstrates:

- image and language instruction input;
- a pretrained SigLIP visual encoder and multimodal projector;
- a Qwen2-0.5B language-model backbone;
- 256-bin discretization of normalized continuous actions;
- autoregressive prediction of seven robot action tokens;
- action-only cross-entropy supervision;
- overfitting and constrained action decoding on three synthetic examples.

![Training loss and action comparison](assets/training-results.png)

## Verified result

The experiment was run on an Apple M5 GPU through PyTorch MPS for 100 optimizer steps.

| Metric | Before training | After 100 steps |
| --- | ---: | ---: |
| Mean action loss | 15.2092 | 0.2834 |
| Teacher-forced action-token accuracy | 0.0% | 90.5% |

Two of the three autoregressive action predictions closely matched their training targets:

```text
red target = [-0.750, -0.500,  0.200, 0, 0, 0, 1]
red pred   = [-0.746, -0.496,  0.199, 0.004, 0.004, 0.004, 0.996]

blue target = [0.650, 0.600, -0.150, 0, 0, 0, 1]
blue pred   = [0.652, 0.598, -0.152, 0.004, 0.004, 0.004, 0.996]
```

The green example was confused with the blue example. This limitation is reported deliberately: the run proves that the complete multimodal training and decoding path works and strongly overfits, but the three-example run does not reach 100% sequence memorization.

## Architecture

```text
synthetic image ─→ SigLIP vision tower ─→ LLaVA projector ─┐
                                                           ├→ Qwen2-0.5B → 7 action tokens
language instruction ──────────────────────────────────────┘

7 action tokens → 256-bin decoder → normalized 7-D continuous action
```

Compared with the Stanford OpenVLA paper, this experiment replaces the fused DINOv2+SigLIP encoder and Llama-2 7B backbone with the smaller LLaVA-OneVision SigLIP + Qwen2-0.5B stack. It preserves the central VLA mechanism rather than the exact model architecture or Open-X training recipe.

## Repository layout

```text
.
├── minimal_openvla_qwen05b.py   # complete training and evaluation experiment
├── requirements.txt             # verified Python dependencies
└── assets/training-results.png  # plots from the verified 100-step run
```

The script expects a local checkout of [`IP127000/openVLA-Qwen2-0.5B`](https://github.com/IP127000/openVLA-Qwen2-0.5B) next to it:

```text
parent-directory/
├── minimal-openvla-qwen05b/
└── qwen2-openvla/
```

## Installation

Python 3.10-3.12 is recommended.

```bash
git clone https://github.com/IP127000/openVLA-Qwen2-0.5B.git qwen2-openvla
git clone https://github.com/IP127000/minimal-openvla-qwen05b.git

python3 -m venv .venv-openvla
source .venv-openvla/bin/activate
pip install --upgrade pip
pip install -r minimal-openvla-qwen05b/requirements.txt
```

The Hugging Face model is public. A token is normally unnecessary. If your environment requires authentication, pass it through the environment rather than putting it in source code:

```bash
export HF_TOKEN="your-token"
```

## Run

From the parent directory containing both repositories:

```bash
PYTORCH_ENABLE_MPS_FALLBACK=1 \
python minimal-openvla-qwen05b/minimal_openvla_qwen05b.py \
  --steps 100 \
  --learning-rate 5e-5 \
  --save-dir minimal-openvla-qwen05b/output/minimal-openvla-overfit
```

On CUDA, `PYTORCH_ENABLE_MPS_FALLBACK` can be omitted.

The first run downloads approximately 1.8GB of LLaVA-Qwen weights plus the separate SigLIP vision tower. Later runs use the Hugging Face cache.

## What is trained

To keep memory and runtime manageable, the script freezes most parameters and trains:

- the multimodal projector;
- input token embeddings;
- the language-model output head;
- the final Qwen transformer block.

On Apple Silicon the model is trained in FP32 because FP16 AdamW produced NaNs during real testing. CUDA uses BF16 when supported and otherwise FP16.

## Synthetic dataset

The script creates three deterministic 224×224 images containing red, green, or blue squares. Each image/instruction pair is assigned a normalized seven-dimensional action:

```text
[Δx, Δy, Δz, Rx, Ry, Rz, gripper]
```

This dataset is only intended to verify that the multimodal model can learn and decode action tokens. It does not measure robotics performance or generalization.

## Known limitations

- Uses SigLIP only, not OpenVLA's fused DINOv2+SigLIP visual encoder.
- Uses Qwen2-0.5B instead of Llama-2 7B.
- Uses synthetic images rather than Open-X/RLDS demonstrations.
- Assumes actions are already normalized to `[-1, 1]`.
- Does not implement per-dataset quantile normalization or robot-specific unnormalization.
- The verified three-sample run reached 90.5%, not 100%, action-token accuracy.

## References

- [OpenVLA paper](https://arxiv.org/abs/2406.09246)
- [Official OpenVLA repository](https://github.com/openvla/openvla)
- [LLaVA-OneVision Qwen2-0.5B checkpoint](https://huggingface.co/lmms-lab/llava-onevision-qwen2-0.5b-ov)
- [openVLA-Qwen2-0.5B source](https://github.com/IP127000/openVLA-Qwen2-0.5B)

## License

MIT. Model checkpoints and upstream components remain subject to their respective licenses.