Sinestro38/qwen3-1p7b-sgd-polaris-step1550-final

Qwen3-1.7B-Base + Polaris RL (PLAIN SGD, step 1550, final / collapsed)

⚠️ This checkpoint is the FINAL state of the run, not the best. Val acc collapsed back to 0% in the second half of training (after peaking at 14.29% around step 300). For a usable model from the same run see Sinestro38/qwen3-1p7b-sgd-polaris-step300-best-val.

This is the final checkpoint (step 1550, ~5 epochs) of a plain-SGD RL fine-tune of Qwen/Qwen3-1.7B-Base on POLARIS-Project/Polaris-Dataset-53K. Released alongside the peak-val ckpt for late-trajectory ΔW analysis in an ICML-2026 study on the low-rank structure of SGD vs Adam RL updates.

Why this (collapsed) checkpoint is still useful

For the SVD-compressibility study, comparing the spectral profile of ΔW_step300 (peak) vs ΔW_step1550 (collapsed) is informative — the question is whether plain-SGD updates stay low-rank as the run drifts, or whether the rank profile broadens during collapse. Both ckpts are needed for this trajectory analysis.

If you just want a model to use for inference, stop reading and use the step-300 checkpoint instead.

Training recipe

field	value
base model	Qwen/Qwen3-1.7B-Base
dataset	POLARIS-Project/Polaris-Dataset-53K (52,779 train / 512 val)
algorithm	GRPO (adv_estimator=grpo, use_kl_loss=False, entropy_coeff=0, use_kl_in_reward=False)
optimizer	PLAIN SGD — momentum=0.0, nesterov=false, dampening=0.0, weight_decay=0.0
learning rate	1e-1 (constant)
train batch size	128 (1 grad step per rollout batch)
ppo_micro_batch_size_per_gpu	4
rollout.n	4
rollout.temperature	1.0
max_prompt_length	1024
max_response_length	8192
total epochs	5 (step 1550, end of run)
hardware	4× B200 (179 GB)
step time	~65 s/step
trainer	verl (FSDP + vLLM rollout)

Results — full val acc trajectory

step	val acc
0	0.0% (baseline)
50	0.0%
100	0.2%
150	10.0%
200	10.6%
250	12.3%
300	14.29% ← peak
350	10.8%
400	12.7%
450	8.4%
500	12.1%
550	13.7%
600	12.5%
650	10.8%
700	2.7% (collapse begins)
750	8.0%
800-1100	4-8% (drift)
1150+	~0% (fully collapsed)
1550	0% (this checkpoint)

This is a textbook plain-SGD-without-momentum drift on noisy GRPO advantages — the update is the pure first-order gradient (no smoothing), so it follows large noise excursions and eventually wanders into a degenerate region of weight space.

Use

⚠️ Inference quality is poor at this checkpoint. Provided primarily for ΔW research.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

m = AutoModelForCausalLM.from_pretrained(
    "Sinestro38/qwen3-1p7b-sgd-polaris-step1550-final",
    torch_dtype=torch.bfloat16,
    device_map="cuda",
)
tok = AutoTokenizer.from_pretrained("Sinestro38/qwen3-1p7b-sgd-polaris-step1550-final")

prompt = "Find all integer solutions to x^2 + y^2 = 25. Let's think step by step and output the final answer within \\boxed{}."
msgs = [{"role": "user", "content": prompt}]
ids = tok.apply_chat_template(msgs, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
out = m.generate(ids, max_new_tokens=2048, do_sample=True, temperature=0.6)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

Caveats

• Model is in a collapsed RL state — val acc 0%. Use only for ΔW spectral analysis or trajectory studies.
• ΔW magnitude is larger than the step-300 ckpt (more cumulative SGD steps), useful for analyzing how the spectrum evolves.

Citation context

Work in progress — being submitted to ICML 2026 with a paper on plain-SGD RL update compressibility for batched-LoRA serving.

Related models in this study:

• Sinestro38/qwen3-1p7b-sgd-polaris-step300-best-val — same run, peak-validation checkpoint
• Sinestro38/dsr1-qwen7b-sgd-polaris-step100-best-val — same recipe, scaled to 7B with DS-R1-Distill base
• Sinestro38/dsr1-qwen7b-sgd-polaris-step412-final — 7B final