🤝 Open to Collab

Rick Holmberg PRO

juiceb0xc0de

30 27 136

https://github.com/JuiceB0xC0de

AI & ML interests

Mechanistic interpretation SAE weight training Novel architecture exploration Brain atlas builds

Recent Activity

updated a model about 3 hours ago

juiceb0xc0de/qwen3-0.6b-SAE

updated a dataset about 4 hours ago

juiceb0xc0de/qwen3-8b-atlas

updated a dataset about 5 hours ago

juiceb0xc0de/nova-1-1.3b-code-atlas

View all activity

Organizations

reacted to badaoui's post with 😎 3 days ago

Post

2258

432 GB of ultra-fast HBM4 and up to 23.3 TB/s of memory bandwidth on a single GPU 🤯.

Two weeks ago, we got early access to AMD's new Instinct MI455X, and our first goal was simple: make sure 🤗 Transformers works on day one.

Over the past few weeks, we worked closely with the AMD team to validate the platform, enable Flash Attention, add torchcodec support for multimodal models, and resolve issues uncovered during testing.

The result:
✅ 99.5% success rate across our 24 core Transformers model architectures - already on par with our daily CI on previous AMD and NVIDIA platforms.

The hardware is just as exciting. With 432 GB of HBM per GPU, our early capacity experiments showed more than 3× the concurrent long-context requests compared to MI300, thanks to the much larger KV cache capacity.

A huge thanks to the AMD team for the early access and the great collaboration!

Read the full blog 👇
https://huggingface.co/blog/badaoui/transformers-on-amd-mi455

1 reply

replied to kingabzpro's post 8 days ago

Hey! If you're looking for a more in depth break down regarding pricing or any of the other features on Hugging Face the Docs section is a great resource as well. The Hugging Chat model is free to use for queries on a lot more than just the platform. You can find everything from optimizing inference speed to working with newer architectures like Mamba, or getting started with a Reachy Mini. Hope that helps!

reacted to danielhanchen's post with 😎 11 days ago

Post

5618

Gemma 4 is now faster and much more accurate! 🚀

Google made huge improvements to tool-calling and chat accuracy, reliability + speed.
To get fixes, re-download our updated GGUF, MLX, NVFP4 quants!

Unsloth quants: https://huggingface.co/collections/unsloth/gemma-4
Gemma 4 Guide: https://unsloth.ai/docs/models/gemma-4

8 replies

reacted to Hari5115's post with 🚀 14 days ago

Post

230

Fair warning space/photograhy lovers: this is weirdly addictive, Just one date becomes thirty !

There's a spot on the internet that shows you the exact image nasa shared with the world on any day since 1995. Type your birthday and see what the universe was up to.

(Born before 1995? Ah — the cosmos wasn't posting yet. You're officially more vintage than the dataset. 😎 Try a big life date instead.) 😅

🔭 See yours → Hari5115/cosmic-moment
Or hit Surprise me and let the universe pick.

Post the image you got below 👇 — let's see whose day space showed off for. 🌠

Built on the open nasa apod dataset · public domain · not affiliated with nasa

Dataset: 📦 Hari5115/nasa-apod

#space #photography #astro #cosmic

reacted to danielhanchen's post with 🚀 14 days ago

Post

4603

We’re releasing Gemma 4 NVFP4 quants that run 1.5× faster on your GPU.

Gemma-4-12B NVFP4 works on 11GB VRAM.
26B-A4B hits 13K tok/s (B200).

Unsloth NVFP4 enables faster, more accurate 4-bit Blackwell inference.

Blog: https://unsloth.ai/docs/basics/nvfp4
Gemma NVFP4: https://huggingface.co/collections/unsloth/nvfp4

2 replies

reacted to Quazim0t0's post with 🤯 15 days ago

Post

4274

I never really posted about my DaisyChain project because it's still work in progress. I decided to post a small bit about it and the demo.
DaisyChain Genomics: four small DNA/RNA specialists chained behind a learned router that behave like one big genomics model, at ~7× less active compute.
I built a modular genomics model chasing a 500M-parameter foundation model, then caught myself measuring it wrong. Here's the honest version.
DaisyChain is a different bet: instead of one monolithic DNA model, it's four ~74M specialists (eukaryote, prokaryote, mRNA, splice) chained behind a learned router, each distilled per-domain from HuggingFaceBio's Carbon-500M. Every specialist reports how surprised it is (bits/base) and the router hands each sequence to the link most at home with it. In lineage it's a cluster Branch-Train-Merge mixture of experts, so you can chain on a new domain without retraining the others.
The pitch: ~295M total params (under Carbon-500M), but only one ~74M specialist runs per query, so ~7× cheaper per token, routing at 100% held-out.
The mistake: Carbon works in 6-mers, and I'd been scoring likelihood as 6-mer cross-entropy. By that number I was +0.043 bits/base behind, splice even "beating" Carbon. But Carbon scores at the base-pair level, which is harder and more honest. Re-run their way:
Real gap: 1.862 vs 1.787 bits/base, +0.089 behind, not +0.043
No domain actually beats Carbon; the "splice win" was an artifact
Seq recovery: euk 31.5% vs 38.9%, bacteria 40.9% vs 54.1%

DaisyChain is still behind Carbon-500M (itself a draft model, not built to top benchmarks), but by a number I can defend, and the gap closes with every per-domain pass. 🌼

DaisyChainAI
DaisyChainAI/Daisychain-Genomics-Demo
DaisyChainAI/daisychain-genomics

reacted to Quazim0t0's post with 🤗 19 days ago

Post

5420

Disabling Gated Access for some of my models today. I will update this post with the list as I go. I had to go back recently and make updates to a lot of models and bit off more than I could chew with managing all the releases. I didn't realize many of you asked for access and I apologize for not accepting your access to the models you were wanting to look at. I don't like to release something fully unless I feel I put what I could into it for the moment. Some models will remain on gated access, but I will now be accepting those who request to view the repo.

Disabled Gated Access:
Quazim0t0/Byrne-VLM-131M - v2 Updates + Training Instructions
Quazim0t0/Byrne-Speech - 12M Tiny Speech model
Quazim0t0/Byrne-ASR-English - 12M Tiny ASR Model
Quazim0t0/Byrne-VE - Byrne-VE — Tiny Self-Distilled Vision Encoder (39M)
Quazim0t0/Positronic-144M - Research Artifact
Quazim0t0/SpikeWhale-SNN-216M - Research Artifact
Quazim0t0/Mycel-LM-79M - Research Artifact
Quazim0t0/Chimera-64M - Research Artifact

Accepting Gated Access Requests (7/9):
https://huggingface.co/Quazim0t0/Wheeler-63M

Also uploaded my Neural Photonic Project:
Three trained nets in series: light interferes through the MZI2.pt optical core (verified 256/256), is measured by the PD.pt neural photodetector (verified 1024/1024), and folded into a single OUTPUT byte by the real ADC8 neural-CPU adder. Every value below is computed end-to-end by the three loaded, verified nets — no analytic formulas.
Demo: https://quazim0t0-neural-photonic-hybrid.hf.space/
Model Weights: Quazim0t0/neural-photonic

AND!

A work in progress:
Ashen Depths
https://quazim0t0-ashendepths.static.hf.space/index.html

reacted to danielhanchen's post with 🚀 20 days ago

Post

6103

DeepSeek-V4 can now run locally with Unsloth GGUFs! 🐳

Run lossless DeepSeek-V4-Flash on 168GB RAM or
3-bit works on 110GB Mac, RAM, VRAM setups.

Run via Unsloth Studio or llama.cpp.

GGUF: unsloth/DeepSeek-V4-Flash-GGUF
Guide: https://unsloth.ai/docs/models/deepseek-v4

replied to HannesVonEssen's post 20 days ago

Sick update. I've been using your visualizer for awhile now. Its cool to see your adding community focused features like this.

reacted to davidmezzetti's post with 👀 21 days ago

Post

1061

CeleBERTy Small: Domain model for Pop Culture, Art, Music and Entertainment.

Build vector embeddings that perform better than larger models.

https://huggingface.co/blog/NeuML/celeberty-small

reacted to Quazim0t0's post with 👀 22 days ago

Post

2400

Created research language model whose channel-mixing block is not an MLP. It is a differentiable Neighbour-Sensing fungal-colony-growth model: each token is expanded into a colony of hyphal tips that grow in a bounded latent region, sense a shared density field, and steer their own growth — the "MLP" is replaced by a few differentiable steps of colony growth, read back out into the hidden state.

Quazim0t0/Mycel-LM-79M

Also the original SpikeWhale project — the one that sparked all the other SpikeWhale related projects. Every spiking primitive here is hand-written in plain PyTorch: the leaky integrate-and-fire (LIF) neuron dynamics, the fast-sigmoid surrogate gradient, and the backprop-through-time training loop. No snntorch, no spikingjelly, no norse, no bindsnet — the network is a genuine from-scratch SNN.

Quazim0t0/SpikeWhale-SNN-216M

5 replies

replied to albertvillanova's post 22 days ago

Absolutely stoked! Huge milestone for the team. Enjoy a bit of peace now that its over. Or just keep on the grind it's what I always do!

reacted to albertvillanova's post with 🔥 22 days ago

Post

3592

🎉 KTO is now part of the stable TRL API

As of Promote KTO to stable API, KTOTrainer and KTOConfig have graduated from trl.experimental to the stable trl API. https://github.com/huggingface/trl/pull/6175

This one closes out a long road. Over the past 6+ months, the "Align KTO with DPO" effort landed ~90 PRs methodically bringing KTO up to the standard we hold for stable trainers, one carefully-scoped change at a time:
- Feature parity with DPO: full VLM support (incl. multi-image), sync_ref_model, PEFT + Liger, ZeRO-3 + PEFT dtype fix, pad_to_multiple_of, activation offloading, IterableDataset and dict eval_dataset, remove_unused_columns, and reference-logprob precomputation at init.
- Consistency with DPO: aligned method order and signatures, tokenization, _prepare_dataset, PEFT handling, ref-model preparation for distributed training, and config layout — plus a new DataCollatorForKTO and output format. Metrics moved into _compute_loss and simplified to direct averages via the shared _metrics attribute.
- Removing legacy baggage: dropped encoder-decoder support, BOS/EOS handling, null_ref_context, generate_during_eval, model_init, preprocess_logits_for_metrics, model/ref adapter names, and several dead config knobs.
- Coverage: a full test suite mirroring DPO, text collator tests, VLM tests, and slow tests.
- The promotion itself: the experimental → stable move (#6175) and shim cleanup (#6287), handled so downstream users get a clean deprecation path.

Honestly, this has been one of the more complex tasks I've taken on since joining the team, not because any single change was hard, but because it demanded sustained consistency across a ~2,000-line trainer, with every branch, comment, and edge case kept in lockstep with DPO.

Huge thanks to everyone who reviewed along the way (especially @qgallouedec ), the incremental review cadence is exactly what kept this maintainable.

KTO now sits on equal footing with our other flagship trainers. 🚀

2 replies

posted an update 25 days ago

Post

184

HF community! Thank you so much the overwhelming flood of downloads I've received on my model atlases over the last couple of weeks. It makes me so happy to know other people out there are enjoying my work. With that being said, I am looking for an opportunity from the builders and trainers out there.

I've recently had the chance to do a mid pre training snapshot durin process and am looking for the opportunity to get some more of these images . This is a symbiotic trade. I am more than happy to do full reports on the status of your model and translate what your model may stand to gain from as you head towards the finish line. The atlas building process uses 8965 different prompt over a wide variety of behavioural features. I combines many different methods of interpreting what is going on inside the model between prompt and response that we are unable to see.

For anyone else interested in neural network imaging and mechanistic interpretation check out my library of atlases and my new work translating the numbers into a 3d visual format to accompany the sqlite database. If you've gpt a model that you wish to have atlased leave a commen and I'kk be sure to ge arounf to it as soon as possible.

https://huggingface.co/collections/juiceb0xc0de/cloud-atlases

reacted to AbstractPhil's post with 👀 25 days ago

Post

105

Understanding the Aleph Fibonacci in visual form with full rotary.

https://claude.ai/public/artifacts/0d536427-bc7d-464a-890d-bddd02ce42dc

This ought to clear up much of the confusion as to what is actually happening under the hood, converted to an understandable 2d visual format. There have been multiple iterations, this is the current format and mathematics behind it as I attempt to solve the fibonacci curve related to negative imaginary numeric inversion that causes the statistics instability.

replied to mmhamdy's post 29 days ago

Haha no shit. I just finished writing an article on scaling ssm mamba style models and popped over to see what's new in posts. I guess there's a theme today.

posted an update about 2 months ago

Post

253

😅 You ever fumble on a project? Please someone tell me I'm not alone. I fumbled at step one and remained oblivious for the remainder of the project. Funny story, I was under the assumption that Qwen/Qwen3-8B was the base model that the paired with the Qwen SAE released by Alibaba. I didn't realize there was a Qwen3-8b-Base model until after the 12 hours of independent mapping techniques I had applied to the model that was missing the -Base suffix. 🤗 My bad, I'm just a bartender. I should not be unsupervised.

Not all is lost however. The outcome was a very in depth neural network atlas complete with its own SQLite queryable database for the Qwen3-8B model I can now share with you all. The data base combines these methods for a full in depth dive:

- Neuron Taxonomy
- Category Separation Scoring
- Co-activation Analysis
- Per-Head Decomposition
- Component Comparison
- Attribution Patching
- Sparse Non-negative Matrix Factorization
- NeuronLens
- DAS SVD rotation
- Cross-layer Coherence
- SQLite database

So if you've ever wondered where a specific behaviour or ability lives in the hidden dimensions of Qwen-8B or perhaps wanted to make informed quantization decisions please enjoy the fruits of my ill-informed labour lol. 😂

juiceb0xc0de/qwen3-8b-atlas
Qwen/Qwen3-8B

replied to appvoid's post 2 months ago

I applaud you in your journey into the void with small models. I too am deeply fascinated with the optimization of smaller models rather than asking for more parameters and terabytes of scraped internet data. I hope to see what you've come up with in a few weeks time.

I just finished designing a sparsity training scheduler that trains on average 35% of a models available weights with almost no hidden dimensions between transformers adjoined and zero throughput while randomizing trainable locations. It cuts VRAM and training time down and the models set higher benchmarks on mathematics than FFT models trained on the same corpus. I discovered this while fucking around for fun.

I don't doubt the discoveries to be made with training smaller architectures have many more surprises in store for us.

replied to their post 2 months ago

@danielhanchen what happened to this magnificent model!? I had the perfect place to slot it in to my team of AI bros! I would love to see this back on HF. 🤗

reacted to kalyan-ks's post with 👀 2 months ago

Post

1687

LLM Guardrail Models are Less Robust Against Text Mutation Attacks

Blog post - https://huggingface.co/blog/kalyan-ks/llm-guardrail-models-less-robust

Evaluated the robustness of three LLM guardrail models (GLiGuard, LlamaGuard3 and MiniGuard).

Evaluation is done using 16 text mutation attacks over three datasets (AEGIS 2.0, WildGuard and ExpGuard).

Achieved average Unsafe ASR score of up to 33% and average Safe ASR score of up to 25% against GLiGuard model.

Achieved average Unsafe ASR score of up to 35% and average Safe ASR score of up to 17% against LlamaGuard3-8B model.

Achieved average Unsafe ASR score of up to 45% and average Safe ASR score of up to 15% against MiniGuard v0.1 model.

Rick Holmberg PRO

AI & ML interests

Recent Activity

Organizations

juiceb0xc0de's activity