Instructions to use 0xSero/Qwen3.5-99B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 0xSero/Qwen3.5-99B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="0xSero/Qwen3.5-99B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("0xSero/Qwen3.5-99B")
model = AutoModelForMultimodalLM.from_pretrained("0xSero/Qwen3.5-99B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 0xSero/Qwen3.5-99B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "0xSero/Qwen3.5-99B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0xSero/Qwen3.5-99B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/0xSero/Qwen3.5-99B

SGLang

How to use 0xSero/Qwen3.5-99B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "0xSero/Qwen3.5-99B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0xSero/Qwen3.5-99B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "0xSero/Qwen3.5-99B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0xSero/Qwen3.5-99B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 0xSero/Qwen3.5-99B with Docker Model Runner:
```
docker model run hf.co/0xSero/Qwen3.5-99B
```

Donation cost to run Heretic or otherwise Abliterate this?

by dissociativity - opened 18 days ago

Discussion

dissociativity

18 days ago

How much would it cost you to Heretic or otherwise abliterate it in one of the more modern methods of doing so that is less destructive?
I'd consider paying, or at least contributing to it, this REAP fine-tune, if your claims are true seems like a very promising model.

0xSero

Owner 18 days ago

I am taking donations for a program to get DGX Sparks into libraries and community centers around the United States. I would appreciate contributions to this https://donate.sybilsolutions.ai

Depending on the model and size it takes between 500$ to 5k USD, I fortunately have RTX Pro 6000s so I can take that cost on but it would lock up my hardware for some time.

Please submit a specific model, end target size, specialisation, and any additional wants/datasets.

josephmayo

18 days ago

i think he isn't talking of community centers or a different model
he's specifically saying he would donate to get an abliterated / uncensored version of this model using the newer methods that preserve model's capabilities (like dr schimdit, RepE or surgical orthogonalization) and he's happy to pay for it

0xSero

Owner 18 days ago

No I get what he's asking.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment