LAXMAYDAY/anime_background_edit_data_webdataset

Anime Background Edit Pairs (WebDataset)

This repository is the canonical distribution of 20,221 paired anime background edits, packaged as 10 streaming-friendly WebDataset TAR shards. It is self-contained and does not require a separate ImageFolder repository.

Each sample contains a source image (ref1), an English editing instruction, and the desired edited image (target). Source and target images come from the same background group, so the instruction describes the visual transformation while preserving the underlying scene composition.

Dataset summary

Property	Value
Samples	20,221
TAR shards	10
TAR members	80,884
Total shard size	7,118,909,440 bytes (about 6.63 GiB)
Image format	WebP, quality 95
Instruction language	English
Invalid-pair flags retained	183
Abandoned pairs excluded	9

All TAR members are regular files. Hardlinks used by the local source dataset were dereferenced when the shards were created, making the archives safe for sequential WebDataset streaming.

Sample structure

Every sample uses a six-digit key and has four members:

000000.ref1.webp
000000.target.webp
000000.prompt.txt
000000.json

• ref1.webp: source image to edit.
• target.webp: desired edited image.
• prompt.txt: executable English editing instruction.
• json: complete sample metadata and provenance.

The JSON member includes the following main fields:

• id, sample_id, pair_id, and group_id
• edit_instruction
• difference_summary, changes, and preserve
• annotation_model and annotation_source
• confidence, invalid_pair, and review_status
• source/target dimensions and SHA-256 hashes
• ref1_member, target_member, and prompt_member

Dataset creation

The source illustrations were organized into background groups containing multiple visual variants of the same scene. Images within each group were paired to form source/target edit examples. A vision-language model compared both images and produced a structured English instruction describing the changes to apply while preserving the scene's composition and unaffected objects.

Images were converted to WebP at quality 95 while retaining their original dimensions. The completed records were assigned stable keys from 000000 to 020220, then packed into 10 deterministic shards. Each image is stored as a regular TAR member, including samples that reuse the same underlying source image, so the dataset can be consumed safely by sequential streaming readers.

Visual examples

Each row below is one complete training example: the model receives the source image and instruction, and learns to produce the target image.

Daytime to nighttime (000073, group BG000003)

Instruction: Change the scene from daytime to nighttime, applying a blue-tinted darkening effect to the entire view.

Source (ref1)	Target

Green landscape to autumn (001492, group BG000292)

Instruction: Change the landscape colors to an autumnal palette, replacing green foliage with warm orange and red tones in the trees and grass.

Source (ref1)	Target

Loading with WebDataset

Because this repository is manually gated, first request access on the dataset page and authenticate with Hugging Face:

hf auth login
pip install webdataset huggingface_hub pillow

Then download or reuse the authenticated local Hub cache and stream the shards:

from pathlib import Path

import webdataset as wds
from huggingface_hub import snapshot_download

repo_dir = snapshot_download(
    repo_id="LAXMAYDAY/anime_background_edit_data_webdataset",
    repo_type="dataset",
    allow_patterns="train-*.tar",
)
shards = sorted(str(path) for path in Path(repo_dir).glob("train-*.tar"))

dataset = (
    wds.WebDataset(shards, shardshuffle=True)
    .shuffle(1000)
    .decode("pil")
    .to_tuple("ref1.webp", "target.webp", "prompt.txt", "json")
)

for ref1, target, instruction, metadata in dataset:
    print(instruction)
    print(metadata["pair_id"], metadata["group_id"])
    break

For distributed training, split shards by worker/node using the normal WebDataset pipeline utilities appropriate to your training framework.

Annotation provenance

• doubao-seed-2-0-mini-260428: 18,131 samples
• gpt-5.5: 2,090 samples

Annotation sources:

• doubao_initial: 15,699
• doubao_final: 2,432
• jarless_middle: 2,090

Quality notes

The dataset retains 183 completed samples marked invalid_pair=true. These may represent unrelated, unusable, or effectively unchanged pairs. They are kept so downstream users can decide whether identity/no-op examples are useful; filter them through the JSON metadata when needed.

Nine pairs that did not pass annotation validation are excluded from all shards. Multiple edit pairs may reuse the same underlying image when one background group contains more than two visual variants.

Shards

train-00000-of-00010.tar
train-00001-of-00010.tar
train-00002-of-00010.tar
train-00003-of-00010.tar
train-00004-of-00010.tar
train-00005-of-00010.tar
train-00006-of-00010.tar
train-00007-of-00010.tar
train-00008-of-00010.tar
train-00009-of-00010.tar

The first shard contains 2,023 samples; each remaining shard contains 2,022.