Skip to content

vllm.model_executor.models.d3

Decision 3.0 (vllm-sr/d3): a Qwen3.5 backbone with an answer-code readout.

The checkpoint is a bare Qwen3_5Model plus readout.safetensors, one row per answer code, and decision_config.json. The readout scores the codes at the last prompt token and is served as the score head of a classification model. D3ForSequenceClassificationConfig applies the checkpoint's attention mode and label space.

Under the prefix_bidir attention mode a prompt is a state block (the system turn, the media, State: and the state) followed by a question block. The full attention layers read each block in both directions and the question block also reads the state block, so the state never sees a question; the linear-attention layers stay causal. Each request carries its state block's token count in PoolingParams.extra_kwargs[STATE_TOKENS], and D3PrefixBidirModelState makes the two blocks the PrefixLM ranges of its attention.

Classes:

D3PrefixBidirModelState

Bases: DefaultModelState

Model Runner V2 state of a prefix_bidir checkpoint: a request's state block and question block are the PrefixLM ranges of its attention, so the attention is causal across them and bidirectional within each.

Source code in vllm/model_executor/models/d3.py 
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
class D3PrefixBidirModelState(DefaultModelState):
    """Model Runner V2 state of a `prefix_bidir` checkpoint: a request's state
    block and question block are the PrefixLM ranges of its attention, so the
    attention is causal across them and bidirectional within each."""

    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.blocks: dict[str, list[tuple[int, int]]] = {}

    def add_request(self, req_index: int, new_req_data: NewRequestData) -> None:
        super().add_request(req_index, new_req_data)
        params = new_req_data.pooling_params
        state = (params.extra_kwargs or {}).get(STATE_TOKENS) if params else None
        length = len(new_req_data.prompt_token_ids or ())
        if isinstance(state, int) and 0 < state < length:
            blocks = [(0, state - 1), (state, length - 1)]
        else:
            # Without a state block size (warm-up runs, or a request that did
            # not go through /v1/systemone) the whole prompt is one block.
            logger.debug_once("A prefix_bidir request has no state block size")
            blocks = [(0, length - 1)]
        self.blocks[new_req_data.req_id] = blocks

    def remove_request(self, req_id: str) -> None:
        super().remove_request(req_id)
        self.blocks.pop(req_id, None)

    def get_mm_prefix_ranges(
        self, input_batch: InputBatch
    ) -> dict[int, list[tuple[int, int]]] | None:
        ranges = {
            i: self.blocks[req_id]
            for i, req_id in enumerate(input_batch.req_ids)
            if req_id in self.blocks
        }
        return ranges or None