1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
//! # Qwen3
//!
//! The Qwen3 decoder-only language model, as `transformers`' `Qwen3Model`
//! defines it: grouped-query causal self-attention with per-head query/key
//! RMS normalization before a rotary embedding, a `SwiGLU` feed-forward, and a
//! head tied to the embedding or a linear map of its own
//! (`tie_word_embeddings`). [`Qwen3Model`] is built from a
//! [`Qwen3Config`] out of the shared
//! [`CausalSelfAttention`](crate::blocks::transformers::attention::csa::CausalSelfAttention),
//! [`TransformerGatedMlp`](crate::blocks::transformers::mlp::TransformerGatedMlp) and
//! [`RotaryEmbedding`](crate::blocks::transformers::embedding::RotaryEmbedding),
//! wrapping an attention-only
//! [`LmDecoderModel`](crate::blocks::transformers::models::language_decoder::LmDecoderModel), and
//! decodes through a [`KVCache`](crate::ops::transformers::attention::KVCache).
//!
//! It is the model of the dense Qwen3 checkpoints up to 8B: `Qwen3-0.6B`,
//! `Qwen3-1.7B`, `Qwen3-4B`, their `-Base` builds and `Qwen3-4B`'s `2507`
//! builds, `Qwen3-4B-SafeRL`, and `Qwen3Guard-Gen-0.6B` and `-4B`, whose heads
//! are tied, and `Qwen3-8B`, its `-Base` and `Qwen3Guard-Gen-8B`, whose head is
//! its own. It is also the model of any dense Qwen3 an `hf:` repo holds, and
//! the text model of
//! [`kits::speech::asr::qwen3_asr`](crate::kits::speech::asr::qwen3_asr).
//!
//! - [`blocks`]: the model, built from a [`Qwen3Config`].
//! - Decoding: the model is a
//! [`StackedLm`](crate::kits::support::decoding::StackedLm), its decode state
//! its stack's
//! [`DecoderStackCache`](crate::blocks::transformers::decoders::stack::DecoderStackCache),
//! so [`decode_cached`](crate::kits::support::decoding::decode_cached)
//! decodes it under any search.
//! - [`Qwen3Construct`]: a model with the positions it was trained on and its
//! checkpoint's tokenizer and generation files, unread.
//! - [`weights`] (feature `store`): loading tensors named as `transformers`'
//! `Qwen3ForCausalLM` state dict, and reading its `config.json`.
//! - [`pretrained`] (features `store` and `cache`): the named checkpoints,
//! `qwen/qwen3-0.6b` the default, and the factory that loads one.
//!
//! A tokenizer and a chat template are not part of the kit; the searches are
//! [`kits::support::decoding`](crate::kits::support::decoding)'s.
pub use *;
pub use *;