# Granite Models Catalog
# Generated: 2026-09-01 17:30:25 UTC
# Source: HuggingFace ibm-granite organization
- id: granite-3.0-1b-a400m-instruct
family: "Granite Language"
version: "3.0"
size: 1334628352
context_length: 4096
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.0-1b-a400m-instruct"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 24
hidden_size: 1024
num_attention_heads: 16
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 24
supported_functions:
- Chat
- ToolCalling
variants:
- format: safetensors
precision: bfloat16
size_gb: 2.669
url: "https://huggingface.co/ibm-granite/granite-3.0-1b-a400m-instruct"
- format: Ollama
precision: Q4_K_M
size_gb: 0.803
url: "https://ollama.com/library/granite3-moe:1b"
description: |
Granite-3.0-1B-A400M-Instruct is an 1B parameter model finetuned from *Granite-3.0-1B-A400M-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging.
tags:
- efficient
- instruct
- text
- id: granite-3.0-2b-instruct
family: "Granite Language"
version: "3.0"
size: 2634201088
context_length: 4096
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.0-2b-instruct"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
variants:
- format: safetensors
precision: bfloat16
size_gb: 5.268
url: "https://huggingface.co/ibm-granite/granite-3.0-2b-instruct"
- format: Ollama
precision: Q4_K_M
size_gb: 1.6
url: "https://ollama.com/library/granite3-dense:2b"
description: |
Granite-3.0-2B-Instruct is a 2B parameter model finetuned from *Granite-3.0-2B-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging.
tags:
- efficient
- instruct
- text
- id: granite-3.0-3b-a800m-instruct
family: "Granite Language"
version: "3.0"
size: 3374295552
context_length: 4096
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.0-3b-a800m-instruct"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 32
hidden_size: 1536
num_attention_heads: 24
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 32
supported_functions:
- Chat
- ToolCalling
variants:
- format: safetensors
precision: bfloat16
size_gb: 6.749
url: "https://huggingface.co/ibm-granite/granite-3.0-3b-a800m-instruct"
- format: Ollama
precision: Q4_K_M
size_gb: 2.1
url: "https://ollama.com/library/granite3-moe:3b"
description: |
Granite-3.0-3B-A800M-Instruct is a 3B parameter model finetuned from *Granite-3.0-3B-A800M-Base-4K* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging.
tags:
- efficient
- instruct
- text
- id: granite-3.0-8b-instruct
family: "Granite Language"
version: "3.0"
size: 8170848256
context_length: 4096
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.0-8b-instruct"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
variants:
- format: safetensors
precision: bfloat16
size_gb: 16.342
url: "https://huggingface.co/ibm-granite/granite-3.0-8b-instruct"
- format: Ollama
precision: Q4_K_M
size_gb: 4.9
url: "https://ollama.com/library/granite3-dense:8b"
description: |
Granite-3.0-8B-Instruct is a 8B parameter model finetuned from *Granite-3.0-8B-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging.
tags:
- general-purpose
- instruct
- text
- id: granite-3.1-1b-a400m-instruct
family: "Granite Language"
version: "3.1"
size: 1334628352
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.1-1b-a400m-instruct"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 24
hidden_size: 1024
num_attention_heads: 16
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 24
supported_functions:
- Chat
- ToolCalling
variants:
- format: safetensors
precision: bfloat16
size_gb: 2.669
url: "https://huggingface.co/ibm-granite/granite-3.1-1b-a400m-instruct"
- format: Ollama
precision: Q8_0
size_gb: 1.4
url: "https://ollama.com/library/granite3.1-moe:1b"
description: |
Granite-3.1-1B-A400M-Instruct is a 1B parameter long-context instruct model finetuned from Granite-3.1-1B-A400M-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets tailored for solving long context problems. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging.
tags:
- efficient
- instruct
- text
- id: granite-3.1-2b-instruct
family: "Granite Language"
version: "3.1"
size: 2533531648
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.1-2b-instruct"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
variants:
- format: safetensors
precision: bfloat16
size_gb: 5.067
url: "https://huggingface.co/ibm-granite/granite-3.1-2b-instruct"
- format: Ollama
precision: Q4_K_M
size_gb: 1.6
url: "https://ollama.com/library/granite3.1-dense:2b"
description: |
Granite-3.1-2B-Instruct is a 2B parameter long-context instruct model finetuned from Granite-3.1-2B-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets tailored for solving long context problems. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging.
tags:
- efficient
- instruct
- text
- id: granite-3.1-3b-a800m-instruct
family: "Granite Language"
version: "3.1"
size: 3298793472
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.1-3b-a800m-instruct"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 32
hidden_size: 1536
num_attention_heads: 24
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 32
supported_functions:
- Chat
- ToolCalling
variants:
- format: safetensors
precision: bfloat16
size_gb: 6.598
url: "https://huggingface.co/ibm-granite/granite-3.1-3b-a800m-instruct"
- format: Ollama
precision: Q4_K_M
size_gb: 2.0
url: "https://ollama.com/library/granite3.1-moe:3b"
description: |
Granite-3.1-3B-A800M-Instruct is a 3B parameter long-context instruct model finetuned from Granite-3.1-3B-A800M-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets tailored for solving long context problems. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging.
tags:
- efficient
- instruct
- text
- id: granite-3.1-8b-instruct
family: "Granite Language"
version: "3.1"
size: 8170848256
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.1-8b-instruct"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
variants:
- format: safetensors
precision: bfloat16
size_gb: 16.342
url: "https://huggingface.co/ibm-granite/granite-3.1-8b-instruct"
- format: Ollama
precision: Q4_K_M
size_gb: 5.0
url: "https://ollama.com/library/granite3.1-dense:8b"
description: |
Granite-3.1-8B-Instruct is a 8B parameter long-context instruct model finetuned from Granite-3.1-8B-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets tailored for solving long context problems. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging.
tags:
- general-purpose
- instruct
- text
- id: granite-3.2-2b-instruct
family: "Granite Language"
version: "3.2"
size: 2533531648
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.2-2b-instruct"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
- Thinking
variants:
- format: safetensors
precision: bfloat16
size_gb: 5.067
url: "https://huggingface.co/ibm-granite/granite-3.2-2b-instruct"
- format: Ollama
precision: Q4_K_M
size_gb: 1.5
url: "https://ollama.com/library/granite3.2:2b"
description: |
Granite-3.2-2B-Instruct is an 2-billion-parameter, long-context AI model fine-tuned for thinking capabilities. Built on top of [Granite-3.1-2B-Instruct](https://huggingface.co/ibm-granite/granite-3.1-2b-instruct), it has been trained using a mix of permissively licensed open-source datasets and internally generated synthetic data designed for reasoning tasks. The model allows controllability of its thinking capability, ensuring it is applied only when required.
tags:
- efficient
- instruct
- text
- id: granite-3.2-8b-instruct
family: "Granite Language"
version: "3.2"
size: 8170848256
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.2-8b-instruct"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
- Thinking
variants:
- format: safetensors
precision: bfloat16
size_gb: 16.342
url: "https://huggingface.co/ibm-granite/granite-3.2-8b-instruct"
- format: Ollama
precision: Q4_K_M
size_gb: 4.9
url: "https://ollama.com/library/granite3.2:8b"
description: |
Granite-3.2-8B-Instruct is an 8-billion-parameter, long-context AI model fine-tuned for thinking capabilities. Built on top of [Granite-3.1-8B-Instruct](https://huggingface.co/ibm-granite/granite-3.1-8b-instruct), it has been trained using a mix of permissively licensed open-source datasets and internally generated synthetic data designed for reasoning tasks. The model allows controllability of its thinking capability, ensuring it is applied only when required.
tags:
- general-purpose
- instruct
- text
- id: granite-3.2-8b-instruct-preview
family: "Granite Language"
version: "3.2"
size: 8170848256
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.2-8b-instruct-preview"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
- Thinking
variants:
- format: safetensors
precision: bfloat16
size_gb: 16.342
url: "https://huggingface.co/ibm-granite/granite-3.2-8b-instruct-preview"
- format: Ollama
precision: Q4_K_M
size_gb: 4.9
url: "https://ollama.com/library/granite3.2:8b"
description: |
Granite-3.2-8B-Instruct-Preview is an early release of an 8B long-context model fine-tuned for enhanced reasoning (thinking) capabilities. Built on top of [Granite-3.1-8B-Instruct](https://huggingface.co/ibm-granite/granite-3.1-8b-instruct), it has been trained using a mix of permissively licensed open-source datasets and internally generated synthetic data designed for reasoning tasks. The model allows controllability of its thinking capability, ensuring it is applied only when required.
tags:
- general-purpose
- instruct
- text
- id: granite-3.3-2b-instruct
family: "Granite Language"
version: "3.3"
size: 2533539840
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.3-2b-instruct"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
- Thinking
variants:
- format: GGUF
precision: Q2_K
size_gb: 0.978
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 1.357
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 1.252
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 1.13
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 1.453
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 1.605
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 1.545
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 1.464
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 1.757
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 1.91
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 1.805
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 1.757
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 2.081
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 2.694
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-Q8_0.gguf"
- format: GGUF
precision: F16
size_gb: 5.069
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct-GGUF/blob/main/granite-3.3-2b-instruct-f16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 5.067
url: "https://huggingface.co/ibm-granite/granite-3.3-2b-instruct"
- format: MLX
precision: float16
size_gb: 5.067
url: "https://huggingface.co/mlx-community/granite-3.3-2b-instruct-fp16"
- format: MLX
precision: 8bit
size_gb: 2.692
url: "https://huggingface.co/mlx-community/granite-3.3-2b-instruct-8bit"
- format: MLX
precision: 6bit
size_gb: 2.059
url: "https://huggingface.co/mlx-community/granite-3.3-2b-instruct-6bit"
- format: MLX
precision: 4bit
size_gb: 1.425
url: "https://huggingface.co/mlx-community/granite-3.3-2b-instruct-4bit"
- format: Ollama
precision: Q4_K_M
size_gb: 1.5
url: "https://ollama.com/library/granite3.3:2b"
description: |
Granite-3.3-2B-Instruct is a 2-billion parameter 128K context length language model fine-tuned for improved reasoning and instruction-following capabilities. Built on top of Granite-3.3-2B-Base, the model delivers significant gains on benchmarks for measuring generic performance including AlpacaEval-2.0 and Arena-Hard, and improvements in mathematics, coding, and instruction following. It supports structured reasoning through \<think\>\<\/think\> and \<response\>\<\/response\> tags, providing clear separation between internal thoughts and final outputs. The model has been trained on a carefully balanced combination of permissively licensed data and curated synthetic tasks.
tags:
- efficient
- instruct
- text
- id: granite-3.3-8b-instruct
family: "Granite Language"
version: "3.3"
size: 8170864640
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-3.3-8b-instruct"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
- Thinking
variants:
- format: GGUF
precision: Q2_K
size_gb: 3.104
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 4.349
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 3.997
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 3.593
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 4.651
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 5.149
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 4.943
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 4.686
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 5.647
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 6.145
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 5.797
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 5.647
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 6.705
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 8.684
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-Q8_0.gguf"
- format: GGUF
precision: F16
size_gb: 16.344
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct-GGUF/blob/main/granite-3.3-8b-instruct-f16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 16.342
url: "https://huggingface.co/ibm-granite/granite-3.3-8b-instruct"
- format: MLX
precision: float16
size_gb: 16.342
url: "https://huggingface.co/mlx-community/granite-3.3-8b-instruct-fp16"
- format: MLX
precision: 8bit
size_gb: 8.682
url: "https://huggingface.co/mlx-community/granite-3.3-8b-instruct-8bit"
- format: MLX
precision: 6bit
size_gb: 6.639
url: "https://huggingface.co/mlx-community/granite-3.3-8b-instruct-6bit"
- format: MLX
precision: 4bit
size_gb: 4.597
url: "https://huggingface.co/mlx-community/granite-3.3-8b-instruct-4bit"
- format: Ollama
precision: Q4_K_M
size_gb: 4.9
url: "https://ollama.com/library/granite3.3:8b"
description: |
Granite-3.3-8B-Instruct is a 8-billion parameter 128K context length language model fine-tuned for improved reasoning and instruction-following capabilities. Built on top of Granite-3.3-8B-Base, the model delivers significant gains on benchmarks for measuring generic performance including AlpacaEval-2.0 and Arena-Hard, and improvements in mathematics, coding, and instruction following. It supports structured reasoning through \<think\>\<\/think\> and \<response\>\<\/response\> tags, providing clear separation between internal thoughts and final outputs. The model has been trained on a carefully balanced combination of permissively licensed data and curated synthetic tasks.
tags:
- general-purpose
- instruct
- text
- id: granite-4.0-1b
family: "Granite Language"
version: "4.0"
size: 1631750144
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.0-1b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 16
num_key_value_heads: 4
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
variants:
- format: GGUF
precision: Q2_K
size_gb: 0.702
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 0.926
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 0.86
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 0.786
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 0.975
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 1.064
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 1.024
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 0.981
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 1.153
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 1.242
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 1.178
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 1.153
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 1.343
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 1.738
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 3.267
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-GGUF/blob/main/granite-4.0-1b-bf16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 3.264
url: "https://huggingface.co/ibm-granite/granite-4.0-1b"
- format: MLX
precision: 3bit
size_gb: 0.714
url: "https://huggingface.co/mlx-community/granite-4.0-1b-3bit"
- format: MLX
precision: 4bit
size_gb: 0.918
url: "https://huggingface.co/mlx-community/granite-4.0-1b-4bit"
- format: MLX
precision: 5bit
size_gb: 1.122
url: "https://huggingface.co/mlx-community/granite-4.0-1b-5bit"
- format: MLX
precision: 6bit
size_gb: 1.326
url: "https://huggingface.co/mlx-community/granite-4.0-1b-6bit"
- format: MLX
precision: 8bit
size_gb: 1.734
url: "https://huggingface.co/mlx-community/granite-4.0-1b-8bit"
- format: MLX
precision: bfloat16
size_gb: 3.264
url: "https://huggingface.co/mlx-community/granite-4.0-1b-bf16"
- format: Ollama
precision: BF16
size_gb: 3.3
url: "https://ollama.com/library/granite4:1b"
description: |
Granite-4.0-1B is a lightweight instruct model finetuned from *Granite-4.0-1B-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques including supervised finetuning, reinforcement learning, and model merging.
tags:
- efficient
- text
- id: granite-4.0-1b-speech
family: "Granite Speech"
version: "4.0"
size: 2313207132
context_length: 8192
model_type: "Speech"
huggingface_repo: "ibm-granite/granite-4.0-1b-speech"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 16
num_key_value_heads: 4
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- Transcription
variants:
- format: GGUF
precision: Q4_K_M
size_gb: 1.139
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-speech-GGUF/blob/main/granite-4.0-1b-speech-Q4_K_M.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 1.32
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-speech-GGUF/blob/main/granite-4.0-1b-speech-Q5_K_M.gguf"
- format: GGUF
precision: Q6_K
size_gb: 1.511
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-speech-GGUF/blob/main/granite-4.0-1b-speech-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 1.956
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-speech-GGUF/blob/main/granite-4.0-1b-speech-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 3.678
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-speech-GGUF/blob/main/granite-4.0-1b-speech-bf16.gguf"
- format: GGUF
precision: F16
size_gb: 1.159
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-speech-GGUF/blob/main/mmproj-model-f16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 4.626
url: "https://huggingface.co/ibm-granite/granite-4.0-1b-speech"
- format: Ollama
precision: BF16
size_gb: 3.3
url: "https://ollama.com/library/granite4:1b"
description: |
Granite-4.0-1b-speech is a compact and efficient speech-language model, specifically designed for multilingual automatic speech recognition (ASR) and bidirectional automatic speech translation (AST).
tags:
- audio
- efficient
- speech
- transcription
- id: granite-4.0-350m
family: "Granite Language"
version: "4.0"
size: 352379904
context_length: 32768
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.0-350m"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 28
hidden_size: 1024
num_attention_heads: 16
num_key_value_heads: 4
head_dim: 64
layer_types:
- kind: full_attention
count: 28
supported_functions:
- Chat
- ToolCalling
variants:
- format: GGUF
precision: Q2_K
size_gb: 0.181
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 0.22
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 0.208
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 0.195
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 0.228
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 0.244
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 0.237
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 0.229
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 0.26
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 0.275
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 0.264
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 0.26
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 0.293
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 0.378
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 0.708
url: "https://huggingface.co/ibm-granite/granite-4.0-350m-GGUF/blob/main/granite-4.0-350m-bf16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 0.705
url: "https://huggingface.co/ibm-granite/granite-4.0-350m"
- format: MLX
precision: 3bit
size_gb: 0.154
url: "https://huggingface.co/mlx-community/granite-4.0-350m-3bit"
- format: MLX
precision: 4bit
size_gb: 0.198
url: "https://huggingface.co/mlx-community/granite-4.0-350m-4bit"
- format: MLX
precision: 5bit
size_gb: 0.242
url: "https://huggingface.co/mlx-community/granite-4.0-350m-5bit"
- format: MLX
precision: 6bit
size_gb: 0.286
url: "https://huggingface.co/mlx-community/granite-4.0-350m-6bit"
- format: MLX
precision: 8bit
size_gb: 0.374
url: "https://huggingface.co/mlx-community/granite-4.0-350m-8bit"
- format: MLX
precision: bfloat16
size_gb: 0.705
url: "https://huggingface.co/mlx-community/granite-4.0-350m-bf16"
- format: Ollama
precision: BF16
size_gb: 0.691
url: "https://ollama.com/library/granite4:350m"
description: |
Granite-4.0-350M is a lightweight instruct model finetuned from *Granite-4.0-350M-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques including supervised finetuning, reinforcement learning, and model merging.
tags:
- efficient
- text
- id: granite-4.0-3b-vision
family: "Granite Vision"
version: "4.0"
size: 3997206464
context_length: 8192
model_type: "Vision"
huggingface_repo: "ibm-granite/granite-4.0-3b-vision"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2560
num_attention_heads: 40
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ImageUnderstanding
- ToolCalling
variants:
- format: safetensors
precision: null
size_gb: 3.997
url: "https://huggingface.co/ibm-granite/granite-4.0-3b-vision"
- format: MLX
precision: 4bit
size_gb: 3.01
url: "https://huggingface.co/mlx-community/granite-4.0-3b-vision-4bit"
- format: MLX
precision: 5bit
size_gb: 3.488
url: "https://huggingface.co/mlx-community/granite-4.0-3b-vision-5bit"
- format: MLX
precision: 6bit
size_gb: 3.966
url: "https://huggingface.co/mlx-community/granite-4.0-3b-vision-6bit"
- format: MLX
precision: 8bit
size_gb: 4.923
url: "https://huggingface.co/mlx-community/granite-4.0-3b-vision-8bit"
- format: MLX
precision: mxfp4
size_gb: 3.01
url: "https://huggingface.co/mlx-community/granite-4.0-3b-vision-mxfp4"
- format: MLX
precision: mxfp8
size_gb: 4.803
url: "https://huggingface.co/mlx-community/granite-4.0-3b-vision-mxfp8"
- format: MLX
precision: nvfp4
size_gb: 3.249
url: "https://huggingface.co/mlx-community/granite-4.0-3b-vision-nvfp4"
description: |
Granite-4.0-3B-Vision is a vision-language model (VLM) designed for enterprise-grade
document data extraction. It focuses on specialized, complex extraction tasks that
ultracompact models often struggle with:
tags:
- efficient
- image
- multimodal
- vision
- id: granite-4.0-h-1b
family: "Granite Language"
version: "4.0"
size: 1461538368
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.0-h-1b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 1536
num_attention_heads: 12
num_key_value_heads: 4
head_dim: 128
layer_types:
- kind: full_attention
count: 4
- kind: recurrent
count: 36
mamba: { d_conv: 4, d_state: 128, d_inner: 3072, n_groups: 1 }
supported_functions:
- Chat
- ToolCalling
variants:
- format: GGUF
precision: Q2_K
size_gb: 0.589
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 0.763
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 0.732
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 0.695
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 0.868
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 0.95
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 0.901
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 0.873
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 1.032
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 1.113
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 1.049
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 1.032
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 1.205
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 1.559
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 2.928
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b-GGUF/blob/main/granite-4.0-h-1b-bf16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 2.923
url: "https://huggingface.co/ibm-granite/granite-4.0-h-1b"
- format: MLX
precision: 4bit
size_gb: 0.823
url: "https://huggingface.co/mlx-community/granite-4.0-h-1b-4bit"
- format: MLX
precision: 3bit
size_gb: 0.641
url: "https://huggingface.co/mlx-community/granite-4.0-h-1b-3bit"
- format: MLX
precision: 5bit
size_gb: 1.006
url: "https://huggingface.co/mlx-community/granite-4.0-h-1b-5bit"
- format: MLX
precision: 6bit
size_gb: 1.188
url: "https://huggingface.co/mlx-community/granite-4.0-h-1b-6bit"
- format: MLX
precision: 8bit
size_gb: 1.554
url: "https://huggingface.co/mlx-community/granite-4.0-h-1b-8bit"
- format: MLX
precision: bfloat16
size_gb: 2.923
url: "https://huggingface.co/mlx-community/granite-4.0-h-1b-bf16"
- format: Ollama
precision: Q8_0
size_gb: 1.6
url: "https://ollama.com/library/granite4:1b-h"
description: |
Granite-4.0-H-1B is a lightweight instruct model finetuned from *Granite-4.0-H-1B-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques including supervised finetuning, reinforcement learning, and model merging.
tags:
- efficient
- text
- id: granite-4.0-h-350m
family: "Granite Language"
version: "4.0"
size: 340332224
context_length: 32768
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.0-h-350m"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 32
hidden_size: 768
num_attention_heads: 12
num_key_value_heads: 4
head_dim: 64
layer_types:
- kind: full_attention
count: 4
- kind: recurrent
count: 28
mamba: { d_conv: 4, d_state: 128, d_inner: 1536, n_groups: 1 }
supported_functions:
- Chat
- ToolCalling
variants:
- format: GGUF
precision: Q2_K
size_gb: 0.16
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 0.195
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 0.189
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 0.181
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 0.216
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 0.233
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 0.223
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 0.217
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 0.249
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 0.265
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 0.252
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 0.249
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 0.284
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 0.366
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 0.685
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m-GGUF/blob/main/granite-4.0-h-350m-bf16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 0.681
url: "https://huggingface.co/ibm-granite/granite-4.0-h-350m"
- format: MLX
precision: 3bit
size_gb: 0.149
url: "https://huggingface.co/mlx-community/granite-4.0-h-350m-3bit"
- format: MLX
precision: 4bit
size_gb: 0.192
url: "https://huggingface.co/mlx-community/granite-4.0-h-350m-4bit"
- format: MLX
precision: 5bit
size_gb: 0.234
url: "https://huggingface.co/mlx-community/granite-4.0-h-350m-5bit"
- format: MLX
precision: 6bit
size_gb: 0.277
url: "https://huggingface.co/mlx-community/granite-4.0-h-350m-6bit"
- format: MLX
precision: 8bit
size_gb: 0.362
url: "https://huggingface.co/mlx-community/granite-4.0-h-350m-8bit"
- format: MLX
precision: bfloat16
size_gb: 0.681
url: "https://huggingface.co/mlx-community/granite-4.0-h-350m-bf16"
- format: Ollama
precision: Q8_0
size_gb: 0.357
url: "https://ollama.com/library/granite4:350m-h"
description: |
Granite-4.0-H-350M is a lightweight instruct model finetuned from *Granite-4.0-H-350M-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques including supervised finetuning, reinforcement learning, and model merging.
tags:
- efficient
- text
- id: granite-4.0-h-micro
family: "Granite Language"
version: "4.0"
size: 3191396096
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.0-h-micro"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 4
- kind: recurrent
count: 36
mamba: { d_conv: 4, d_state: 128, d_inner: 4096, n_groups: 1 }
supported_functions:
- Chat
- ToolCalling
variants:
- format: GGUF
precision: Q2_K
size_gb: 1.226
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 1.638
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 1.555
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 1.459
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 1.856
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 2.042
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 1.943
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 1.867
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 2.229
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 2.415
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 2.273
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 2.229
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 2.625
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 3.398
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 6.389
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro-GGUF/blob/main/granite-4.0-h-micro-bf16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 6.383
url: "https://huggingface.co/ibm-granite/granite-4.0-h-micro"
- format: MLX
precision: 8bit
size_gb: 3.392
url: "https://huggingface.co/mlx-community/granite-4.0-h-micro-8bit"
- format: MLX
precision: 6bit
size_gb: 2.594
url: "https://huggingface.co/mlx-community/granite-4.0-h-micro-6bit"
- format: MLX
precision: 4bit
size_gb: 1.797
url: "https://huggingface.co/mlx-community/granite-4.0-h-micro-4bit"
- format: Ollama
precision: Q4_K_M
size_gb: 1.9
url: "https://ollama.com/library/granite4:micro-h"
- format: LMStudio
precision: Q4_1
size_gb: 2.083
url: "https://lmstudio.ai/models/ibm/granite-4-h-micro"
- format: OpenRouter
precision:
size_gb: null
url: "https://openrouter.ai/ibm-granite/granite-4.0-h-micro"
description: |
Granite-4.0-H-Micro is a 3B parameter long-context instruct model finetuned from *Granite-4.0-H-Micro-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging. Granite 4.0 instruct models feature improved *instruction following (IF)* and *tool-calling* capabilities, making them more effective in enterprise applications.
tags:
- efficient
- text
- id: granite-4.0-h-small
family: "Granite Language"
version: "4.0"
size: 32207337984
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.0-h-small"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 4
- kind: recurrent
count: 36
mamba: { d_conv: 4, d_state: 128, d_inner: 8192, n_groups: 1 }
supported_functions:
- Chat
- ToolCalling
variants:
- format: GGUF
precision: Q2_K
size_gb: 11.779
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 16.475
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 15.36
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 14.053
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 18.274
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 20.261
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 19.477
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 18.422
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 22.247
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 24.233
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 22.866
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 22.247
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 26.468
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 34.265
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-Q8_0.gguf"
- format: GGUF
precision: F16
size_gb: 64.446
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF/blob/main/granite-4.0-h-small-f16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 64.415
url: "https://huggingface.co/ibm-granite/granite-4.0-h-small"
- format: MLX
precision: 8bit
size_gb: 34.222
url: "https://huggingface.co/mlx-community/granite-4.0-h-small-8bit"
- format: MLX
precision: 4bit
size_gb: 18.126
url: "https://huggingface.co/mlx-community/granite-4.0-h-small-4bit"
- format: Ollama
precision: Q4_K_M
size_gb: 19
url: "https://ollama.com/library/granite4:small-h"
- format: LMStudio
precision: Q4_1
size_gb: 20.938
url: "https://lmstudio.ai/models/ibm/granite-4-h-small"
description: |
Granite-4.0-H-Small is a 32B parameter long-context instruct model finetuned from *Granite-4.0-H-Small-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging. Granite 4.0 instruct models feature improved *instruction following (IF)* and *tool-calling* capabilities, making them more effective in enterprise applications.
tags:
- reasoning
- text
- id: granite-4.0-h-tiny
family: "Granite Language"
version: "4.0"
size: 6939037248
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.0-h-tiny"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 1536
num_attention_heads: 12
num_key_value_heads: 4
head_dim: 128
layer_types:
- kind: full_attention
count: 4
- kind: recurrent
count: 36
mamba: { d_conv: 4, d_state: 128, d_inner: 3072, n_groups: 1 }
supported_functions:
- Chat
- ToolCalling
variants:
- format: GGUF
precision: Q2_K
size_gb: 2.586
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 3.601
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 3.353
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 3.062
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 3.963
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 4.387
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 4.231
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 3.996
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 4.81
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 5.234
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 4.949
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 4.81
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 5.711
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 7.39
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 13.891
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny-GGUF/blob/main/granite-4.0-h-tiny-bf16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 13.878
url: "https://huggingface.co/ibm-granite/granite-4.0-h-tiny"
- format: MLX
precision: 8bit
size_gb: 7.374
url: "https://huggingface.co/mlx-community/granite-4.0-h-tiny-8bit"
- format: MLX
precision: 4bit
size_gb: 3.906
url: "https://huggingface.co/mlx-community/Granite-4.0-H-Tiny-4bit-DWQ"
- format: MLX
precision: 3bit
size_gb: 3.04
url: "https://huggingface.co/mlx-community/granite-4.0-h-tiny-3bit-MLX"
- format: MLX
precision: 6bit
size_gb: 5.64
url: "https://huggingface.co/mlx-community/granite-4.0-h-tiny-6bit-MLX"
- format: MLX
precision: 5bit
size_gb: 4.773
url: "https://huggingface.co/mlx-community/granite-4.0-h-tiny-5bit-MLX"
- format: Ollama
precision: Q4_K_M
size_gb: 4.2
url: "https://ollama.com/library/granite4:tiny-h"
- format: LMStudio
precision: Q4_1
size_gb: 4.542
url: "https://lmstudio.ai/models/ibm/granite-4-h-tiny"
description: |
Granite-4.0-H-Tiny is a 7B parameter long-context instruct model finetuned from *Granite-4.0-H-Tiny-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging. Granite 4.0 instruct models feature improved *instruction following (IF)* and *tool-calling* capabilities, making them more effective in enterprise applications.
tags:
- general-purpose
- text
- id: granite-4.0-micro
family: "Granite Language"
version: "4.0"
size: 3402836480
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.0-micro"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2560
num_attention_heads: 40
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
variants:
- format: GGUF
precision: Q2_K
size_gb: 1.371
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 1.864
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 1.726
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 1.567
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 1.985
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 2.181
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 2.1
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 1.998
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 2.378
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 2.574
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 2.437
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 2.378
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 2.796
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 3.62
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 6.81
url: "https://huggingface.co/ibm-granite/granite-4.0-micro-GGUF/blob/main/granite-4.0-micro-bf16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 6.806
url: "https://huggingface.co/ibm-granite/granite-4.0-micro"
- format: MLX
precision: 8bit
size_gb: 3.616
url: "https://huggingface.co/mlx-community/granite-4.0-micro-8bit"
- format: Ollama
precision: Q4_K_M
size_gb: 2.1
url: "https://ollama.com/library/granite4:micro"
- format: LMStudio
precision: Q4_1
size_gb: 2.255
url: "https://lmstudio.ai/models/ibm/granite-4-micro"
description: |
Granite-4.0-Micro is a 3B parameter long-context instruct model finetuned from *Granite-4.0-Micro-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. This model is developed using a diverse set of techniques with a structured chat format, including supervised finetuning, model alignment using reinforcement learning, and model merging. Granite 4.0 instruct models feature improved *instruction following (IF)* and *tool-calling* capabilities, making them more effective in enterprise applications.
tags:
- efficient
- text
- id: granite-4.0-tiny-preview
family: "Granite Language"
version: "4.0"
size: 6671539776
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.0-tiny-preview"
native_dtype: "unknown"
architecture:
num_hidden_layers: 40
hidden_size: 1536
num_attention_heads: 12
num_key_value_heads: 4
head_dim: 128
layer_types:
- kind: full_attention
count: 4
- kind: recurrent
count: 36
mamba: { d_conv: 4, d_state: 128, d_inner: 3072, n_groups: 1 }
supported_functions:
- Chat
- ToolCalling
- Thinking
variants:
- format: GGUF
precision: Q2_K
size_gb: 2.451
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 3.437
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 3.196
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 2.915
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 3.79
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 4.202
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 4.05
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 3.822
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 4.614
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 5.026
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 4.748
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 4.614
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 5.489
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 7.104
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-Q8_0.gguf"
- format: GGUF
precision: F16
size_gb: 13.354
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview-GGUF/blob/main/granite-4.0-tiny-preview-f16.gguf"
- format: safetensors
precision: null
size_gb: 6.672
url: "https://huggingface.co/ibm-granite/granite-4.0-tiny-preview"
- format: MLX
precision: 8bit
size_gb: 7.089
url: "https://huggingface.co/mlx-community/granite-4.0-tiny-preview-8bit"
- format: MLX
precision: 4bit
size_gb: 3.756
url: "https://huggingface.co/mlx-community/granite-4.0-tiny-preview-4bit"
description: |
Granite-4-Tiny-Preview is a 7B parameter fine-grained hybrid mixture-of-experts (MoE) instruct model fine-tuned from Granite-4.0-Tiny-Base-Preview using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets tailored for solving long context problems. This model is developed using a diverse set of techniques with a structured chat format, including supervised fine-tuning, and model alignment using reinforcement learning.
tags:
- general-purpose
- text
- id: granite-4.1-30b
family: "Granite Language"
version: "4.1"
size: 28865728512
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.1-30b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 64
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 64
supported_functions:
- Chat
- ToolCalling
variants:
- format: GGUF
precision: Q2_K
size_gb: 10.724
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 15.13
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 13.957
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 12.569
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 16.348
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 18.127
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 17.49
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 16.485
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 19.905
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 21.683
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 20.493
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 19.905
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 23.684
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 30.675
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 57.736
url: "https://huggingface.co/ibm-granite/granite-4.1-30b-GGUF/blob/main/granite-4.1-30b-bf16-00001-of-00005.gguf"
- format: safetensors
precision: bfloat16
size_gb: 57.731
url: "https://huggingface.co/ibm-granite/granite-4.1-30b"
- format: MLX
precision: bfloat16
size_gb: 57.731
url: "https://huggingface.co/mlx-community/granite-4.1-30b-bf16"
- format: MLX
precision: nvfp4
size_gb: 18.042
url: "https://huggingface.co/mlx-community/granite-4.1-30b-nvfp4"
- format: MLX
precision: mxfp4
size_gb: 16.238
url: "https://huggingface.co/mlx-community/granite-4.1-30b-mxfp4"
- format: MLX
precision: mxfp8
size_gb: 29.768
url: "https://huggingface.co/mlx-community/granite-4.1-30b-mxfp8"
- format: MLX
precision: 6bit
size_gb: 25.258
url: "https://huggingface.co/mlx-community/granite-4.1-30b-6bit"
- format: MLX
precision: 4bit
size_gb: 18.042
url: "https://huggingface.co/mlx-community/granite-4.1-30b-4bit"
- format: MLX
precision: 5bit
size_gb: 21.65
url: "https://huggingface.co/mlx-community/granite-4.1-30b-5bit"
- format: MLX
precision: 8bit
size_gb: 32.474
url: "https://huggingface.co/mlx-community/granite-4.1-30b-8bit"
- format: Ollama
precision: Q4_K_M
size_gb: 17
url: "https://ollama.com/library/granite4.1:30b"
- format: LMStudio
precision: Q4_K_M
size_gb: 17.5
url: "https://lmstudio.ai/models/ibm/granite-4.1-30b"
description: |
Granite-4.1-30B is a 30B parameter long-context instruct model finetuned from *Granite-4.1-30B-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. Granite 4.1 models have gone through an improved post-training pipeline, including supervised finetuning and reinforcement learning alignment, resulting in enhanced tool calling, instruction following, and chat capabilities.
tags:
- reasoning
- text
- id: granite-4.1-3b
family: "Granite Language"
version: "4.1"
size: 3402836480
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.1-3b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2560
num_attention_heads: 40
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
variants:
- format: GGUF
precision: Q2_K
size_gb: 1.371
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 1.864
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 1.726
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 1.567
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 1.985
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 2.181
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 2.1
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 1.998
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 2.378
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 2.574
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 2.437
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 2.378
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 2.796
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 3.62
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 6.81
url: "https://huggingface.co/ibm-granite/granite-4.1-3b-GGUF/blob/main/granite-4.1-3b-bf16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 6.806
url: "https://huggingface.co/ibm-granite/granite-4.1-3b"
- format: MLX
precision: bfloat16
size_gb: 6.806
url: "https://huggingface.co/mlx-community/granite-4.1-3b-bf16"
- format: MLX
precision: nvfp4
size_gb: 2.127
url: "https://huggingface.co/mlx-community/granite-4.1-3b-nvfp4"
- format: MLX
precision: mxfp4
size_gb: 1.914
url: "https://huggingface.co/mlx-community/granite-4.1-3b-mxfp4"
- format: MLX
precision: mxfp8
size_gb: 3.509
url: "https://huggingface.co/mlx-community/granite-4.1-3b-mxfp8"
- format: MLX
precision: 4bit
size_gb: 2.127
url: "https://huggingface.co/mlx-community/granite-4.1-3b-4bit"
- format: MLX
precision: 5bit
size_gb: 2.552
url: "https://huggingface.co/mlx-community/granite-4.1-3b-5bit"
- format: MLX
precision: 6bit
size_gb: 2.978
url: "https://huggingface.co/mlx-community/granite-4.1-3b-6bit"
- format: MLX
precision: 8bit
size_gb: 3.828
url: "https://huggingface.co/mlx-community/granite-4.1-3b-8bit"
- format: Ollama
precision: Q4_K_M
size_gb: 2.1
url: "https://ollama.com/library/granite4.1:3b"
- format: LMStudio
precision: Q4_K_M
size_gb: 2.1
url: "https://lmstudio.ai/models/ibm/granite-4.1-3b"
description: |
Granite-4.1-3B is a 3B parameter long-context instruct model finetuned from *Granite-4.1-3B-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. Granite 4.1 models have gone through an improved post-training pipeline, including supervised finetuning and reinforcement learning alignment, resulting in enhanced tool calling, instruction following, and chat capabilities.
tags:
- efficient
- text
- id: granite-4.1-8b
family: "Granite Language"
version: "4.1"
size: 8791592960
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.1-8b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
variants:
- format: GGUF
precision: Q2_K
size_gb: 3.412
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 4.7
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 4.347
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 3.943
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 5.056
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 5.58
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 5.348
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 5.091
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 6.103
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 6.627
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 6.254
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 6.103
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 7.216
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 9.346
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 17.587
url: "https://huggingface.co/ibm-granite/granite-4.1-8b-GGUF/blob/main/granite-4.1-8b-bf16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 17.583
url: "https://huggingface.co/ibm-granite/granite-4.1-8b"
- format: MLX
precision: bfloat16
size_gb: 16.761
url: "https://huggingface.co/mlx-community/granite-4.1-8b-bf16"
- format: MLX
precision: nvfp4
size_gb: 5.238
url: "https://huggingface.co/mlx-community/granite-4.1-8b-nvfp4"
- format: MLX
precision: mxfp4
size_gb: 4.715
url: "https://huggingface.co/mlx-community/granite-4.1-8b-mxfp4"
- format: MLX
precision: mxfp8
size_gb: 8.643
url: "https://huggingface.co/mlx-community/granite-4.1-8b-mxfp8"
- format: MLX
precision: 4bit
size_gb: 5.238
url: "https://huggingface.co/mlx-community/granite-4.1-8b-4bit"
- format: MLX
precision: 5bit
size_gb: 6.286
url: "https://huggingface.co/mlx-community/granite-4.1-8b-5bit"
- format: MLX
precision: 6bit
size_gb: 7.333
url: "https://huggingface.co/mlx-community/granite-4.1-8b-6bit"
- format: MLX
precision: 8bit
size_gb: 9.428
url: "https://huggingface.co/mlx-community/granite-4.1-8b-8bit"
- format: Ollama
precision: Q4_K_M
size_gb: 5.3
url: "https://ollama.com/library/granite4.1:8b"
- format: LMStudio
precision: Q4_K_M
size_gb: 5.3
url: "https://lmstudio.ai/models/ibm/granite-4.1-8b"
- format: OpenRouter
precision:
size_gb: null
url: "https://openrouter.ai/ibm-granite/granite-4.1-8b"
description: |
Granite-4.1-8B is a 8B parameter long-context instruct model finetuned from *Granite-4.1-8B-Base* using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets. Granite 4.1 models have gone through an improved post-training pipeline, including supervised finetuning and reinforcement learning alignment, resulting in enhanced tool calling, instruction following, and chat capabilities.
tags:
- general-purpose
- text
- id: granite-4.2-30b
family: "Granite Language"
version: "4.2"
size: 29276770304
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.2-30b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 64
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 64
supported_functions:
- Chat
- ToolCalling
- Thinking
variants:
- format: GGUF
precision: Q2_K
size_gb: 10.859
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 15.307
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 14.133
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 12.746
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 16.58
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 18.384
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 17.721
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 16.716
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 20.188
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 21.992
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 20.776
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 20.188
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 24.021
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 31.112
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 58.558
url: "https://huggingface.co/ibm-granite/granite-4.2-30b-GGUF/blob/main/granite-4.2-30b-bf16-00001-of-00002.gguf"
- format: safetensors
precision: bfloat16
size_gb: 58.554
url: "https://huggingface.co/ibm-granite/granite-4.2-30b"
- format: Ollama
precision: Q4_K_M
size_gb: 18
url: "https://ollama.com/library/granite4.2:30b"
description: |
| | |
|---|---|
| **Developers** | Granite Team, IBM |
| **Model Type** | Decoder-only Dense Transformer (Reasoning) |
| **Architecture** | GraniteForCausalLM |
| **Base Model** | [Granite-4.1-30B-Base](https://huggingface.co/ibm-granite/granite-4.1-30b-base) |
| **Parameters** | 30B |
| **Context Length** | Natively Supports 128K (Long-context extension to 512K) |
| **Precision** | bfloat16 |
| **Tested Languages** | English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, Chinese (other languages may work but have not been fully tested) |
| **Reasoning Mode** | Built-in `<think>...</think>` chain-of-thought |
| **Best For** | Reasoning, Code Generation, Tool Calling, Agentic Workflows, Multilingual Dialog |
| **License** | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
| **HF Collection** | [Granite 4.2 Language Models](https://huggingface.co/collections/ibm-granite/granite-42-language-models) |
| **Release Date** | August 25, 2026 |
tags:
- reasoning
- text
- id: granite-4.2-3b
family: "Granite Language"
version: "4.2"
size: 3659737600
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.2-3b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2560
num_attention_heads: 40
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
- Thinking
variants:
- format: GGUF
precision: Q2_K
size_gb: 1.456
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 1.975
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 1.836
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 1.677
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 2.129
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 2.342
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 2.244
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 2.143
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 2.554
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 2.767
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 2.614
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 2.554
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 3.006
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 3.893
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 7.323
url: "https://huggingface.co/ibm-granite/granite-4.2-3b-GGUF/blob/main/granite-4.2-3b-bf16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 7.319
url: "https://huggingface.co/ibm-granite/granite-4.2-3b"
- format: Ollama
precision: Q4_K_M
size_gb: 2.2
url: "https://ollama.com/library/granite4.2:3b"
description: |
| | |
|---|---|
| **Developers** | Granite Team, IBM |
| **Model Type** | Decoder-only Dense Transformer (Reasoning) |
| **Architecture** | GraniteForCausalLM |
| **Base Model** | [Granite-4.1-3B-Base](https://huggingface.co/ibm-granite/granite-4.1-3b-base) |
| **Parameters** | 3B |
| **Context Length** | Natively Supports 128K (Long-context extension to 512K) |
| **Precision** | bfloat16 |
| **Tested Languages** | English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, Chinese (other languages may work but have not been fully tested) |
| **Reasoning Mode** | Built-in `<think>...</think>` chain-of-thought |
| **Best For** | Reasoning, Code Generation, Tool Calling, Agentic Workflows, Multilingual Dialog |
| **License** | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
| **HF Collection** | [Granite 4.2 Language Models](https://huggingface.co/collections/ibm-granite/granite-42-language-models) |
| **Release Date** | August 25, 2026 |
tags:
- efficient
- text
- id: granite-4.2-8b
family: "Granite Language"
version: "4.2"
size: 8791592960
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-4.2-8b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ToolCalling
- Thinking
variants:
- format: GGUF
precision: Q2_K
size_gb: 3.412
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 4.7
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 4.347
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 3.943
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 5.056
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 5.58
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 5.348
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 5.091
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 6.103
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 6.627
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 6.254
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 6.103
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 7.216
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 9.346
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 17.587
url: "https://huggingface.co/ibm-granite/granite-4.2-8b-GGUF/blob/main/granite-4.2-8b-bf16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 17.583
url: "https://huggingface.co/ibm-granite/granite-4.2-8b"
- format: Ollama
precision: Q4_K_M
size_gb: 5.3
url: "https://ollama.com/library/granite4.2:8b"
- format: OpenRouter
precision:
size_gb: null
url: "https://openrouter.ai/ibm-granite/granite-4.2-8b"
description: |
| | |
|---|---|
| **Developers** | Granite Team, IBM |
| **Model Type** | Decoder-only Dense Transformer (Reasoning) |
| **Architecture** | GraniteForCausalLM |
| **Base Model** | [Granite-4.1-8B-Base](https://huggingface.co/ibm-granite/granite-4.1-8b-base) |
| **Parameters** | 8B |
| **Context Length** | Natively Supports 128K (Long-context extension to 512K) |
| **Precision** | bfloat16 |
| **Tested Languages** | English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, Chinese (other languages may work but have not been fully tested) |
| **Reasoning Mode** | Built-in `<think>...</think>` chain-of-thought |
| **Best For** | Reasoning, Code Generation, Tool Calling, Agentic Workflows, Multilingual Dialog |
| **License** | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
| **HF Collection** | [Granite 4.2 Language Models](https://huggingface.co/collections/ibm-granite/granite-42-language-models) |
| **Release Date** | August 25, 2026 |
tags:
- general-purpose
- text
- id: granite-docling-258M
family: "Granite Docling"
version: ""
size: 257517120
context_length: 8192
model_type: "Vision"
huggingface_repo: "ibm-granite/granite-docling-258M"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 30
hidden_size: 576
num_attention_heads: 9
num_key_value_heads: 3
head_dim: 64
layer_types:
- kind: full_attention
count: 30
supported_functions:
- Chat
- ImageUnderstanding
variants:
- format: GGUF
precision: BF16
size_gb: 0.332
url: "https://huggingface.co/ibm-granite/granite-docling-258M-GGUF/blob/main/granite-docling-258M-BF16.gguf"
- format: GGUF
precision: BF16
size_gb: 0.332
url: "https://huggingface.co/ibm-granite/granite-docling-258M-GGUF/blob/main/granite-docling-258M-bf16.gguf"
- format: GGUF
precision: F16
size_gb: 0.19
url: "https://huggingface.co/ibm-granite/granite-docling-258M-GGUF/blob/main/mmproj-model-f16.gguf"
- format: safetensors
precision: null
size_gb: 0.258
url: "https://huggingface.co/ibm-granite/granite-docling-258M"
description: |
Granite Docling 258M for visual analysis and image understanding.
tags:
- efficient
- image
- multimodal
- vision
- id: granite-docling-258M-mlx
family: "Granite Docling"
version: ""
size: 315319872
context_length: 8192
model_type: "Vision"
huggingface_repo: "ibm-granite/granite-docling-258M-mlx"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 30
hidden_size: 576
num_attention_heads: 9
num_key_value_heads: 3
head_dim: 64
layer_types:
- kind: full_attention
count: 30
supported_functions:
- Chat
- ImageUnderstanding
variants:
- format: safetensors
precision: null
size_gb: 0.315
url: "https://huggingface.co/ibm-granite/granite-docling-258M-mlx"
description: |
Granite Docling 258M for visual analysis and image understanding.
tags:
- efficient
- image
- multimodal
- vision
- id: granite-embedding-107m-multilingual
family: "Granite Embedding"
version: ""
size: 106994304
context_length: 514
model_type: "Embedding"
huggingface_repo: "ibm-granite/granite-embedding-107m-multilingual"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 6
hidden_size: 384
num_attention_heads: 12
num_key_value_heads: 12
head_dim: 32
layer_types:
- kind: full_attention
count: 6
supported_functions:
- Embeddings
variants:
- format: safetensors
precision: bfloat16
size_gb: 0.214
url: "https://huggingface.co/ibm-granite/granite-embedding-107m-multilingual"
- format: Ollama
precision: F16
size_gb: 0.216
url: "https://ollama.com/ibm/granite-embedding:107m"
description: |
Granite-Embedding-107M-Multilingual is a 107M parameter dense biencoder embedding model from the Granite Embeddings suite that can be used to generate high quality text embeddings. This model produces embedding vectors of size 384 and is trained using a combination of open source relevance-pair datasets with permissive, enterprise-friendly license, and IBM collected and generated datasets. This model is developed using contrastive finetuning, knowledge distillation and model merging for improved performance.
tags:
- efficient
- embedding
- retrieval
- id: granite-embedding-125m-english
family: "Granite Embedding"
version: ""
size: 124645632
context_length: 514
model_type: "Embedding"
huggingface_repo: "ibm-granite/granite-embedding-125m-english"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 12
hidden_size: 768
num_attention_heads: 12
num_key_value_heads: 12
head_dim: 64
layer_types:
- kind: full_attention
count: 12
supported_functions:
- Embeddings
variants:
- format: safetensors
precision: bfloat16
size_gb: 0.249
url: "https://huggingface.co/ibm-granite/granite-embedding-125m-english"
- format: Ollama
precision: F16
size_gb: 0.245
url: "https://ollama.com/ibm/granite-embedding:125m"
description: |
Granite-Embedding-125m-English is a 125M parameter dense biencoder embedding model from the Granite Embeddings suite that can be used to generate high quality text embeddings. This model produces embedding vectors of size 768. Compared to most other open-source models, this model was only trained using open-source relevance-pair datasets with permissive, enterprise-friendly license, plus IBM collected and generated datasets. While maintaining competitive scores on academic benchmarks such as BEIR, this model also performs well on many enterprise use cases. This model is developed using retrieval oriented pretraining, contrastive finetuning and knowledge distillation.
tags:
- efficient
- embedding
- retrieval
- id: granite-embedding-278m-multilingual
family: "Granite Embedding"
version: ""
size: 278043648
context_length: 514
model_type: "Embedding"
huggingface_repo: "ibm-granite/granite-embedding-278m-multilingual"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 12
hidden_size: 768
num_attention_heads: 12
num_key_value_heads: 12
head_dim: 64
layer_types:
- kind: full_attention
count: 12
supported_functions:
- Embeddings
variants:
- format: safetensors
precision: bfloat16
size_gb: 0.556
url: "https://huggingface.co/ibm-granite/granite-embedding-278m-multilingual"
- format: Ollama
precision: F16
size_gb: 0.55
url: "https://ollama.com/library/granite-embedding:278m"
description: |
Granite-Embedding-278M-Multilingual is a 278M parameter model from the Granite Embeddings suite that can be used to generate high quality text embeddings. This model produces embedding vectors of size 768 and is trained using a combination of open source relevance-pair datasets with permissive, enterprise-friendly license, and IBM collected and generated datasets. This model is developed using contrastive finetuning, knowledge distillation and model merging for improved performance.
tags:
- efficient
- embedding
- retrieval
- id: granite-embedding-30m-english
family: "Granite Embedding"
version: ""
size: 30295296
context_length: 514
model_type: "Embedding"
huggingface_repo: "ibm-granite/granite-embedding-30m-english"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 6
hidden_size: 384
num_attention_heads: 12
num_key_value_heads: 12
head_dim: 32
layer_types:
- kind: full_attention
count: 6
supported_functions:
- Embeddings
variants:
- format: safetensors
precision: bfloat16
size_gb: 0.061
url: "https://huggingface.co/ibm-granite/granite-embedding-30m-english"
- format: Ollama
precision: F16
size_gb: 0.062
url: "https://ollama.com/library/granite-embedding:30m"
description: |
Granite-Embedding-30m-English is a 30M parameter dense bi-encoder embedding model from the Granite Embeddings suite that can be used to generate high quality text embeddings. This model produces embedding vectors of size 384 and is trained using a combination of open source relevance-pair datasets with permissive, enterprise-friendly license, and IBM collected and generated datasets. While maintaining competitive scores on academic benchmarks such as BEIR, this model also performs well on many enterprise use cases. This model is developed using retrieval oriented pre-training, contrastive fine-tuning, knowledge distillation and model merging for improved performance.
tags:
- efficient
- embedding
- retrieval
- id: granite-embedding-311m-multilingual-r2
family: "Granite Embedding"
version: "r2"
size: 311664384
context_length: 32768
model_type: "Embedding"
huggingface_repo: "ibm-granite/granite-embedding-311m-multilingual-r2"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 22
hidden_size: 768
num_attention_heads: 12
num_key_value_heads: 12
head_dim: 64
layer_types:
- kind: full_attention
count: 22
supported_functions:
- Embeddings
variants:
- format: safetensors
precision: null
size_gb: 0.312
url: "https://huggingface.co/ibm-granite/granite-embedding-311m-multilingual-r2"
description: |
Granite-Embedding-311M-Multilingual-R2 is a 311M parameter dense embedding model from the Granite Embeddings collection for high-quality multilingual text embeddings. It produces 768-dimensional vectors with a context length of up to 32,768 tokens. The model supports **200+ languages** (based on the multilingual pretraining corpus of the underlying encoder), with enhanced support for 52 languages and programming code that receive explicit retrieval-pair and cross-lingual training. All training data uses permissive, enterprise-friendly licenses, plus IBM-collected and IBM-generated datasets.
tags:
- efficient
- embedding
- retrieval
- id: granite-embedding-97m-multilingual-r2
family: "Granite Embedding"
version: "r2"
size: 97441152
context_length: 32768
model_type: "Embedding"
huggingface_repo: "ibm-granite/granite-embedding-97m-multilingual-r2"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 12
hidden_size: 384
num_attention_heads: 12
num_key_value_heads: 12
head_dim: 32
layer_types:
- kind: full_attention
count: 12
supported_functions:
- Embeddings
variants:
- format: safetensors
precision: null
size_gb: 0.097
url: "https://huggingface.co/ibm-granite/granite-embedding-97m-multilingual-r2"
description: |
Granite-Embedding-97M-Multilingual-R2 is a 97M parameter dense embedding model from the Granite Embeddings collection for high-quality multilingual text embeddings at minimal compute cost. It produces 384-dimensional vectors with a context length of up to 32,768 tokens. The model supports **200+ languages** (based on the multilingual pretraining corpus of the underlying encoder), with **enhanced support for 52 languages and programming code** that receive explicit retrieval-pair and cross-lingual training. All training data uses permissive, enterprise-friendly licenses, plus IBM-collected and IBM-generated datasets.
tags:
- efficient
- embedding
- retrieval
- id: granite-embedding-english-r2
family: "Granite Embedding"
version: "r2"
size: 149014272
context_length: 8192
model_type: "Embedding"
huggingface_repo: "ibm-granite/granite-embedding-english-r2"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 22
hidden_size: 768
num_attention_heads: 12
num_key_value_heads: 12
head_dim: 64
layer_types:
- kind: full_attention
count: 22
supported_functions:
- Embeddings
variants:
- format: safetensors
precision: bfloat16
size_gb: 0.298
url: "https://huggingface.co/ibm-granite/granite-embedding-english-r2"
description: |
Granite-embedding-english-r2 is a 149M parameter dense biencoder embedding model from the Granite Embeddings collection that can be used to generate high quality text embeddings. This model produces embedding vectors of size 768 based on context length of upto 8192 tokens. Compared to most other open-source models, this model was only trained using open-source relevance-pair datasets with permissive, enterprise-friendly license, plus IBM collected and generated datasets.
tags:
- efficient
- embedding
- retrieval
- id: granite-embedding-reranker-english-r2
family: "Granite Embedding"
version: "r2"
size: 149605633
context_length: 8192
model_type: "Embedding"
huggingface_repo: "ibm-granite/granite-embedding-reranker-english-r2"
native_dtype: "float32"
architecture:
num_hidden_layers: 22
hidden_size: 768
num_attention_heads: 12
num_key_value_heads: 12
head_dim: 64
layer_types:
- kind: full_attention
count: 22
supported_functions:
- Embeddings
variants:
- format: safetensors
precision: float32
size_gb: 0.598
url: "https://huggingface.co/ibm-granite/granite-embedding-reranker-english-r2"
description: |
_granite-embedding-reranker-english-r2_ is a 149M parameter dense cross-encoder model from the Granite Embeddings collection that can be used to generate high quality text embeddings. This model produces embedding vectors of size 768 based on context length of upto 8192 tokens. Compared to most other open-source models, this model was only trained using open-source relevance-pair datasets with permissive, enterprise-friendly license, plus IBM collected and generated datasets.
tags:
- efficient
- embedding
- retrieval
- id: granite-embedding-small-english-r2
family: "Granite Embedding"
version: "r2"
size: 47662464
context_length: 8192
model_type: "Embedding"
huggingface_repo: "ibm-granite/granite-embedding-small-english-r2"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 12
hidden_size: 384
num_attention_heads: 12
num_key_value_heads: 12
head_dim: 32
layer_types:
- kind: full_attention
count: 12
supported_functions:
- Embeddings
variants:
- format: safetensors
precision: bfloat16
size_gb: 0.095
url: "https://huggingface.co/ibm-granite/granite-embedding-small-english-r2"
description: |
Granite-embedding-small-english-r2 is a 47M parameter dense biencoder embedding model from the Granite Embeddings collection that can be used to generate high quality text embeddings. This model produces embedding vectors of size 384 based on context length of upto 8192 tokens. Compared to most other open-source models, this model was only trained using open-source relevance-pair datasets with permissive, enterprise-friendly license, plus IBM collected and generated datasets.
tags:
- efficient
- embedding
- retrieval
- id: granite-guardian-3.0-2b
family: "Granite Guardian"
version: "3.0"
size: 2533531648
context_length: 8192
model_type: "Text"
huggingface_repo: "ibm-granite/granite-guardian-3.0-2b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Guardian
variants:
- format: safetensors
precision: bfloat16
size_gb: 5.067
url: "https://huggingface.co/ibm-granite/granite-guardian-3.0-2b"
description: |
**Granite Guardian 3.0 2B** is a fine-tuned Granite 3.0 2B Instruct model designed to detect risks in prompts and responses.
It can help with risk detection along many key dimensions catalogued in the [IBM AI Risk Atlas](https://www.ibm.com/docs/en/watsonx/saas?topic=ai-risk-atlas).
It is trained on unique data comprising human annotations and synthetic data informed by internal red-teaming.
It outperforms other open-source models in the same space on standard benchmarks.
tags:
- efficient
- guardian
- moderation
- safety
- text
- id: granite-guardian-3.0-8b
family: "Granite Guardian"
version: "3.0"
size: 8170848256
context_length: 8192
model_type: "Text"
huggingface_repo: "ibm-granite/granite-guardian-3.0-8b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Guardian
variants:
- format: safetensors
precision: bfloat16
size_gb: 16.342
url: "https://huggingface.co/ibm-granite/granite-guardian-3.0-8b"
description: |
**Granite Guardian 3.0 8B** is a fine-tuned Granite 3.0 8B Instruct model designed to detect risks in prompts and responses.
It can help with risk detection along many key dimensions catalogued in the [IBM AI Risk Atlas](https://www.ibm.com/docs/en/watsonx/saas?topic=ai-risk-atlas).
It is trained on unique data comprising human annotations and synthetic data informed by internal red-teaming.
It outperforms other open-source models in the same space on standard benchmarks.
tags:
- general-purpose
- guardian
- moderation
- safety
- text
- id: granite-guardian-3.1-2b
family: "Granite Guardian"
version: "3.1"
size: 2533531648
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-guardian-3.1-2b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Guardian
variants:
- format: safetensors
precision: bfloat16
size_gb: 5.067
url: "https://huggingface.co/ibm-granite/granite-guardian-3.1-2b"
- format: Ollama
precision: Q6_K
size_gb: 2.1
url: "https://ollama.com/ibm/granite3.1-guardian:2b"
description: |
**Granite Guardian 3.1 2B** is a fine-tuned Granite 3.1 2B Instruct model designed to detect risks in prompts and responses.
It can help with risk detection along many key dimensions catalogued in the [IBM AI Risk Atlas](https://www.ibm.com/docs/en/watsonx/saas?topic=ai-risk-atlas).
It is trained on unique data comprising human annotations and synthetic data informed by internal red-teaming.
It outperforms other open-source models in the same space on standard benchmarks.
tags:
- efficient
- guardian
- moderation
- safety
- text
- id: granite-guardian-3.1-8b
family: "Granite Guardian"
version: "3.1"
size: 8170848256
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-guardian-3.1-8b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Guardian
variants:
- format: safetensors
precision: bfloat16
size_gb: 16.342
url: "https://huggingface.co/ibm-granite/granite-guardian-3.1-8b"
- format: Ollama
precision: Q6_K
size_gb: 6.7
url: "https://ollama.com/ibm/granite3.1-guardian:8b"
description: |
**Granite Guardian 3.1 8B** is a fine-tuned Granite 3.1 8B Instruct model designed to detect risks in prompts and responses.
It can help with risk detection along many key dimensions catalogued in the [IBM AI Risk Atlas](https://www.ibm.com/docs/en/watsonx/saas?topic=ai-risk-atlas).
It is trained on unique data comprising human annotations and synthetic data informed by internal red-teaming.
It outperforms other open-source models in the same space on standard benchmarks.
tags:
- general-purpose
- guardian
- moderation
- safety
- text
- id: granite-guardian-3.2-3b-a800m
family: "Granite Guardian"
version: "3.2"
size: 3298793472
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-guardian-3.2-3b-a800m"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 32
hidden_size: 1536
num_attention_heads: 24
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 32
supported_functions:
- Guardian
variants:
- format: safetensors
precision: bfloat16
size_gb: 6.598
url: "https://huggingface.co/ibm-granite/granite-guardian-3.2-3b-a800m"
- format: Ollama
precision: Q6_K
size_gb: 2.7
url: "https://ollama.com/ibm/granite3.2-guardian:3b"
description: |
**Granite Guardian 3.2 3B-A800M** is a fine-tuned Granite 3.2 3B-A800M instruct model designed to detect risks in prompts and responses.
It can help with risk detection along many key dimensions catalogued in the [IBM AI Risk Atlas](https://www.ibm.com/docs/en/watsonx/saas?topic=ai-risk-atlas).
It is trained on unique data comprising human annotations and synthetic data informed by internal red-teaming.
It outperforms other open-source models in the same space on standard benchmarks.
tags:
- efficient
- guardian
- moderation
- safety
- text
- id: granite-guardian-3.2-5b
family: "Granite Guardian"
version: "3.2"
size: 5779996672
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-guardian-3.2-5b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 28
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 28
supported_functions:
- Guardian
variants:
- format: safetensors
precision: bfloat16
size_gb: 11.56
url: "https://huggingface.co/ibm-granite/granite-guardian-3.2-5b"
- format: Ollama
precision: Q6_K
size_gb: 4.7
url: "https://ollama.com/ibm/granite3.2-guardian:5b"
description: |
**Granite Guardian 3.2 5B** is a thinned down version of Granite Guardian 3.1 8B designed to detect risks in prompts and responses.
It can help with risk detection along many key dimensions catalogued in the [IBM AI Risk Atlas](https://www.ibm.com/docs/en/watsonx/saas?topic=ai-risk-atlas).
tags:
- general-purpose
- guardian
- moderation
- safety
- text
- id: granite-guardian-3.2-8b-factuality-detection
family: "Granite Guardian"
version: "3.2"
size: 8170848256
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-guardian-3.2-8b-factuality-detection"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Guardian
variants:
- format: safetensors
precision: bfloat16
size_gb: 16.342
url: "https://huggingface.co/ibm-granite/granite-guardian-3.2-8b-factuality-detection"
description: |
**Granite Guardian 3.2 8B Factuality Detection** is a model based on [ibm-granite/granite-3.2-8b-instruct](https://huggingface.co/ibm-granite/granite-3.2-8b-instruct), fine-tuned to safely detect an LLM response as unfactual.
tags:
- general-purpose
- guardian
- moderation
- safety
- text
- id: granite-guardian-3.3-8b
family: "Granite Guardian"
version: "3.3"
size: 8170864640
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-guardian-3.3-8b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Guardian
variants:
- format: GGUF
precision: Q4_K_M
size_gb: 4.943
url: "https://huggingface.co/ibm-granite/granite-guardian-3.3-8b-GGUF/blob/main/granite-guardian-3.3-8b-Q4_K_M.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 5.797
url: "https://huggingface.co/ibm-granite/granite-guardian-3.3-8b-GGUF/blob/main/granite-guardian-3.3-8b-Q5_K_M.gguf"
- format: GGUF
precision: Q6_K
size_gb: 6.705
url: "https://huggingface.co/ibm-granite/granite-guardian-3.3-8b-GGUF/blob/main/granite-guardian-3.3-8b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 8.684
url: "https://huggingface.co/ibm-granite/granite-guardian-3.3-8b-GGUF/blob/main/granite-guardian-3.3-8b-Q8_0.gguf"
- format: GGUF
precision: F16
size_gb: 16.344
url: "https://huggingface.co/ibm-granite/granite-guardian-3.3-8b-GGUF/blob/main/granite-guardian-3.3-8b-f16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 16.342
url: "https://huggingface.co/ibm-granite/granite-guardian-3.3-8b"
- format: Ollama
precision: Q6_K
size_gb: 6.7
url: "https://ollama.com/ibm/granite3.3-guardian:8b"
description: |
Granite Guardian 3.3 8b is a specialized Granite 3.3 8B model designed to judge if the input prompts and the output responses of an LLM based system meet specified criteria. The model comes pre-baked with certain criteria including but not limited to: jailbreak attempts, profanity, and hallucinations related to tool calls and retrieval augmented generation in agent-based systems. Additionally, the model also allows users to bring their own criteria and tailor the judging behavior to specific use-cases.
tags:
- general-purpose
- guardian
- moderation
- safety
- text
- id: granite-guardian-4.1-8b
family: "Granite Guardian"
version: "4.1"
size: 8380551168
context_length: 131072
model_type: "Text"
huggingface_repo: "ibm-granite/granite-guardian-4.1-8b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Guardian
variants:
- format: GGUF
precision: Q2_K
size_gb: 3.277
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q2_K.gguf"
- format: GGUF
precision: Q3_K_L
size_gb: 4.523
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q3_K_L.gguf"
- format: GGUF
precision: Q3_K_M
size_gb: 4.17
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q3_K_M.gguf"
- format: GGUF
precision: Q3_K_S
size_gb: 3.766
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q3_K_S.gguf"
- format: GGUF
precision: Q4_0
size_gb: 4.825
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q4_0.gguf"
- format: GGUF
precision: Q4_1
size_gb: 5.323
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q4_1.gguf"
- format: GGUF
precision: Q4_K_M
size_gb: 5.117
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q4_K_M.gguf"
- format: GGUF
precision: Q4_K_S
size_gb: 4.86
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q4_K_S.gguf"
- format: GGUF
precision: Q5_0
size_gb: 5.821
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q5_0.gguf"
- format: GGUF
precision: Q5_1
size_gb: 6.319
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q5_1.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 5.971
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q5_K_M.gguf"
- format: GGUF
precision: Q5_K_S
size_gb: 5.821
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q5_K_S.gguf"
- format: GGUF
precision: Q6_K
size_gb: 6.879
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 8.909
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 16.765
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b-GGUF/blob/main/granite-guardian-4.1-8b-bf16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 16.761
url: "https://huggingface.co/ibm-granite/granite-guardian-4.1-8b"
- format: Ollama
precision: Q6_K
size_gb: 6.9
url: "https://ollama.com/library/granite4.1-guardian:8b"
description: |
**Granite Guardian 4.1 8B** is a specialized safety model fine-tuned from [ibm-granite/granite-4.1-8b](https://huggingface.co/ibm-granite/granite-4.1-8b), designed to judge if the input prompts and the output responses of an LLM-based system meet specified criteria. The model comes pre-baked with certain criteria including but not limited to: jailbreak attempts, profanity, and hallucinations related to tool calls and retrieval augmented generation in agent-based systems. Additionally, the model also allows users to bring their own criteria and tailor the judging behavior to specific use cases.
tags:
- general-purpose
- guardian
- moderation
- safety
- text
- id: granite-guardian-hap-125m
family: "Granite Guardian"
version: ""
size: 124647170
context_length: 514
model_type: "Text"
huggingface_repo: "ibm-granite/granite-guardian-hap-125m"
native_dtype: "float32"
architecture:
num_hidden_layers: 12
hidden_size: 768
num_attention_heads: 12
num_key_value_heads: 12
head_dim: 64
layer_types:
- kind: full_attention
count: 12
supported_functions:
- Guardian
variants:
- format: safetensors
precision: float32
size_gb: 0.499
url: "https://huggingface.co/ibm-granite/granite-guardian-hap-125m"
description: |
This model is IBM's 12-layer toxicity binary classifier for English, intended to be used as a guardrail for any large language model. It has been trained on several benchmark datasets in English, specifically for detecting hateful, abusive, profane and other toxic content in plain text.
tags:
- efficient
- guardian
- moderation
- safety
- text
- id: granite-guardian-hap-38m
family: "Granite Guardian"
version: ""
size: 38455682
context_length: 514
model_type: "Text"
huggingface_repo: "ibm-granite/granite-guardian-hap-38m"
native_dtype: "float32"
architecture:
num_hidden_layers: 4
hidden_size: 576
num_attention_heads: 12
num_key_value_heads: 12
head_dim: 48
layer_types:
- kind: full_attention
count: 4
supported_functions:
- Guardian
variants:
- format: safetensors
precision: float32
size_gb: 0.154
url: "https://huggingface.co/ibm-granite/granite-guardian-hap-38m"
description: |
This model is IBM's lightweight, 4-layer toxicity binary classifier for English. Its latency characteristics make it a suitable guardrail for any large language model. It can also be used for bulk processing of data where high throughput is needed. It has been trained on several benchmark datasets in English, specifically for detecting hateful, abusive, profane and other toxic content in plain text.
tags:
- efficient
- guardian
- moderation
- safety
- text
- id: granite-speech-3.2-8b
family: "Granite Speech"
version: "3.2"
size: 8483498282
context_length: 8192
model_type: "Speech"
huggingface_repo: "ibm-granite/granite-speech-3.2-8b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- Transcription
- ToolCalling
- Thinking
variants:
- format: safetensors
precision: bfloat16
size_gb: 16.967
url: "https://huggingface.co/ibm-granite/granite-speech-3.2-8b"
- format: Ollama
precision: Q4_K_M
size_gb: 4.9
url: "https://ollama.com/library/granite3.2:8b"
description: |
Granite-speech-3.2-8b is a compact and efficient speech-language model, specifically designed for automatic speech recognition (ASR) and automatic speech translation (AST). Granite-speech-3.2-8b uses a two-pass design, unlike integrated models that combine speech and language into a single pass. Initial calls to granite-speech-3.2-8b will transcribe audio files into text. To process the transcribed text using the underlying Granite language model, users must make a second call as each step must be explicitly initiated.
tags:
- audio
- general-purpose
- speech
- transcription
- id: granite-speech-3.3-2b
family: "Granite Speech"
version: "3.3"
size: 3009285376
context_length: 8192
model_type: "Speech"
huggingface_repo: "ibm-granite/granite-speech-3.3-2b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- Transcription
- ToolCalling
- Thinking
variants:
- format: safetensors
precision: bfloat16
size_gb: 6.019
url: "https://huggingface.co/ibm-granite/granite-speech-3.3-2b"
- format: Ollama
precision: Q4_K_M
size_gb: 1.5
url: "https://ollama.com/library/granite3.3:2b"
description: |
Granite-speech-3.3-2b is a compact and efficient speech-language model, specifically designed for automatic speech recognition (ASR) and automatic speech translation (AST). Granite-speech-3.3-2b uses a two-pass design, unlike integrated models that combine speech and language into a single pass. Initial calls to granite-speech-3.3-2b will transcribe audio files into text. To process the transcribed text using the underlying Granite language model, users must make a second call as each step must be explicitly initiated.
tags:
- audio
- efficient
- speech
- transcription
- id: granite-speech-3.3-8b
family: "Granite Speech"
version: "3.3"
size: 8648711424
context_length: 8192
model_type: "Speech"
huggingface_repo: "ibm-granite/granite-speech-3.3-8b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 4096
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- Transcription
- ToolCalling
- Thinking
variants:
- format: safetensors
precision: bfloat16
size_gb: 17.297
url: "https://huggingface.co/ibm-granite/granite-speech-3.3-8b"
- format: Ollama
precision: Q4_K_M
size_gb: 4.9
url: "https://ollama.com/library/granite3.3:8b"
description: |
Granite-speech-3.3-8b is a compact and efficient speech-language model, specifically designed for automatic speech recognition (ASR) and automatic speech translation (AST). Granite-speech-3.3-8b uses a two-pass design, unlike integrated models that combine speech and language into a single pass. Initial calls to granite-speech-3.3-8b will transcribe audio files into text. To process the transcribed text using the underlying Granite language model, users must make a second call as each step must be explicitly initiated.
tags:
- audio
- general-purpose
- speech
- transcription
- id: granite-speech-4.1-2b
family: "Granite Speech"
version: "4.1"
size: 2313207132
context_length: 8192
model_type: "Speech"
huggingface_repo: "ibm-granite/granite-speech-4.1-2b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 16
num_key_value_heads: 4
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- Transcription
variants:
- format: GGUF
precision: Q4_K_M
size_gb: 1.139
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-GGUF/blob/main/granite-speech-4.1-2b-Q4_K_M.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 1.32
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-GGUF/blob/main/granite-speech-4.1-2b-Q5_K_M.gguf"
- format: GGUF
precision: Q6_K
size_gb: 1.511
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-GGUF/blob/main/granite-speech-4.1-2b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 1.956
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-GGUF/blob/main/granite-speech-4.1-2b-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 3.678
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-GGUF/blob/main/granite-speech-4.1-2b-bf16.gguf"
- format: GGUF
precision: F16
size_gb: 1.159
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-GGUF/blob/main/mmproj-model-f16.gguf"
- format: safetensors
precision: null
size_gb: 2.313
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b"
description: |
Granite Speech 4.1 2B is a compact and efficient speech-language model, specifically designed for multilingual automatic speech recognition (ASR) and bidirectional automatic speech translation (AST) for English, French, German, Spanish, Portuguese and Japanese.
tags:
- audio
- efficient
- speech
- transcription
- id: granite-speech-4.1-2b-plus
family: "Granite Speech"
version: "4.1"
size: 2111812956
context_length: 8192
model_type: "Speech"
huggingface_repo: "ibm-granite/granite-speech-4.1-2b-plus"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 16
num_key_value_heads: 4
head_dim: 128
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- Transcription
- ToolCalling
variants:
- format: GGUF
precision: Q4_K_M
size_gb: 1.024
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plus-GGUF/blob/main/granite-speech-4.1-2b-plus-Q4_K_M.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 1.178
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plus-GGUF/blob/main/granite-speech-4.1-2b-plus-Q5_K_M.gguf"
- format: GGUF
precision: Q6_K
size_gb: 1.343
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plus-GGUF/blob/main/granite-speech-4.1-2b-plus-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 1.738
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plus-GGUF/blob/main/granite-speech-4.1-2b-plus-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 3.267
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plus-GGUF/blob/main/granite-speech-4.1-2b-plus-bf16.gguf"
- format: GGUF
precision: F16
size_gb: 1.168
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plus-GGUF/blob/main/mmproj-model-f16.gguf"
- format: safetensors
precision: null
size_gb: 2.112
url: "https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plus"
description: |
Granite-Speech-4.1-2B-Plus has similar capabilities to the [Granite-Speech-4.1-2B](https://huggingface.co/ibm-granite/granite-speech-4.1-2b) model. The plus model adds two new community-requested rich transcription features that can be activated with a simple prompt change: speaker-attributed ASR (speaker labels and word transcripts) and word-level timing information. Unlike the base mode, the plus model doesn't provide punctuation and capitalization.
tags:
- audio
- efficient
- speech
- transcription
- id: granite-speech-5.0-470m-turboctc
family: "Granite Speech"
version: "5.0"
size: 472993792
context_length: 8192
model_type: "Speech"
huggingface_repo: "ibm-granite/granite-speech-5.0-470m-turboctc"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 0
hidden_size: 0
num_attention_heads: 0
num_key_value_heads: 0
head_dim: 0
layer_types:
- kind: full_attention
count: 0
supported_functions:
- Chat
- Transcription
variants:
- format: safetensors
precision: null
size_gb: 0.473
url: "https://huggingface.co/ibm-granite/granite-speech-5.0-470m-turboctc"
description: |
Granite Speech 5.0 TurboCTC is a compact 470 million parameter English ASR model with very high inference speed that is well suited for deployment on laptops, smartphones and other edge devices.
The model consists of a conformer acoustic encoder with block self-attention, self-conditioning and temporal downsampling with an output layer corresponding to 16,384 BPE units.
It was trained on approximately 60,000 hours of English audio from public corpora using Connectionist Temporal Classification (CTC) and inference is done non-autoregressively with greedy decoding.
tags:
- audio
- efficient
- speech
- transcription
- id: granite-speech-5.0-470m-turboctc-nc
family: "Granite Speech"
version: "5.0"
size: 472993792
context_length: 8192
model_type: "Speech"
huggingface_repo: "ibm-granite/granite-speech-5.0-470m-turboctc-nc"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 0
hidden_size: 0
num_attention_heads: 0
num_key_value_heads: 0
head_dim: 0
layer_types:
- kind: full_attention
count: 0
supported_functions:
- Chat
- Transcription
variants:
- format: safetensors
precision: null
size_gb: 0.473
url: "https://huggingface.co/ibm-granite/granite-speech-5.0-470m-turboctc-nc"
description: |
Granite Speech 5.0 TurboCTC Non-commercial is a compact 470 million parameter English ASR model with very high inference speed that is released for research and noncommercial purposes only.
We invite users to refer to [granite-speech-5.0-470m-turboctc](https://huggingface.co/ibm-granite/granite-speech-5.0-470m-turboctc) for other use cases.
tags:
- audio
- efficient
- speech
- transcription
- id: granite-vision-3.1-2b-preview
family: "Granite Vision"
version: "3.1"
size: 2975396928
context_length: 8192
model_type: "Vision"
huggingface_repo: "ibm-granite/granite-vision-3.1-2b-preview"
native_dtype: "unknown"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ImageUnderstanding
- ToolCalling
variants:
- format: safetensors
precision: null
size_gb: 2.975
url: "https://huggingface.co/ibm-granite/granite-vision-3.1-2b-preview"
description: |
granite-vision-3.1-2b-preview is a compact and efficient vision-language model, specifically designed for visual document understanding, enabling automated content extraction from tables, charts, infographics, plots, diagrams, and more. The model was trained on a meticulously curated instruction-following dataset, comprising diverse public datasets and synthetic datasets tailored to support a wide range of document understanding and general image tasks. It was trained by fine-tuning a Granite large language model (https://huggingface.co/ibm-granite/granite-3.1-2b-instruct) with both image and text modalities.
tags:
- efficient
- image
- multimodal
- vision
- id: granite-vision-3.2-2b
family: "Granite Vision"
version: "3.2"
size: 2975396928
context_length: 8192
model_type: "Vision"
huggingface_repo: "ibm-granite/granite-vision-3.2-2b"
native_dtype: "unknown"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ImageUnderstanding
- ToolCalling
variants:
- format: safetensors
precision: null
size_gb: 2.975
url: "https://huggingface.co/ibm-granite/granite-vision-3.2-2b"
- format: MLX
precision: bfloat16
size_gb: 6.152
url: "https://huggingface.co/mlx-community/granite-vision-3.2-2b-bf16"
- format: MLX
precision: 4bit
size_gb: 2.346
url: "https://huggingface.co/mlx-community/granite-vision-3.2-2b-4bit"
- format: MLX
precision: 5bit
size_gb: 2.677
url: "https://huggingface.co/mlx-community/granite-vision-3.2-2b-5bit"
- format: MLX
precision: 6bit
size_gb: 3.008
url: "https://huggingface.co/mlx-community/granite-vision-3.2-2b-6bit"
- format: MLX
precision: 8bit
size_gb: 3.67
url: "https://huggingface.co/mlx-community/granite-vision-3.2-2b-8bit"
- format: MLX
precision: mxfp4
size_gb: 2.346
url: "https://huggingface.co/mlx-community/granite-vision-3.2-2b-mxfp4"
- format: MLX
precision: mxfp8
size_gb: 3.587
url: "https://huggingface.co/mlx-community/granite-vision-3.2-2b-mxfp8"
- format: MLX
precision: nvfp4
size_gb: 2.512
url: "https://huggingface.co/mlx-community/granite-vision-3.2-2b-nvfp4"
- format: Ollama
precision: Q4_K_M
size_gb: 2.4
url: "https://ollama.com/library/granite3.2-vision:2b"
- format: LMStudio
precision: null
size_gb: 1.7
url: "https://lmstudio.ai/models/ibm/granite-vision-3.2-2b"
description: |
granite-vision-3.2-2b is a compact and efficient vision-language model, specifically designed for visual document understanding, enabling automated content extraction from tables, charts, infographics, plots, diagrams, and more. The model was trained on a meticulously curated instruction-following dataset, comprising diverse public datasets and synthetic datasets tailored to support a wide range of document understanding and general image tasks. It was trained by fine-tuning a Granite large language model with both image and text modalities.
tags:
- efficient
- image
- multimodal
- vision
- id: granite-vision-3.3-2b
family: "Granite Vision"
version: "3.3"
size: 2975396928
context_length: 8192
model_type: "Vision"
huggingface_repo: "ibm-granite/granite-vision-3.3-2b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 32
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ImageUnderstanding
- ToolCalling
variants:
- format: GGUF
precision: Q4_K_M
size_gb: 1.545
url: "https://huggingface.co/ibm-granite/granite-vision-3.3-2b-GGUF/blob/main/granite-vision-3.3-2b-Q4_K_M.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 1.805
url: "https://huggingface.co/ibm-granite/granite-vision-3.3-2b-GGUF/blob/main/granite-vision-3.3-2b-Q5_K_M.gguf"
- format: GGUF
precision: Q6_K
size_gb: 2.081
url: "https://huggingface.co/ibm-granite/granite-vision-3.3-2b-GGUF/blob/main/granite-vision-3.3-2b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 2.694
url: "https://huggingface.co/ibm-granite/granite-vision-3.3-2b-GGUF/blob/main/granite-vision-3.3-2b-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 5.069
url: "https://huggingface.co/ibm-granite/granite-vision-3.3-2b-GGUF/blob/main/granite-vision-3.3-2b-bf16.gguf"
- format: GGUF
precision: F16
size_gb: 0.893
url: "https://huggingface.co/ibm-granite/granite-vision-3.3-2b-GGUF/blob/main/mmproj-model-f16.gguf"
- format: safetensors
precision: bfloat16
size_gb: 5.951
url: "https://huggingface.co/ibm-granite/granite-vision-3.3-2b"
- format: Ollama
precision: Q8_0
size_gb: 3.6
url: "https://ollama.com/ibm/granite3.3-vision:2b"
description: |
Granite Vision 3.3 2b for visual analysis and image understanding.
tags:
- efficient
- image
- multimodal
- vision
- id: granite-vision-3.3-2b-embedding
family: "Granite Vision"
version: "3.3"
size: 2975659200
context_length: 8192
model_type: "Vision"
huggingface_repo: "ibm-granite/granite-vision-3.3-2b-embedding"
native_dtype: "float32"
architecture:
num_hidden_layers: 40
hidden_size: 2048
num_attention_heads: 0
num_key_value_heads: 8
head_dim: 0
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ImageUnderstanding
- ToolCalling
variants:
- format: safetensors
precision: float32
size_gb: 11.903
url: "https://huggingface.co/ibm-granite/granite-vision-3.3-2b-embedding"
- format: Ollama
precision: Q8_0
size_gb: 3.6
url: "https://ollama.com/ibm/granite3.3-vision:2b"
description: |
Granite-vision-3.3-2b-embedding is an efficient embedding model based on [granite-vision-3.3-2b](https://huggingface.co/ibm-granite/granite-vision-3.3-2b). This model is specifically designed for multimodal document retrieval, enabling queries on documents with tables, charts, infographics, and complex layouts. The model generates ColBERT-style multi-vector representations of pages.
By removing the need for OCR-based text extractions, granite-vision-3.3-2b-embedding can help simplify and accelerate RAG pipelines.
tags:
- efficient
- image
- multimodal
- vision
- id: granite-vision-4.1-4b
family: "Granite Vision"
version: "4.1"
size: 3997203904
context_length: 8192
model_type: "Vision"
huggingface_repo: "ibm-granite/granite-vision-4.1-4b"
native_dtype: "bfloat16"
architecture:
num_hidden_layers: 40
hidden_size: 2560
num_attention_heads: 40
num_key_value_heads: 8
head_dim: 64
layer_types:
- kind: full_attention
count: 40
supported_functions:
- Chat
- ImageUnderstanding
- ToolCalling
variants:
- format: GGUF
precision: Q4_K_M
size_gb: 2.1
url: "https://huggingface.co/ibm-granite/granite-vision-4.1-4b-GGUF/blob/main/granite-vision-4.1-4b-Q4_K_M.gguf"
- format: GGUF
precision: Q5_K_M
size_gb: 2.437
url: "https://huggingface.co/ibm-granite/granite-vision-4.1-4b-GGUF/blob/main/granite-vision-4.1-4b-Q5_K_M.gguf"
- format: GGUF
precision: Q6_K
size_gb: 2.796
url: "https://huggingface.co/ibm-granite/granite-vision-4.1-4b-GGUF/blob/main/granite-vision-4.1-4b-Q6_K.gguf"
- format: GGUF
precision: Q8_0
size_gb: 3.62
url: "https://huggingface.co/ibm-granite/granite-vision-4.1-4b-GGUF/blob/main/granite-vision-4.1-4b-Q8_0.gguf"
- format: GGUF
precision: BF16
size_gb: 6.81
url: "https://huggingface.co/ibm-granite/granite-vision-4.1-4b-GGUF/blob/main/granite-vision-4.1-4b-bf16.gguf"
- format: GGUF
precision: F16
size_gb: 1.162
url: "https://huggingface.co/ibm-granite/granite-vision-4.1-4b-GGUF/blob/main/mmproj-model-f16.gguf"
- format: safetensors
precision: null
size_gb: 3.997
url: "https://huggingface.co/ibm-granite/granite-vision-4.1-4b"
description: |
Granite Vision 4.1 4B is a vision-language model (VLM) that delivers frontier-level
performance on structured document extraction tasks — chart extraction, table extraction,
and semantic key-value pair extraction — in a compact 4B parameter footprint, providing
a lightweight alternative to much larger frontier models for these tasks:
tags:
- efficient
- image
- multimodal
- vision