tree-sitter-language-pack 1.6.2

Core library for tree-sitter language pack - provides compiled parsers for 305 languages
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182

<!-- Generated by scripts/generate_readme.py. DO NOT EDIT. -->

# tree-sitter-language-pack — Rust

<div align="center" style="display: flex; flex-wrap: wrap; gap: 8px; justify-content: center; margin: 20px 0;">
  <!-- Language Bindings -->  <a href="https://crates.io/crates/tree-sitter-language-pack">
    <img src="https://img.shields.io/crates/v/tree-sitter-language-pack?label=Rust&color=007ec6" alt="Rust">
  </a>  <a href="https://pypi.org/project/tree-sitter-language-pack/">
    <img src="https://img.shields.io/pypi/v/tree-sitter-language-pack?label=Python&color=007ec6" alt="Python">
  </a>  <a href="https://www.npmjs.com/package/@kreuzberg/tree-sitter-language-pack">
    <img src="https://img.shields.io/npm/v/@kreuzberg/tree-sitter-language-pack?label=Node.js&color=007ec6" alt="Node">
  </a>  <a href="https://www.npmjs.com/package/@kreuzberg/tree-sitter-language-pack-wasm">
    <img src="https://img.shields.io/npm/v/@kreuzberg/tree-sitter-language-pack-wasm?label=WASM&color=007ec6" alt="Wasm">
  </a>  <a href="https://central.sonatype.com/artifact/dev.kreuzberg/tree-sitter-language-pack">
    <img src="https://img.shields.io/maven-central/v/dev.kreuzberg/tree-sitter-language-pack?label=Java&color=007ec6" alt="Java">
  </a>  <a href="https://github.com/kreuzberg-dev/tree-sitter-language-pack/tree/main/packages/go">
    <img src="https://img.shields.io/github/v/tag/kreuzberg-dev/tree-sitter-language-pack?label=Go&color=007ec6" alt="Go">
  </a>  <a href="https://www.nuget.org/packages/TreeSitterLanguagePack">
    <img src="https://img.shields.io/nuget/v/TreeSitterLanguagePack?label=C%23&color=007ec6" alt="Csharp">
  </a>  <a href="https://packagist.org/packages/kreuzberg/tree-sitter-language-pack">
    <img src="https://img.shields.io/packagist/v/kreuzberg/tree-sitter-language-pack?label=PHP&color=007ec6" alt="Php">
  </a>  <a href="https://rubygems.org/gems/tree_sitter_language_pack">
    <img src="https://img.shields.io/gem/v/tree_sitter_language_pack?label=Ruby&color=007ec6" alt="Ruby">
  </a>  <a href="https://hex.pm/packages/tree_sitter_language_pack">
    <img src="https://img.shields.io/hexpm/v/tree_sitter_language_pack?label=Elixir&color=007ec6" alt="Elixir">
  </a>  <a href="https://github.com/kreuzberg-dev/tree-sitter-language-pack/pkgs/container/tree-sitter-language-pack">
    <img src="https://img.shields.io/badge/Docker-007ec6?logo=docker&logoColor=white" alt="Docker">
  </a>  <a href="https://github.com/kreuzberg-dev/tree-sitter-language-pack/tree/main/crates/ts-pack-ffi">
    <img src="https://img.shields.io/badge/C-FFI-007ec6" alt="Ffi">
  </a>
  <!-- Project Info -->
  <a href="https://github.com/kreuzberg-dev/tree-sitter-language-pack/actions">
    <img src="https://img.shields.io/github/actions/workflow/status/kreuzberg-dev/tree-sitter-language-pack/ci-rust.yaml?branch=main&label=CI&color=007ec6" alt="CI">
  </a>
  <a href="https://github.com/kreuzberg-dev/tree-sitter-language-pack/blob/main/LICENSE">
    <img src="https://img.shields.io/badge/License-MIT-007ec6.svg" alt="License">
  </a>
  <a href="https://github.com/kreuzberg-dev/homebrew-tap">
    <img src="https://img.shields.io/badge/homebrew-ts--pack-007ec6?logo=homebrew" alt="Homebrew">
  </a>
  <a href="https://docs.tree-sitter-language-pack.kreuzberg.dev">
    <img src="https://img.shields.io/badge/docs-kreuzberg.dev-007ec6" alt="Docs">
  </a>
</div>

<div align="center">
  <img width="3384" height="573" alt="Banner" src="https://github.com/user-attachments/assets/478a83da-237b-446b-b3a8-e564c13e00a8" />
</div>

<div align="center">
  <a href="https://discord.gg/xt9WY3GnKR">
    <img height="22" src="https://img.shields.io/badge/Discord-Join%20our%20community-7289da?logo=discord&logoColor=white" alt="Discord">
  </a>
</div>

Rust core library providing access to 305 tree-sitter parsers with on-demand download and caching support.

## Installation

```sh
cargo add ts-pack-core
```

## Quick Start

```rust
use ts_pack_core::{get_language, get_parser, available_languages};

// Initialize and download specific languages (optional)
ts_pack_core::init(&["python", "javascript", "rust"]).unwrap();

// Get a language (auto-downloads if needed)
let lang = get_language("python").unwrap();

// Get a pre-configured parser
let mut parser = get_parser("python").unwrap();
let tree = parser.parse("def hello(): pass", None).unwrap();
println!("{}", tree.root_node().to_sexp());

// List all available languages
for lang in available_languages() {
    println!("{}", lang);
}

// Process source code (auto-downloads language if needed)
let config = ts_pack_core::ProcessConfig::new("python").all();
let result = ts_pack_core::process("def hello(): pass", &config).unwrap();
println!("Functions: {}", result.structure.len());
println!("Imports: {}", result.imports.len());

// Pre-download languages for offline use
ts_pack_core::download(&["python", "javascript"]).unwrap();

// With chunking
let config = ts_pack_core::ProcessConfig::new("python").all().with_chunking(1000);
let result = ts_pack_core::process(source, &config).unwrap();
println!("Chunks: {}", result.chunks.len());
```

## Features

| Feature | Description |
| --- | --- |
| **305+ Languages** | Pre-compiled parsers for 305+ programming languages |
| **On-Demand Downloads** | Parsers are downloaded on-demand and cached locally for fast reuse |
| **Selective Installation** | Download only the languages you need; unused parsers never downloaded |
| **Polyglot Bindings** | Native bindings for Rust, Python, Node.js, Go, Java, Elixir, and C/C++ |
| **Automatic Caching** | Downloaded parsers cached in platform-specific directories for offline use |
| **CLI Tool** | `ts-pack download` to pre-download parsers for offline/CI/Docker use |

### Downloading Parsers

Parsers are downloaded automatically on first use. For production, CI, or Docker, pre-download them:

```rust
// Download specific languages
ts_pack_core::download(&["python", "javascript", "rust"])?;

// Download all 305 languages
ts_pack_core::download_all()?;
```

Or use the CLI:

```sh
ts-pack download python javascript rust
ts-pack download --all
ts-pack download --groups web,systems
```

### Static Compilation

To statically compile parsers into your binary (instead of downloading at runtime), set the `TSLP_LANGUAGES` environment variable at build time:

```sh
TSLP_LANGUAGES=python,rust,javascript cargo build
```

## API Reference

### Language Discovery

- `available_languages()` -- list all supported language names
- `has_language(name)` -- check if a language is available
- `language_count()` -- total number of supported languages

### Language Detection

- `detect_language(path)` -- detect language from file path
- `detect_language_from_extension(ext)` -- detect language from a bare file extension
- `detect_language_from_path(path)` -- detect language from a file path
- `detect_language_from_content(content)` -- detect language from shebang line
- `extension_ambiguity(ext)` -- check if an extension is ambiguous (returns assigned language + alternatives)

### Parsing

- `get_parser(name)` / `parse_string(source, language)` -- parse source code into a syntax tree

### Download API

- `init(languages)` -- pre-download specific languages for offline use
- `download(languages)` -- download parsers on demand

### Intelligence

- `process(source, config)` -- extract structured analysis (functions, classes, imports, comments, chunks) from source code

### Syntax Highlighting Queries

- `get_highlights_query(language)` -- get bundled highlights.scm query for a language
- `get_injections_query(language)` -- get bundled injections.scm query
- `get_locals_query(language)` -- get bundled locals.scm query

For full documentation, see [kreuzberg.dev](https://docs.tree-sitter-language-pack.kreuzberg.dev).

## License

MIT -- see [LICENSE](https://github.com/kreuzberg-dev/tree-sitter-language-pack/blob/main/LICENSE) for details.

---

Part of [tree-sitter-language-pack](https://github.com/kreuzberg-dev/tree-sitter-language-pack) -- A comprehensive collection of tree-sitter language parsers with polyglot bindings.