Expand description
Code analysis on top of the C++ syntax tree — and on top of cpp_toolchain, which is what reads a file the way a
compiler does.
§What this crate is for
cpp_parser answers “what does this file look like” — a lossless tree of every token, with no opinion about what
any of it means. cpp_toolchain answers “what does the compiler read” — which project this is, which include
paths it has, which file includes which, what the macros are at a position, and what one file looks like once they
are applied. This crate answers “what does any of it mean”: scopes, declarations, types, members, overloads, and
the diagnostics a reader is shown.
§Why the preprocessor is a layer of its own, and why it is a crate
The obvious design — run a preprocessor, then parse its output — is the one a compiler uses, and it is unusable for an editor:
- The tree must stay lossless. Expansion results cannot be written back into the tree without losing the correspondence between a byte of source and a byte of the file. So expansion produces a shadow token stream — the rendering — and the tree is left alone.
- Both branches of
#ifmust stay visible. A compiler picks one; an editor cannot. Code inside#if defined(_WIN32)is code a person is writing, on whatever platform they happen to be on, and making it disappear is the failure mode users notice first. - Every edit would invalidate everything. Expansion is not local — one
#definechanges the meaning of every file that includes it — so doing it before parsing would make the parse itself un-incremental.
It is a separate crate because the direction of the dependency is the design: the preprocessor may not ask this layer what a name means, and while both lived in one crate nothing stopped it. What that buys is the unit the cache is built around — one file’s rendering, keyed by its text and its environment — which is a thing the toolchain can produce without a project, an index, or a session.
§What this layer answers with
a rendering, mapped back -> the file's declarations (index, summary)
a name at an offset -> a declaration, or why not (sema::resolve, index::project)
a call -> the overload, and what it returns (index::project::calls)
a cursor -> members, parameters, a card (completion, inlay, hover)
a project -> an index, its cache, its session (index::store, cache, session)The vocabulary the preprocessor produced — directives, guards, the shadow stream, the rendering — is re-exported
from here (crate::condition, crate::macros, crate::preprocess::cooked, …) so that a consumer of this crate
does not have to know which crate below it defined a type it is looking at.
Re-exports§
pub use cache::align;pub use cache::alloc;pub use cache::env;pub use cache::memory;pub use cache::summary;pub use cache::summary_codec;pub use cache::unit_codec;pub use index::effect;pub use index::surface;pub use session::completion;pub use session::cuda;pub use session::definition;pub use session::doc;pub use session::folding;pub use session::format;pub use session::inlay;pub use session::semantic;pub use session::signature;pub use cache::FORMAT_VERSION;pub use cache::READING_FINGERPRINT;pub use cache::SummaryKey;pub use align::Alignment;pub use align::Difference;pub use align::DiskTexts;pub use align::MAX_EDIT_DISTANCE;pub use align::Reason;pub use align::Report;pub use align::Side;pub use align::StreamToken;pub use align::UnitTexts;pub use align::align;pub use align::files_of;pub use align::normalized;pub use align::tokenize_preprocessed;pub use align::unit_tokens;pub use align::without_pragmas;pub use completion::BUILTIN_TYPES;pub use completion::CompletionContext;pub use completion::CompletionItem;pub use completion::CompletionSet;pub use completion::DIRECTIVES;pub use completion::DirectiveName;pub use completion::Header;pub use completion::HeaderIndex;pub use completion::HeaderKind;pub use completion::ItemKind;pub use completion::KEYWORDS;pub use completion::Keyword;pub use completion::KeywordUse;pub use completion::SNIPPETS;pub use completion::Snippet;pub use completion::completion_at;pub use completion::context_at;pub use completion::detail_of;pub use completion::header_index;pub use completion::why_no_members;pub use sema::check::CheckConfig;pub use sema::check::CheckContext;pub use sema::check::DiagnosticCode;pub use sema::check::Finding;pub use sema::declarations::build_facts;pub use sema::declarations::by_name;pub use sema::declarations::declared_type_of;pub use sema::declarations::scope_of;pub use sema::check;pub use sema::module_info;pub use sema::modules;pub use sema::parser_symbols;pub use sema::scopes;pub use sema::symbol;pub use memory::MemoryReport;pub use summary::ConditionalBody;pub use summary::ClosureEvidence;pub use summary::ConditionAt;pub use summary::ConditionalRegion;pub use summary::CookedDiagnostic;pub use summary::DeclFact;pub use summary::DeclKind;pub use summary::DeclModifiers;pub use summary::FactGuard;pub use summary::FileSummary;pub use summary::GuardBranch;pub use summary::IncludeFact;pub use summary::IndexedRendering;pub use summary::IndexedUnit;pub use summary::MacroAt;pub use summary::MacroDefinitions;pub use summary::MacroFact;pub use summary::MacroKind;pub use summary::MacroView;pub use summary::MapReport;pub use summary::OutlineSymbol;pub use summary::ReflectionFact;pub use summary::RenderedUnit;pub use summary::SummaryGuards;pub use summary::TranslationUnit;pub use summary::UnitDefinitions;pub use summary::UnitReading;pub use summary::UnitSpan;pub use summary::macros_from_direct_includes;pub use summary::macros_from_direct_includes_with_bodies;pub use summary::macros_from_the_closure_with_bodies;pub use summary::macros_in_force_before_the_include;pub use summary::outline_of;pub use file::FileAnalysis;pub use file::FileTokens;pub use index::walk_cache::WalkCache;pub use index::ChangeBatch;pub use index::CookedFile;pub use index::EventKind;pub use index::FileEvent;pub use index::FileIndexer;pub use index::FileReferences;pub use index::HeaderTarget;pub use index::IncludeBudget;pub use index::IncludeIndex;pub use index::IncludeVisibility;pub use index::MacroReferences;pub use index::MemberCompletions;pub use index::MemberList;pub use index::NameCompletions;pub use index::NameProvenance;pub use index::NotIndexed;pub use index::NotIndexedReason;pub use index::OfferedName;pub use index::PathPattern;pub use index::Priority;pub use index::ProjectDefinition;pub use index::ProjectDefinitions;pub use index::ProjectIndex;pub use index::ProjectMacro;pub use index::ProjectMember;pub use index::ProjectSymbol;pub use index::Reference;pub use index::ReferenceBudget;pub use index::ReferenceKind;pub use index::Rename;pub use index::Response;pub use index::Step;pub use index::StepOutcome;pub use index::StoreStats;pub use index::SummaryReadError;pub use index::SummaryStore;pub use index::SymbolReferences;pub use index::SymbolToFind;pub use index::UnlistedBase;pub use index::UnresolvedEdge;pub use index::VisibleDeclaration;pub use index::WatchFilter;pub use index::Worklist;pub use index::definition_across_files;pub use index::header_at;pub use index::macro_across_files;pub use index::macro_references;pub use index::member_across_files;pub use index::member_completions_at;pub use index::members_of;pub use index::name_completions_at;pub use index::outcome_of;pub use index::summarize;pub use index::symbol_references;pub use module_info::ImportDeclaration;pub use module_info::ImportTarget;pub use module_info::ModuleInfo;pub use module_info::ModuleUnit;pub use modules::ImportEdge;pub use modules::ImportOutcome;pub use modules::MAX_IMPORT_DEPTH;pub use modules::ModuleGraph;pub use modules::ModuleScanner;pub use modules::ModuleUnitEntry;pub use modules::scan_imports;pub use summary::ModuleReading;pub use scopes::build_scopes;pub use scopes::declared_module_names;pub use inlay::ParameterHint;pub use inlay::TypeHint;pub use inlay::TypeHintKind;pub use session::CookedReading;pub use session::DiagnosticReading;pub use session::ExpressionType;pub use session::FileDiagnostic;pub use session::FileDiagnostics;pub use session::ModuleImport;pub use session::ModuleNote;pub use session::ModuleUnitTarget;pub use session::OpenDocuments;pub use session::Session;pub use session::SessionFiles;pub use session::UnitStats;pub use session::WrittenAttribute;pub use file::vfs::Vfs;pub use file::vfs::VfsFile;pub use file::view::FileView;pub use symbol::Binding;pub use symbol::BindingKind;pub use symbol::BindingOrigin;pub use symbol::DeclName;pub use symbol::HeaderName;pub use symbol::Known;pub use symbol::MaybeName;pub use symbol::Name;pub use symbol::NameKind;pub use symbol::QualifiedName;pub use symbol::Scope;pub use symbol::ScopeId;pub use symbol::ScopeKind;pub use symbol::ScopeTree;pub use symbol::UnknownReason;
Modules§
- cache
- The on-disk cache: one file per file, keyed by a hash of everything that determines a summary.
- condition
#if, evaluated the way a preprocessor evaluates it — including the parts that are not obvious.- config
- Where the files are, and what the compiler was told about them.
- cooked
- The cooked token stream: what a compiler would hand to its parser.
- directive
- Preprocessor directives, structured.
- expand
- Macro expansion: a shadow token stream, with every token knowing where it really came from.
- file
- One file, analysed: its tokens, its macros, its conditionals.
- graph
- Following includes: the dependency graph, and the visits that build it.
- guard
- Guards: which conditional regions a piece of code sits inside, and whether it is being compiled.
- guards
- Include guards: recognising that a header protects itself, and remembering what a header did to the macro table.
- include
- Finding the file an
#includenames. - index
- Building one file’s
FileSummary— the join nothing else in this crate performs. - macros
- A
#define: the macro’s name, its parameters, and its replacement tokens. - msvc
- Microsoft’s toolchain: where it is, what its
INCLUDEis, and what it predefines. - paths
- Paths, file identities, and reading files — behind a trait so that all of it is testable.
- preprocess
- One traversal of a file that produces everything the layers above need.
- probes
- Every diagnostic probe in this crate, and the one place each is read from the environment.
- project
- What this project is, and how it is built
- sema
- Names, scopes and modules: what the parsed text declares, and what a name written in it refers to.
- session
- The driver: one open project, with the layers below joined and a queue that notifications re-seed.
- system_
headers - Where a system’s headers are, when no compiler can be asked.
- token
- A token as the analysis layer sees it: owned text, file coordinates, and nothing else.
- toolchain
- Finding the compiler, and asking it where its own headers are.
Structs§
- Branch
- One branch of a conditional region.
- Build
Statement - What the project says about how this file is built — the two statements a compiler has to be asked about, and the reason they travel together.
- Cached
Files - One read of each file per run — the layer every other layer’s text comes from.
- Command
Line Macro - A macro the compiler was given on the command line.
- Compile
Command - One entry of a
compile_commands.json. - Compile
Commands - A parsed compile database, and what was skipped.
- Compile
Section [compile]— how this project is built, when the discovered answer is not the right one.- Compiler
Config - What the compiler was told.
- Config
Problem - Config
Report - A configuration file, read: what it said, where it was, and everything wrong with it.
- Configuration
- What a file’s includes contribute, as the table
cook_withstarts from. - Cooked
Stream - One file, preprocessed: the tokens a compiler would parse, and what was removed on the way.
- Database
- A compile database, and how it was found.
- Define
- A
#define. - Diagnostic
- A note, and where it happened.
- Diagnostics
Section [diagnostics]— read by the language server.- Discovery
Problem - One thing about a project’s discovery that a person should be told.
- Disk
Commands - The real thing.
- Disk
Files - The real filesystem.
- Edge
- An
#includethat resolved. - Environment
- The environment a compiler is looked for in, as data.
- Expanded
Token - A token in an expanded stream.
- Expansion
- The result of expanding a token stream.
- File
Entry - A file the walk reached.
- File
Graph - The result of walking one translation unit.
- File
Guard - What analysing a header did to the macro table, so a second visit can be decided.
- FileId
- A file’s identity within one analysis.
- File
Macros - What one file of a walked unit starts with, as the cooker asks it: the unit’s definitions as that file sees them, with the compilation’s own builtins underneath.
- File
Preprocessing - Everything one pass over a file produces.
- Format
Strings Section [format_strings]— where this project’s format strings are, read by the language server.- Formatting
Section [format]— the formatter this project uses. SeeDEFAULT_FORMATTERfor whatformattermeans and why.- Guard
- The conditional regions an item sits inside, outermost first.
- Guard
Analysis - What was found in one file’s directives.
- Guard
Stack - Builds guards by walking a file’s directives in order.
- Hover
Section [hover]— read by the language server.- Include
- An
#includeor#include_next. - Include
Path - An include search directory.
- Include
Resolver - Resolves includes against a provider and a configuration.
- Index
Section [index]— the cache and the scan.- Macro
Body - The replacement list of a macro, in the form expansion needs it.
- Macro
Def - A macro definition.
- Macro
Invocation - One macro call that was expanded.
- Macro
Table - The macros visible at one point in a file.
- Marked
- The macro names in force at a point in a walk.
- Memory
Files - Files held in memory.
- NoMacro
Bodies - The answer for a caller with no macro evidence at all — a buffer parsed on its own, or a test that is about something else.
- NoMacros
- A table that defines nothing, so every identifier is
0. - Output
- What a program wrote to its two streams.
- Over
- Two layers of starting state, newest first — a materialised environment over the compiler’s builtins.
- Overlay
Files - One provider in front of another: open documents first, then the disk.
- Parameter
- One declared parameter of a function-like macro.
- Parsed
Definitions - A
#define, parsed once per run rather than once per file that sees it. - Path
Interner - Mints a
FileIdper distinct path, and remembers which path that was. - Positional
Macros - A macro table that answers as of one offset.
- Project
Config - Everything
.cppls.tomlcan say, after parsing. - Project
Discovery - Everything that can be worked out about a project without asking a compiler.
- Region
- One conditional region: the chain of branches written for a single
#if. - Rendered
Cooked - A cooked stream spelled out as text, and where each token in it came from.
- Rendered
Span - One token’s place in the rendering, the place it was written, and the place to act on it.
- Resolved
- A resolved include.
- Spanned
Directive - A directive together with where it is.
- Token
- One token, with its text detached from the tree.
- Toolchain
- A compiler that answered, and what it said.
- Unresolved
- An include that could not be resolved, and what was tried.
- Unresolved
Include - An
#includethat did not resolve, and where it was written. - Workspace
Section [workspace]— what the project is made of.
Enums§
- Condition
Expr - One node of a parsed condition.
- Database
Origin - Where the database was, which is the first thing to check when it is the wrong one.
- Directive
- One directive, with its arguments read out.
- Directive
Kind - The directive’s name, as a typed value.
- Eval
Error - Why a condition could not be parsed.
- Expansion
Note - Why the expander declined to expand something.
- Format
Syntax - The three grammars a format string can be written in, as the configuration spells them.
- FoundIn
- Where in the search order a file was found.
- Function
Entry - One of a project’s own format functions — the syntax it takes, and which argument is the format string.
- Include
Form - How an
#includespelled its target. - Lookup
- What a table has to say about one name.
- Origin
- Where a token in an expanded stream came from.
- Parameter
Kind - Resolution
- The outcome of resolving an include.
- Severity
- How bad a configuration problem is.
- Skip
Reason - Why an include was not followed.
- Toolchain
Source - How a toolchain was found, strongest first.
- Value
- The value of a conditional expression, or the reason there is not one.
- Visibility
- Whether the code inside a guard is being compiled.
- Visit
- What happened at the other end of an include.
- Walk
Scope - How much of the project a walk is allowed to read.
Constants§
- CACHE_
DIRECTORY - The directory name this crate keeps its cache in, under the project root.
- CONFIGURATION_
FILE_ NAMES - The file names that configure an analysis rather than being part of it.
- DEFAULT_
FORMATTER [format]— which formattertextDocument/formattingruns, read by the language server.- MAX_
INCLUDE_ DEPTH - How deep includes are followed before the walk gives up.
- PROJECT_
CONFIG_ FILE - The name a project’s configuration is conventionally found under, in the workspace root.
Traits§
- Command
Runner - Runs a program. The second thing in this crate that touches the world outside it.
- File
Provider - Reads files. The only thing in this layer that touches a filesystem.
- Macro
Bindings - What a cook reads its macros from — the definitions a file starts with, asked by name and offset.
- Macro
Bodies - What a name in a macro’s body expands to, for the one question a shape reader has to ask about it.
- Macro
Values - Look up macros while evaluating.
- Unit
Sources - Where the text of a unit’s files comes from — everything a one-walk cook needs from the world.
Functions§
- analyse_
guards - Recognise a file’s guard and collect the macros it defines.
- config_
from_ arguments - Read a compiler configuration out of a list of arguments.
- configuration_
from_ environment - Build the table a level-2 cook starts from, out of what the includes contribute.
- configuration_
from_ environment_ and configuration_from_environment_with, with the run’s definition cache.- configuration_
from_ environment_ with configuration_from_environment, with a decision the measurement left open.- content_
hash - A file’s text as one number — the content half of a key.
- cook
- Cook one file, with nothing defined before it — the architecture’s level 0.
- cook_
with - Cook one file with the macros a configuration starts it with.
- detect_
guard - Recognise a file’s guard, given its directives in source order.
- discover
- Find a compiler and ask it. The one call a caller needs.
- discover_
with discoverwith compilers the project named, tried before the environment’s.- evaluate
- Parse and evaluate in one step.
- expand
- Expand the macros in a token stream.
- expand_
with_ budget - Expand with no limit on the number of tokens, for callers that pass their own bound.
- file_
only - Analyse one file and nothing it includes.
- find_
compiler - The compiler to ask, or
Nonewhen nothing usable was found. - find_
database - fnv1a64
- FNV-1a over bytes — the hash a cache key is built from.
- join_
normalized - Join a directory and a relative path, then normalize.
- load_
config - Read a project’s configuration from a provider.
- normalize_
path - Normalize a path for comparison: separators unified,
.and..resolved, case folded if asked. - parent_
normalized - The directory a path lives in, normalized, or an empty path when there is none.
- parse_
compile_ commands - Read a
compile_commands.json. - parse_
condition - Parse a condition from the tokens that follow
#ifor#elif. - parse_
config - Read a configuration file’s text.
- parse_
directive_ tokens - Read one directive from its tokens, including the leading
#. - parse_
search_ list - The directories a compiler listed between its markers.
- parse_
version - The compiler’s version line, when it printed one this can recognise.
- predefined_
macros_ of - The macros a configuration decides by itself — before a single file is read, and whether or not a compiler was found.
- preprocess
- Read a file’s directives, macros, and conditionals in one pass.
- project_
config_ from_ flags - Apply a project’s
[compile]section to a list of flags and read the result. - scan_
directives - Read every directive in a file, in source order.
- search_
paths - split_
command_ line - Split a shell command line into arguments.
- walk
- Follow a translation unit’s includes.
- walk_
scoped - Follow a translation unit’s includes, to the extent
scopeallows.