Skip to main content

Crate cpp_code_analysis

Crate cpp_code_analysis 

Source
Expand description

Code analysis on top of the C++ syntax tree — and on top of cpp_toolchain, which is what reads a file the way a compiler does.

§What this crate is for

cpp_parser answers “what does this file look like” — a lossless tree of every token, with no opinion about what any of it means. cpp_toolchain answers “what does the compiler read” — which project this is, which include paths it has, which file includes which, what the macros are at a position, and what one file looks like once they are applied. This crate answers “what does any of it mean”: scopes, declarations, types, members, overloads, and the diagnostics a reader is shown.

§Why the preprocessor is a layer of its own, and why it is a crate

The obvious design — run a preprocessor, then parse its output — is the one a compiler uses, and it is unusable for an editor:

  • The tree must stay lossless. Expansion results cannot be written back into the tree without losing the correspondence between a byte of source and a byte of the file. So expansion produces a shadow token stream — the rendering — and the tree is left alone.
  • Both branches of #if must stay visible. A compiler picks one; an editor cannot. Code inside #if defined(_WIN32) is code a person is writing, on whatever platform they happen to be on, and making it disappear is the failure mode users notice first.
  • Every edit would invalidate everything. Expansion is not local — one #define changes the meaning of every file that includes it — so doing it before parsing would make the parse itself un-incremental.

It is a separate crate because the direction of the dependency is the design: the preprocessor may not ask this layer what a name means, and while both lived in one crate nothing stopped it. What that buys is the unit the cache is built around — one file’s rendering, keyed by its text and its environment — which is a thing the toolchain can produce without a project, an index, or a session.

§What this layer answers with

a rendering, mapped back      ->  the file's declarations            (index, summary)
a name at an offset           ->  a declaration, or why not          (sema::resolve, index::project)
a call                        ->  the overload, and what it returns  (index::project::calls)
a cursor                      ->  members, parameters, a card        (completion, inlay, hover)
a project                     ->  an index, its cache, its session   (index::store, cache, session)

The vocabulary the preprocessor produced — directives, guards, the shadow stream, the rendering — is re-exported from here (crate::condition, crate::macros, crate::preprocess::cooked, …) so that a consumer of this crate does not have to know which crate below it defined a type it is looking at.

Re-exports§

pub use cache::align;
pub use cache::alloc;
pub use cache::env;
pub use cache::memory;
pub use cache::summary;
pub use cache::summary_codec;
pub use cache::unit_codec;
pub use index::effect;
pub use index::surface;
pub use session::completion;
pub use session::cuda;
pub use session::definition;
pub use session::doc;
pub use session::folding;
pub use session::format;
pub use session::inlay;
pub use session::semantic;
pub use session::signature;
pub use cache::FORMAT_VERSION;
pub use cache::READING_FINGERPRINT;
pub use cache::SummaryKey;
pub use align::Alignment;
pub use align::Difference;
pub use align::DiskTexts;
pub use align::MAX_EDIT_DISTANCE;
pub use align::Reason;
pub use align::Report;
pub use align::Side;
pub use align::StreamToken;
pub use align::UnitTexts;
pub use align::align;
pub use align::files_of;
pub use align::normalized;
pub use align::shared;
pub use align::tokenize_preprocessed;
pub use align::unit_tokens;
pub use align::without_pragmas;
pub use completion::BUILTIN_TYPES;
pub use completion::CompletionContext;
pub use completion::CompletionItem;
pub use completion::CompletionSet;
pub use completion::DIRECTIVES;
pub use completion::DirectiveName;
pub use completion::Header;
pub use completion::HeaderIndex;
pub use completion::HeaderKind;
pub use completion::ItemKind;
pub use completion::KEYWORDS;
pub use completion::Keyword;
pub use completion::KeywordUse;
pub use completion::SNIPPETS;
pub use completion::SharedHeaders;
pub use completion::Snippet;
pub use completion::completion_at;
pub use completion::context_at;
pub use completion::detail_of;
pub use completion::header_index;
pub use completion::why_no_members;
pub use sema::check::CheckConfig;
pub use sema::check::CheckContext;
pub use sema::check::DiagnosticCode;
pub use sema::check::Finding;
pub use sema::declarations::build_facts;
pub use sema::declarations::by_name;
pub use sema::declarations::declared_type_of;
pub use sema::declarations::scope_of;
pub use sema::check;
pub use sema::module_info;
pub use sema::modules;
pub use sema::parser_symbols;
pub use sema::scopes;
pub use sema::symbol;
pub use memory::MemoryReport;
pub use summary::ConditionalBody;
pub use summary::ClosureEvidence;
pub use summary::ConditionAt;
pub use summary::ConditionalRegion;
pub use summary::CookedDiagnostic;
pub use summary::DeclFact;
pub use summary::DeclKind;
pub use summary::DeclModifiers;
pub use summary::FactGuard;
pub use summary::FileSummary;
pub use summary::GuardBranch;
pub use summary::IncludeFact;
pub use summary::IndexedRendering;
pub use summary::IndexedUnit;
pub use summary::MacroAt;
pub use summary::MacroDefinitions;
pub use summary::MacroFact;
pub use summary::MacroKind;
pub use summary::MacroView;
pub use summary::MapReport;
pub use summary::OutlineSymbol;
pub use summary::ReflectionFact;
pub use summary::RenderedUnit;
pub use summary::SummaryGuards;
pub use summary::TranslationUnit;
pub use summary::UnitDefinitions;
pub use summary::UnitReading;
pub use summary::UnitSpan;
pub use summary::macros_from_direct_includes;
pub use summary::macros_from_direct_includes_with_bodies;
pub use summary::macros_from_the_closure_with_bodies;
pub use summary::macros_in_force_before_the_include;
pub use summary::outline_of;
pub use file::FileAnalysis;
pub use file::FileTokens;
pub use index::walk_cache::WalkCache;
pub use index::ChangeBatch;
pub use index::CookedFile;
pub use index::EventKind;
pub use index::FileEvent;
pub use index::FileIndexer;
pub use index::FileReferences;
pub use index::HeaderTarget;
pub use index::IncludeBudget;
pub use index::IncludeIndex;
pub use index::IncludeVisibility;
pub use index::MacroReferences;
pub use index::MemberCompletions;
pub use index::MemberList;
pub use index::NameCompletions;
pub use index::NameProvenance;
pub use index::NotIndexed;
pub use index::NotIndexedReason;
pub use index::OfferedName;
pub use index::PathPattern;
pub use index::Priority;
pub use index::ProjectDefinition;
pub use index::ProjectDefinitions;
pub use index::ProjectIndex;
pub use index::ProjectMacro;
pub use index::ProjectMember;
pub use index::ProjectSymbol;
pub use index::Reference;
pub use index::ReferenceBudget;
pub use index::ReferenceKind;
pub use index::Rename;
pub use index::Response;
pub use index::Step;
pub use index::StepOutcome;
pub use index::StoreStats;
pub use index::SummaryReadError;
pub use index::SummaryStore;
pub use index::SymbolReferences;
pub use index::SymbolToFind;
pub use index::UnlistedBase;
pub use index::UnresolvedEdge;
pub use index::VisibleDeclaration;
pub use index::WatchFilter;
pub use index::Worklist;
pub use index::definition_across_files;
pub use index::header_at;
pub use index::macro_across_files;
pub use index::macro_references;
pub use index::member_across_files;
pub use index::member_completions_at;
pub use index::members_of;
pub use index::name_completions_at;
pub use index::outcome_of;
pub use index::summarize;
pub use index::symbol_references;
pub use module_info::ImportDeclaration;
pub use module_info::ImportTarget;
pub use module_info::ModuleInfo;
pub use module_info::ModuleUnit;
pub use modules::ImportEdge;
pub use modules::ImportOutcome;
pub use modules::MAX_IMPORT_DEPTH;
pub use modules::ModuleGraph;
pub use modules::ModuleScanner;
pub use modules::ModuleUnitEntry;
pub use modules::scan_imports;
pub use summary::ModuleReading;
pub use scopes::build_scopes;
pub use scopes::declared_module_names;
pub use inlay::ParameterHint;
pub use inlay::TypeHint;
pub use inlay::TypeHintKind;
pub use session::CookedReading;
pub use session::DiagnosticReading;
pub use session::ExpressionType;
pub use session::FileDiagnostic;
pub use session::FileDiagnostics;
pub use session::ModuleImport;
pub use session::ModuleNote;
pub use session::ModuleUnitTarget;
pub use session::OpenDocuments;
pub use session::Session;
pub use session::SessionFiles;
pub use session::UnitStats;
pub use session::WrittenAttribute;
pub use file::vfs::Vfs;
pub use file::vfs::VfsFile;
pub use file::view::FileView;
pub use symbol::Binding;
pub use symbol::BindingKind;
pub use symbol::BindingOrigin;
pub use symbol::DeclName;
pub use symbol::HeaderName;
pub use symbol::Known;
pub use symbol::MaybeName;
pub use symbol::Name;
pub use symbol::NameKind;
pub use symbol::QualifiedName;
pub use symbol::Scope;
pub use symbol::ScopeId;
pub use symbol::ScopeKind;
pub use symbol::ScopeTree;
pub use symbol::UnknownReason;

Modules§

cache
The on-disk cache: one file per file, keyed by a hash of everything that determines a summary.
condition
#if, evaluated the way a preprocessor evaluates it — including the parts that are not obvious.
config
Where the files are, and what the compiler was told about them.
cooked
The cooked token stream: what a compiler would hand to its parser.
directive
Preprocessor directives, structured.
expand
Macro expansion: a shadow token stream, with every token knowing where it really came from.
file
One file, analysed: its tokens, its macros, its conditionals.
graph
Following includes: the dependency graph, and the visits that build it.
guard
Guards: which conditional regions a piece of code sits inside, and whether it is being compiled.
guards
Include guards: recognising that a header protects itself, and remembering what a header did to the macro table.
include
Finding the file an #include names.
index
Building one file’s FileSummary — the join nothing else in this crate performs.
macros
A #define: the macro’s name, its parameters, and its replacement tokens.
msvc
Microsoft’s toolchain: where it is, what its INCLUDE is, and what it predefines.
paths
Paths, file identities, and reading files — behind a trait so that all of it is testable.
preprocess
One traversal of a file that produces everything the layers above need.
probes
Every diagnostic probe in this crate, and the one place each is read from the environment.
project
What this project is, and how it is built
sema
Names, scopes and modules: what the parsed text declares, and what a name written in it refers to.
session
The driver: one open project, with the layers below joined and a queue that notifications re-seed.
system_headers
Where a system’s headers are, when no compiler can be asked.
token
A token as the analysis layer sees it: owned text, file coordinates, and nothing else.
toolchain
Finding the compiler, and asking it where its own headers are.

Structs§

Branch
One branch of a conditional region.
BuildStatement
What the project says about how this file is built — the two statements a compiler has to be asked about, and the reason they travel together.
CachedFiles
One read of each file per run — the layer every other layer’s text comes from.
CommandLineMacro
A macro the compiler was given on the command line.
CompileCommand
One entry of a compile_commands.json.
CompileCommands
A parsed compile database, and what was skipped.
CompileSection
[compile] — how this project is built, when the discovered answer is not the right one.
CompilerConfig
What the compiler was told.
ConfigProblem
ConfigReport
A configuration file, read: what it said, where it was, and everything wrong with it.
Configuration
What a file’s includes contribute, as the table cook_with starts from.
CookedStream
One file, preprocessed: the tokens a compiler would parse, and what was removed on the way.
Database
A compile database, and how it was found.
Define
A #define.
Diagnostic
A note, and where it happened.
DiagnosticsSection
[diagnostics] — read by the language server.
DiscoveryProblem
One thing about a project’s discovery that a person should be told.
DiskCommands
The real thing.
DiskFiles
The real filesystem.
Edge
An #include that resolved.
Environment
The environment a compiler is looked for in, as data.
ExpandedToken
A token in an expanded stream.
Expansion
The result of expanding a token stream.
FileEntry
A file the walk reached.
FileGraph
The result of walking one translation unit.
FileGuard
What analysing a header did to the macro table, so a second visit can be decided.
FileId
A file’s identity within one analysis.
FileMacros
What one file of a walked unit starts with, as the cooker asks it: the unit’s definitions as that file sees them, with the compilation’s own builtins underneath.
FilePreprocessing
Everything one pass over a file produces.
FormatStringsSection
[format_strings] — where this project’s format strings are, read by the language server.
FormattingSection
[format] — the formatter this project uses. See DEFAULT_FORMATTER for what formatter means and why.
Guard
The conditional regions an item sits inside, outermost first.
GuardAnalysis
What was found in one file’s directives.
GuardStack
Builds guards by walking a file’s directives in order.
HoverSection
[hover] — read by the language server.
Include
An #include or #include_next.
IncludePath
An include search directory.
IncludeResolver
Resolves includes against a provider and a configuration.
IndexSection
[index] — the cache and the scan.
MacroBody
The replacement list of a macro, in the form expansion needs it.
MacroDef
A macro definition.
MacroInvocation
One macro call that was expanded.
MacroTable
The macros visible at one point in a file.
Marked
The macro names in force at a point in a walk.
MemoryFiles
Files held in memory.
NoMacroBodies
The answer for a caller with no macro evidence at all — a buffer parsed on its own, or a test that is about something else.
NoMacros
A table that defines nothing, so every identifier is 0.
Output
What a program wrote to its two streams.
Over
Two layers of starting state, newest first — a materialised environment over the compiler’s builtins.
OverlayFiles
One provider in front of another: open documents first, then the disk.
Parameter
One declared parameter of a function-like macro.
ParsedDefinitions
A #define, parsed once per run rather than once per file that sees it.
PathInterner
Mints a FileId per distinct path, and remembers which path that was.
PositionalMacros
A macro table that answers as of one offset.
ProjectConfig
Everything .cppls.toml can say, after parsing.
ProjectDiscovery
Everything that can be worked out about a project without asking a compiler.
Region
One conditional region: the chain of branches written for a single #if.
RenderedCooked
A cooked stream spelled out as text, and where each token in it came from.
RenderedSpan
One token’s place in the rendering, the place it was written, and the place to act on it.
Resolved
A resolved include.
SpannedDirective
A directive together with where it is.
Token
One token, with its text detached from the tree.
Toolchain
A compiler that answered, and what it said.
Unresolved
An include that could not be resolved, and what was tried.
UnresolvedInclude
An #include that did not resolve, and where it was written.
WorkspaceSection
[workspace] — what the project is made of.

Enums§

ConditionExpr
One node of a parsed condition.
DatabaseOrigin
Where the database was, which is the first thing to check when it is the wrong one.
Directive
One directive, with its arguments read out.
DirectiveKind
The directive’s name, as a typed value.
EvalError
Why a condition could not be parsed.
ExpansionNote
Why the expander declined to expand something.
FormatSyntax
The three grammars a format string can be written in, as the configuration spells them.
FoundIn
Where in the search order a file was found.
FunctionEntry
One of a project’s own format functions — the syntax it takes, and which argument is the format string.
IncludeForm
How an #include spelled its target.
Lookup
What a table has to say about one name.
Origin
Where a token in an expanded stream came from.
ParameterKind
Resolution
The outcome of resolving an include.
Severity
How bad a configuration problem is.
SkipReason
Why an include was not followed.
ToolchainSource
How a toolchain was found, strongest first.
Value
The value of a conditional expression, or the reason there is not one.
Visibility
Whether the code inside a guard is being compiled.
Visit
What happened at the other end of an include.
WalkScope
How much of the project a walk is allowed to read.

Constants§

CACHE_DIRECTORY
The directory name this crate keeps its cache in, under the project root.
CONFIGURATION_FILE_NAMES
The file names that configure an analysis rather than being part of it.
DEFAULT_FORMATTER
[format] — which formatter textDocument/formatting runs, read by the language server.
MAX_INCLUDE_DEPTH
How deep includes are followed before the walk gives up.
PROJECT_CONFIG_FILE
The name a project’s configuration is conventionally found under, in the workspace root.

Traits§

CommandRunner
Runs a program. The second thing in this crate that touches the world outside it.
FileProvider
Reads files. The only thing in this layer that touches a filesystem.
MacroBindings
What a cook reads its macros from — the definitions a file starts with, asked by name and offset.
MacroBodies
What a name in a macro’s body expands to, for the one question a shape reader has to ask about it.
MacroValues
Look up macros while evaluating.
UnitSources
Where the text of a unit’s files comes from — everything a one-walk cook needs from the world.

Functions§

analyse_guards
Recognise a file’s guard and collect the macros it defines.
config_from_arguments
Read a compiler configuration out of a list of arguments.
configuration_from_environment
Build the table a level-2 cook starts from, out of what the includes contribute.
configuration_from_environment_and
configuration_from_environment_with, with the run’s definition cache.
configuration_from_environment_with
configuration_from_environment, with a decision the measurement left open.
content_hash
A file’s text as one number — the content half of a key.
cook
Cook one file, with nothing defined before it — the architecture’s level 0.
cook_with
Cook one file with the macros a configuration starts it with.
detect_guard
Recognise a file’s guard, given its directives in source order.
discover
Find a compiler and ask it. The one call a caller needs.
discover_with
discover with compilers the project named, tried before the environment’s.
evaluate
Parse and evaluate in one step.
expand
Expand the macros in a token stream.
expand_with_budget
Expand with no limit on the number of tokens, for callers that pass their own bound.
file_only
Analyse one file and nothing it includes.
find_compiler
The compiler to ask, or None when nothing usable was found.
find_database
fnv1a64
FNV-1a over bytes — the hash a cache key is built from.
join_normalized
Join a directory and a relative path, then normalize.
load_config
Read a project’s configuration from a provider.
normalize_path
Normalize a path for comparison: separators unified, . and .. resolved, case folded if asked.
parent_normalized
The directory a path lives in, normalized, or an empty path when there is none.
parse_compile_commands
Read a compile_commands.json.
parse_condition
Parse a condition from the tokens that follow #if or #elif.
parse_config
Read a configuration file’s text.
parse_directive_tokens
Read one directive from its tokens, including the leading #.
parse_search_list
The directories a compiler listed between its markers.
parse_version
The compiler’s version line, when it printed one this can recognise.
predefined_macros_of
The macros a configuration decides by itself — before a single file is read, and whether or not a compiler was found.
preprocess
Read a file’s directives, macros, and conditionals in one pass.
project_config_from_flags
Apply a project’s [compile] section to a list of flags and read the result.
scan_directives
Read every directive in a file, in source order.
search_paths
split_command_line
Split a shell command line into arguments.
walk
Follow a translation unit’s includes.
walk_scoped
Follow a translation unit’s includes, to the extent scope allows.