Expand description
Code analysis on top of the C++ syntax tree.
§What this crate is for
cpp_parser answers “what does this file look like” — a lossless tree of every token, with no
opinion about what any of it means. This crate answers “what does it mean”, which in C++ starts
with the preprocessor: the text a compiler sees is not the text in the file.
§Why the preprocessor is a separate layer over the tree, not a pass before it
The obvious design — run a preprocessor, then parse its output — is the one a compiler uses, and it is unusable for an editor:
- The tree must stay lossless. Expansion results cannot be written back into the tree without losing the correspondence between a byte of source and a byte of the file. So expansion produces a shadow token stream and the tree is left alone.
- Both branches of
#ifmust stay visible. A compiler picks one; an editor cannot. Code inside#if defined(_WIN32)is code a person is writing, on whatever platform they happen to be on, and making it disappear is the failure mode users notice first. - Every edit would invalidate everything. Expansion is not local — one
#definechanges the meaning of every file that includes it — so doing it before parsing would make the parse itself un-incremental.
§What this layer produces
PreprocessorDirective nodes -> typed Directive values (directive.rs)
#define -> MacroDef, token sequence (macros.rs)
#if / #elif -> Guard conditions over macros (condition.rs, guard.rs)
a macro call -> a shadow token stream, every
token carrying its origin (expand.rs)A Guard travels with everything derived from a conditional region, so that a consumer can ask
“is this code even being compiled?” and get Active, Inactive, or — the common answer —
Unknown, because #if _MSC_VER > 1900 cannot be decided on a machine that is not MSVC.
An ExpandedToken travels with a Origin, so that “where is this?” has an answer even when the
token was written in a different file: navigation reaches the macro, diagnostics reach the call site.
Re-exports§
pub use cache::CACHE_DIRECTORY;pub use cache::FORMAT_VERSION;pub use cache::READING_FINGERPRINT;pub use cache::SummaryKey;pub use cache::content_hash;pub use cache::fnv1a64;pub use align::Alignment;pub use align::Difference;pub use align::DiskTexts;pub use align::MAX_EDIT_DISTANCE;pub use align::Reason;pub use align::Report;pub use align::Side;pub use align::StreamToken;pub use align::UnitTexts;pub use align::align;pub use align::files_of;pub use align::normalized;pub use align::tokenize_preprocessed;pub use align::unit_tokens;pub use align::without_pragmas;pub use completion::BUILTIN_TYPES;pub use completion::CompletionContext;pub use completion::CompletionItem;pub use completion::CompletionSet;pub use completion::DIRECTIVES;pub use completion::DirectiveName;pub use completion::Header;pub use completion::HeaderIndex;pub use completion::HeaderKind;pub use completion::ItemKind;pub use completion::KEYWORDS;pub use completion::Keyword;pub use completion::KeywordUse;pub use completion::SNIPPETS;pub use completion::Snippet;pub use completion::completion_at;pub use completion::context_at;pub use completion::detail_of;pub use completion::header_index;pub use completion::why_no_members;pub use cooked::FileMacros;pub use cooked::Over;pub use cooked::ParsedDefinitions;pub use cooked::configuration_from_environment_and;pub use file::token;pub use include::config;pub use include::graph;pub use include::msvc;pub use include::paths;pub use include::system_headers;pub use include::toolchain;pub use macros::MacroBindings;pub use macros::UnitSources;pub use preprocess::condition;pub use preprocess::cooked;pub use preprocess::directive;pub use preprocess::expand;pub use preprocess::guards;pub use preprocess::macros;pub use sema::check::CheckConfig;pub use sema::check::CheckContext;pub use sema::check::DiagnosticCode;pub use sema::check::Finding;pub use sema::declarations::build_facts;pub use sema::declarations::by_name;pub use sema::declarations::declared_type_of;pub use sema::declarations::scope_of;pub use sema::check;pub use sema::module_info;pub use sema::modules;pub use sema::parser_symbols;pub use sema::scopes;pub use sema::symbol;pub use memory::MemoryReport;pub use summary::ConditionalBody;pub use summary::ClosureEvidence;pub use summary::ConditionAt;pub use summary::ConditionalRegion;pub use summary::CookedDiagnostic;pub use summary::DeclFact;pub use summary::DeclKind;pub use summary::DeclModifiers;pub use summary::FactGuard;pub use summary::FileSummary;pub use summary::GuardBranch;pub use summary::IncludeFact;pub use summary::IndexedRendering;pub use summary::IndexedUnit;pub use summary::MacroAt;pub use summary::MacroDefinitions;pub use summary::MacroFact;pub use summary::MacroKind;pub use summary::MacroView;pub use summary::MapReport;pub use summary::OutlineSymbol;pub use summary::ReflectionFact;pub use summary::RenderedUnit;pub use summary::SummaryGuards;pub use summary::TranslationUnit;pub use summary::UnitDefinitions;pub use summary::UnitReading;pub use summary::UnitSpan;pub use summary::macros_from_direct_includes;pub use summary::macros_from_direct_includes_with_bodies;pub use summary::macros_from_the_closure_with_bodies;pub use summary::macros_in_force_before_the_include;pub use summary::outline_of;pub use preprocess::guard;pub use condition::ConditionExpr;pub use condition::EvalError;pub use condition::Lookup;pub use condition::MacroValues;pub use condition::NoMacros;pub use condition::Value;pub use condition::evaluate;pub use condition::parse_condition;pub use config::CommandLineMacro;pub use config::CompileCommand;pub use config::CompileCommands;pub use config::CompilerConfig;pub use config::IncludePath;pub use config::config_from_arguments;pub use config::parse_compile_commands;pub use config::predefined_macros_of;pub use config::project_config_from_flags;pub use config::split_command_line;pub use directive::Define;pub use directive::Directive;pub use directive::DirectiveKind;pub use directive::Include;pub use directive::IncludeForm;pub use directive::SpannedDirective;pub use directive::parse_directive_tokens;pub use directive::scan_directives;pub use expand::Diagnostic;pub use expand::ExpandedToken;pub use expand::Expansion;pub use expand::ExpansionNote;pub use expand::MacroInvocation;pub use expand::Origin;pub use expand::expand;pub use expand::expand_with_budget;pub use file::FileAnalysis;pub use file::FileTokens;pub use graph::Edge;pub use graph::FileEntry;pub use graph::FileGraph;pub use graph::MAX_INCLUDE_DEPTH;pub use graph::Marked;pub use graph::SkipReason;pub use graph::UnresolvedInclude;pub use graph::Visit;pub use graph::WalkScope;pub use graph::file_only;pub use graph::walk;pub use graph::walk_scoped;pub use guard::Branch;pub use guard::Guard;pub use guard::GuardStack;pub use guard::Region;pub use guard::Visibility;pub use guards::FileGuard;pub use guards::GuardAnalysis;pub use guards::analyse_guards;pub use guards::detect_guard;pub use include::toolchain::BuildStatement;pub use include::FoundIn;pub use include::IncludeResolver;pub use include::Resolution;pub use include::Resolved;pub use include::Unresolved;pub use toolchain::CommandRunner;pub use toolchain::DiskCommands;pub use toolchain::Environment;pub use toolchain::Output;pub use toolchain::Toolchain;pub use toolchain::ToolchainSource;pub use toolchain::discover;pub use toolchain::discover_with;pub use toolchain::find_compiler;pub use toolchain::parse_search_list;pub use toolchain::parse_version;pub use toolchain::search_paths;pub use cooked::Configuration;pub use cooked::CookedStream;pub use cooked::RenderedCooked;pub use cooked::RenderedSpan;pub use cooked::configuration_from_environment;pub use cooked::configuration_from_environment_with;pub use cooked::cook;pub use cooked::cook_with;pub use index::walk_cache::WalkCache;pub use index::ChangeBatch;pub use index::CookedFile;pub use index::EventKind;pub use index::FileEvent;pub use index::FileIndexer;pub use index::FileReferences;pub use index::HeaderTarget;pub use index::IncludeBudget;pub use index::IncludeIndex;pub use index::IncludeVisibility;pub use index::MacroReferences;pub use index::MemberCompletions;pub use index::MemberList;pub use index::NameCompletions;pub use index::NameProvenance;pub use index::NotIndexed;pub use index::NotIndexedReason;pub use index::OfferedName;pub use index::PathPattern;pub use index::Priority;pub use index::ProjectDefinition;pub use index::ProjectDefinitions;pub use index::ProjectIndex;pub use index::ProjectMacro;pub use index::ProjectMember;pub use index::ProjectSymbol;pub use index::Reference;pub use index::ReferenceBudget;pub use index::ReferenceKind;pub use index::Rename;pub use index::Response;pub use index::Step;pub use index::StepOutcome;pub use index::StoreStats;pub use index::SummaryReadError;pub use index::SummaryStore;pub use index::SymbolReferences;pub use index::SymbolToFind;pub use index::UnlistedBase;pub use index::UnresolvedEdge;pub use index::VisibleDeclaration;pub use index::WatchFilter;pub use index::Worklist;pub use index::definition_across_files;pub use index::header_at;pub use index::macro_across_files;pub use index::macro_references;pub use index::member_across_files;pub use index::member_completions_at;pub use index::members_of;pub use index::name_completions_at;pub use index::outcome_of;pub use index::summarize;pub use index::symbol_references;pub use macros::MacroBody;pub use macros::MacroDef;pub use macros::MacroTable;pub use macros::Parameter;pub use macros::ParameterKind;pub use module_info::ImportDeclaration;pub use module_info::ImportTarget;pub use module_info::ModuleInfo;pub use module_info::ModuleUnit;pub use modules::ImportEdge;pub use modules::ImportOutcome;pub use modules::MAX_IMPORT_DEPTH;pub use modules::ModuleGraph;pub use modules::ModuleScanner;pub use modules::ModuleUnitEntry;pub use modules::scan_imports;pub use paths::CachedFiles;pub use paths::DiskFiles;pub use paths::FileId;pub use paths::FileProvider;pub use paths::MemoryFiles;pub use paths::OverlayFiles;pub use paths::PathInterner;pub use paths::join_normalized;pub use paths::normalize_path;pub use paths::parent_normalized;pub use preprocess::FilePreprocessing;pub use preprocess::PositionalMacros;pub use preprocess::preprocess;pub use project::CONFIGURATION_FILE_NAMES;pub use project::CompileSection;pub use project::ConfigProblem;pub use project::ConfigReport;pub use project::Database;pub use project::DatabaseOrigin;pub use project::DiagnosticsSection;pub use project::DiscoveryProblem;pub use project::FormatSection;pub use project::FormatSyntax;pub use project::FunctionEntry;pub use project::HoverSection;pub use project::IndexSection;pub use project::PROJECT_CONFIG_FILE;pub use project::ProjectConfig;pub use project::ProjectDiscovery;pub use project::Severity;pub use project::WorkspaceSection;pub use project::find_database;pub use project::load_config;pub use project::parse_config;pub use summary::ModuleReading;pub use scopes::build_scopes;pub use scopes::declared_module_names;pub use inlay::ParameterHint;pub use inlay::TypeHint;pub use inlay::TypeHintKind;pub use session::CookedReading;pub use session::DiagnosticReading;pub use session::ExpressionType;pub use session::FileDiagnostic;pub use session::FileDiagnostics;pub use session::ModuleImport;pub use session::ModuleNote;pub use session::OpenDocuments;pub use session::Session;pub use session::SessionFiles;pub use session::UnitStats;pub use file::vfs::Vfs;pub use file::vfs::VfsFile;pub use file::view::FileView;pub use symbol::Binding;pub use symbol::BindingKind;pub use symbol::BindingOrigin;pub use symbol::DeclName;pub use symbol::HeaderName;pub use symbol::Known;pub use symbol::MaybeName;pub use symbol::Name;pub use symbol::NameKind;pub use symbol::QualifiedName;pub use symbol::Scope;pub use symbol::ScopeId;pub use symbol::ScopeKind;pub use symbol::ScopeTree;pub use symbol::UnknownReason;pub use token::Token;
Modules§
- align
- Alignment against a real compiler: run the true preprocessor over the same file, put the two token streams side by side, and report where they differ — by kind, not as an opinion.
- cache
- The on-disk cache: one file per file, keyed by a hash of everything that determines a summary.
- completion
textDocument/completion— the list a cursor is asking for, and why each item is in it- cuda
- CUDA’s own vocabulary: the words a CUDA project reads as part of the language and a C++ compiler does not.
- effect
- A file’s effect on the macro table: what its directives do to the environment they are read in.
- file
- One file, analysed: its tokens, its macros, its conditionals.
- folding
- Folding — the regions of a file a client may hide
- format
- The format string — which
{…}and%…of a literal is what. - include
- Finding the file an
#includenames. - index
- Building one file’s
FileSummary— the join nothing else in this crate performs. - inlay
- Inlay hints: what the file does not write and a reader still needs.
- memory
- What the analysis is holding, in bytes — the number a report like “five gigabytes” needs before anything can be called a leak.
- preprocess
- One traversal of a file that produces everything the layers above need.
- probes
- Every diagnostic probe in this crate, and the one place each is read from the environment.
- project
- What this project is, and how it is built
- sema
- Names, scopes and modules: what the parsed text declares, and what a name written in it refers to.
- semantic
- Semantic classification: what each name in a file is, as far as this analysis can tell.
- session
- The driver: one open project, with the four layers joined and a queue that notifications re-seed.
- signature
- Call signatures: which function is being called, and which of its parameters the cursor is in.
- summary
- What the index remembers about one file — the unit the whole cache is built from.
- summary_
codec - Reading and writing
FileSummaryas bytes, so the index survives a restart. - surface
- A file’s directives as data: what it defines, what it undefines, what it includes, and the conditions it reads its environment through — with the tokens needed to evaluate them.
- unit_
codec - A translation unit’s rendering, as bytes — what makes the second read of a unit a decode instead of a cook.
Structs§
- NoMacro
Bodies - The answer for a caller with no macro evidence at all — a buffer parsed on its own, or a test that is about something else.
Traits§
- Macro
Bodies - What a name in a macro’s body expands to, for the one question a shape reader has to ask about it.