RustedSciThe 0.4.20

//! # Task Parser Module
//!
//! A comprehensive document parser for structured text with sections and key-value pairs.
//! Parses documents with format: `title1 key1: value1, value2 key2: value2 title2 key3: value3, value4`
//! into nested HashMaps with type-safe value handling and advanced features.
//!
//! ## Core Functionality
//! - **Structured Parsing**: Converts text documents into `HashMap<String, HashMap<String, Option<Vec<Value>>>>`
//! - **Type Safety**: Supports multiple value types (String, Float, Integer, Boolean, Vector, Optional)
//! - **Template System**: Pre-defines expected structure with `TemplateMap` for validation
//! - **Pseudonym Support**: Allows aliases for section headers and field names
//! - **Selective Parsing**: Parse only specific sections by title
//! - **Comment Filtering**: Automatically filters lines starting with //, #, %, or ;
//!
//! ## Main Structures and Enums
//!
//! ### `DocumentParser`
//! Main parser struct with state management and error handling:
//! - `new(input)` - Create parser with input text
//! - `with_template(template)` - Set parsing template
//! - `with_pseudonims(headers, fields)` - Configure aliases
//! - `parse_document()` - Parse entire document
//! - `parse_document_as()` - Parse with template support
//! - `parse_this_sections(titles)` - Parse specific sections only
//! - `parse_document_as_strings()` - Parse all values as strings
//!
//! ### `Value` Enum
//! Type-safe value representation with variants:
//! - `String(String)` - Text values
//! - `Float(f64)` - Floating point numbers
//! - `Integer(i64)` - Signed integers
//! - `Usize(usize)` - Unsigned size type
//! - `Vector(Vec<f64>)` - Arrays of floats in `[1.0, 2.0, 3.0]` format
//! - `Boolean(bool)` - True/false values
//! - `Optional(Option<Box<Value>>)` - Nullable values with `None` or `Some(value)` syntax
//!
//! Helper methods: `as_string()`, `as_float()`, `as_integer()`, `as_boolean()`, `as_vector()`, etc.
//!
//! ### `TemplateType` Enum
//! Simplified enum for template specification without data:
//! - Used in `TemplateMap` to define expected structure
//! - Variants match `Value` enum but without actual data
//!
//! ## Key Functions
//!
//! ### Core Parsing Functions
//! - `parse_document(input)` - Parse entire document into DocumentMap
//! - `parse_document_as(input, template)` - Parse with optional template
//! - `parse_this_sections(input, titles)` - Parse only specified sections
//! - `parse_document_as_strings(input, template)` - Parse all values as strings
//!
//! ### Low-level Parsers (using nom crate)
//! - `parse_title(input)` - Parse section headers
//! - `parse_key(input)` - Parse field names
//! - `parse_value(input)` - Parse single values with type detection
//! - `parse_value_list(input)` - Parse comma-separated value lists
//! - `parse_key_value_pair(input)` - Parse `key: value1, value2` pairs
//! - `parse_section(input)` - Parse complete sections
//!
//! ### Utility Functions
//! - `invert_vec_map(map)` - Convert `HashMap<String, Vec<String>>` to `HashMap<String, String>`
//! - `filter_comments(input)` - Remove comment lines from input
//!
//! ## Type Aliases
//! - `DocumentMap` = `HashMap<String, SectionMap>`
//! - `SectionMap` = `HashMap<String, Option<Vec<Value>>>`
//! - `TemplateMap` = `HashMap<String, HashMap<String, Option<Vec<TemplateType>>>>`
//!
//! ## Non-obvious Features and Tips
//!
//! ### 1. Bracket-Aware Value Parsing
//! The `parse_single_value()` function tracks bracket `[]` and parenthesis `()` depth to correctly
//! parse complex nested structures like vectors and optional values without breaking on internal commas.
//!
//! ### 2. Pseudonym System Complexity
//! - One real name can map to multiple pseudonyms
//! - Parser must discover which pseudonyms are actually used in input
//! - `find_actual_pseudonyms()` method resolves real names to actual pseudonyms in document
//! - Templates are converted from real names to pseudonyms before parsing
//!
//! ### 3. Template-Pseudonym Integration
//! When both templates and pseudonyms are used:
//! - Templates are defined with real names
//! - Input uses pseudonyms
//! - `convert_template_to_pseudonyms()` dynamically adapts template structure
//! - Results are converted back to real names after parsing
//!
//! ### 4. Optional Value Parsing
//! Supports both `None` and `Some(value)` syntax with recursive inner value parsing.
//! The parser can handle nested optionals like `Some(Some(42))`.
//!
//! ### 5. Vector Parsing
//! Vectors use `[value1, value2, value3]` syntax and are parsed as `Vec<f64>`.
//! Empty vectors `[]` are supported.
//!
//! ### 6. Comment Filtering
//! Automatically removes lines starting with `//`, `#`, `%`, or `;` before parsing.
//! Empty lines are also filtered out.
//!
//! ### 7. Error Handling Strategy
//! - Parser stores last error in `error` field
//! - `is_success()` method checks both error state and result presence
//! - Graceful fallbacks when pseudonym resolution fails
//!
//! ### 8. Memory Efficiency
//! - Uses `Box<Value>` for optional values to reduce enum size
//! - Reuses parsing functions through composition
//! - Minimal string allocations during parsing
//!
//! ## Usage Examples
//!
//! ```rust, ignore
//! // Basic parsing
//! let mut parser = DocumentParser::new("section1 key1: value1, value2".to_string());
//! let result = parser.parse_document().unwrap();
//!
//! // With template
//! let template = HashMap::from([("section1".to_string(),
//!     HashMap::from([("key1".to_string(), Some(vec![TemplateType::String]))]));
//! let mut parser = DocumentParser::new(input).with_template(template);
//! let result = parser.parse_document_as().unwrap();
//!
//! // With pseudonyms
//! let mut parser = DocumentParser::new(input);
//! parser.with_pseudonims(
//!     Some(HashMap::from([("section1".to_string(), vec!["sec1".to_string()])])),
//!     Some(HashMap::from([("key1".to_string(), vec!["k1".to_string()])]))
//! );
//! let result = parser.parse_document().unwrap();
//! ```
use nom::{
    branch::alt,
    bytes::complete::tag,
    character::complete::{alpha1, alphanumeric1},
    combinator::{map, recognize},
    multi::many0,
    sequence::pair,
    IResult, Parser,
};
use std::collections::HashMap;
use std::fmt::Debug;
use std::fmt::Display;
use std::fs;
use std::path::PathBuf;
use tabled::{Table, Tabled};

/// Comprehensive error type for document parsing operations.
///
/// This error type provides detailed information about parsing failures including:
/// - The specific type of error that occurred
/// - Location information (line and column numbers when available)
/// - Contextual information about what was being parsed
/// - A snippet of the input around the error location for debugging
///
/// # Examples
///
/// ```rust,ignore
/// let error = ParseError::new(
///     ParseErrorKind::InvalidValue,
///     Some(5),
///     Some(12),
///     "Expected integer but found 'abc'".to_string(),
///     Some("key: abc, next_key".to_string())
/// );
/// ```
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ParseError {
    /// The specific category of parsing error
    pub kind: ParseErrorKind,
    /// Line number where the error occurred (1-based indexing)
    pub line: Option<usize>,
    /// Column number where the error occurred (1-based indexing)
    pub column: Option<usize>,
    /// Human-readable description of what went wrong and context
    pub context: String,
    /// A snippet of the input text around the error location for debugging
    pub input_snippet: Option<String>,
}

/// Categorizes different types of parsing errors that can occur during document processing.
///
/// Each variant represents a specific class of parsing failure with distinct characteristics:
///
/// # Error Categories
///
/// ## Structural Errors
/// - `InvalidSection`: Problems with section headers or structure
/// - `InvalidKey`: Issues with field names or key formatting
/// - `MissingColon`: Missing or malformed key-value separators
///
/// ## Value Errors  
/// - `InvalidValue`: Problems parsing individual values (type mismatches, format issues)
/// - `UnexpectedToken`: Encountering unexpected characters or symbols
///
/// ## Validation Errors
/// - `TemplateValidation`: Mismatches between parsed data and expected template structure
/// - `PseudonymResolution`: Issues resolving pseudonyms to real names
///
/// ## System Errors
/// - `FileError`: File system operations (reading, permissions, not found)
/// - `CommentFilter`: Problems during comment line filtering
///
/// # Usage Notes
///
/// - Use `InvalidValue` for type conversion failures (e.g., "abc" when expecting number)
/// - Use `TemplateValidation` for missing required fields or unexpected sections
/// - Use `PseudonymResolution` when pseudonym mappings fail or are ambiguous
/// - Use `FileError` for any file I/O related issues during document loading
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ParseErrorKind {
    /// Section header is malformed, missing, or contains invalid characters.
    ///
    /// Common causes:
    /// - Section name contains forbidden characters
    /// - Section header appears in wrong location
    /// - Duplicate section names
    InvalidSection,

    /// Field name/key is malformed, missing, or contains invalid characters.
    ///
    /// Common causes:
    /// - Key contains spaces or forbidden characters  
    /// - Key appears without corresponding value
    /// - Duplicate keys within same section
    InvalidKey,

    /// Individual value cannot be parsed or converted to expected type.
    ///
    /// Common causes:
    /// - Type mismatch (e.g., "abc" when expecting number)
    /// - Malformed vector syntax (e.g., "[1, 2, abc]")
    /// - Invalid optional syntax (e.g., "Some(" without closing ")")
    /// - Boolean values other than "true"/"false"
    InvalidValue,

    /// Missing or malformed colon separator between key and value.
    ///
    /// Common causes:
    /// - Key without colon (e.g., "key value" instead of "key: value")
    /// - Multiple colons in key-value pair
    /// - Colon in wrong position
    MissingColon,

    /// Unexpected character or token encountered during parsing.
    ///
    /// Common causes:
    /// - Special characters in wrong context
    /// - Unclosed brackets or parentheses
    /// - Invalid escape sequences
    /// - Encoding issues
    UnexpectedToken,

    /// Parsed document doesn't match expected template structure.
    ///
    /// Common causes:
    /// - Required sections missing from input
    /// - Required fields missing from sections
    /// - Value types don't match template expectations
    /// - Extra sections/fields not allowed by template
    TemplateValidation,

    /// Cannot resolve pseudonyms to real names or ambiguous mappings.
    ///
    /// Common causes:
    /// - Pseudonym maps to multiple real names
    /// - Real name not found in pseudonym mapping
    /// - Circular pseudonym references
    /// - Pseudonym configuration conflicts
    PseudonymResolution,

    /// Two section or field aliases resolve to the same canonical name.
    AliasCollision,

    /// Two names become identical after the configured case normalization.
    CaseNormalizationCollision,

    /// The parser configuration itself is invalid before input parsing starts.
    InvalidConfiguration,

    /// File system operation failed during document loading.
    ///
    /// Common causes:
    /// - File not found or path doesn't exist
    /// - Permission denied reading file
    /// - File is locked by another process
    /// - Disk I/O errors or corrupted file
    /// - Invalid file encoding
    FileError,

    /// Error occurred while filtering comment lines from input.
    ///
    /// Common causes:
    /// - Invalid UTF-8 sequences in input
    /// - Extremely large input causing memory issues
    /// - Malformed line endings
    CommentFilter,
}

impl ParseError {
    /// Creates a new ParseError with all fields specified.
    pub fn new(
        kind: ParseErrorKind,
        line: Option<usize>,
        column: Option<usize>,
        context: String,
        input_snippet: Option<String>,
    ) -> Self {
        Self {
            kind,
            line,
            column,
            context,
            input_snippet,
        }
    }

    /// Creates a simple ParseError with just error kind and context message.
    pub fn simple(kind: ParseErrorKind, context: String) -> Self {
        Self {
            kind,
            line: None,
            column: None,
            context,
            input_snippet: None,
        }
    }
}

impl std::fmt::Display for ParseError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        write!(f, "{:?}: {}", self.kind, self.context)?;

        if let (Some(line), Some(col)) = (self.line, self.column) {
            write!(f, " at line {}, column {}", line, col)?;
        }

        if let Some(snippet) = &self.input_snippet {
            write!(f, "\nNear: {}", snippet)?;
        }

        Ok(())
    }
}

impl std::error::Error for ParseError {}

/// Helper function to create error with position information from nom input.
pub fn create_positioned_error(
    kind: ParseErrorKind,
    context: String,
    input: &str,
    original_input: &str,
) -> ParseError {
    let consumed = original_input.len() - input.len();
    let before_error = &original_input[..consumed];

    let line = before_error.lines().count();
    let column = before_error
        .lines()
        .last()
        .map(|l| l.len() + 1)
        .unwrap_or(1);

    let snippet = extract_error_snippet(original_input, consumed);

    ParseError::new(kind, Some(line), Some(column), context, Some(snippet))
}

/// Extracts a snippet of text around the error position for context.
fn extract_error_snippet(input: &str, error_pos: usize) -> String {
    let start = error_pos.saturating_sub(20);
    let end = (error_pos + 20).min(input.len());

    let snippet = &input[start..end];
    if start > 0 {
        format!("...{}", snippet)
    } else {
        snippet.to_string()
    }
}

#[derive(Debug, Clone, PartialEq)]
pub enum TemplateType {
    String,
    Float,
    Integer,
    Usize,
    Vector,
    Boolean,
    Optional,
}

pub type DocumentMap = HashMap<String, SectionMap>;
type SectionMap = HashMap<String, Option<Vec<Value>>>;
type TemplateMap = HashMap<String, HashMap<String, Option<Vec<TemplateType>>>>;

/// Document parser struct that wraps parsing functionality
#[derive(Debug, Clone)]
pub struct DocumentParser {
    pub input: String,
    pub result: Option<DocumentMap>,
    pub template: Option<TemplateMap>,
    pub headers_pseudonims: Option<HashMap<String, String>>,
    pub field_name_pseudonims: Option<HashMap<String, String>>,
    pub string_result: Option<HashMap<String, HashMap<String, Option<Vec<String>>>>>,
    pub error: Option<String>,
    configuration_error: Option<ParseError>,
}

impl DocumentParser {
    /// Attach a source position to semantic diagnostics that are discovered
    /// after the nom parser has already produced a map. The semantic pass
    /// still knows the offending source token, so diagnostics should not fall
    /// back to the old position-less `ParseError::simple` form.
    fn semantic_error_at_token(
        &self,
        kind: ParseErrorKind,
        context: String,
        token: &str,
    ) -> ParseError {
        self.input
            .find(token)
            .map(|offset| {
                create_positioned_error(
                    kind.clone(),
                    context.clone(),
                    &self.input[offset..],
                    &self.input,
                )
            })
            .unwrap_or_else(|| ParseError::simple(kind, context))
    }

    /// Create a new DocumentParser with input
    pub fn new(input: String) -> Self {
        Self {
            input,
            result: None,
            template: None,
            headers_pseudonims: None,
            field_name_pseudonims: None,
            string_result: None,
            error: None,
            configuration_error: None,
        }
    }

    /// Set template for parsing
    pub fn with_template(mut self, template: TemplateMap) -> Self {
        self.template = Some(template);
        self
    }

    /// Parse the document while preserving the parser error category.
    pub fn parse_document_typed(&mut self) -> Result<&DocumentMap, ParseError> {
        if let Some(error) = &self.configuration_error {
            return Err(error.clone());
        }

        let result = parse_document(&self.input)?;
        let result = self.try_to_real_names_typed(Some(result))?.ok_or_else(|| {
            ParseError::simple(
                ParseErrorKind::InvalidSection,
                "document parser returned no result".to_string(),
            )
        })?;
        self.result = Some(result);
        self.error = None;
        Ok(self.result.as_ref().expect("result was just stored"))
    }

    /// Parse and validate a templated document with a typed error contract.
    pub fn parse_document_as_typed(&mut self) -> Result<&DocumentMap, ParseError> {
        if let Some(error) = &self.configuration_error {
            return Err(error.clone());
        }
        if self.template.is_some() {
            self.validate_template().map_err(|message| {
                ParseError::simple(ParseErrorKind::TemplateValidation, message)
            })?;
        }

        let template_for_parsing = self
            .template
            .as_ref()
            .map(|template| self.convert_template_to_pseudonyms(template));
        let result = parse_document_as(&self.input, template_for_parsing)?;
        let result = self.try_to_real_names_typed(Some(result))?.ok_or_else(|| {
            ParseError::simple(
                ParseErrorKind::InvalidSection,
                "document parser returned no result".to_string(),
            )
        })?;
        if let Some(template) = &self.template {
            self.validate_against_template(&result, template)
                .map_err(|message| {
                    ParseError::simple(ParseErrorKind::TemplateValidation, message)
                })?;
        }
        self.result = Some(result);
        self.error = None;
        Ok(self.result.as_ref().expect("result was just stored"))
    }

    /// Parse the document with enhanced error handling
    pub fn parse_document(&mut self) -> Result<&DocumentMap, String> {
        if let Some(error) = &self.configuration_error {
            let message = error.to_string();
            self.error = Some(message.clone());
            return Err(message);
        }
        match parse_document(&self.input) {
            Ok(result) => {
                let result = self
                    .try_to_real_names_typed(Some(result))
                    .map_err(|error| {
                        let message = error.to_string();
                        self.error = Some(message.clone());
                        message
                    })?;
                let Some(result) = result else {
                    let error = "document parser returned no result".to_string();
                    self.error = Some(error.clone());
                    return Err(error);
                };
                self.result = Some(result);
                self.error = None;
                Ok(self.result.as_ref().unwrap())
            }
            Err(e) => {
                let error_msg = format!("{}", e);
                self.error = Some(error_msg.clone());
                Err(error_msg)
            }
        }
    }

    /// Parse document with template support and validation
    pub fn parse_document_as(&mut self) -> Result<&DocumentMap, String> {
        if let Some(error) = &self.configuration_error {
            let message = error.to_string();
            self.error = Some(message.clone());
            return Err(message);
        }
        // First validate template if present
        if let Some(_) = &self.template {
            if let Err(e) = self.validate_template() {
                self.error = Some(e.clone());
                return Err(e);
            }
        }

        // Convert template to use actual pseudonyms if pseudonyms are configured
        let template_for_parsing = if let Some(template) = &self.template {
            Some(self.convert_template_to_pseudonyms(template))
        } else {
            None
        };

        match parse_document_as(&self.input, template_for_parsing) {
            Ok(result) => {
                let result = self
                    .try_to_real_names_typed(Some(result))
                    .map_err(|error| {
                        let message = error.to_string();
                        self.error = Some(message.clone());
                        message
                    })?;
                let Some(result) = result else {
                    let error = "document parser returned no result".to_string();
                    self.error = Some(error.clone());
                    return Err(error);
                };

                // Validate parsed result against template
                if let Some(template) = &self.template {
                    if let Err(e) = self.validate_against_template(&result, template) {
                        self.error = Some(e.clone());
                        return Err(e);
                    }
                }

                self.result = Some(result);
                self.error = None;
                Ok(self.result.as_ref().unwrap())
            }
            Err(e) => {
                let error_msg = format!("{}", e);
                self.error = Some(error_msg.clone());
                Err(error_msg)
            }
        }
    }

    /// Parse document as strings
    pub fn parse_document_as_strings(
        &mut self,
    ) -> Result<&HashMap<String, HashMap<String, Option<Vec<String>>>>, String> {
        let template = self.template.as_ref().map(|t| {
            t.iter()
                .map(|(k, v)| {
                    let string_map = v.iter().map(|(key, _)| (key.clone(), None)).collect();
                    (k.clone(), string_map)
                })
                .collect()
        });

        match parse_document_as_strings(&self.input, template) {
            Ok(result) => {
                self.string_result = Some(result);
                self.error = None;
                Ok(self.string_result.as_ref().unwrap())
            }
            Err(e) => {
                let error_msg = format!("{}", e);
                self.error = Some(error_msg.clone());
                Err(error_msg)
            }
        }
    }

    /// Parse specific sections by titles (titles should be real names)
    pub fn parse_this_sections(&mut self, titles: Vec<String>) -> Result<&DocumentMap, String> {
        if let Some(error) = &self.configuration_error {
            let message = error.to_string();
            self.error = Some(message.clone());
            return Err(message);
        }
        // First parse the entire document to discover which pseudonyms are actually used
        let full_doc = match parse_document(&self.input) {
            Ok(doc) => doc,
            Err(e) => return Err(format!("{}", e)),
        };

        // Find actual pseudonyms used in input that correspond to requested real names
        let parsing_titles = self.find_actual_pseudonyms(&full_doc, titles);

        match parse_this_sections(&self.input, parsing_titles) {
            Ok(result) => {
                let result = self
                    .try_to_real_names_typed(Some(result))
                    .map_err(|error| {
                        let message = error.to_string();
                        self.error = Some(message.clone());
                        message
                    })?;
                let Some(result) = result else {
                    let error = "document parser returned no result".to_string();
                    self.error = Some(error.clone());
                    return Err(error);
                };
                self.result = Some(result);
                self.error = None;
                Ok(self.result.as_ref().unwrap())
            }
            Err(e) => {
                let error_msg = format!("{}", e);
                self.error = Some(error_msg.clone());
                Err(error_msg)
            }
        }
    }

    /// Get the parsed result
    pub fn get_result(&self) -> Option<&DocumentMap> {
        self.result.as_ref()
    }

    /// Get the string result
    pub fn get_string_result(
        &self,
    ) -> Option<&HashMap<String, HashMap<String, Option<Vec<String>>>>> {
        self.string_result.as_ref()
    }

    /// Get the last error
    pub fn get_error(&self) -> Option<&String> {
        self.error.as_ref()
    }

    /// Get detailed error information with suggestions
    pub fn get_detailed_error(&self) -> Option<String> {
        self.error.as_ref().map(|e| {
            format!(
                "Parsing Error: {}\n\nSuggestions:\n- Check for missing colons after field names\n- Ensure section names don't contain spaces\n- Verify bracket matching in vectors [1.0, 2.0]\n- Check for duplicate section or field names",
                e
            )
        })
    }

    pub fn pretty_print_result(&self) {
        if let Some(result) = &self.result {
            pretty_print_map(result);
        }
    }
    /// Check if parsing was successful
    pub fn is_success(&self) -> bool {
        self.error.is_none() && (self.result.is_some() || self.string_result.is_some())
    }

    /// Validate input before parsing (quick syntax check)
    pub fn validate_syntax(&self) -> Result<(), String> {
        let lines: Vec<&str> = self.input.lines().collect();

        for (line_num, line) in lines.iter().enumerate() {
            let trimmed = line.trim();
            if trimmed.is_empty()
                || trimmed.starts_with("//")
                || trimmed.starts_with('#')
                || trimmed.starts_with('%')
                || trimmed.starts_with(';')
            {
                continue;
            }

            // Check for common syntax issues
            if trimmed.contains(':') {
                let parts: Vec<&str> = trimmed.split(':').collect();
                if parts.len() != 2 {
                    return Err(format!(
                        "Line {}: Multiple colons found. Each line should have exactly one colon separating key and value.",
                        line_num + 1
                    ));
                }
                if parts[0].trim().is_empty() {
                    return Err(format!("Line {}: Empty key before colon.", line_num + 1));
                }
                if parts[1].trim().is_empty() {
                    return Err(format!("Line {}: Empty value after colon.", line_num + 1));
                }
            }

            // Check for unmatched brackets
            let mut bracket_count = 0;
            let mut paren_count = 0;
            for ch in trimmed.chars() {
                match ch {
                    '[' => bracket_count += 1,
                    ']' => bracket_count -= 1,
                    '(' => paren_count += 1,
                    ')' => paren_count -= 1,
                    _ => {}
                }
            }
            if bracket_count != 0 {
                return Err(format!(
                    "Line {}: Unmatched square brackets []. Found {} opening and {} closing brackets.",
                    line_num + 1,
                    trimmed.matches('[').count(),
                    trimmed.matches(']').count()
                ));
            }
            if paren_count != 0 {
                return Err(format!(
                    "Line {}: Unmatched parentheses (). Found {} opening and {} closing parentheses.",
                    line_num + 1,
                    trimmed.matches('(').count(),
                    trimmed.matches(')').count()
                ));
            }
        }

        Ok(())
    }

    /// Validate template structure and types
    pub fn validate_template(&self) -> Result<(), String> {
        let template = match &self.template {
            Some(t) => t,
            None => return Ok(()), // No template to validate
        };

        if template.is_empty() {
            return Err("Template is empty. At least one section must be defined.".to_string());
        }

        for (section_name, fields) in template {
            if section_name.trim().is_empty() {
                return Err("Template contains empty section name.".to_string());
            }

            if fields.is_empty() {
                return Err(format!(
                    "Template section '{}' contains no fields. Each section must have at least one field.",
                    section_name
                ));
            }

            for (field_name, field_types) in fields {
                if field_name.trim().is_empty() {
                    return Err(format!(
                        "Template section '{}' contains empty field name.",
                        section_name
                    ));
                }

                if let Some(types) = field_types {
                    if types.is_empty() {
                        return Err(format!(
                            "Template field '{}.{}' has empty type list. Specify at least one expected type.",
                            section_name, field_name
                        ));
                    }
                }
            }
        }

        Ok(())
    }

    /// Validate parsed document against template
    pub fn validate_against_template(
        &self,
        document: &DocumentMap,
        template: &TemplateMap,
    ) -> Result<(), String> {
        // Check for missing required sections
        for (template_section, template_fields) in template {
            if !document.contains_key(template_section) {
                return Err(format!(
                    "Required section '{}' is missing from document.",
                    template_section
                ));
            }

            let doc_section = &document[template_section];

            // Check for missing required fields
            for (template_field, expected_types) in template_fields {
                if !doc_section.contains_key(template_field) {
                    return Err(format!(
                        "Required field '{}.{}' is missing from document.",
                        template_section, template_field
                    ));
                }

                // Check if the field exists but has None value (indicating it was missing from input)
                if let Some(None) = doc_section.get(template_field) {
                    return Err(format!(
                        "Required field '{}.{}' is missing from document.",
                        template_section, template_field
                    ));
                }

                // Validate field types if specified
                if let Some(expected_types) = expected_types {
                    if let Some(Some(actual_values)) = doc_section.get(template_field) {
                        for (i, actual_value) in actual_values.iter().enumerate() {
                            if let Some(expected_type) = expected_types.get(i) {
                                if let Err(e) = self.validate_value_type(
                                    actual_value,
                                    expected_type,
                                    template_section,
                                    template_field,
                                    i,
                                ) {
                                    return Err(e);
                                }
                            }
                        }
                    }
                }
            }
        }

        Ok(())
    }

    /// Validate individual value against expected type
    fn validate_value_type(
        &self,
        value: &Value,
        expected_type: &TemplateType,
        section: &str,
        field: &str,
        index: usize,
    ) -> Result<(), String> {
        let matches = match (value, expected_type) {
            (Value::String(_), TemplateType::String) => true,
            (Value::Float(_), TemplateType::Float) => true,
            (Value::Integer(_), TemplateType::Integer) => true,
            (Value::Usize(_), TemplateType::Usize) => true,
            (Value::Vector(_), TemplateType::Vector) => true,
            (Value::Boolean(_), TemplateType::Boolean) => true,
            (Value::Optional(_), TemplateType::Optional) => true,
            // Allow some type flexibility
            (Value::Integer(_), TemplateType::Usize) => true, // Integer can be converted to usize
            (Value::Integer(_), TemplateType::Float) => true, // Integer can be converted to float
            _ => false,
        };

        if !matches {
            return Err(format!(
                "Type mismatch in field '{}.{}' at position {}: expected {:?} but found {:?} (value: {})",
                section,
                field,
                index,
                expected_type,
                self.get_value_type_name(value),
                value
            ));
        }

        Ok(())
    }

    /// Get human-readable type name for Value
    fn get_value_type_name(&self, value: &Value) -> &'static str {
        match value {
            Value::String(_) => "String",
            Value::Float(_) => "Float",
            Value::Integer(_) => "Integer",
            Value::Usize(_) => "Usize",
            Value::Vector(_) => "Vector",
            Value::Boolean(_) => "Boolean",
            Value::Optional(_) => "Optional",
        }
    }

    /// Comprehensive validation combining syntax and template checks
    pub fn validate_all(&self) -> Result<(), String> {
        // First check syntax
        self.validate_syntax()?;

        // Then validate template if present
        self.validate_template()?;

        Ok(())
    }

    /// Reset the parser state
    pub fn reset(&mut self) {
        self.result = None;
        self.string_result = None;
        self.error = None;
    }

    /// Update input and reset state
    pub fn set_input(&mut self, input: String) {
        self.input = input;
        self.reset();
    }

    /// Load settings from file
    pub fn setting_from_file(&mut self, path: Option<PathBuf>) -> Result<(), String> {
        let file_path = match path {
            Some(p) => p,
            None => {
                // Search for file starting with "problem" in current directory
                let current_dir = std::env::current_dir()
                    .map_err(|e| format!("Failed to get current directory: {}", e))?;

                let entries = fs::read_dir(&current_dir)
                    .map_err(|e| format!("Failed to read directory: {}", e))?;

                let mut problem_file = None;
                for entry in entries {
                    let entry =
                        entry.map_err(|e| format!("Failed to read directory entry: {}", e))?;
                    let file_name = entry.file_name();
                    let file_name_str = file_name.to_string_lossy();

                    if file_name_str.starts_with("problem") && file_name_str.ends_with(".txt") {
                        problem_file = Some(entry.path());
                        break;
                    }
                }

                problem_file.ok_or_else(|| "No file starting with 'problem' and ending with '.txt' found in current directory".to_string())?
            }
        };

        let content = fs::read_to_string(&file_path)
            .map_err(|e| format!("Failed to read file {:?}: {}", file_path, e))?;

        self.set_input(content);
        Ok(())
    }

    /// pseudonims are given as "real name of field":{vec!["names user can use for the name of field"]}
    /// Converts HashMap<String, Vec<String>> to HashMap<String, String>
    /// Each element of the Vec becomes a key, and the original key becomes its value
    pub fn try_with_pseudonims(
        &mut self,
        headers_pseudonims: Option<HashMap<String, Vec<String>>>,
        field_name_pseudonims: Option<HashMap<String, Vec<String>>>,
    ) -> Result<(), String> {
        self.try_with_pseudonims_typed(headers_pseudonims, field_name_pseudonims)
            .map_err(|error| error.to_string())
    }

    /// Configure aliases while retaining a structured parser error.
    pub fn try_with_pseudonims_typed(
        &mut self,
        headers_pseudonims: Option<HashMap<String, Vec<String>>>,
        field_name_pseudonims: Option<HashMap<String, Vec<String>>>,
    ) -> Result<(), ParseError> {
        let headers = headers_pseudonims
            .as_ref()
            .map(invert_vec_map_checked_typed)
            .transpose()?;
        let fields = field_name_pseudonims
            .as_ref()
            .map(invert_vec_map_checked_typed)
            .transpose()?;
        // Preserve the historical compatibility semantics: `None` means
        // "leave this alias namespace unchanged", not "clear it".
        if headers_pseudonims.is_some() {
            self.headers_pseudonims = headers;
        }
        if field_name_pseudonims.is_some() {
            self.field_name_pseudonims = fields;
        }
        self.configuration_error = None;
        Ok(())
    }

    /// Compatibility wrapper for callers that cannot propagate setup errors.
    pub fn with_pseudonims(
        &mut self,
        headers_pseudonims: Option<HashMap<String, Vec<String>>>,
        field_name_pseudonims: Option<HashMap<String, Vec<String>>>,
    ) {
        if let Err(error) =
            self.try_with_pseudonims_typed(headers_pseudonims, field_name_pseudonims)
        {
            self.configuration_error = Some(error);
        }
    }

    /// Resolve aliases and reject two source keys collapsing to one output key.
    pub fn try_to_real_names(
        &self,
        result: Option<DocumentMap>,
    ) -> Result<Option<DocumentMap>, String> {
        self.try_to_real_names_typed(result)
            .map_err(|error| error.to_string())
    }

    /// Resolve aliases without losing the collision category.
    pub fn try_to_real_names_typed(
        &self,
        result: Option<DocumentMap>,
    ) -> Result<Option<DocumentMap>, ParseError> {
        let Some(mut doc_map) = result else {
            return Ok(None);
        };

        if let Some(aliases) = &self.headers_pseudonims {
            let mut resolved = DocumentMap::new();
            for (header, section) in doc_map {
                let source_header = header.clone();
                let name = aliases.get(&header).cloned().unwrap_or(header);
                if resolved.insert(name.clone(), section).is_some() {
                    return Err(self.semantic_error_at_token(
                        ParseErrorKind::AliasCollision,
                        format!("section alias collision: multiple headers resolve to `{name}`"),
                        &source_header,
                    ));
                }
            }
            doc_map = resolved;
        }

        if let Some(aliases) = &self.field_name_pseudonims {
            for (section, fields) in doc_map.iter_mut() {
                let mut resolved = HashMap::new();
                for (field, values) in fields.drain() {
                    let source_field = field.clone();
                    let name = aliases.get(&field).cloned().unwrap_or(field);
                    if resolved.insert(name.clone(), values).is_some() {
                        return Err(self.semantic_error_at_token(
                            ParseErrorKind::AliasCollision,
                            format!(
                                "field alias collision in section `{section}`: multiple fields resolve to `{name}`"
                            ),
                            &source_field,
                        ));
                    }
                }
                *fields = resolved;
            }
        }

        Ok(Some(doc_map))
    }

    /// Compatibility wrapper preserving the historical `Option` API.
    pub fn to_real_names(&self, result: Option<DocumentMap>) -> Option<DocumentMap> {
        self.try_to_real_names(result).ok().flatten()
    }
    /// Find actual pseudonyms used in the document that correspond to requested real names
    pub fn find_actual_pseudonyms(
        &self,
        full_doc: &DocumentMap,
        real_names: Vec<String>,
    ) -> Vec<String> {
        if let Some(headers_pseudonims) = &self.headers_pseudonims {
            let mut result = Vec::new();

            for real_name in real_names {
                // Find which pseudonym (if any) is actually used in the document for this real name
                let mut found_pseudonym = None;

                // Check all pseudonyms that map to this real name
                for (pseudonym, mapped_real_name) in headers_pseudonims {
                    if mapped_real_name == &real_name && full_doc.contains_key(pseudonym) {
                        found_pseudonym = Some(pseudonym.clone());
                        break;
                    }
                }

                // Use the found pseudonym or the real name if no pseudonym is found
                result.push(found_pseudonym.unwrap_or(real_name));
            }

            result
        } else {
            real_names
        }
    }

    // Convert template from real names to pseudonyms for parsing
    fn convert_template_to_pseudonyms(&self, template: &TemplateMap) -> TemplateMap {
        if let (Some(headers_pseudonims), Some(field_pseudonims)) =
            (&self.headers_pseudonims, &self.field_name_pseudonims)
        {
            // First parse input to discover actual pseudonyms
            let full_doc = match parse_document(&self.input) {
                Ok(doc) => doc,
                Err(_) => return template.clone(), // If parsing fails, return original template
            };

            let mut converted_template = HashMap::new();

            for (real_header, real_fields) in template {
                // Find actual pseudonym for this header
                let actual_header_pseudonym = headers_pseudonims
                    .iter()
                    .find(|&(_, &ref real)| real == real_header)
                    .and_then(|(pseudo, _)| {
                        if full_doc.contains_key(pseudo) {
                            Some(pseudo.clone())
                        } else {
                            None
                        }
                    })
                    .unwrap_or_else(|| real_header.clone());

                // Convert field names to pseudonyms
                let mut converted_fields = HashMap::new();
                for (real_field, template_type) in real_fields {
                    let actual_field_pseudonym = field_pseudonims
                        .iter()
                        .find(|&(_, &ref real)| real == real_field)
                        .map(|(pseudo, _)| pseudo.clone())
                        .unwrap_or_else(|| real_field.clone());

                    converted_fields.insert(actual_field_pseudonym, template_type.clone());
                }

                converted_template.insert(actual_header_pseudonym, converted_fields);
            }

            converted_template
        } else {
            template.clone()
        }
    }
    pub fn input_to_lower_case(&mut self) {
        let mut result = String::new();
        let mut chars = self.input.chars().peekable();

        while let Some(ch) = chars.next() {
            if ch.is_alphabetic() {
                let mut word = String::new();
                word.push(ch);

                // Collect the rest of the word
                while let Some(&next_ch) = chars.peek() {
                    if next_ch.is_alphabetic() {
                        word.push(chars.next().unwrap());
                    } else {
                        break;
                    }
                }

                // Preserve "Some" and "None", lowercase everything else
                if word == "Some" || word == "None" {
                    result.push_str(&word);
                } else {
                    result.push_str(&word.to_lowercase());
                }
            } else {
                result.push(ch);
            }
        }

        self.input = result;
    }

    pub fn try_keys_to_lower_case(&mut self, exception: Option<Vec<String>>) -> Result<(), String> {
        self.try_keys_to_lower_case_typed(exception)
            .map_err(|error| error.to_string())
    }

    /// Normalize control keys without losing case-collision diagnostics.
    pub fn try_keys_to_lower_case_typed(
        &mut self,
        exception: Option<Vec<String>>,
    ) -> Result<(), ParseError> {
        if let Some(ref result) = self.result {
            let mut new_result = DocumentMap::new();
            for (key, value) in result {
                let new_key = key.to_lowercase();
                //  if outer key is in exception vector than don't do anything with nested key -
                // just copy to the result map
                let mut new_section_map = SectionMap::new();
                for (nested_key, nested_value) in value {
                    let new_nested_key = if let Some(ref exceptions) = exception {
                        if exceptions.contains(key) {
                            nested_key.clone() // Don't change nested key if outer key is in exceptions
                        } else {
                            nested_key.to_lowercase()
                        }
                    } else {
                        nested_key.to_lowercase()
                    };
                    if new_section_map
                        .insert(new_nested_key.clone(), nested_value.clone())
                        .is_some()
                    {
                        return Err(self.semantic_error_at_token(
                            ParseErrorKind::CaseNormalizationCollision,
                            format!(
                                "case-normalization collision in section `{key}` for field `{new_nested_key}`"
                            ),
                            &nested_key,
                        ));
                    }
                }
                if new_result
                    .insert(new_key.clone(), new_section_map)
                    .is_some()
                {
                    return Err(self.semantic_error_at_token(
                        ParseErrorKind::CaseNormalizationCollision,
                        format!("case-normalization collision for section `{new_key}`"),
                        &key,
                    ));
                }
            }
            self.result = Some(new_result);
        }
        Ok(())
    }

    /// Compatibility wrapper; task parsers should use the fallible variant.
    pub fn keys_to_lower_case(&mut self, exception: Option<Vec<String>>) {
        if let Err(error) = self.try_keys_to_lower_case(exception) {
            self.error = Some(error);
        }
    }
}
/// Converts HashMap<String, Vec<String>> to HashMap<String, String>
/// Each element of the Vec becomes a key, and the original key becomes its value
pub fn invert_vec_map(map: &HashMap<String, Vec<String>>) -> HashMap<String, String> {
    let mut result = HashMap::new();
    for (key, vec) in map {
        for val in vec {
            result.insert(val.clone(), key.clone());
        }
    }
    result
}

/// Invert aliases without silently selecting the last declaration on collision.
pub fn invert_vec_map_checked(
    map: &HashMap<String, Vec<String>>,
) -> Result<HashMap<String, String>, String> {
    invert_vec_map_checked_typed(map).map_err(|error| error.to_string())
}

/// Invert aliases and retain a typed collision error for task adapters.
pub fn invert_vec_map_checked_typed(
    map: &HashMap<String, Vec<String>>,
) -> Result<HashMap<String, String>, ParseError> {
    let mut result = HashMap::new();
    for (real_name, aliases) in map {
        for alias in aliases {
            if let Some(previous) = result.insert(alias.clone(), real_name.clone()) {
                if previous != *real_name {
                    return Err(ParseError::simple(
                        ParseErrorKind::AliasCollision,
                        format!(
                            "pseudonym collision: `{alias}` maps to both `{previous}` and `{real_name}`"
                        ),
                    ));
                }
            }
        }
    }
    Ok(result)
}
/// enum to represent different value types:
#[derive(Debug, Clone, PartialEq)]
pub enum Value {
    String(String),
    Float(f64),
    Integer(i64),
    Usize(usize),
    Vector(Vec<f64>),
    Boolean(bool),
    Optional(Option<Box<Value>>),
    // Add other types as needed
}
#[allow(dead_code)]
impl Value {
    // Helper functions to access different value types
    pub fn as_string(&self) -> Option<&String> {
        if let Value::String(s) = self {
            Some(s)
        } else {
            None
        }
    }

    pub fn as_float(&self) -> Option<f64> {
        if let Value::Float(f) = self {
            Some(*f)
        } else {
            None
        }
    }
    pub fn as_usize(&self) -> Option<usize> {
        if let Value::Integer(i) = self {
            Some(*i as usize)
        } else {
            None
        }
    }
    pub fn as_integer(&self) -> Option<i64> {
        if let Value::Integer(i) = self {
            Some(*i)
        } else {
            None
        }
    }

    pub fn as_boolean(&self) -> Option<bool> {
        if let Value::Boolean(b) = self {
            Some(*b)
        } else {
            None
        }
    }

    pub fn as_vector(&self) -> Option<&Vec<f64>> {
        if let Value::Vector(v) = self {
            Some(v)
        } else {
            None
        }
    }

    pub fn as_optional(&self) -> Option<&Option<Box<Value>>> {
        if let Value::Optional(opt) = self {
            Some(opt)
        } else {
            None
        }
    }
    pub fn as_option_string(&self) -> Option<&String> {
        if let Value::Optional(opt) = self {
            if let Some(inner) = opt {
                if let Value::String(s) = inner.as_ref() {
                    return Some(s);
                }
            }
        }
        None
    }

    pub fn as_option_float(&self) -> Option<f64> {
        if let Value::Optional(opt) = self {
            if let Some(inner) = opt {
                if let Value::Float(f) = inner.as_ref() {
                    return Some(*f);
                }
            }
        }
        None
    }
    pub fn as_option_integer(&self) -> Option<i64> {
        if let Value::Optional(opt) = self {
            if let Some(inner) = opt {
                if let Value::Integer(i) = inner.as_ref() {
                    return Some(*i);
                }
            }
        }
        None
    }

    pub fn as_option_usize(&self) -> Option<usize> {
        if let Value::Optional(opt) = self {
            if let Some(inner) = opt {
                if let Value::Integer(i) = inner.as_ref() {
                    return Some(*i as usize);
                }
            }
        }
        None
    }
    // Try to convert to string representation
    pub fn to_string_value(&self) -> String {
        match self {
            Value::String(s) => s.clone(),
            Value::Float(f) => f.to_string(),
            Value::Integer(i) => i.to_string(),
            Value::Usize(i) => i.to_string(),
            Value::Vector(v) => v
                .iter()
                .map(|f| f.to_string())
                .collect::<Vec<String>>()
                .join(", "),
            Value::Boolean(b) => b.to_string(),
            Value::Optional(opt) => {
                if let Some(inner) = opt {
                    inner.to_string_value()
                } else {
                    "None".to_string()
                }
            }
        }
    }
}

// Implement Display for Value
impl Display for Value {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Value::String(s) => write!(f, "{}", s),
            Value::Float(val) => write!(f, "{}", val),
            Value::Integer(val) => write!(f, "{}", val),
            Value::Usize(val) => write!(f, "{}", val),
            Value::Vector(val) => write!(f, "{:?}", val),
            Value::Boolean(val) => write!(f, "{}", val),
            Value::Optional(opt) => {
                if let Some(inner) = opt {
                    write!(f, "{}", inner)
                } else {
                    write!(f, "None")
                }
            }
        }
    }
}
/*parse_key_value_pair_new() - Handle colon validation and structure

parse_section_new() - Section-level validation

parse_document_new() - Top-level document parsing */
/// Parses a title (word characters, underscores, and special symbols)
pub fn parse_title<'a>(
    input: &'a str,
    original_input: &str,
) -> Result<(String, &'a str), ParseError> {
    let parser = recognize(pair(
        alt((alpha1::<&str, nom::error::Error<&str>>, tag("_"))),
        many0(alt((
            alphanumeric1,
            tag("_"),
            tag("=>"),
            tag("-"),
            tag(">"),
            tag("="),
            tag("<"),
        ))),
    ));

    let mut parser = map(parser, String::from);

    match parser.parse(input) {
        Ok((remaining, result)) => {
            let trimmed_remaining = remaining.trim();
            Ok((result, trimmed_remaining))
        }
        Err(_) => {
            Err(create_positioned_error(
                ParseErrorKind::InvalidSection,
                "Failed to parse section title. Section names must start with a letter or underscore and contain only alphanumeric characters, underscores, and symbols (=>, -, >, =, <)".to_string(),
                input,
                original_input,
            ))
        }
    }
}

/// Parses a key (word characters, underscores, and special symbols like =>)
pub fn parse_key<'a>(
    input: &'a str,
    original_input: &str,
) -> Result<(String, &'a str), ParseError> {
    let parser = recognize(pair(
        alt((alpha1::<&str, nom::error::Error<&str>>, tag("_"))),
        many0(alt((
            alphanumeric1,
            tag("_"),
            tag("=>"),
            tag("-"),
            tag("+"),
        ))),
    ));

    let mut parser = map(parser, String::from);

    match parser.parse(input) {
        Ok((remaining, result)) => Ok((result, remaining)),
        Err(_) => {
            Err(create_positioned_error(
                ParseErrorKind::InvalidKey,
                "Failed to parse field key. Keys must start with a letter or underscore and contain only alphanumeric characters, underscores, and symbols (=>, -, +)".to_string(),
                input,
                original_input,
            ))
        }
    }
}

fn parse_single_value(input: &str) -> IResult<&str, &str> {
    let chars = input.char_indices();
    let mut bracket_depth = 0;
    let mut paren_depth = 0;
    let mut end_pos = 0;
    let mut saw_top_level_whitespace = false;

    for (pos, ch) in chars {
        match ch {
            '[' => bracket_depth += 1,
            ']' => bracket_depth -= 1,
            '(' => paren_depth += 1,
            ')' => paren_depth -= 1,
            ',' | '\n' | ';' if bracket_depth == 0 && paren_depth == 0 => {
                break;
            }
            // Preserve spaces inside symbolic expressions, but stop before the
            // next inline key-value pair (`... value key2: ...`).
            ' ' | '\t' if bracket_depth == 0 && paren_depth == 0 => {
                let tail = trim_inline_start(&input[pos..]);
                if looks_like_inline_key_value_start(tail) {
                    break;
                }
                // Legacy inline format support:
                // `section1 key1: v1, v2 key2: v3, v4 section2`
                // We only treat this as a boundary on the first top-level
                // whitespace so normal phrases like `value with spaces` keep
                // parsing as one value.
                if !saw_top_level_whitespace && looks_like_standalone_section_title(tail) {
                    break;
                }
                saw_top_level_whitespace = true;
            }
            _ => {}
        }
        end_pos = pos + ch.len_utf8();
    }

    if end_pos == 0 {
        return Err(nom::Err::Error(nom::error::Error::new(
            input,
            nom::error::ErrorKind::TakeWhile1,
        )));
    }

    Ok((&input[end_pos..], &input[..end_pos]))
}

fn trim_inline_start(input: &str) -> &str {
    input.trim_start_matches([' ', '\t'])
}

fn looks_like_inline_key_value_start(input: &str) -> bool {
    let s = trim_inline_start(input);
    if s.is_empty() {
        return false;
    }

    let mut chars = s.char_indices();
    let Some((_, first)) = chars.next() else {
        return false;
    };
    if !(first.is_ascii_alphabetic() || first == '_') {
        return false;
    }

    let mut end = first.len_utf8();
    for (idx, ch) in chars {
        if ch.is_ascii_alphanumeric()
            || ch == '_'
            || ch == '-'
            || ch == '+'
            || ch == '='
            || ch == '>'
        {
            end = idx + ch.len_utf8();
            continue;
        }
        break;
    }

    let rest = trim_inline_start(&s[end..]);
    rest.starts_with(':')
}

fn looks_like_standalone_section_title(input: &str) -> bool {
    let s = trim_inline_start(input);
    if s.is_empty() {
        return false;
    }

    // This branch is only for inline trailing tokens; real multiline section
    // boundaries are handled earlier in the parsing flow.
    if s.contains(':') || s.contains(',') || s.contains(';') || s.contains('\n') || s.contains('\r')
    {
        return false;
    }

    let mut parts = s.split_whitespace();
    let Some(token) = parts.next() else {
        return false;
    };
    if parts.next().is_some() {
        return false;
    }

    let mut chars = token.chars();
    let Some(first) = chars.next() else {
        return false;
    };
    if !(first.is_ascii_alphabetic() || first == '_') {
        return false;
    }

    chars.all(|c| {
        c.is_ascii_alphanumeric() || c == '_' || c == '-' || c == '>' || c == '=' || c == '<'
    })
}

pub fn parse_value<'a>(
    input: &'a str,
    original_input: &str,
) -> Result<(Value, &'a str), ParseError> {
    let (remaining, value_str) = parse_single_value(input).map_err(|_| {
        create_positioned_error(
            ParseErrorKind::InvalidValue,
            "Failed to extract value from input".to_string(),
            input,
            original_input,
        )
    })?;

    let s = value_str.trim();

    let value = if s == "None" {
        Value::Optional(None)
    } else if s.starts_with("Some(") && s.ends_with(')') {
        let inner = &s[5..s.len() - 1];
        if let Ok(val) = inner.parse::<i64>() {
            Value::Optional(Some(Box::new(Value::Integer(val))))
        } else if let Ok(val) = inner.parse::<f64>() {
            Value::Optional(Some(Box::new(Value::Float(val))))
        } else if let Ok(val) = inner.parse::<bool>() {
            Value::Optional(Some(Box::new(Value::Boolean(val))))
        } else if let Ok(val) = inner.parse::<usize>() {
            Value::Optional(Some(Box::new(Value::Usize(val))))
        } else {
            Value::Optional(Some(Box::new(Value::String(inner.to_string()))))
        }
    } else if s.starts_with('[') && s.ends_with(']') {
        let inner = &s[1..s.len() - 1];
        if inner.is_empty() {
            Value::Vector(vec![])
        } else {
            let float_results: Result<Vec<f64>, _> =
                inner.split(',').map(|v| v.trim().parse::<f64>()).collect();

            match float_results {
                Ok(values) => Value::Vector(values),
                Err(_) => {
                    return Err(create_positioned_error(
                        ParseErrorKind::InvalidValue,
                        format!(
                            "Invalid vector format '{}'. Expected comma-separated numbers in brackets like [1.0, 2.0, 3.0]",
                            s
                        ),
                        input,
                        original_input,
                    ));
                }
            }
        }
    } else if let Ok(val) = s.parse::<i64>() {
        Value::Integer(val)
    } else if let Ok(val) = s.parse::<f64>() {
        Value::Float(val)
    } else if let Ok(val) = s.parse::<bool>() {
        Value::Boolean(val)
    } else if let Ok(val) = s.parse::<usize>() {
        Value::Usize(val)
    } else {
        Value::String(s.to_string())
    };

    Ok((value, remaining))
}

/// Parses a value list with error handling
pub fn parse_value_list<'a>(
    input: &'a str,
    original_input: &str,
) -> Result<(Vec<Value>, &'a str), ParseError> {
    let input = input.trim_start();

    if input.is_empty() {
        return Err(create_positioned_error(
            ParseErrorKind::InvalidValue,
            "Expected value list but found empty input".to_string(),
            input,
            original_input,
        ));
    }

    let mut values = Vec::new();
    let mut remaining = input;

    // Parse first value
    let (value, new_remaining) = parse_value(remaining, original_input)?;
    values.push(value);
    remaining = new_remaining.trim_start();

    // Parse additional comma-separated values
    while remaining.starts_with(',') {
        remaining = &remaining[1..].trim_start(); // Skip comma

        if remaining.is_empty() {
            return Err(create_positioned_error(
                ParseErrorKind::InvalidValue,
                "Expected value after comma but found end of input".to_string(),
                remaining,
                original_input,
            ));
        }

        let (value, new_remaining) = parse_value(remaining, original_input)?;
        values.push(value);
        remaining = new_remaining.trim_start();
    }

    Ok((values, remaining))
}

/// Parses a value list with lookahead to detect key-value pair boundaries
fn parse_value_list_with_lookahead<'a>(
    input: &'a str,
    original_input: &str,
) -> Result<(Vec<Value>, &'a str), ParseError> {
    let input = input.trim_start();

    if input.is_empty() {
        return Err(create_positioned_error(
            ParseErrorKind::InvalidValue,
            "Expected value list but found empty input".to_string(),
            input,
            original_input,
        ));
    }

    let mut values = Vec::new();
    let mut remaining = input;

    loop {
        // Parse a value
        let (value, new_remaining) = parse_value(remaining, original_input)?;
        values.push(value);
        remaining = trim_inline_start(new_remaining);

        // Check if we should continue parsing values
        if remaining.starts_with(',') {
            remaining = trim_inline_start(&remaining[1..]); // Skip comma

            if remaining.is_empty() {
                return Err(create_positioned_error(
                    ParseErrorKind::InvalidValue,
                    "Expected value after comma but found end of input".to_string(),
                    remaining,
                    original_input,
                ));
            }
            continue;
        }

        // Reaching a new line terminates this key-value pair.
        if remaining.starts_with('\n') || remaining.starts_with('\r') || remaining.starts_with(';')
        {
            break;
        }

        // Check if the next content looks like a new key-value pair
        if !remaining.is_empty() {
            if looks_like_standalone_section_title(remaining) {
                break;
            }

            // Look for a pattern like "key:" on the same line.
            let current_line = remaining.lines().next().unwrap_or("");
            if let Some(colon_pos) = current_line.find(':') {
                let potential_key = remaining[..colon_pos].trim();
                // If it looks like a valid key, stop parsing values
                if !potential_key.is_empty()
                    && potential_key
                        .chars()
                        .all(|c| c.is_alphanumeric() || c == '_' || c == '-' || c == '+')
                    && !potential_key.contains(' ')
                {
                    break;
                }
            }
        }

        // Otherwise continue parsing space-separated tokens on the same line.
        // This keeps symbolic expressions like `heat - y` intact at parser-adapter level.
        if remaining.is_empty() {
            break;
        }
    }

    Ok((values, remaining))
}

/// Parses a key-value pair with comprehensive error handling
pub fn parse_key_value_pair<'a>(
    input: &'a str,
    original_input: &str,
) -> Result<((String, Vec<Value>), &'a str), ParseError> {
    let input = input.trim_start();

    // Parse key
    let (key, remaining) = parse_key(input, original_input)?;
    let remaining = remaining.trim_start();

    // Check for colon
    if !remaining.starts_with(':') {
        return Err(create_positioned_error(
            ParseErrorKind::MissingColon,
            format!(
                "Expected ':' after key '{}' but found '{}'.",
                key,
                remaining.chars().next().unwrap_or(' ')
            ),
            remaining,
            original_input,
        ));
    }

    let remaining = &remaining[1..].trim_start(); // Skip colon

    // Parse value list with improved handling for mixed spacing
    let (values, remaining) = parse_value_list_with_lookahead(remaining, original_input)?;

    Ok(((key, values), remaining))
}

/// Parses a section with comprehensive error handling
pub fn parse_section<'a>(
    input: &'a str,
    original_input: &str,
) -> Result<((String, HashMap<String, Vec<Value>>), &'a str), ParseError> {
    let input = input.trim_start();

    // Parse section title
    let (title, mut remaining) = parse_title(input, original_input)?;
    remaining = remaining.trim();

    let mut section_map = HashMap::new();

    // Parse key-value pairs until we hit another section or end of input
    while !remaining.is_empty() {
        let trimmed = remaining.trim_start();
        if trimmed.is_empty() {
            break;
        }

        // Try to parse as key-value pair first
        match parse_key_value_pair(remaining, original_input) {
            Ok(((key, values), new_remaining)) => {
                if section_map.contains_key(&key) {
                    return Err(create_positioned_error(
                        ParseErrorKind::InvalidKey,
                        format!("Duplicate key '{}' found in section '{}'", key, title),
                        remaining,
                        original_input,
                    ));
                }
                section_map.insert(key, values);
                remaining = new_remaining.trim();
            }
            Err(_) => {
                // If we can't parse a key-value pair, check if it looks like a new section
                let first_line = trimmed.lines().next().unwrap_or("");
                let words: Vec<&str> = first_line.split_whitespace().collect();

                if !words.is_empty() {
                    let first_word = words[0];
                    // Check if the first word could be a section header
                    if first_word
                        .chars()
                        .all(|c| c.is_alphanumeric() || c == '_' || c == '-')
                        && !first_word.is_empty()
                    {
                        // Try to parse as a title to see if it's a valid section header
                        if let Ok((parsed_title, _)) = parse_title(first_line, original_input) {
                            if parsed_title == first_word {
                                // Looks like a new section, stop parsing this section
                                break;
                            }
                        }
                    }
                }
                // If it's not a new section and we can't parse it as key-value, it's an error
                // But first, let's check if we have any content parsed already
                if section_map.is_empty() {
                    return Err(create_positioned_error(
                        ParseErrorKind::InvalidSection,
                        format!("Section '{}' contains no valid key-value pairs", title),
                        input,
                        original_input,
                    ));
                } else {
                    // We have some content, so this might be the start of a new section
                    break;
                }
            }
        }
    }

    if section_map.is_empty() {
        return Err(create_positioned_error(
            ParseErrorKind::InvalidSection,
            format!("Section '{}' contains no valid key-value pairs", title),
            input,
            original_input,
        ));
    }

    Ok(((title, section_map), remaining))
}

/// Parses the entire document with comprehensive error handling
pub fn parse_document(input: &str) -> Result<DocumentMap, ParseError> {
    let filtered_input = filter_comments(input);
    let mut remaining = filtered_input.trim();
    let original_input = &filtered_input;

    if remaining.is_empty() {
        return Err(ParseError::simple(
            ParseErrorKind::InvalidSection,
            "Document is empty after filtering comments".to_string(),
        ));
    }

    let mut result = HashMap::new();

    while !remaining.is_empty() {
        let ((section_title, section_map), new_remaining) =
            parse_section(remaining, original_input)?;

        if result.contains_key(&section_title) {
            return Err(create_positioned_error(
                ParseErrorKind::InvalidSection,
                format!("Duplicate section '{}' found in document", section_title),
                remaining,
                original_input,
            ));
        }

        // Convert to expected format
        let mut title_map = HashMap::new();
        for (key, values) in section_map {
            title_map.insert(key, Some(values));
        }
        result.insert(section_title, title_map);

        remaining = new_remaining.trim();
    }

    Ok(result)
}

/// Filters out comment lines (starting with //, #, %, or ;)
fn filter_comments(input: &str) -> String {
    input
        .lines()
        .filter(|line| {
            let trimmed = line.trim();
            !trimmed.starts_with("//")
                && !trimmed.starts_with('#')
                && !trimmed.starts_with('%')
                && !trimmed.starts_with(';')
                && !trimmed.is_empty()
        })
        .collect::<Vec<&str>>()
        .join("\n")
}

/// Parses a document and merges with a template HashMap, ensuring all expected keys exist
pub fn parse_document_with_template(
    input: &str,
    template: &TemplateMap,
) -> Result<DocumentMap, ParseError> {
    let mut parsed = parse_document(input)?;

    // Ensure all expected titles and keys exist
    for (title, keys_map) in template {
        if !parsed.contains_key(title) {
            parsed.insert(title.clone(), HashMap::new());
        }

        let section_map = parsed.get_mut(title).unwrap();
        for key in keys_map.keys() {
            if !section_map.contains_key(key) {
                section_map.insert(key.clone(), None);
            }
        }
    }

    Ok(parsed)
}

/// Helper function to parse a document
pub fn parse_document_as(
    input: &str,
    template: Option<TemplateMap>,
) -> Result<DocumentMap, ParseError> {
    match template {
        Some(template) => parse_document_with_template(input, &template),
        None => parse_document(input),
    }
}

/// Parses only specific sections by their titles
pub fn parse_this_sections(input: &str, titles: Vec<String>) -> Result<DocumentMap, ParseError> {
    let filtered_input = filter_comments(input);
    let mut remaining = filtered_input.trim();
    let original_input = &filtered_input;

    if remaining.is_empty() {
        return Err(ParseError::simple(
            ParseErrorKind::InvalidSection,
            "Document is empty after filtering comments".to_string(),
        ));
    }

    let mut result = HashMap::new();

    // Keep parsing sections until input is exhausted
    while !remaining.is_empty() {
        // Try to parse a section
        match parse_section(remaining, original_input) {
            Ok(((title, section_map), new_remaining)) => {
                // Only add to result if title is in the requested titles
                if titles.contains(&title) {
                    let mut title_map = HashMap::new();
                    for (key, values) in section_map {
                        title_map.insert(key, Some(values));
                    }
                    result.insert(title, title_map);
                }

                remaining = new_remaining.trim();
            }
            Err(_) => {
                // If we can't parse a section, we're done
                break;
            }
        }
    }

    Ok(result)
}

/// function to parse document into HashMap<String, HashMap<String, Option<Vec<String>>>
/// the differ is that nested hashmap has type not  HashMap<String, Option<Vec<Value>>>
/// but HashMap<String, Option<Vec<String>>
/// Parses the document and converts all values to strings
pub fn parse_document_as_strings(
    input: &str,
    template: Option<HashMap<String, HashMap<String, Option<Vec<String>>>>>,
) -> Result<HashMap<String, HashMap<String, Option<Vec<String>>>>, ParseError> {
    // First parse with our Value enum
    let value_map = parse_document_as(input, None)?;
    // Convert the Value map to a String map
    let mut string_map = HashMap::new();

    for (title, section_map) in value_map {
        let mut string_section = HashMap::new();

        for (key, value_opt) in section_map {
            let string_values = value_opt.map(|values| {
                values
                    .into_iter()
                    .map(|v| v.to_string_value())
                    .collect::<Vec<String>>()
            });

            string_section.insert(key, string_values);
        }

        string_map.insert(title, string_section);
    }

    // If a template was provided, ensure all expected keys exist
    if let Some(template) = template {
        for (title, keys_map) in template {
            if !string_map.contains_key(&title) {
                string_map.insert(title.clone(), HashMap::new());
            }

            let section_map = string_map.get_mut(&title).unwrap();
            for key in keys_map.keys() {
                if !section_map.contains_key(key) {
                    section_map.insert(key.clone(), None);
                }
            }
        }
    }

    Ok(string_map)
}

/// Pretty print DocumentMap as a formatted table
pub fn pretty_print_map(doc_map: &DocumentMap) {
    #[derive(Tabled)]
    struct TableRow {
        #[tabled(rename = "Section")]
        section: String,
        #[tabled(rename = "Key")]
        key: String,
        #[tabled(rename = "Values")]
        values: String,
    }

    let mut rows = Vec::new();

    for (section_name, section_map) in doc_map {
        for (key, values_opt) in section_map {
            let values_str = match values_opt {
                Some(values) => values
                    .iter()
                    .map(|v| v.to_string())
                    .collect::<Vec<String>>()
                    .join(", "),
                None => "None".to_string(),
            };

            rows.push(TableRow {
                section: section_name.clone(),
                key: key.clone(),
                values: values_str,
            });
        }
    }

    let table = Table::new(rows);
    println!("{}", table);
}

#[cfg(test)]

mod tests {
    // use rand::rand_core::le;
    // use toml::value;

    use super::*;
    #[test]
    fn close_to_life_example1() {
        let input = "
        solver_settings
        scheme: forward
        method: Dense
        strategy: Damped
        linear_sys_method: None
        abs_tolerance: 1e-6
        max_iterations: 100
        loglevel: Some(info)
        bounds
        z: -10.0, 10.0
        y: -7.0, 7.0
        ";
        let res = parse_document(input);
        let max_iterations = res
            .clone()
            .unwrap()
            .get("solver_settings")
            .unwrap()
            .get("max_iterations")
            .unwrap()
            .clone();
        println!("max_iterations: {:?}", max_iterations);
        assert!(max_iterations.is_some());
        let max_iter_value = max_iterations.unwrap()[0].clone();
        println!("max_iter_value: {:?}", max_iter_value);
        let max_iter = max_iter_value.as_usize().unwrap();
        assert!(max_iter == 100);
        let bounds = res.clone().unwrap().get("bounds").unwrap().clone();
        let y_bounds = bounds.get("y").unwrap().clone().unwrap();
        let y0 = y_bounds[0].clone().as_float().unwrap();
        let y1 = y_bounds[1].clone().as_float().unwrap();
        assert_eq!(y0, -7.0);
        assert_eq!(y1, 7.0);
        println!("bounds: {:?}", bounds);
        // assert!(max_iter == 100);
        println!("res {:?}", res);
        let bounds_from_map: HashMap<String, (f64, f64)> = bounds
            .iter()
            .map(|(key, value)| {
                let binding = value.clone().unwrap();
                let value0 = binding[0].as_float().unwrap();
                let value1 = binding[1].as_float().unwrap();
                (key.to_owned(), (value0, value1))
            })
            .collect();
        println!("bounds_from_map: {:?}", bounds_from_map);
        assert!(res.is_ok());
    }
    #[test]
    fn close_to_life_examples2() {
        let task_content2 = "
        process_conditions
        problem_name:HMXTest
        problem_description: HMXdecompositiontest
        substances:HMX, HMXprod
        Tm: 1500.0
        L: 9e-4
        dT: 600.0
        T_scale: 600.0
        P: 1e6
        Cp: 1464.4
        Lambda: 0.07
        m: 0.000770
        M: 0.0342
        thermal_effects: [102000.0]
        boundary_condition
        HMX: 0.999
        HMXprod: 0.001
        T: 800.0
        groups:true
        diffusion_coefficients
        HMX: 4.7619e-8
        HMXprod: 4.7619e-8
        HMX
        H: 4
        N: 8
        C: 8
        O: 8
        HMXprod
        H: 6
        C: 1
        O: 1
        reactions
        HMX=>HMXprod: [130000.0, 0.0, 20920.0, 102000.0]
        ";
        let res = parse_document(task_content2);
        println!("Parse result: {:?}", res);
        assert!(res.is_ok());
    }
    #[allow(non_upper_case_globals)]
    const task_content: &str = r#"
        process_conditions
        problem_name: Some(HMXTest)
        problem_description: Some(HMXdecompositiontest)
        substances: HMX, HMXprod
        Tm: 1500.0
        L: 9e-4
        dT: 600.0
        T_scale: 600.0
        P: 1e6
        Cp: 1464.4
        Lambda: 0.07
        m: 0.0043
        M: 0.0342
        thermal_effects: [102000.0]
        groups:true
        boundary_condition
        HMX: 0.999
        HMXprod: 0.001
        T: 800.0
        diffusion_coefficients
        HMX: 0.000009296
        HMXprod: 0.000009296
        HMX
        H: 4
        N: 8
        C: 8
        O: 8
        HMXprod
        H: 6
        C: 1
        O: 1
        reactions
        HMX=>10HMXprod: [130000.0, 0.0, 20920.0, 102000.0]
        solver_settings
        scheme: forward
        method: Sparse
        strategy: Damped
        linear_sys_method: None
        abs_tolerance: 1e-5
        max_iterations: 100
        loglevel: Some(info)
        bounds
        C: [-10.0, 10.0]
        J:  [-1e20, 1e20]
        Teta: [-100.0, 100.0]
        q: [-1e20, 1e20] 
        rel_tolerance
        C: 1e-5
        J: 1e-5
        Teta: 1e-5
        q:  1e-5
        strategy_params
        max_jac: Some(3)
        max_damp_iter: Some(10)
        damp_factor: Some(0.5)
        adaptive: None
        "#;
    #[test]
    fn close_to_life_examples3() {
        let res = parse_document(task_content);
        println!("Parse result: {:?}", res);
        assert!(res.is_ok());
        match res {
            Ok(map) => pretty_print_map(&map),
            Err(e) => println!("Error: {}", e),
        }
    }
    #[allow(non_upper_case_globals)]
    const task_content3: &str = r#"
        process_conditions
        problem_name: Some(HMXTest)
        problem_description: Some(HMXdecompositiontest)
        substances: HMX, HMXprod
        t0: 0.0
        t_end: 1.0
        n_steps: 200
        arg:x
        Tm: 1500.0
        L: 9e-4
        dT: 600.0
        T_scale: 600.0
        P: 1e6
        Cp: 1464.4
        Lambda: 0.07
        m: 0.0043
        M: 0.0342
        thermal_effects: [102000.0]
        groups:true
        boundary_condition
        HMX: 0.999
        HMXprod: 0.001
        T: 800.0
        diffusion_coefficients
        HMX: 0.000009296
        HMXprod: 0.000009296
        HMX
        H: 4
        N: 8
        C: 8
        O: 8
        HMXprod
        H: 6
        C: 1
        O: 1
        reactions
        HMX=>10HMXprod: [130000.0, 0.0, 20920.0, 102000.0]
        solver_settings
        scheme: forward
        method: Sparse
        strategy: Damped
        linear_sys_method: None
        abs_tolerance: 1e-5
        max_iterations: 100
        loglevel: Some(info)
        dont_save_logs: true
        bounds
        C: -10.0, 10.0
        J:  -1e20, 1e20
        Teta:-100.0, 100.0
        q: -1e20, 1e20
        rel_tolerance
        C: 1e-5
        J: 1e-5
        Teta: 1e-5
        q:  1e-5
        strategy_params
        max_jac: Some(3)
        max_damp_iter: Some(10)
        damp_factor: Some(0.5)
        adaptive: None
        postprocessing
        gnuplot:true
        save_to_csv:false
        filename: meow
        "#;
    #[test]
    fn close_to_life_examples4() {
        let res = parse_document(task_content3);
        println!("Parse result: {:?}", res);
        assert!(res.is_ok());
        match res {
            Ok(map) => pretty_print_map(&map),
            Err(e) => println!("Error: {}", e),
        }
    }
}
/*


*/
#[test]
fn test_as_option_usize() {
    // Test Some(integer) -> Some(usize)
    let value = Value::Optional(Some(Box::new(Value::Integer(42))));
    assert_eq!(value.as_option_usize(), Some(42));

    // Test None -> None
    let value = Value::Optional(None);
    assert_eq!(value.as_option_usize(), None);

    // Test non-optional value -> None
    let value = Value::String("test".to_string());
    assert_eq!(value.as_option_usize(), None);

    // Test Some(non-integer) -> None
    let value = Value::Optional(Some(Box::new(Value::String("test".to_string()))));
    assert_eq!(value.as_option_usize(), None);

    // Test Some(negative integer) - should still work as it casts i64 to usize
    let value = Value::Optional(Some(Box::new(Value::Integer(-1))));
    assert_eq!(value.as_option_usize(), Some(usize::MAX)); // -1 as usize wraps around
}

#[test]
fn test_as_option_usize_edge_cases() {
    // Test zero
    let value = Value::Optional(Some(Box::new(Value::Integer(0))));
    assert_eq!(value.as_option_usize(), Some(0));

    // Test large positive number
    let value = Value::Optional(Some(Box::new(Value::Integer(i64::MAX))));
    assert_eq!(value.as_option_usize(), Some(i64::MAX as usize));

    // Test nested optional (should return None as it's not directly Some(Integer))
    let nested = Value::Optional(Some(Box::new(Value::Optional(Some(Box::new(
        Value::Integer(42),
    ))))));
    assert_eq!(nested.as_option_usize(), None);
}