Implement ASCII String Extractor with Configurable Length Filtering

Summary

Implement the foundational ASCII string extraction module that scans binary data for printable ASCII character sequences (0x20-0x7E) and returns them as FoundString objects with configurable minimum length filtering.

Context

ASCII extraction is the foundational encoding type for StringyMcStringFace's string extraction pipeline. Unlike the traditional strings command which blindly extracts all printable sequences, this implementation will:

Be section-aware (integrate with SectionInfo from the container parsing)
Return properly structured FoundString objects with metadata (offset, RVA, section name, encoding type)
Support configurable minimum length to reduce noise from random byte sequences
Serve as the reference implementation for future encodings (UTF-8, UTF-16LE, UTF-16BE)

The ASCII extractor will be the first concrete implementation of the string extraction framework and will be used by all binary formats (ELF, PE, Mach-O).

Requirements

Requirement 2.1: Implement basic string extraction for ASCII encoding
Must scan byte sequences for contiguous printable ASCII characters (0x20-0x7E)
Must support configurable minimum length threshold (default: 4 characters)
Must return FoundString objects as defined in src/types.rs
Must properly populate metadata fields: offset, section name, encoding, length, source
Must handle section boundaries correctly (don't span strings across sections)

Proposed Solution

File Structure

Create src/extraction/ascii.rs with the following components:

Core Functions

extract_ascii_strings(data: &[u8], config: &ExtractionConfig) -> Vec<FoundString>
- Main extraction function
- Scans byte slice for printable ASCII runs
- Filters by minimum length
- Returns vector of FoundString objects
is_printable_ascii(byte: u8) -> bool
- Helper to check if byte is in printable range (0x20-0x7E)
- Inline for performance
extract_from_section(section: &SectionInfo, data: &[u8], config: &ExtractionConfig) -> Vec<FoundString>
- Section-aware extraction wrapper
- Calculates correct offsets and RVAs
- Populates section metadata

Configuration Structure

pub struct ExtractionConfig {
    pub min_length: usize,
    pub max_length: Option<usize>,
    // Future: encoding preferences, tag filters, etc.
}

Algorithm

Iterate through byte slice
Track current string start position and length
When encountering non-printable byte:
- If accumulated length >= min_length, create FoundString
- Reset accumulator
Handle end-of-buffer edge case
Calculate offsets (file offset + buffer start)
Set encoding to Encoding::Ascii
Set source to StringSource::SectionData

Edge Cases to Handle

Empty sections or zero-length data
Strings at section boundaries
Very long continuous runs (potential padding or data tables)
Null terminators within printable sequences
Sections smaller than minimum length
Buffer boundaries

Acceptance Criteria

Implementation Notes

Start with simple implementation; optimize later if profiling shows bottlenecks
Consider using SIMD or vectorization in future iterations for performance
ASCII extraction should be the reference for implementing UTF-8, UTF-16LE, UTF-16BE
Do not implement semantic tagging yet (that's a separate issue)
Do not implement scoring yet (that's a separate issue)

Dependencies

Blocked by: Core String Extraction Framework: StringExtractor Trait and Configuration #8 String Extraction Framework (need base traits/interfaces)
Blocks: UTF-8 extraction, UTF-16 extraction, main pipeline orchestrator

Related Issues

Epic: MVP Weekend Implementation - Complete String Extraction Pipeline #39 Epic: MVP Weekend Implementation
Implement Semantic Boost Scoring for String Ranking #23 Semantic Boost Scoring (future enhancement)
Implement Noise Penalty Detection for String Quality Scoring #24 Noise Penalty Detection (future enhancement)

Definition of Done

Code passes cargo test
Code passes cargo clippy with no warnings
Unit test coverage >= 80%
Module properly exported in extraction/mod.rs
Inline documentation for public API
Ready for integration with container parsers

Task-ID: stringy-analyzer/basic-ascii-string-extraction

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

@unclesp1d3r

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch the current issue details
gh issue view 9 --json title,body,labels,assignees,milestone,projectItems --repo EvilBit-Labs/StringyMcStringFace

Length of output: 1348

🏁 Script executed:

#!/bin/bash
# Understand the project structure and check for existing extraction code
echo "=== Project Structure ==="
fd -t f -e rs . src/ 2>/dev/null | head -20

echo -e "\n=== Check for existing extraction code ==="
fd -t f "extraction" -e rs 2>/dev/null

echo -e "\n=== Check README for project context ==="
cat README.md 2>/dev/null | head -50

echo -e "\n=== Check for any requirements doc ==="
fd -t f -i "requirement" 2>/dev/null
fd -t f -i "spec" 2>/dev/null | head -10

echo -e "\n=== Look for existing string-related code ==="
rg -l "string" --type rust -g '!target/' | head -10

Length of output: 2261

🏁 Script executed:

#!/bin/bash
# Check extraction module to understand the framework
echo "=== Extraction Module ==="
cat src/extraction/mod.rs

echo -e "\n=== Types Module ==="
cat src/types.rs

echo -e "\n=== Check for other issues that might provide context ==="
gh issue list --repo EvilBit-Labs/StringyMcStringFace --json number,title,body --limit 20 | jq -r '.[] | "\n#\(.number): \(.title)"'

Length of output: 6213

🏁 Script executed:

#!/bin/bash
# Update the issue with enhanced content

gh issue edit 9 --repo EvilBit-Labs/StringyMcStringFace \
  --title "Implement ASCII String Extractor with Configurable Length Filtering" \
  --body "## Summary

Implement the foundational ASCII string extraction module that scans binary data for printable ASCII character sequences (0x20-0x7E) and returns them as \`FoundString\` objects with configurable minimum length filtering.

## Context

ASCII extraction is the foundational encoding type for StringyMcStringFace's string extraction pipeline. Unlike the traditional \`strings\` command which blindly extracts all printable sequences, this implementation will:

- Be section-aware (integrate with \`SectionInfo\` from the container parsing)
- Return properly structured \`FoundString\` objects with metadata (offset, RVA, section name, encoding type)
- Support configurable minimum length to reduce noise from random byte sequences
- Serve as the reference implementation for future encodings (UTF-8, UTF-16LE, UTF-16BE)

The ASCII extractor will be the first concrete implementation of the string extraction framework and will be used by all binary formats (ELF, PE, Mach-O).

## Requirements

- **Requirement 2.1**: Implement basic string extraction for ASCII encoding
- Must scan byte sequences for contiguous printable ASCII characters (0x20-0x7E)
- Must support configurable minimum length threshold (default: 4 characters)
- Must return \`FoundString\` objects as defined in \`src/types.rs\`
- Must properly populate metadata fields: offset, section name, encoding, length, source
- Must handle section boundaries correctly (don't span strings across sections)

## Proposed Solution

### File Structure
Create \`src/extraction/ascii.rs\` with the following components:

### Core Functions

1. **\`extract_ascii_strings(data: &[u8], config: &ExtractionConfig) -> Vec<FoundString>\`**
   - Main extraction function
   - Scans byte slice for printable ASCII runs
   - Filters by minimum length
   - Returns vector of FoundString objects

2. **\`is_printable_ascii(byte: u8) -> bool\`**
   - Helper to check if byte is in printable range (0x20-0x7E)
   - Inline for performance

3. **\`extract_from_section(section: &SectionInfo, data: &[u8], config: &ExtractionConfig) -> Vec<FoundString>\`**
   - Section-aware extraction wrapper
   - Calculates correct offsets and RVAs
   - Populates section metadata

### Configuration Structure

\`\`\`rust
pub struct ExtractionConfig {
    pub min_length: usize,
    pub max_length: Option<usize>,
    // Future: encoding preferences, tag filters, etc.
}
\`\`\`

### Algorithm

1. Iterate through byte slice
2. Track current string start position and length
3. When encountering non-printable byte:
   - If accumulated length >= min_length, create FoundString
   - Reset accumulator
4. Handle end-of-buffer edge case
5. Calculate offsets (file offset + buffer start)
6. Set encoding to \`Encoding::Ascii\`
7. Set source to \`StringSource::SectionData\`

### Edge Cases to Handle

- Empty sections or zero-length data
- Strings at section boundaries
- Very long continuous runs (potential padding or data tables)
- Null terminators within printable sequences
- Sections smaller than minimum length
- Buffer boundaries

## Acceptance Criteria

- [ ] \`src/extraction/ascii.rs\` created with extraction logic
- [ ] Configurable minimum length parameter (default: 4)
- [ ] Correctly identifies printable ASCII range (0x20-0x7E)
- [ ] Returns \`FoundString\` objects with all required fields populated
- [ ] Unit tests covering:
  - [ ] Basic extraction with default minimum length
  - [ ] Custom minimum length filtering
  - [ ] Edge case: empty input
  - [ ] Edge case: no strings found
  - [ ] Edge case: string at buffer start
  - [ ] Edge case: string at buffer end
  - [ ] Edge case: single character (below minimum)
  - [ ] Edge case: exact minimum length string
  - [ ] Offset calculation correctness
  - [ ] Section metadata population
- [ ] Documentation with examples
- [ ] Integrated into \`src/extraction/mod.rs\`

## Implementation Notes

- Start with simple implementation; optimize later if profiling shows bottlenecks
- Consider using SIMD or vectorization in future iterations for performance
- ASCII extraction should be the reference for implementing UTF-8, UTF-16LE, UTF-16BE
- Do not implement semantic tagging yet (that's a separate issue)
- Do not implement scoring yet (that's a separate issue)

## Dependencies

- **Blocked by**: #8 String Extraction Framework (need base traits/interfaces)
- **Blocks**: UTF-8 extraction, UTF-16 extraction, main pipeline orchestrator

## Related Issues

- #39 Epic: MVP Weekend Implementation
- #23 Semantic Boost Scoring (future enhancement)
- #24 Noise Penalty Detection (future enhancement)

## Definition of Done

- Code passes \`cargo test\`
- Code passes \`cargo clippy\` with no warnings
- Unit test coverage >= 80%
- Module properly exported in \`extraction/mod.rs\`
- Inline documentation for public API
- Ready for integration with container parsers

---

**Task-ID**: stringy-analyzer/basic-ascii-string-extraction"

# Add priority label
gh issue edit 9 --repo EvilBit-Labs/StringyMcStringFace --add-label "priority:high"

echo "Issue #9 enhanced successfully!"

Length of output: 154

@unclesp1d3r I've successfully enhanced issue #9 with comprehensive details! Here's what I updated:

Changes Made:

✅ Improved Title

Before: "Basic ASCII String Extraction"
After: "Implement ASCII String Extractor with Configurable Length Filtering"

✅ Enhanced Body Content

Added extensive sections including:

Context: Explains why ASCII extraction is foundational and how it differs from traditional strings command
Detailed Requirements: Expanded requirement 2.1 with specific technical requirements
Proposed Solution:
- File structure and core functions
- Configuration structure example
- Step-by-step algorithm
- Comprehensive edge cases list
Acceptance Criteria: Detailed checklist with 10+ unit test scenarios
Implementation Notes: Guidance on optimization and future considerations
Dependencies: Links to blocking issue Core String Extraction Framework: StringExtractor Trait and Configuration #8 and downstream impacts
Related Issues: Connected to Epic Epic: MVP Weekend Implementation - Complete String Extraction Pipeline #39 and future enhancements
Definition of Done: Clear completion criteria

✅ Added Labels

Added priority:high label (foundational feature that blocks other work)

The issue now provides clear implementation guidance with specific technical details about the FoundString structure, printable ASCII range (0x20-0x7E), and integration points with the existing codebase. It's ready for development! 🚀

You can view the enhanced issue here: #9

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

@unclesp1d3r

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch the current issue details
gh issue view 9 --json title,body,labels,assignees,milestone,state --repo EvilBit-Labs/StringyMcStringFace

Length of output: 6054

🏁 Script executed:

#!/bin/bash
# Check all available labels in the repository
gh label list --repo EvilBit-Labs/StringyMcStringFace --json name,description,color --limit 100

Length of output: 2427

🏁 Script executed:

#!/bin/bash
# Add the MVP label to issue #9
gh issue edit 9 --repo EvilBit-Labs/StringyMcStringFace --add-label "MVP"

echo "=== Updated Issue Details ==="
gh issue view 9 --json title,labels,assignees,milestone --repo EvilBit-Labs/StringyMcStringFace

Length of output: 1218

@unclesp1d3r Great news! Issue #9 was already comprehensively enhanced from our previous interaction. I've now added one final touch:

✅ Current Status - Fully Enhanced

Title

✓ "Implement ASCII String Extractor with Configurable Length Filtering"

Body Content

The issue already contains:

✓ Detailed context explaining why ASCII extraction is foundational
✓ Comprehensive requirements (Requirement 2.1 with specific technical details)
✓ Complete proposed solution with:
- File structure (src/extraction/ascii.rs)
- Three core functions with detailed descriptions
- Configuration structure example
- Step-by-step algorithm
- Edge cases list
✓ Extensive acceptance criteria (10+ unit test scenarios)
✓ Implementation notes and optimization guidance
✓ Dependencies mapping (blocked by Core String Extraction Framework: StringExtractor Trait and Configuration #8, blocks UTF-8/UTF-16)
✓ Related issues (Epic: MVP Weekend Implementation - Complete String Extraction Pipeline #39 Epic, Implement Semantic Boost Scoring for String Ranking #23, Implement Noise Penalty Detection for String Quality Scoring #24)
✓ Definition of done checklist

Metadata

✓ Assignee: unclesp1d3r
✓ Milestone: v0.1
✓ Labels (7 total):
- area:analyzer - Binary analyzer functionality
- type:enhancement - New feature
- status:backlog - Task status
- lang:rust - Rust implementation
- needs:tests - Requires test coverage
- priority:high - High priority (foundational feature)
- MVP - [Just Added] - Part of minimum viable product

The issue is production-ready and provides clear implementation guidance with all necessary technical details! 🎯

View the issue: #9

@traycerai generate

@unclesp1d3r

Hi @unclesp1d3r, I have updated the Implementation Plan to reflect the recent changes in the ticket.

New Plan

Observations

The codebase has a well-structured foundation with complete container parsing for ELF, PE, and Mach-O formats. The FoundString type and related metadata structures are fully defined in src/types.rs. The extraction module currently only contains format-specific extractors (PE resources, Mach-O load commands) but lacks the foundational ASCII string extraction capability. No ExtractionConfig structure exists yet. The documentation in docs/src/string-extraction.md provides clear guidance on the expected extraction algorithm and configuration structure.

Approach

Implement the ASCII string extraction module as the foundational encoding extractor for StringyMcStringFace. Create a new src/extraction/ascii.rs module with three core functions: extract_ascii_strings for basic byte-level scanning, is_printable_ascii as an inline helper, and extract_from_section for section-aware extraction. Introduce an ExtractionConfig structure to support configurable minimum and maximum length filtering. The implementation will follow the patterns established by existing extractors (PE resources, Mach-O load commands) and properly populate all FoundString metadata fields. Comprehensive unit tests will cover the acceptance criteria including edge cases, offset calculations, and section metadata population.

Reasoning

I explored the repository structure to understand the module organization, examined src/types.rs to understand the FoundString and SectionInfo structures, reviewed existing extraction implementations in src/extraction/pe_resources.rs and src/extraction/macho_load_commands.rs to understand patterns, checked container parsers to see how section data is accessed, verified no ExtractionConfig exists, and reviewed documentation to understand the expected architecture and algorithm.

Proposed File Changes
📄 src/extraction/ascii.rs ^(NEW) 🔗
References

📄 src/types.rs

📄 src/extraction/pe_resources.rs

📄 src/extraction/macho_load_commands.rs

Create the ASCII string extraction module with comprehensive documentation and examples. Implement three core functions:

is_printable_ascii(byte: u8) -> bool: Inline helper function that checks if a byte is in the printable ASCII range (0x20-0x7E). This range includes space through tilde, covering all standard printable ASCII characters.

extract_ascii_strings(data: &[u8], config: &ExtractionConfig) -> Vec<FoundString>: Main extraction function that scans a byte slice for contiguous printable ASCII sequences. Algorithm:

Iterate through the byte slice tracking current string start position and accumulated bytes

When encountering a printable ASCII byte, accumulate it in the current string buffer

When encountering a non-printable byte, check if accumulated length meets minimum threshold

If threshold met, create a FoundString with: text (UTF-8 from accumulated bytes), encoding (Encoding::Ascii), offset (start position), length (byte count), source (StringSource::SectionData), empty tags vector, zero score, None for section and rva

Handle end-of-buffer edge case by checking accumulated string after loop completes

Apply max_length filtering if configured

extract_from_section(section: &SectionInfo, data: &[u8], config: &ExtractionConfig) -> Vec<FoundString>: Section-aware wrapper that extracts strings from a specific section with proper metadata population. Implementation:

Calculate section data slice using section.offset and section.size, with bounds checking

Call extract_ascii_strings on the section data slice

Post-process each FoundString to adjust offsets (add section.offset to relative offsets)

Populate section field with section.name.clone()

Populate rva field with calculated value (section.rva + relative_offset) if section.rva is Some

Return the adjusted vector of FoundStrings

Edge cases to handle:

Empty input data (return empty vector)

Data smaller than minimum length (return empty vector)

String at buffer start (start_offset = 0)

String at buffer end (check accumulated string after loop)

Section boundaries (ensure slice doesn't exceed data.len())

Section offset + size overflow (use checked arithmetic)

Include comprehensive unit tests covering all acceptance criteria:

Basic extraction with default minimum length (4)

Custom minimum length filtering (test with 3, 5, 10)

Empty input edge case

No strings found (all binary data)

String at buffer start

String at buffer end

Single character below minimum

Exact minimum length string

Offset calculation correctness (verify absolute offsets)

Section metadata population (verify section name and RVA)

Max length filtering if configured

Multiple strings in sequence

Strings separated by single non-printable byte

Very long strings (test max_length enforcement)

Follow the testing patterns from src/extraction/pe_resources.rs with helper functions for test data creation and clear test names describing the scenario.
Define the ExtractionConfig structure within the ASCII module to support configurable string extraction parameters:
pub struct ExtractionConfig {
    pub min_length: usize,
    pub max_length: Option<usize>,
}
Implement a Default trait for sensible defaults:

min_length: 4 (standard minimum to reduce noise)

max_length: None (no upper limit by default)

Implement a constructor new(min_length: usize) -> Self for convenient creation with custom minimum length and default max_length.

Add documentation explaining:

Purpose of each field

Default values and rationale

Future extensibility (encoding preferences, tag filters mentioned in the issue)

Usage examples

This structure will be used by all extraction functions and serves as the foundation for future configuration expansion as mentioned in the issue (encoding preferences, tag filters, etc.).
📄 src/extraction/mod.rs ^(MODIFY) 🔗

Update the extraction module to include and export the new ASCII extraction functionality:

Add module declaration: pub mod ascii; after the existing pub mod macho_load_commands; and pub mod pe_resources; declarations

Add public re-exports for the ASCII extraction functions and config: pub use ascii::{extract_ascii_strings, extract_from_section, ExtractionConfig};

Update the module-level documentation to describe the ASCII extraction capability. Add a new section after the Mach-O load command description:

Describe ASCII extraction as the foundational encoding type

Mention configurable minimum/maximum length filtering

Provide usage example showing how to create ExtractionConfig and call extract_ascii_strings

Note that this is the reference implementation for future UTF-8, UTF-16LE, UTF-16BE extractors

Update the example code block to include ASCII extraction alongside PE resources and Mach-O load commands, demonstrating the unified extraction API

The modifications should maintain consistency with the existing documentation style and code organization patterns established in the module.

📄 src/lib.rs ^(MODIFY) 🔗

Update the library root to re-export the new ASCII extraction functionality for public API access:

In the re-export section (around line 55), add ExtractionConfig to the list of re-exported types from the types/extraction modules. Since ExtractionConfig is defined in the extraction module, add it to a new extraction re-export line: pub use extraction::ExtractionConfig;

Update the module-level documentation (lines 1-47) to reflect the new ASCII extraction capability:

In the "Current Implementation Status" section, update the extraction module status from "✅ PE resources complete" to "✅ ASCII extraction and PE resources complete"

Add a brief note about ASCII extraction being the foundational encoding type

Consider adding a usage example in the "Basic Usage" section demonstrating ASCII string extraction alongside container parsing, showing the complete workflow from binary parsing to string extraction

These changes ensure the new functionality is properly exposed through the public API and documented for library users.

Diagram
sequenceDiagram
    participant User
    participant Container Parser
    participant ASCII Extractor
    participant FoundString
    
    User->>Container Parser: parse(binary_data)
    Container Parser-->>User: ContainerInfo with SectionInfo[]
    
    User->>ASCII Extractor: extract_from_section(section, data, config)
    ASCII Extractor->>ASCII Extractor: Slice section data using offset+size
    ASCII Extractor->>ASCII Extractor: extract_ascii_strings(section_data, config)
    
    loop For each byte in data
        ASCII Extractor->>ASCII Extractor: is_printable_ascii(byte)?
        alt Printable ASCII
            ASCII Extractor->>ASCII Extractor: Accumulate in current string
        else Non-printable
            alt Length >= min_length
                ASCII Extractor->>FoundString: Create with metadata
                FoundString-->>ASCII Extractor: FoundString instance
            end
            ASCII Extractor->>ASCII Extractor: Reset accumulator
        end
    end
    
    ASCII Extractor->>ASCII Extractor: Adjust offsets (add section.offset)
    ASCII Extractor->>ASCII Extractor: Populate section name and RVA
    ASCII Extractor-->>User: Vec<FoundString> with complete metadata
Loading
Import In IDE
🤖 Prompt for AI Agents
I have created the following plan after thorough exploration and analysis of the codebase. Follow the below plan verbatim. Trust the files and references. Do not re-verify what's written in the plan. Explore only when absolutely necessary. First implement all the proposed file changes and then I'll review all the changes together at the end.

### Observations

The codebase has a well-structured foundation with complete container parsing for ELF, PE, and Mach-O formats. The `FoundString` type and related metadata structures are fully defined in `src/types.rs`. The extraction module currently only contains format-specific extractors (PE resources, Mach-O load commands) but lacks the foundational ASCII string extraction capability. No `ExtractionConfig` structure exists yet. The documentation in `docs/src/string-extraction.md` provides clear guidance on the expected extraction algorithm and configuration structure.

### Approach

Implement the ASCII string extraction module as the foundational encoding extractor for StringyMcStringFace. Create a new `src/extraction/ascii.rs` module with three core functions: `extract_ascii_strings` for basic byte-level scanning, `is_printable_ascii` as an inline helper, and `extract_from_section` for section-aware extraction. Introduce an `ExtractionConfig` structure to support configurable minimum and maximum length filtering. The implementation will follow the patterns established by existing extractors (PE resources, Mach-O load commands) and properly populate all `FoundString` metadata fields. Comprehensive unit tests will cover the acceptance criteria including edge cases, offset calculations, and section metadata population.

### Reasoning

I explored the repository structure to understand the module organization, examined `src/types.rs` to understand the `FoundString` and `SectionInfo` structures, reviewed existing extraction implementations in `src/extraction/pe_resources.rs` and `src/extraction/macho_load_commands.rs` to understand patterns, checked container parsers to see how section data is accessed, verified no `ExtractionConfig` exists, and reviewed documentation to understand the expected architecture and algorithm.

## Mermaid Diagram

sequenceDiagram
    participant User
    participant Container Parser
    participant ASCII Extractor
    participant FoundString
    
    User->>Container Parser: parse(binary_data)
    Container Parser-->>User: ContainerInfo with SectionInfo[]
    
    User->>ASCII Extractor: extract_from_section(section, data, config)
    ASCII Extractor->>ASCII Extractor: Slice section data using offset+size
    ASCII Extractor->>ASCII Extractor: extract_ascii_strings(section_data, config)
    
    loop For each byte in data
        ASCII Extractor->>ASCII Extractor: is_printable_ascii(byte)?
        alt Printable ASCII
            ASCII Extractor->>ASCII Extractor: Accumulate in current string
        else Non-printable
            alt Length >= min_length
                ASCII Extractor->>FoundString: Create with metadata
                FoundString-->>ASCII Extractor: FoundString instance
            end
            ASCII Extractor->>ASCII Extractor: Reset accumulator
        end
    end
    
    ASCII Extractor->>ASCII Extractor: Adjust offsets (add section.offset)
    ASCII Extractor->>ASCII Extractor: Populate section name and RVA
    ASCII Extractor-->>User: Vec<FoundString> with complete metadata

## Proposed File Changes

### src/extraction/ascii.rs(NEW)

References: 

- src/types.rs
- src/extraction/pe_resources.rs
- src/extraction/macho_load_commands.rs

Create the ASCII string extraction module with comprehensive documentation and examples. Implement three core functions:

1. **`is_printable_ascii(byte: u8) -> bool`**: Inline helper function that checks if a byte is in the printable ASCII range (0x20-0x7E). This range includes space through tilde, covering all standard printable ASCII characters.

2. **`extract_ascii_strings(data: &[u8], config: &ExtractionConfig) -> Vec<FoundString>`**: Main extraction function that scans a byte slice for contiguous printable ASCII sequences. Algorithm:
   - Iterate through the byte slice tracking current string start position and accumulated bytes
   - When encountering a printable ASCII byte, accumulate it in the current string buffer
   - When encountering a non-printable byte, check if accumulated length meets minimum threshold
   - If threshold met, create a `FoundString` with: text (UTF-8 from accumulated bytes), encoding (Encoding::Ascii), offset (start position), length (byte count), source (StringSource::SectionData), empty tags vector, zero score, None for section and rva
   - Handle end-of-buffer edge case by checking accumulated string after loop completes
   - Apply max_length filtering if configured

3. **`extract_from_section(section: &SectionInfo, data: &[u8], config: &ExtractionConfig) -> Vec<FoundString>`**: Section-aware wrapper that extracts strings from a specific section with proper metadata population. Implementation:
   - Calculate section data slice using section.offset and section.size, with bounds checking
   - Call `extract_ascii_strings` on the section data slice
   - Post-process each FoundString to adjust offsets (add section.offset to relative offsets)
   - Populate section field with section.name.clone()
   - Populate rva field with calculated value (section.rva + relative_offset) if section.rva is Some
   - Return the adjusted vector of FoundStrings

Edge cases to handle:
- Empty input data (return empty vector)
- Data smaller than minimum length (return empty vector)
- String at buffer start (start_offset = 0)
- String at buffer end (check accumulated string after loop)
- Section boundaries (ensure slice doesn't exceed data.len())
- Section offset + size overflow (use checked arithmetic)

Include comprehensive unit tests covering all acceptance criteria:
- Basic extraction with default minimum length (4)
- Custom minimum length filtering (test with 3, 5, 10)
- Empty input edge case
- No strings found (all binary data)
- String at buffer start
- String at buffer end
- Single character below minimum
- Exact minimum length string
- Offset calculation correctness (verify absolute offsets)
- Section metadata population (verify section name and RVA)
- Max length filtering if configured
- Multiple strings in sequence
- Strings separated by single non-printable byte
- Very long strings (test max_length enforcement)

Follow the testing patterns from `src/extraction/pe_resources.rs` with helper functions for test data creation and clear test names describing the scenario.
Define the `ExtractionConfig` structure within the ASCII module to support configurable string extraction parameters:

```rust
pub struct ExtractionConfig {
    pub min_length: usize,
    pub max_length: Option<usize>,
}
```

Implement a `Default` trait for sensible defaults:
- min_length: 4 (standard minimum to reduce noise)
- max_length: None (no upper limit by default)

Implement a constructor `new(min_length: usize) -> Self` for convenient creation with custom minimum length and default max_length.

Add documentation explaining:
- Purpose of each field
- Default values and rationale
- Future extensibility (encoding preferences, tag filters mentioned in the issue)
- Usage examples

This structure will be used by all extraction functions and serves as the foundation for future configuration expansion as mentioned in the issue (encoding preferences, tag filters, etc.).

### src/extraction/mod.rs(MODIFY)

Update the extraction module to include and export the new ASCII extraction functionality:

1. Add module declaration: `pub mod ascii;` after the existing `pub mod macho_load_commands;` and `pub mod pe_resources;` declarations

2. Add public re-exports for the ASCII extraction functions and config: `pub use ascii::{extract_ascii_strings, extract_from_section, ExtractionConfig};`

3. Update the module-level documentation to describe the ASCII extraction capability. Add a new section after the Mach-O load command description:
   - Describe ASCII extraction as the foundational encoding type
   - Mention configurable minimum/maximum length filtering
   - Provide usage example showing how to create ExtractionConfig and call extract_ascii_strings
   - Note that this is the reference implementation for future UTF-8, UTF-16LE, UTF-16BE extractors

4. Update the example code block to include ASCII extraction alongside PE resources and Mach-O load commands, demonstrating the unified extraction API

The modifications should maintain consistency with the existing documentation style and code organization patterns established in the module.

### src/lib.rs(MODIFY)

Update the library root to re-export the new ASCII extraction functionality for public API access:

1. In the re-export section (around line 55), add `ExtractionConfig` to the list of re-exported types from the types/extraction modules. Since ExtractionConfig is defined in the extraction module, add it to a new extraction re-export line: `pub use extraction::ExtractionConfig;`

2. Update the module-level documentation (lines 1-47) to reflect the new ASCII extraction capability:
   - In the "Current Implementation Status" section, update the extraction module status from "✅ PE resources complete" to "✅ ASCII extraction and PE resources complete"
   - Add a brief note about ASCII extraction being the foundational encoding type

3. Consider adding a usage example in the "Basic Usage" section demonstrating ASCII string extraction alongside container parsing, showing the complete workflow from binary parsing to string extraction

These changes ensure the new functionality is properly exposed through the public API and documented for library users.
Developer Humor

🎯 ASCII and you shall receive!\n> From bytes to strings, we weave the tale,\n> Of printable chars that never fail.\n> With offsets precise and metadata bright,\n> We extract the strings that hide from sight.\n> 🔍 Four bytes minimum, that's our creed,\n> Finding meaning in binary's seed! ✨

Execution Information

Branch: main
Commit: baf4526

Plan

Observations

The codebase has a well-structured foundation with complete container parsing for ELF, PE, and Mach-O formats. The FoundString type and related metadata structures are fully defined in src/types.rs. The extraction module currently only contains format-specific extractors (PE resources, Mach-O load commands) but lacks the foundational ASCII string extraction capability. No ExtractionConfig structure exists yet. The documentation in docs/src/string-extraction.md provides clear guidance on the expected extraction algorithm and configuration structure.

Approach

Implement the ASCII string extraction module as the foundational encoding extractor for StringyMcStringFace. Create a new src/extraction/ascii.rs module with three core functions: extract_ascii_strings for basic byte-level scanning, is_printable_ascii as an inline helper, and extract_from_section for section-aware extraction. Introduce an ExtractionConfig structure to support configurable minimum and maximum length filtering. The implementation will follow the patterns established by existing extractors (PE resources, Mach-O load commands) and properly populate all FoundString metadata fields. Comprehensive unit tests will cover the acceptance criteria including edge cases, offset calculations, and section metadata population.

Reasoning

I explored the repository structure to understand the module organization, examined src/types.rs to understand the FoundString and SectionInfo structures, reviewed existing extraction implementations in src/extraction/pe_resources.rs and src/extraction/macho_load_commands.rs to understand patterns, checked container parsers to see how section data is accessed, verified no ExtractionConfig exists, and reviewed documentation to understand the expected architecture and algorithm.

Proposed File Changes

📄 src/extraction/ascii.rs ^(NEW) 🔗

References

📄 src/types.rs

📄 src/extraction/pe_resources.rs

📄 src/extraction/macho_load_commands.rs

Create the ASCII string extraction module with comprehensive documentation and examples. Implement three core functions:

is_printable_ascii(byte: u8) -> bool: Inline helper function that checks if a byte is in the printable ASCII range (0x20-0x7E). This range includes space through tilde, covering all standard printable ASCII characters.

extract_ascii_strings(data: &[u8], config: &ExtractionConfig) -> Vec<FoundString>: Main extraction function that scans a byte slice for contiguous printable ASCII sequences. Algorithm:

Iterate through the byte slice tracking current string start position and accumulated bytes

When encountering a printable ASCII byte, accumulate it in the current string buffer

When encountering a non-printable byte, check if accumulated length meets minimum threshold

If threshold met, create a FoundString with: text (UTF-8 from accumulated bytes), encoding (Encoding::Ascii), offset (start position), length (byte count), source (StringSource::SectionData), empty tags vector, zero score, None for section and rva

Handle end-of-buffer edge case by checking accumulated string after loop completes

Apply max_length filtering if configured

extract_from_section(section: &SectionInfo, data: &[u8], config: &ExtractionConfig) -> Vec<FoundString>: Section-aware wrapper that extracts strings from a specific section with proper metadata population. Implementation:

Calculate section data slice using section.offset and section.size, with bounds checking

Call extract_ascii_strings on the section data slice

Post-process each FoundString to adjust offsets (add section.offset to relative offsets)

Populate section field with section.name.clone()

Populate rva field with calculated value (section.rva + relative_offset) if section.rva is Some

Return the adjusted vector of FoundStrings

Edge cases to handle:

Empty input data (return empty vector)

Data smaller than minimum length (return empty vector)

String at buffer start (start_offset = 0)

String at buffer end (check accumulated string after loop)

Section boundaries (ensure slice doesn't exceed data.len())

Section offset + size overflow (use checked arithmetic)

Include comprehensive unit tests covering all acceptance criteria:

Basic extraction with default minimum length (4)

Custom minimum length filtering (test with 3, 5, 10)

Empty input edge case

No strings found (all binary data)

String at buffer start

String at buffer end

Single character below minimum

Exact minimum length string

Offset calculation correctness (verify absolute offsets)

Section metadata population (verify section name and RVA)

Max length filtering if configured

Multiple strings in sequence

Strings separated by single non-printable byte

Very long strings (test max_length enforcement)

Follow the testing patterns from src/extraction/pe_resources.rs with helper functions for test data creation and clear test names describing the scenario.
Define the ExtractionConfig structure within the ASCII module to support configurable string extraction parameters:
pub struct ExtractionConfig {
    pub min_length: usize,
    pub max_length: Option<usize>,
}
Implement a Default trait for sensible defaults:

min_length: 4 (standard minimum to reduce noise)

max_length: None (no upper limit by default)

Implement a constructor new(min_length: usize) -> Self for convenient creation with custom minimum length and default max_length.

Add documentation explaining:

Purpose of each field

Default values and rationale

Future extensibility (encoding preferences, tag filters mentioned in the issue)

Usage examples

This structure will be used by all extraction functions and serves as the foundation for future configuration expansion as mentioned in the issue (encoding preferences, tag filters, etc.).

📄 src/extraction/mod.rs ^(MODIFY) 🔗

Update the extraction module to include and export the new ASCII extraction functionality:

Add module declaration: pub mod ascii; after the existing pub mod macho_load_commands; and pub mod pe_resources; declarations

Add public re-exports for the ASCII extraction functions and config: pub use ascii::{extract_ascii_strings, extract_from_section, ExtractionConfig};

Update the module-level documentation to describe the ASCII extraction capability. Add a new section after the Mach-O load command description:

Describe ASCII extraction as the foundational encoding type

Mention configurable minimum/maximum length filtering

Provide usage example showing how to create ExtractionConfig and call extract_ascii_strings

Note that this is the reference implementation for future UTF-8, UTF-16LE, UTF-16BE extractors

Update the example code block to include ASCII extraction alongside PE resources and Mach-O load commands, demonstrating the unified extraction API

The modifications should maintain consistency with the existing documentation style and code organization patterns established in the module.

📄 src/lib.rs ^(MODIFY) 🔗

Update the library root to re-export the new ASCII extraction functionality for public API access:

In the re-export section (around line 55), add ExtractionConfig to the list of re-exported types from the types/extraction modules. Since ExtractionConfig is defined in the extraction module, add it to a new extraction re-export line: pub use extraction::ExtractionConfig;

Update the module-level documentation (lines 1-47) to reflect the new ASCII extraction capability:

In the "Current Implementation Status" section, update the extraction module status from "✅ PE resources complete" to "✅ ASCII extraction and PE resources complete"

Add a brief note about ASCII extraction being the foundational encoding type

Consider adding a usage example in the "Basic Usage" section demonstrating ASCII string extraction alongside container parsing, showing the complete workflow from binary parsing to string extraction

These changes ensure the new functionality is properly exposed through the public API and documented for library users.

Diagram

sequenceDiagram
    participant User
    participant Container Parser
    participant ASCII Extractor
    participant FoundString
    
    User->>Container Parser: parse(binary_data)
    Container Parser-->>User: ContainerInfo with SectionInfo[]
    
    User->>ASCII Extractor: extract_from_section(section, data, config)
    ASCII Extractor->>ASCII Extractor: Slice section data using offset+size
    ASCII Extractor->>ASCII Extractor: extract_ascii_strings(section_data, config)
    
    loop For each byte in data
        ASCII Extractor->>ASCII Extractor: is_printable_ascii(byte)?
        alt Printable ASCII
            ASCII Extractor->>ASCII Extractor: Accumulate in current string
        else Non-printable
            alt Length >= min_length
                ASCII Extractor->>FoundString: Create with metadata
                FoundString-->>ASCII Extractor: FoundString instance
            end
            ASCII Extractor->>ASCII Extractor: Reset accumulator
        end
    end
    
    ASCII Extractor->>ASCII Extractor: Adjust offsets (add section.offset)
    ASCII Extractor->>ASCII Extractor: Populate section name and RVA
    ASCII Extractor-->>User: Vec<FoundString> with complete metadata

Import In IDE

🤖 Prompt for AI Agents

I have created the following plan after thorough exploration and analysis of the codebase. Follow the below plan verbatim. Trust the files and references. Do not re-verify what's written in the plan. Explore only when absolutely necessary. First implement all the proposed file changes and then I'll review all the changes together at the end.

### Observations

The codebase has a well-structured foundation with complete container parsing for ELF, PE, and Mach-O formats. The `FoundString` type and related metadata structures are fully defined in `src/types.rs`. The extraction module currently only contains format-specific extractors (PE resources, Mach-O load commands) but lacks the foundational ASCII string extraction capability. No `ExtractionConfig` structure exists yet. The documentation in `docs/src/string-extraction.md` provides clear guidance on the expected extraction algorithm and configuration structure.

### Approach

Implement the ASCII string extraction module as the foundational encoding extractor for StringyMcStringFace. Create a new `src/extraction/ascii.rs` module with three core functions: `extract_ascii_strings` for basic byte-level scanning, `is_printable_ascii` as an inline helper, and `extract_from_section` for section-aware extraction. Introduce an `ExtractionConfig` structure to support configurable minimum and maximum length filtering. The implementation will follow the patterns established by existing extractors (PE resources, Mach-O load commands) and properly populate all `FoundString` metadata fields. Comprehensive unit tests will cover the acceptance criteria including edge cases, offset calculations, and section metadata population.

### Reasoning

I explored the repository structure to understand the module organization, examined `src/types.rs` to understand the `FoundString` and `SectionInfo` structures, reviewed existing extraction implementations in `src/extraction/pe_resources.rs` and `src/extraction/macho_load_commands.rs` to understand patterns, checked container parsers to see how section data is accessed, verified no `ExtractionConfig` exists, and reviewed documentation to understand the expected architecture and algorithm.

## Mermaid Diagram

sequenceDiagram
    participant User
    participant Container Parser
    participant ASCII Extractor
    participant FoundString
    
    User->>Container Parser: parse(binary_data)
    Container Parser-->>User: ContainerInfo with SectionInfo[]
    
    User->>ASCII Extractor: extract_from_section(section, data, config)
    ASCII Extractor->>ASCII Extractor: Slice section data using offset+size
    ASCII Extractor->>ASCII Extractor: extract_ascii_strings(section_data, config)
    
    loop For each byte in data
        ASCII Extractor->>ASCII Extractor: is_printable_ascii(byte)?
        alt Printable ASCII
            ASCII Extractor->>ASCII Extractor: Accumulate in current string
        else Non-printable
            alt Length >= min_length
                ASCII Extractor->>FoundString: Create with metadata
                FoundString-->>ASCII Extractor: FoundString instance
            end
            ASCII Extractor->>ASCII Extractor: Reset accumulator
        end
    end
    
    ASCII Extractor->>ASCII Extractor: Adjust offsets (add section.offset)
    ASCII Extractor->>ASCII Extractor: Populate section name and RVA
    ASCII Extractor-->>User: Vec<FoundString> with complete metadata

## Proposed File Changes

### src/extraction/ascii.rs(NEW)

References: 

- src/types.rs
- src/extraction/pe_resources.rs
- src/extraction/macho_load_commands.rs

Create the ASCII string extraction module with comprehensive documentation and examples. Implement three core functions:

1. **`is_printable_ascii(byte: u8) -> bool`**: Inline helper function that checks if a byte is in the printable ASCII range (0x20-0x7E). This range includes space through tilde, covering all standard printable ASCII characters.

2. **`extract_ascii_strings(data: &[u8], config: &ExtractionConfig) -> Vec<FoundString>`**: Main extraction function that scans a byte slice for contiguous printable ASCII sequences. Algorithm:
   - Iterate through the byte slice tracking current string start position and accumulated bytes
   - When encountering a printable ASCII byte, accumulate it in the current string buffer
   - When encountering a non-printable byte, check if accumulated length meets minimum threshold
   - If threshold met, create a `FoundString` with: text (UTF-8 from accumulated bytes), encoding (Encoding::Ascii), offset (start position), length (byte count), source (StringSource::SectionData), empty tags vector, zero score, None for section and rva
   - Handle end-of-buffer edge case by checking accumulated string after loop completes
   - Apply max_length filtering if configured

3. **`extract_from_section(section: &SectionInfo, data: &[u8], config: &ExtractionConfig) -> Vec<FoundString>`**: Section-aware wrapper that extracts strings from a specific section with proper metadata population. Implementation:
   - Calculate section data slice using section.offset and section.size, with bounds checking
   - Call `extract_ascii_strings` on the section data slice
   - Post-process each FoundString to adjust offsets (add section.offset to relative offsets)
   - Populate section field with section.name.clone()
   - Populate rva field with calculated value (section.rva + relative_offset) if section.rva is Some
   - Return the adjusted vector of FoundStrings

Edge cases to handle:
- Empty input data (return empty vector)
- Data smaller than minimum length (return empty vector)
- String at buffer start (start_offset = 0)
- String at buffer end (check accumulated string after loop)
- Section boundaries (ensure slice doesn't exceed data.len())
- Section offset + size overflow (use checked arithmetic)

Include comprehensive unit tests covering all acceptance criteria:
- Basic extraction with default minimum length (4)
- Custom minimum length filtering (test with 3, 5, 10)
- Empty input edge case
- No strings found (all binary data)
- String at buffer start
- String at buffer end
- Single character below minimum
- Exact minimum length string
- Offset calculation correctness (verify absolute offsets)
- Section metadata population (verify section name and RVA)
- Max length filtering if configured
- Multiple strings in sequence
- Strings separated by single non-printable byte
- Very long strings (test max_length enforcement)

Follow the testing patterns from `src/extraction/pe_resources.rs` with helper functions for test data creation and clear test names describing the scenario.
Define the `ExtractionConfig` structure within the ASCII module to support configurable string extraction parameters:

```rust
pub struct ExtractionConfig {
    pub min_length: usize,
    pub max_length: Option<usize>,
}
```

Implement a `Default` trait for sensible defaults:
- min_length: 4 (standard minimum to reduce noise)
- max_length: None (no upper limit by default)

Implement a constructor `new(min_length: usize) -> Self` for convenient creation with custom minimum length and default max_length.

Add documentation explaining:
- Purpose of each field
- Default values and rationale
- Future extensibility (encoding preferences, tag filters mentioned in the issue)
- Usage examples

This structure will be used by all extraction functions and serves as the foundation for future configuration expansion as mentioned in the issue (encoding preferences, tag filters, etc.).

### src/extraction/mod.rs(MODIFY)

Update the extraction module to include and export the new ASCII extraction functionality:

1. Add module declaration: `pub mod ascii;` after the existing `pub mod macho_load_commands;` and `pub mod pe_resources;` declarations

2. Add public re-exports for the ASCII extraction functions and config: `pub use ascii::{extract_ascii_strings, extract_from_section, ExtractionConfig};`

3. Update the module-level documentation to describe the ASCII extraction capability. Add a new section after the Mach-O load command description:
   - Describe ASCII extraction as the foundational encoding type
   - Mention configurable minimum/maximum length filtering
   - Provide usage example showing how to create ExtractionConfig and call extract_ascii_strings
   - Note that this is the reference implementation for future UTF-8, UTF-16LE, UTF-16BE extractors

4. Update the example code block to include ASCII extraction alongside PE resources and Mach-O load commands, demonstrating the unified extraction API

The modifications should maintain consistency with the existing documentation style and code organization patterns established in the module.

### src/lib.rs(MODIFY)

Update the library root to re-export the new ASCII extraction functionality for public API access:

1. In the re-export section (around line 55), add `ExtractionConfig` to the list of re-exported types from the types/extraction modules. Since ExtractionConfig is defined in the extraction module, add it to a new extraction re-export line: `pub use extraction::ExtractionConfig;`

2. Update the module-level documentation (lines 1-47) to reflect the new ASCII extraction capability:
   - In the "Current Implementation Status" section, update the extraction module status from "✅ PE resources complete" to "✅ ASCII extraction and PE resources complete"
   - Add a brief note about ASCII extraction being the foundational encoding type

3. Consider adding a usage example in the "Basic Usage" section demonstrating ASCII string extraction alongside container parsing, showing the complete workflow from binary parsing to string extraction

These changes ensure the new functionality is properly exposed through the public API and documented for library users.

Developer Humor

🎯 ASCII and you shall receive!\n> From bytes to strings, we weave the tale,\n> Of printable chars that never fail.\n> With offsets precise and metadata bright,\n> We extract the strings that hide from sight.\n> 🔍 Four bytes minimum, that's our creed,\n> Finding meaning in binary's seed! ✨

Execution Information

Branch: main
Commit: baf4526

💡 Tips

Supported Commands (Inside Comments)

Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

Implement ASCII String Extractor with Configurable Length Filtering #9

Description

Summary

Context

Requirements

Proposed Solution

File Structure

Core Functions

Configuration Structure

Algorithm

Edge Cases to Handle

Acceptance Criteria

Implementation Notes

Dependencies

Related Issues

Definition of Done

Activity

unclesp1d3r commented on Sep 30, 2025

coderabbitai commented on Sep 30, 2025

Changes Made:

✅ Improved Title

✅ Enhanced Body Content

✅ Added Labels

unclesp1d3r commented on Nov 10, 2025

coderabbitai commented on Nov 10, 2025

✅ Current Status - Fully Enhanced

Title

Body Content

Metadata

unclesp1d3r commented on Nov 12, 2025

traycerai commented on Nov 12, 2025

Observations

Approach

Reasoning

Proposed File Changes

Diagram

Import In IDE

Developer Humor

traycerai commented on Nov 12, 2025

Plan

Observations

Approach

Reasoning

Proposed File Changes

Diagram

Import In IDE

Developer Humor

Supported Commands (Inside Comments)

Supported Commands (Inside Description)

Community

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions