Implement YARA Rule Output Formatter with String Escaping and Modifiers

Overview

This task implements a YARA rule formatter for StringyMcStringFace that outputs extracted strings in valid YARA rule syntax. YARA is a widely-used pattern matching tool in malware research and incident response, making YARA-compatible output a critical feature for security analysts who want to create detection rules from extracted strings.

Context

The binary analyzer currently extracts strings from ELF, PE, and Mach-O binaries with rich metadata including:

Encoding types (ASCII, UTF-8, UTF-16LE, UTF-16BE)
Semantic tags (URLs, domains, IPs, file paths, etc.)
Offsets, RVAs, and section information
Relevance scores

To maximize utility for security practitioners, we need to format this data as valid YARA rules that can be immediately used for threat hunting and detection.

Technical Requirements

1. String Escaping

Implement proper C-style escaping for YARA text strings:

Escape double quotes: \"
Escape backslashes: \\
Escape newlines: \n
Escape carriage returns: \r
Escape tabs: \t
Non-printable bytes: \xNN (hex notation)

2. Encoding Mapping

Map FoundString encoding types to YARA string modifiers:

Encoding::Ascii / Encoding::Utf8 → ascii (default)
Encoding::Utf16Le / Encoding::Utf16Be → wide modifier
Consider adding ascii wide for broader matching when appropriate

3. String Truncation

Apply truncation rules for excessively long strings:

Maximum string length: 256 bytes (configurable)
Truncate with indicator comment (e.g., // truncated from N bytes)
Consider hex string format for binary data or very long strings

4. Semantic Tag Integration

Leverage semantic tags to enhance rule conditions:

Group strings by tag type in rule conditions
Add metadata comments indicating tag classifications
Generate compound conditions (e.g., any of ($url*))

5. Rule Structure

Generate complete, valid YARA rules:

rule binary_name_strings {
    meta:
        description = "Extracted strings from binary_name"
        format = "ELF/PE/MachO"
        generated_by = "StringyMcStringFace"
    
    strings:
        $s1 = "extracted_string" ascii
        $s2 = "wide_string" wide
        $url1 = "https://example.com" nocase
    
    condition:
        any of them
}

Proposed Implementation

File: `src/output/yara.rs`

pub struct YaraFormatter {
    max_string_length: usize,
    include_metadata: bool,
    rule_name: String,
}

impl YaraFormatter {
    pub fn format_rule(&self, strings: &[FoundString], binary_info: &ContainerInfo) -> String;
    fn escape_string(&self, s: &str) -> String;
    fn get_string_modifiers(&self, string: &FoundString) -> Vec<&str>;
    fn truncate_if_needed(&self, s: &str) -> (String, bool);
    fn generate_condition(&self, strings: &[FoundString]) -> String;
}

Module Registration: `src/output/mod.rs`

pub mod yara;

pub enum OutputFormat {
    Json,
    Yara,
    // ... other formats
}

pub trait Formatter {
    fn format(&self, strings: &[FoundString], info: &ContainerInfo) -> String;
}

Example Output

Given extracted strings from a binary, the formatter should produce:

rule suspicious_binary_strings {
    meta:
        description = "Strings extracted from suspicious.exe"
        format = "PE"
        generated_by = "StringyMcStringFace v0.1"
        extracted_count = 47
    
    strings:
        // URLs and network indicators
        $url1 = "http://malicious.example.com/payload" nocase
        $ip1 = "192.168.1.100" ascii
        
        // File paths
        $path1 = "C:\\Windows\\System32\\evil.dll" nocase
        
        // Wide strings (UTF-16)
        $wide1 = "WideString" wide
        
        // Import/Export names
        $imp1 = "CreateRemoteThread" ascii
        $imp2 = "VirtualAllocEx" ascii
    
    condition:
        any of ($url*) or 
        2 of ($imp*) or
        any of ($path*)
}

Test Scenarios

Unit tests should cover:

String Escaping
- Quotes, backslashes, newlines in strings
- Non-ASCII characters (hex escape)
- Already-escaped content (no double-escaping)
Encoding Handling
- ASCII strings → ascii modifier
- UTF-16LE/BE → wide modifier
- Mixed encoding in single rule
Truncation
- Strings under limit → no truncation
- Strings over limit → truncated with comment
- Extreme cases (empty strings, very long strings)
Rule Generation
- Valid YARA syntax (parseable by YARA)
- Proper section formatting (meta, strings, condition)
- Special characters in rule names
Semantic Tags
- URLs grouped and commented
- Network indicators (IPs, domains)
- Import/Export grouping

Dependencies

Blocked by: Output Formatting Framework (Implement Output Formatter Framework with Trait-Based Architecture #25)
- Need base OutputFormat enum and Formatter trait
- May proceed with initial implementation if framework is defined

Acceptance Criteria

src/output/yara.rs implements YaraFormatter
All special characters properly escaped per YARA specification
Encoding types correctly mapped to YARA modifiers
Long strings truncated at configurable threshold
Generated rules pass YARA syntax validation
Comprehensive unit tests with >90% coverage
Integration test with sample binaries (ELF, PE, Mach-O)
Documentation with usage examples

References

Related Issues

Output Formatting Framework (dependency)
Requirement 6.3 implementation

Task ID: stringy-analyzer/yara-friendly-output

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

Plan

Observations

The codebase has a well-defined structure with all necessary types in src/types.rs. The src/output/mod.rs module is currently empty and ready for the output framework. The project uses serde for serialization, thiserror for errors, and insta for snapshot testing. The extraction modules already return Vec<FoundString> with complete metadata including encoding, tags, source, offset, RVA, and section information. The documentation clearly describes the expected YARA output format with proper escaping, modifiers, and rule structure.

Approach

Implement a two-layer solution: (1) Define the output formatting framework with OutputFormat enum and Formatter trait in src/output/mod.rs, and (2) Implement YaraFormatter in src/output/yara.rs with proper string escaping, encoding-to-modifier mapping, truncation logic, semantic tag grouping, and complete YARA rule generation. The formatter will be configurable via a YaraConfig struct and will generate valid YARA rules that can be immediately used for threat hunting.

Reasoning

I explored the codebase structure, examined existing types and extraction modules, reviewed documentation for YARA output requirements, researched YARA string escaping specifications, and analyzed the testing patterns used in the project. I confirmed that all necessary data structures exist and the output module is ready for implementation.

Proposed File Changes

📄 src/output/mod.rs ^(MODIFY) 🔗

References

📄 src/types.rs

Define the output formatting framework by creating:

OutputFormat enum with variants: Json, Yara, Human (for future expansion)

Formatter trait with a format method that takes &[FoundString] and &ContainerInfo and returns Result<String>

Module declarations for the yara submodule

Re-exports for public types like YaraFormatter and YaraConfig

The trait design should be simple and extensible, allowing future formatters (JSON, human-readable) to be added easily. The format method signature should accept string data and container metadata, returning a formatted string or error.

📄 src/output/yara.rs ^(NEW) 🔗

References

📄 src/types.rs

📄 src/output/mod.rs ^(MODIFY)

📄 tests/integration_pe.rs

Implement the YARA rule formatter with the following components:

YaraConfig struct with fields:

max_string_length: usize (default 256)

include_metadata: bool (default true)

rule_name: String (derived from binary name or custom)

min_score: Option<i32> (filter strings by score)

group_by_tags: bool (default true)

YaraFormatter struct that holds a YaraConfig and implements the Formatter trait

String escaping function escape_string(&str) -> String that implements C-style escaping:

Escape double quotes as \"

Escape backslashes as \\

Escape newlines as \n

Escape carriage returns as \r

Escape tabs as \t

Convert non-printable bytes (< 0x20 or > 0x7E) to \xNN hex notation

Handle already-escaped content to avoid double-escaping

Encoding mapper get_string_modifiers(&FoundString) -> Vec<&str> that maps:

Encoding::Ascii or Encoding::Utf8 → ["ascii"]

Encoding::Utf16Le or Encoding::Utf16Be → ["wide"]

Add "nocase" modifier for URLs, domains, and file paths based on tags

Consider "ascii", "wide" for broader matching when appropriate

Truncation function truncate_if_needed(&str, max_len) -> (String, bool) that:

Returns (original, false) if length <= max_len

Returns (truncated, true) if length > max_len

Truncates at byte boundary to avoid breaking UTF-8 sequences

Tag grouping logic that organizes strings by semantic tags:

Group URLs together with $url_N naming

Group IPs with $ip_N naming

Group file paths with $path_N naming

Group imports/exports with $imp_N and $exp_N naming

Ungrouped strings use $s_N naming

Condition generator generate_condition(&[FoundString]) -> String that creates:

Simple any of them for small rule sets

Compound conditions like any of ($url*) or 2 of ($imp*) for grouped strings

Tag-based logic when strings are grouped by semantic meaning

Rule generator format_rule(&[FoundString], &ContainerInfo) -> String that produces:

Rule header with sanitized rule name (replace non-alphanumeric with underscore)

Meta section with description, format (ELF/PE/MachO), generated_by, timestamp

Strings section with properly escaped strings, modifiers, and comments

Condition section with appropriate matching logic

Comments indicating truncation, scores, and tag classifications

The implementation should handle edge cases like empty string lists, very long strings, special characters in rule names, and strings with mixed encodings. Reference the YARA specification for valid syntax at https://yara.readthedocs.io/en/stable/writingrules.html
Add comprehensive unit tests in a #[cfg(test)] module covering:

String escaping tests:

Test escaping of quotes: "hello" → \"hello\"

Test escaping of backslashes: C:\\path → C:\\\\path

Test escaping of newlines: line1\nline2 → line1\\nline2

Test escaping of tabs and carriage returns

Test non-printable characters: \x01\x02 → \\x01\\x02

Test already-escaped content to ensure no double-escaping

Test mixed printable and non-printable characters

Encoding modifier tests:

Test ASCII encoding → ascii modifier

Test UTF-8 encoding → ascii modifier (YARA treats UTF-8 as ASCII)

Test UTF-16LE encoding → wide modifier

Test UTF-16BE encoding → wide modifier

Test nocase modifier for URLs and domains

Truncation tests:

Test strings under limit (no truncation)

Test strings at exact limit (no truncation)

Test strings over limit (truncated with indicator)

Test UTF-8 boundary handling (don't break multi-byte sequences)

Test very long strings (1000+ bytes)

Rule generation tests:

Test empty string list (should return valid minimal rule)

Test single string (simple rule)

Test multiple strings with different encodings

Test strings with various tags (URL, IP, filepath, import)

Test rule name sanitization (spaces, special chars → underscores)

Test metadata generation with different BinaryFormat values

Tag grouping tests:

Test URL grouping ($url_1, $url_2, etc.)

Test IP grouping ($ip_1, $ip_2, etc.)

Test import/export grouping

Test mixed tagged and untagged strings

Condition generation tests:

Test simple condition for ungrouped strings

Test compound condition with multiple tag groups

Test condition with score filtering

Integration tests:

Create synthetic FoundString vectors mimicking real extraction results

Test complete rule generation with realistic data

Verify generated rules are syntactically valid YARA (can be parsed)

Use helper functions to create test FoundString instances with various properties. Consider using insta for snapshot testing of complete rule output similar to patterns in tests/integration_pe.rs.

📄 tests/yara_formatter_integration.rs ^(NEW) 🔗

References

📄 tests/integration_pe.rs

📄 src/extraction/mod.rs

Create integration tests that combine real extraction with YARA formatting:

Test with PE fixture:

Use tests/fixtures/test_binary_with_resources.exe

Extract strings using stringy::extraction::extract_resource_strings

Format as YARA rule using YaraFormatter

Verify rule structure with snapshot testing

Check that version strings, string table entries are properly formatted

Test with ELF fixture:

Use tests/fixtures/test_binary_elf

Parse with ElfParser to get ContainerInfo

Create sample FoundString entries (extraction not yet implemented)

Format as YARA rule

Verify ELF-specific metadata in rule

Test with Mach-O fixture:

Use tests/fixtures/test_binary_macho

Extract load command strings using extract_load_command_strings

Format as YARA rule

Verify framework paths and dylib paths are properly formatted

Snapshot tests:

Use insta::assert_snapshot! to capture complete YARA rule output

Create snapshots for different binary formats

Verify rule syntax remains stable across changes

Edge case tests:

Test with empty extraction results

Test with very large string sets (100+ strings)

Test with strings containing only special characters

Test with mixed encodings in single binary

Follow the pattern established in tests/integration_pe.rs for fixture loading and test structure. Use the get_fixture_path helper pattern for consistent fixture access.

📄 src/lib.rs ^(MODIFY) 🔗

References

📄 src/output/mod.rs ^(MODIFY)

Update the library documentation and re-exports:
Update the module documentation comment to reflect that the output module now has YARA formatter implementation

Add re-exports for the new output types:

pub use output::{Formatter, OutputFormat, YaraFormatter, YaraConfig};

Update the architecture section to mark output module as partially complete (YARA formatter done)
Add a usage example showing how to use the YARA formatter:
use stringy::output::{YaraFormatter, YaraConfig};
use stringy::extraction::extract_resource_strings;

let data = std::fs::read("binary.exe")?;
let strings = extract_resource_strings(&data);
let config = YaraConfig::default().with_rule_name("binary_strings");
let formatter = YaraFormatter::new(config);
let yara_rule = formatter.format(&strings, &container_info)?;
println!("{}", yara_rule);
This makes the YARA formatter easily accessible to library users and documents its usage pattern.

📄 README.md ^(MODIFY) 🔗

Update the README to reflect the new YARA output capability:
Add YARA formatter to the features list
Include a quick example showing YARA rule generation:
use stringy::output::{YaraFormatter, YaraConfig};

// Extract and format as YARA rule
let formatter = YaraFormatter::default();
let yara_rule = formatter.format(&strings, &info)?;
Update the roadmap or status section to mark YARA output as implemented

Add a note about YARA rule validation and usage in the security analysis workflow
Keep the changes minimal and focused on documenting the new capability without overhauling the entire README.

Diagram

sequenceDiagram
    participant User
    participant YaraFormatter
    participant Escaper
    participant ModifierMapper
    participant RuleGenerator
    
    User->>YaraFormatter: format(strings, container_info)
    YaraFormatter->>YaraFormatter: Filter by min_score
    YaraFormatter->>YaraFormatter: Group strings by tags
    
    loop For each string
        YaraFormatter->>Escaper: escape_string(text)
        Escaper-->>YaraFormatter: escaped_text
        YaraFormatter->>ModifierMapper: get_string_modifiers(encoding, tags)
        ModifierMapper-->>YaraFormatter: ["ascii"/"wide", "nocase"]
        YaraFormatter->>YaraFormatter: truncate_if_needed(text, max_len)
    end
    
    YaraFormatter->>RuleGenerator: generate_rule_header(rule_name)
    YaraFormatter->>RuleGenerator: generate_meta_section(container_info)
    YaraFormatter->>RuleGenerator: generate_strings_section(grouped_strings)
    YaraFormatter->>RuleGenerator: generate_condition(grouped_strings)
    
    RuleGenerator-->>YaraFormatter: Complete YARA rule
    YaraFormatter-->>User: Result<String>

Import In IDE

🤖 Prompt for AI Agents

I have created the following plan after thorough exploration and analysis of the codebase. Follow the below plan verbatim. Trust the files and references. Do not re-verify what's written in the plan. Explore only when absolutely necessary. First implement all the proposed file changes and then I'll review all the changes together at the end.

### Observations

The codebase has a well-defined structure with all necessary types in `src/types.rs`. The `src/output/mod.rs` module is currently empty and ready for the output framework. The project uses `serde` for serialization, `thiserror` for errors, and `insta` for snapshot testing. The extraction modules already return `Vec<FoundString>` with complete metadata including encoding, tags, source, offset, RVA, and section information. The documentation clearly describes the expected YARA output format with proper escaping, modifiers, and rule structure.

### Approach

Implement a two-layer solution: (1) Define the output formatting framework with `OutputFormat` enum and `Formatter` trait in `src/output/mod.rs`, and (2) Implement `YaraFormatter` in `src/output/yara.rs` with proper string escaping, encoding-to-modifier mapping, truncation logic, semantic tag grouping, and complete YARA rule generation. The formatter will be configurable via a `YaraConfig` struct and will generate valid YARA rules that can be immediately used for threat hunting.

### Reasoning

I explored the codebase structure, examined existing types and extraction modules, reviewed documentation for YARA output requirements, researched YARA string escaping specifications, and analyzed the testing patterns used in the project. I confirmed that all necessary data structures exist and the output module is ready for implementation.

## Mermaid Diagram

sequenceDiagram
    participant User
    participant YaraFormatter
    participant Escaper
    participant ModifierMapper
    participant RuleGenerator
    
    User->>YaraFormatter: format(strings, container_info)
    YaraFormatter->>YaraFormatter: Filter by min_score
    YaraFormatter->>YaraFormatter: Group strings by tags
    
    loop For each string
        YaraFormatter->>Escaper: escape_string(text)
        Escaper-->>YaraFormatter: escaped_text
        YaraFormatter->>ModifierMapper: get_string_modifiers(encoding, tags)
        ModifierMapper-->>YaraFormatter: ["ascii"/"wide", "nocase"]
        YaraFormatter->>YaraFormatter: truncate_if_needed(text, max_len)
    end
    
    YaraFormatter->>RuleGenerator: generate_rule_header(rule_name)
    YaraFormatter->>RuleGenerator: generate_meta_section(container_info)
    YaraFormatter->>RuleGenerator: generate_strings_section(grouped_strings)
    YaraFormatter->>RuleGenerator: generate_condition(grouped_strings)
    
    RuleGenerator-->>YaraFormatter: Complete YARA rule
    YaraFormatter-->>User: Result<String>

## Proposed File Changes

### src/output/mod.rs(MODIFY)

References: 

- src/types.rs

Define the output formatting framework by creating:

1. **OutputFormat enum** with variants: Json, Yara, Human (for future expansion)
2. **Formatter trait** with a `format` method that takes `&[FoundString]` and `&ContainerInfo` and returns `Result<String>`
3. **Module declarations** for the yara submodule
4. **Re-exports** for public types like `YaraFormatter` and `YaraConfig`

The trait design should be simple and extensible, allowing future formatters (JSON, human-readable) to be added easily. The `format` method signature should accept string data and container metadata, returning a formatted string or error.

### src/output/yara.rs(NEW)

References: 

- src/types.rs
- src/output/mod.rs(MODIFY)
- tests/integration_pe.rs

Implement the YARA rule formatter with the following components:

1. **YaraConfig struct** with fields:
   - `max_string_length: usize` (default 256)
   - `include_metadata: bool` (default true)
   - `rule_name: String` (derived from binary name or custom)
   - `min_score: Option<i32>` (filter strings by score)
   - `group_by_tags: bool` (default true)

2. **YaraFormatter struct** that holds a `YaraConfig` and implements the `Formatter` trait

3. **String escaping function** `escape_string(&str) -> String` that implements C-style escaping:
   - Escape double quotes as `\"`
   - Escape backslashes as `\\`
   - Escape newlines as `\n`
   - Escape carriage returns as `\r`
   - Escape tabs as `\t`
   - Convert non-printable bytes (< 0x20 or > 0x7E) to `\xNN` hex notation
   - Handle already-escaped content to avoid double-escaping

4. **Encoding mapper** `get_string_modifiers(&FoundString) -> Vec<&str>` that maps:
   - `Encoding::Ascii` or `Encoding::Utf8` → `["ascii"]`
   - `Encoding::Utf16Le` or `Encoding::Utf16Be` → `["wide"]`
   - Add `"nocase"` modifier for URLs, domains, and file paths based on tags
   - Consider `"ascii", "wide"` for broader matching when appropriate

5. **Truncation function** `truncate_if_needed(&str, max_len) -> (String, bool)` that:
   - Returns (original, false) if length <= max_len
   - Returns (truncated, true) if length > max_len
   - Truncates at byte boundary to avoid breaking UTF-8 sequences

6. **Tag grouping logic** that organizes strings by semantic tags:
   - Group URLs together with `$url_N` naming
   - Group IPs with `$ip_N` naming
   - Group file paths with `$path_N` naming
   - Group imports/exports with `$imp_N` and `$exp_N` naming
   - Ungrouped strings use `$s_N` naming

7. **Condition generator** `generate_condition(&[FoundString]) -> String` that creates:
   - Simple `any of them` for small rule sets
   - Compound conditions like `any of ($url*) or 2 of ($imp*)` for grouped strings
   - Tag-based logic when strings are grouped by semantic meaning

8. **Rule generator** `format_rule(&[FoundString], &ContainerInfo) -> String` that produces:
   - Rule header with sanitized rule name (replace non-alphanumeric with underscore)
   - Meta section with description, format (ELF/PE/MachO), generated_by, timestamp
   - Strings section with properly escaped strings, modifiers, and comments
   - Condition section with appropriate matching logic
   - Comments indicating truncation, scores, and tag classifications

The implementation should handle edge cases like empty string lists, very long strings, special characters in rule names, and strings with mixed encodings. Reference the YARA specification for valid syntax at https://yara.readthedocs.io/en/stable/writingrules.html
Add comprehensive unit tests in a `#[cfg(test)]` module covering:

1. **String escaping tests**:
   - Test escaping of quotes: `"hello"` → `\"hello\"`
   - Test escaping of backslashes: `C:\\path` → `C:\\\\path`
   - Test escaping of newlines: `line1\nline2` → `line1\\nline2`
   - Test escaping of tabs and carriage returns
   - Test non-printable characters: `\x01\x02` → `\\x01\\x02`
   - Test already-escaped content to ensure no double-escaping
   - Test mixed printable and non-printable characters

2. **Encoding modifier tests**:
   - Test ASCII encoding → `ascii` modifier
   - Test UTF-8 encoding → `ascii` modifier (YARA treats UTF-8 as ASCII)
   - Test UTF-16LE encoding → `wide` modifier
   - Test UTF-16BE encoding → `wide` modifier
   - Test nocase modifier for URLs and domains

3. **Truncation tests**:
   - Test strings under limit (no truncation)
   - Test strings at exact limit (no truncation)
   - Test strings over limit (truncated with indicator)
   - Test UTF-8 boundary handling (don't break multi-byte sequences)
   - Test very long strings (1000+ bytes)

4. **Rule generation tests**:
   - Test empty string list (should return valid minimal rule)
   - Test single string (simple rule)
   - Test multiple strings with different encodings
   - Test strings with various tags (URL, IP, filepath, import)
   - Test rule name sanitization (spaces, special chars → underscores)
   - Test metadata generation with different BinaryFormat values

5. **Tag grouping tests**:
   - Test URL grouping (`$url_1`, `$url_2`, etc.)
   - Test IP grouping (`$ip_1`, `$ip_2`, etc.)
   - Test import/export grouping
   - Test mixed tagged and untagged strings

6. **Condition generation tests**:
   - Test simple condition for ungrouped strings
   - Test compound condition with multiple tag groups
   - Test condition with score filtering

7. **Integration tests**:
   - Create synthetic `FoundString` vectors mimicking real extraction results
   - Test complete rule generation with realistic data
   - Verify generated rules are syntactically valid YARA (can be parsed)

Use helper functions to create test `FoundString` instances with various properties. Consider using `insta` for snapshot testing of complete rule output similar to patterns in `tests/integration_pe.rs`.

### tests/yara_formatter_integration.rs(NEW)

References: 

- tests/integration_pe.rs
- src/extraction/mod.rs

Create integration tests that combine real extraction with YARA formatting:

1. **Test with PE fixture**:
   - Use `tests/fixtures/test_binary_with_resources.exe`
   - Extract strings using `stringy::extraction::extract_resource_strings`
   - Format as YARA rule using `YaraFormatter`
   - Verify rule structure with snapshot testing
   - Check that version strings, string table entries are properly formatted

2. **Test with ELF fixture**:
   - Use `tests/fixtures/test_binary_elf`
   - Parse with `ElfParser` to get `ContainerInfo`
   - Create sample `FoundString` entries (extraction not yet implemented)
   - Format as YARA rule
   - Verify ELF-specific metadata in rule

3. **Test with Mach-O fixture**:
   - Use `tests/fixtures/test_binary_macho`
   - Extract load command strings using `extract_load_command_strings`
   - Format as YARA rule
   - Verify framework paths and dylib paths are properly formatted

4. **Snapshot tests**:
   - Use `insta::assert_snapshot!` to capture complete YARA rule output
   - Create snapshots for different binary formats
   - Verify rule syntax remains stable across changes

5. **Edge case tests**:
   - Test with empty extraction results
   - Test with very large string sets (100+ strings)
   - Test with strings containing only special characters
   - Test with mixed encodings in single binary

Follow the pattern established in `tests/integration_pe.rs` for fixture loading and test structure. Use the `get_fixture_path` helper pattern for consistent fixture access.

### src/lib.rs(MODIFY)

References: 

- src/output/mod.rs(MODIFY)

Update the library documentation and re-exports:

1. Update the module documentation comment to reflect that the output module now has YARA formatter implementation
2. Add re-exports for the new output types:
   - `pub use output::{Formatter, OutputFormat, YaraFormatter, YaraConfig};`
3. Update the architecture section to mark output module as partially complete (YARA formatter done)
4. Add a usage example showing how to use the YARA formatter:
   ```rust
   use stringy::output::{YaraFormatter, YaraConfig};
   use stringy::extraction::extract_resource_strings;
   
   let data = std::fs::read("binary.exe")?;
   let strings = extract_resource_strings(&data);
   let config = YaraConfig::default().with_rule_name("binary_strings");
   let formatter = YaraFormatter::new(config);
   let yara_rule = formatter.format(&strings, &container_info)?;
   println!("{}", yara_rule);
   ```

This makes the YARA formatter easily accessible to library users and documents its usage pattern.

### README.md(MODIFY)

Update the README to reflect the new YARA output capability:

1. Add YARA formatter to the features list
2. Include a quick example showing YARA rule generation:
   ```rust
   use stringy::output::{YaraFormatter, YaraConfig};
   
   // Extract and format as YARA rule
   let formatter = YaraFormatter::default();
   let yara_rule = formatter.format(&strings, &info)?;
   ```
3. Update the roadmap or status section to mark YARA output as implemented
4. Add a note about YARA rule validation and usage in the security analysis workflow

Keep the changes minimal and focused on documenting the new capability without overhauling the entire README.

Developer Humor

🔍 From binary chaos to YARA zen,\n> We escape the quotes and backslash them.\n> Wide strings get wide, ASCII stays tight,\n> Truncate the long ones, keep rules just right!\n> 🎯 Now threat hunters can detect with delight! ✨

Execution Information

Branch: main
Commit: baf4526

💡 Tips

Supported Commands (Inside Comments)

Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

@unclesp1d3r

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch current issue details
gh issue view 28 --json title,body,labels,assignees,milestone,state

# Check repository structure
echo -e "\n=== Repository Structure ==="
fd -t f -e rs | head -20

# Look for existing output formatters
echo -e "\n=== Checking for output directory ==="
fd -t d "output"

# Check if there's an output module
echo -e "\n=== Looking for output related files ==="
fd -t f "output" -e rs

# Check main lib or main file to understand project structure
echo -e "\n=== Checking main entry points ==="
fd -t f "main.rs\|lib.rs" | head -5

Length of output: 1766

🏁 Script executed:

#!/bin/bash
# Check output module to understand framework
echo "=== Output Module ==="
cat src/output/mod.rs

# Check types to understand what we're working with
echo -e "\n=== Types Module ==="
cat src/types.rs

# Check if there are other formatters to use as reference
echo -e "\n=== Looking for other output formats ==="
fd -t f -e rs . src/output/

# Check main.rs to see how output is used
echo -e "\n=== Main.rs (first 100 lines) ==="
head -100 src/main.rs

Length of output: 5904

🌐 Web query:

YARA rule string syntax escaping requirements

💡 Result:

Short summary — how to escape strings in YARA:

Text (double‑quoted) strings: use C‑style escapes. Escape double quote and backslash with a backslash (" and \). Other supported escapes: \n, \r, \t and hex byte \xNN (where NN are two hex digits). Use these when you need literal control characters or to include quotes/backslashes. (yara.readthedocs.io)
Hex (curly‑brace) strings: enclosed in { ... } and represent raw bytes; do not use C escapes inside — specify bytes in hex (e.g. { 41 42 43 }), with wildcards (?) , ranges/jumps ([n] or [n-m]) and alternatives ((AA|BB)). (deepwiki.com)
Regular expressions in YARA follow PCRE‑like rules: use backslash to escape regex metacharacters when needed. (yara.readthedocs.io)

Notes:

YARA text strings are treated as byte sequences (historically ASCII-only; versions ≥4.1 warn about non‑ASCII in source). If you need specific byte values, prefer \xNN or hex strings. (yara.readthedocs.io)

Sources:
[1] YARA documentation — Writing YARA rules (text strings, escapes). (yara.readthedocs.io)
[2] YARA grammar / hex string syntax summary. (deepwiki.com)
[3] YARA regex/escape behavior (PCRE‑like). (yara.readthedocs.io)

Citations:

🌐 Web query:

YARA string modifiers ascii wide fullword nocase

💡 Result:

Quick summary of YARA text/regex string modifiers and how they behave:

ascii — match the plain ASCII form (default for non-wide). Use with wide to include both forms (e.g., "foo" wide ascii). [1][2]
wide — match the UTF‑16/2‑byte form produced by interleaving 0x00 bytes (e.g., "A" matches 41 00). Does not handle full UTF‑16 multibyte characters beyond simple interleaving. Can be combined with ascii and nocase. [1][2]
fullword — require the matched string be delimited by non‑alphanumeric characters (so "domain" fullword won’t match inside "mydomain"). Cannot be used with base64/base64wide. [1][2]
nocase — case‑insensitive matching for text and regex. Cannot be combined with xor, base64, or base64wide. Works with wide/ascii/fullword. [1][2]

Examples

$s = "password" nocase — case‑insensitive ASCII match. [1]
$s = "password" wide — match ASCII interleaved with 0x00 (UTF‑16 LE style). [1]
$s = "password" wide ascii nocase — match both ASCII and wide, case‑insensitive. Order of ascii/wide doesn’t matter. [1][2]
$s = "domain" fullword — match only when delimited by non‑alphanumerics. [1]

Notes / restrictions

ascii is usually unnecessary alone (ASCII is default) but needed when you want both ASCII+wide. [1]
Certain modifiers are mutually exclusive with base64/xor (see docs). [1][2]

Sources: YARA official docs (writing rules / string modifiers) and YARA‑X text patterns docs. [1][2]

🏁 Script executed:

#!/bin/bash
# Update the issue with enhanced content
gh issue edit 28 \
  --title "Implement YARA Rule Output Formatter with String Escaping and Modifiers" \
  --body "## Overview

This task implements a YARA rule formatter for StringyMcStringFace that outputs extracted strings in valid YARA rule syntax. YARA is a widely-used pattern matching tool in malware research and incident response, making YARA-compatible output a critical feature for security analysts who want to create detection rules from extracted strings.

## Context

The binary analyzer currently extracts strings from ELF, PE, and Mach-O binaries with rich metadata including:
- Encoding types (ASCII, UTF-8, UTF-16LE, UTF-16BE)
- Semantic tags (URLs, domains, IPs, file paths, etc.)
- Offsets, RVAs, and section information
- Relevance scores

To maximize utility for security practitioners, we need to format this data as valid YARA rules that can be immediately used for threat hunting and detection.

## Technical Requirements

### 1. String Escaping
Implement proper C-style escaping for YARA text strings:
- Escape double quotes: \`\\\"\`
- Escape backslashes: \`\\\\\`
- Escape newlines: \`\\n\`
- Escape carriage returns: \`\\r\`
- Escape tabs: \`\\t\`
- Non-printable bytes: \`\\xNN\` (hex notation)

### 2. Encoding Mapping
Map \`FoundString\` encoding types to YARA string modifiers:
- \`Encoding::Ascii\` / \`Encoding::Utf8\` → \`ascii\` (default)
- \`Encoding::Utf16Le\` / \`Encoding::Utf16Be\` → \`wide\` modifier
- Consider adding \`ascii wide\` for broader matching when appropriate

### 3. String Truncation
Apply truncation rules for excessively long strings:
- Maximum string length: 256 bytes (configurable)
- Truncate with indicator comment (e.g., \`// truncated from N bytes\`)
- Consider hex string format for binary data or very long strings

### 4. Semantic Tag Integration
Leverage semantic tags to enhance rule conditions:
- Group strings by tag type in rule conditions
- Add metadata comments indicating tag classifications
- Generate compound conditions (e.g., \`any of (\$url*)\`)

### 5. Rule Structure
Generate complete, valid YARA rules:
\`\`\`yara
rule binary_name_strings {
    meta:
        description = \"Extracted strings from binary_name\"
        format = \"ELF/PE/MachO\"
        generated_by = \"StringyMcStringFace\"
    
    strings:
        \$s1 = \"extracted_string\" ascii
        \$s2 = \"wide_string\" wide
        \$url1 = \"https://example.com\" nocase
    
    condition:
        any of them
}
\`\`\`

## Proposed Implementation

### File: \`src/output/yara.rs\`

\`\`\`rust
pub struct YaraFormatter {
    max_string_length: usize,
    include_metadata: bool,
    rule_name: String,
}

impl YaraFormatter {
    pub fn format_rule(&self, strings: &[FoundString], binary_info: &ContainerInfo) -> String;
    fn escape_string(&self, s: &str) -> String;
    fn get_string_modifiers(&self, string: &FoundString) -> Vec<&str>;
    fn truncate_if_needed(&self, s: &str) -> (String, bool);
    fn generate_condition(&self, strings: &[FoundString]) -> String;
}
\`\`\`

### Module Registration: \`src/output/mod.rs\`

\`\`\`rust
pub mod yara;

pub enum OutputFormat {
    Json,
    Yara,
    // ... other formats
}

pub trait Formatter {
    fn format(&self, strings: &[FoundString], info: &ContainerInfo) -> String;
}
\`\`\`

## Example Output

Given extracted strings from a binary, the formatter should produce:

\`\`\`yara
rule suspicious_binary_strings {
    meta:
        description = \"Strings extracted from suspicious.exe\"
        format = \"PE\"
        generated_by = \"StringyMcStringFace v0.1\"
        extracted_count = 47
    
    strings:
        // URLs and network indicators
        \$url1 = \"http://malicious.example.com/payload\" nocase
        \$ip1 = \"192.168.1.100\" ascii
        
        // File paths
        \$path1 = \"C:\\\\Windows\\\\System32\\\\evil.dll\" nocase
        
        // Wide strings (UTF-16)
        \$wide1 = \"WideString\" wide
        
        // Import/Export names
        \$imp1 = \"CreateRemoteThread\" ascii
        \$imp2 = \"VirtualAllocEx\" ascii
    
    condition:
        any of (\$url*) or 
        2 of (\$imp*) or
        any of (\$path*)
}
\`\`\`

## Test Scenarios

Unit tests should cover:

1. **String Escaping**
   - Quotes, backslashes, newlines in strings
   - Non-ASCII characters (hex escape)
   - Already-escaped content (no double-escaping)

2. **Encoding Handling**
   - ASCII strings → \`ascii\` modifier
   - UTF-16LE/BE → \`wide\` modifier
   - Mixed encoding in single rule

3. **Truncation**
   - Strings under limit → no truncation
   - Strings over limit → truncated with comment
   - Extreme cases (empty strings, very long strings)

4. **Rule Generation**
   - Valid YARA syntax (parseable by YARA)
   - Proper section formatting (meta, strings, condition)
   - Special characters in rule names

5. **Semantic Tags**
   - URLs grouped and commented
   - Network indicators (IPs, domains)
   - Import/Export grouping

## Dependencies

- **Blocked by**: Output Formatting Framework (#TBD)
  - Need base \`OutputFormat\` enum and \`Formatter\` trait
  - May proceed with initial implementation if framework is defined

## Acceptance Criteria

- [ ] \`src/output/yara.rs\` implements \`YaraFormatter\`
- [ ] All special characters properly escaped per YARA specification
- [ ] Encoding types correctly mapped to YARA modifiers
- [ ] Long strings truncated at configurable threshold
- [ ] Generated rules pass YARA syntax validation
- [ ] Comprehensive unit tests with >90% coverage
- [ ] Integration test with sample binaries (ELF, PE, Mach-O)
- [ ] Documentation with usage examples

## References

- [YARA Documentation - Writing Rules](https://yara.readthedocs.io/en/stable/writingrules.html)
- [YARA String Syntax](https://yara.readthedocs.io/en/stable/writingrules.html#strings)
- [YARA String Modifiers](https://yara.readthedocs.io/en/stable/writingrules.html#text-strings)

## Related Issues

- Output Formatting Framework (dependency)
- Requirement 6.3 implementation

---

**Task ID**: stringy-analyzer/yara-friendly-output"

# Verify the update
echo -e "\n=== Updated Issue ==="
gh issue view 28 --json title,body --jq '{title: .title, body: .body}'

Length of output: 6365

@unclesp1d3r I've successfully enhanced issue #28! Here's what was updated:

Changes Made

📝 Updated Title

"Implement YARA Rule Output Formatter with String Escaping and Modifiers"

More descriptive and technically specific

📋 Enhanced Issue Body

Added comprehensive sections including:

Overview & Context - Explains YARA's importance in malware analysis and how it fits into StringyMcStringFace's architecture
Technical Requirements - Five detailed sections covering:
- C-style string escaping (\", \\, \n, \r, \t, \xNN)
- Encoding-to-modifier mapping (ASCII, UTF-8, UTF-16)
- String truncation rules (256 byte default)
- Semantic tag integration for smart rule conditions
- Complete YARA rule structure
Proposed Implementation - Concrete API design for YaraFormatter with key methods:
- format_rule() - Main entry point
- escape_string() - Handle special characters
- get_string_modifiers() - Map encodings to YARA modifiers
- truncate_if_needed() - Handle long strings
- generate_condition() - Build rule conditions
Example Output - Shows realistic YARA rule with:
- Metadata section
- Categorized strings (URLs, paths, wide strings, imports)
- Intelligent conditions using wildcards
Test Scenarios - Five comprehensive test categories with specific cases
Acceptance Criteria - Clear checklist with 8 measurable items
References - Links to official YARA documentation

🏷️ Metadata Status

The issue already had excellent metadata:

✅ Labels: area:analyzer, type:enhancement, status:backlog, lang:rust, needs:tests
✅ Assigned: unclesp1d3r
✅ Milestone: v0.1

No changes were needed to labels or assignments.

The issue is now ready for implementation with clear technical specifications, example code structure, and comprehensive test requirements! 🎯

🐰✨
(Ready to help security analysts turn extracted strings into detection rules!)

I have updated the Implementation Plan to reflect the recent changes in the ticket.

New Plan

Observations

The codebase has a well-defined structure with all necessary types in src/types.rs. The src/output/mod.rs module is currently empty and ready for the output framework. The project uses serde for serialization, thiserror for errors, and insta for snapshot testing. The extraction modules already return Vec<FoundString> with complete metadata including encoding, tags, source, offset, RVA, and section information. The documentation clearly describes the expected YARA output format with proper escaping, modifiers, and rule structure.

Approach

Implement a two-layer solution: (1) Define the output formatting framework with OutputFormat enum and Formatter trait in src/output/mod.rs, and (2) Implement YaraFormatter in src/output/yara.rs with proper string escaping, encoding-to-modifier mapping, truncation logic, semantic tag grouping, and complete YARA rule generation. The formatter will be configurable via a YaraConfig struct and will generate valid YARA rules that can be immediately used for threat hunting.

Reasoning

I explored the codebase structure, examined existing types and extraction modules, reviewed documentation for YARA output requirements, researched YARA string escaping specifications, and analyzed the testing patterns used in the project. I confirmed that all necessary data structures exist and the output module is ready for implementation.

Proposed File Changes

📄 src/output/mod.rs ^(MODIFY) 🔗

References

📄 src/types.rs

Define the output formatting framework by creating:

OutputFormat enum with variants: Json, Yara, Human (for future expansion)

Formatter trait with a format method that takes &[FoundString] and &ContainerInfo and returns Result<String>

Module declarations for the yara submodule

Re-exports for public types like YaraFormatter and YaraConfig

The trait design should be simple and extensible, allowing future formatters (JSON, human-readable) to be added easily. The format method signature should accept string data and container metadata, returning a formatted string or error.

📄 src/output/yara.rs ^(NEW) 🔗

References

📄 src/types.rs

📄 src/output/mod.rs ^(MODIFY)

📄 tests/integration_pe.rs

Implement the YARA rule formatter with the following components:

YaraConfig struct with fields:

max_string_length: usize (default 256)

include_metadata: bool (default true)

rule_name: String (derived from binary name or custom)

min_score: Option<i32> (filter strings by score)

group_by_tags: bool (default true)

YaraFormatter struct that holds a YaraConfig and implements the Formatter trait

String escaping function escape_string(&str) -> String that implements C-style escaping:

Escape double quotes as \"

Escape backslashes as \\

Escape newlines as \n

Escape carriage returns as \r

Escape tabs as \t

Convert non-printable bytes (< 0x20 or > 0x7E) to \xNN hex notation

Handle already-escaped content to avoid double-escaping

Encoding mapper get_string_modifiers(&FoundString) -> Vec<&str> that maps:

Encoding::Ascii or Encoding::Utf8 → ["ascii"]

Encoding::Utf16Le or Encoding::Utf16Be → ["wide"]

Add "nocase" modifier for URLs, domains, and file paths based on tags

Consider "ascii", "wide" for broader matching when appropriate

Truncation function truncate_if_needed(&str, max_len) -> (String, bool) that:

Returns (original, false) if length <= max_len

Returns (truncated, true) if length > max_len

Truncates at byte boundary to avoid breaking UTF-8 sequences

Tag grouping logic that organizes strings by semantic tags:

Group URLs together with $url_N naming

Group IPs with $ip_N naming

Group file paths with $path_N naming

Group imports/exports with $imp_N and $exp_N naming

Ungrouped strings use $s_N naming

Condition generator generate_condition(&[FoundString]) -> String that creates:

Simple any of them for small rule sets

Compound conditions like any of ($url*) or 2 of ($imp*) for grouped strings

Tag-based logic when strings are grouped by semantic meaning

Rule generator format_rule(&[FoundString], &ContainerInfo) -> String that produces:

Rule header with sanitized rule name (replace non-alphanumeric with underscore)

Meta section with description, format (ELF/PE/MachO), generated_by, timestamp

Strings section with properly escaped strings, modifiers, and comments

Condition section with appropriate matching logic

Comments indicating truncation, scores, and tag classifications

The implementation should handle edge cases like empty string lists, very long strings, special characters in rule names, and strings with mixed encodings. Reference the YARA specification for valid syntax at https://yara.readthedocs.io/en/stable/writingrules.html
Add comprehensive unit tests in a #[cfg(test)] module covering:

String escaping tests:

Test escaping of quotes: "hello" → \"hello\"

Test escaping of backslashes: C:\\path → C:\\\\path

Test escaping of newlines: line1\nline2 → line1\\nline2

Test escaping of tabs and carriage returns

Test non-printable characters: \x01\x02 → \\x01\\x02

Test already-escaped content to ensure no double-escaping

Test mixed printable and non-printable characters

Encoding modifier tests:

Test ASCII encoding → ascii modifier

Test UTF-8 encoding → ascii modifier (YARA treats UTF-8 as ASCII)

Test UTF-16LE encoding → wide modifier

Test UTF-16BE encoding → wide modifier

Test nocase modifier for URLs and domains

Truncation tests:

Test strings under limit (no truncation)

Test strings at exact limit (no truncation)

Test strings over limit (truncated with indicator)

Test UTF-8 boundary handling (don't break multi-byte sequences)

Test very long strings (1000+ bytes)

Rule generation tests:

Test empty string list (should return valid minimal rule)

Test single string (simple rule)

Test multiple strings with different encodings

Test strings with various tags (URL, IP, filepath, import)

Test rule name sanitization (spaces, special chars → underscores)

Test metadata generation with different BinaryFormat values

Tag grouping tests:

Test URL grouping ($url_1, $url_2, etc.)

Test IP grouping ($ip_1, $ip_2, etc.)

Test import/export grouping

Test mixed tagged and untagged strings

Condition generation tests:

Test simple condition for ungrouped strings

Test compound condition with multiple tag groups

Test condition with score filtering

Integration tests:

Create synthetic FoundString vectors mimicking real extraction results

Test complete rule generation with realistic data

Verify generated rules are syntactically valid YARA (can be parsed)

Use helper functions to create test FoundString instances with various properties. Consider using insta for snapshot testing of complete rule output similar to patterns in tests/integration_pe.rs.

📄 tests/yara_formatter_integration.rs ^(NEW) 🔗

References

📄 tests/integration_pe.rs

📄 src/extraction/mod.rs

Create integration tests that combine real extraction with YARA formatting:

Test with PE fixture:

Use tests/fixtures/test_binary_with_resources.exe

Extract strings using stringy::extraction::extract_resource_strings

Format as YARA rule using YaraFormatter

Verify rule structure with snapshot testing

Check that version strings, string table entries are properly formatted

Test with ELF fixture:

Use tests/fixtures/test_binary_elf

Parse with ElfParser to get ContainerInfo

Create sample FoundString entries (extraction not yet implemented)

Format as YARA rule

Verify ELF-specific metadata in rule

Test with Mach-O fixture:

Use tests/fixtures/test_binary_macho

Extract load command strings using extract_load_command_strings

Format as YARA rule

Verify framework paths and dylib paths are properly formatted

Snapshot tests:

Use insta::assert_snapshot! to capture complete YARA rule output

Create snapshots for different binary formats

Verify rule syntax remains stable across changes

Edge case tests:

Test with empty extraction results

Test with very large string sets (100+ strings)

Test with strings containing only special characters

Test with mixed encodings in single binary

Follow the pattern established in tests/integration_pe.rs for fixture loading and test structure. Use the get_fixture_path helper pattern for consistent fixture access.
📄 src/lib.rs ^(MODIFY) 🔗
References

📄 src/output/mod.rs ^(MODIFY)

Update the library documentation and re-exports:
Update the module documentation comment to reflect that the output module now has YARA formatter implementation

Add re-exports for the new output types:

pub use output::{Formatter, OutputFormat, YaraFormatter, YaraConfig};

Update the architecture section to mark output module as partially complete (YARA formatter done)
Add a usage example showing how to use the YARA formatter:
use stringy::output::{YaraFormatter, YaraConfig};
use stringy::extraction::extract_resource_strings;

let data = std::fs::read("binary.exe")?;
let strings = extract_resource_strings(&data);
let config = YaraConfig::default().with_rule_name("binary_strings");
let formatter = YaraFormatter::new(config);
let yara_rule = formatter.format(&strings, &container_info)?;
println!("{}", yara_rule);
This makes the YARA formatter easily accessible to library users and documents its usage pattern.
📄 README.md ^(MODIFY) 🔗
Update the README to reflect the new YARA output capability:
Add YARA formatter to the features list
Include a quick example showing YARA rule generation:
use stringy::output::{YaraFormatter, YaraConfig};

// Extract and format as YARA rule
let formatter = YaraFormatter::default();
let yara_rule = formatter.format(&strings, &info)?;
Update the roadmap or status section to mark YARA output as implemented

Add a note about YARA rule validation and usage in the security analysis workflow
Keep the changes minimal and focused on documenting the new capability without overhauling the entire README.
Diagram
sequenceDiagram
    participant User
    participant YaraFormatter
    participant Escaper
    participant ModifierMapper
    participant RuleGenerator
    
    User->>YaraFormatter: format(strings, container_info)
    YaraFormatter->>YaraFormatter: Filter by min_score
    YaraFormatter->>YaraFormatter: Group strings by tags
    
    loop For each string
        YaraFormatter->>Escaper: escape_string(text)
        Escaper-->>YaraFormatter: escaped_text
        YaraFormatter->>ModifierMapper: get_string_modifiers(encoding, tags)
        ModifierMapper-->>YaraFormatter: ["ascii"/"wide", "nocase"]
        YaraFormatter->>YaraFormatter: truncate_if_needed(text, max_len)
    end
    
    YaraFormatter->>RuleGenerator: generate_rule_header(rule_name)
    YaraFormatter->>RuleGenerator: generate_meta_section(container_info)
    YaraFormatter->>RuleGenerator: generate_strings_section(grouped_strings)
    YaraFormatter->>RuleGenerator: generate_condition(grouped_strings)
    
    RuleGenerator-->>YaraFormatter: Complete YARA rule
    YaraFormatter-->>User: Result<String>
Loading
Import In IDE
🤖 Prompt for AI Agents
I have created the following plan after thorough exploration and analysis of the codebase. Follow the below plan verbatim. Trust the files and references. Do not re-verify what's written in the plan. Explore only when absolutely necessary. First implement all the proposed file changes and then I'll review all the changes together at the end.

### Observations

The codebase has a well-defined structure with all necessary types in `src/types.rs`. The `src/output/mod.rs` module is currently empty and ready for the output framework. The project uses `serde` for serialization, `thiserror` for errors, and `insta` for snapshot testing. The extraction modules already return `Vec<FoundString>` with complete metadata including encoding, tags, source, offset, RVA, and section information. The documentation clearly describes the expected YARA output format with proper escaping, modifiers, and rule structure.

### Approach

Implement a two-layer solution: (1) Define the output formatting framework with `OutputFormat` enum and `Formatter` trait in `src/output/mod.rs`, and (2) Implement `YaraFormatter` in `src/output/yara.rs` with proper string escaping, encoding-to-modifier mapping, truncation logic, semantic tag grouping, and complete YARA rule generation. The formatter will be configurable via a `YaraConfig` struct and will generate valid YARA rules that can be immediately used for threat hunting.

### Reasoning

I explored the codebase structure, examined existing types and extraction modules, reviewed documentation for YARA output requirements, researched YARA string escaping specifications, and analyzed the testing patterns used in the project. I confirmed that all necessary data structures exist and the output module is ready for implementation.

## Mermaid Diagram

sequenceDiagram
    participant User
    participant YaraFormatter
    participant Escaper
    participant ModifierMapper
    participant RuleGenerator
    
    User->>YaraFormatter: format(strings, container_info)
    YaraFormatter->>YaraFormatter: Filter by min_score
    YaraFormatter->>YaraFormatter: Group strings by tags
    
    loop For each string
        YaraFormatter->>Escaper: escape_string(text)
        Escaper-->>YaraFormatter: escaped_text
        YaraFormatter->>ModifierMapper: get_string_modifiers(encoding, tags)
        ModifierMapper-->>YaraFormatter: ["ascii"/"wide", "nocase"]
        YaraFormatter->>YaraFormatter: truncate_if_needed(text, max_len)
    end
    
    YaraFormatter->>RuleGenerator: generate_rule_header(rule_name)
    YaraFormatter->>RuleGenerator: generate_meta_section(container_info)
    YaraFormatter->>RuleGenerator: generate_strings_section(grouped_strings)
    YaraFormatter->>RuleGenerator: generate_condition(grouped_strings)
    
    RuleGenerator-->>YaraFormatter: Complete YARA rule
    YaraFormatter-->>User: Result<String>

## Proposed File Changes

### src/output/mod.rs(MODIFY)

References: 

- src/types.rs

Define the output formatting framework by creating:

1. **OutputFormat enum** with variants: Json, Yara, Human (for future expansion)
2. **Formatter trait** with a `format` method that takes `&[FoundString]` and `&ContainerInfo` and returns `Result<String>`
3. **Module declarations** for the yara submodule
4. **Re-exports** for public types like `YaraFormatter` and `YaraConfig`

The trait design should be simple and extensible, allowing future formatters (JSON, human-readable) to be added easily. The `format` method signature should accept string data and container metadata, returning a formatted string or error.

### src/output/yara.rs(NEW)

References: 

- src/types.rs
- src/output/mod.rs(MODIFY)
- tests/integration_pe.rs

Implement the YARA rule formatter with the following components:

1. **YaraConfig struct** with fields:
   - `max_string_length: usize` (default 256)
   - `include_metadata: bool` (default true)
   - `rule_name: String` (derived from binary name or custom)
   - `min_score: Option<i32>` (filter strings by score)
   - `group_by_tags: bool` (default true)

2. **YaraFormatter struct** that holds a `YaraConfig` and implements the `Formatter` trait

3. **String escaping function** `escape_string(&str) -> String` that implements C-style escaping:
   - Escape double quotes as `\"`
   - Escape backslashes as `\\`
   - Escape newlines as `\n`
   - Escape carriage returns as `\r`
   - Escape tabs as `\t`
   - Convert non-printable bytes (< 0x20 or > 0x7E) to `\xNN` hex notation
   - Handle already-escaped content to avoid double-escaping

4. **Encoding mapper** `get_string_modifiers(&FoundString) -> Vec<&str>` that maps:
   - `Encoding::Ascii` or `Encoding::Utf8` → `["ascii"]`
   - `Encoding::Utf16Le` or `Encoding::Utf16Be` → `["wide"]`
   - Add `"nocase"` modifier for URLs, domains, and file paths based on tags
   - Consider `"ascii", "wide"` for broader matching when appropriate

5. **Truncation function** `truncate_if_needed(&str, max_len) -> (String, bool)` that:
   - Returns (original, false) if length <= max_len
   - Returns (truncated, true) if length > max_len
   - Truncates at byte boundary to avoid breaking UTF-8 sequences

6. **Tag grouping logic** that organizes strings by semantic tags:
   - Group URLs together with `$url_N` naming
   - Group IPs with `$ip_N` naming
   - Group file paths with `$path_N` naming
   - Group imports/exports with `$imp_N` and `$exp_N` naming
   - Ungrouped strings use `$s_N` naming

7. **Condition generator** `generate_condition(&[FoundString]) -> String` that creates:
   - Simple `any of them` for small rule sets
   - Compound conditions like `any of ($url*) or 2 of ($imp*)` for grouped strings
   - Tag-based logic when strings are grouped by semantic meaning

8. **Rule generator** `format_rule(&[FoundString], &ContainerInfo) -> String` that produces:
   - Rule header with sanitized rule name (replace non-alphanumeric with underscore)
   - Meta section with description, format (ELF/PE/MachO), generated_by, timestamp
   - Strings section with properly escaped strings, modifiers, and comments
   - Condition section with appropriate matching logic
   - Comments indicating truncation, scores, and tag classifications

The implementation should handle edge cases like empty string lists, very long strings, special characters in rule names, and strings with mixed encodings. Reference the YARA specification for valid syntax at https://yara.readthedocs.io/en/stable/writingrules.html
Add comprehensive unit tests in a `#[cfg(test)]` module covering:

1. **String escaping tests**:
   - Test escaping of quotes: `"hello"` → `\"hello\"`
   - Test escaping of backslashes: `C:\\path` → `C:\\\\path`
   - Test escaping of newlines: `line1\nline2` → `line1\\nline2`
   - Test escaping of tabs and carriage returns
   - Test non-printable characters: `\x01\x02` → `\\x01\\x02`
   - Test already-escaped content to ensure no double-escaping
   - Test mixed printable and non-printable characters

2. **Encoding modifier tests**:
   - Test ASCII encoding → `ascii` modifier
   - Test UTF-8 encoding → `ascii` modifier (YARA treats UTF-8 as ASCII)
   - Test UTF-16LE encoding → `wide` modifier
   - Test UTF-16BE encoding → `wide` modifier
   - Test nocase modifier for URLs and domains

3. **Truncation tests**:
   - Test strings under limit (no truncation)
   - Test strings at exact limit (no truncation)
   - Test strings over limit (truncated with indicator)
   - Test UTF-8 boundary handling (don't break multi-byte sequences)
   - Test very long strings (1000+ bytes)

4. **Rule generation tests**:
   - Test empty string list (should return valid minimal rule)
   - Test single string (simple rule)
   - Test multiple strings with different encodings
   - Test strings with various tags (URL, IP, filepath, import)
   - Test rule name sanitization (spaces, special chars → underscores)
   - Test metadata generation with different BinaryFormat values

5. **Tag grouping tests**:
   - Test URL grouping (`$url_1`, `$url_2`, etc.)
   - Test IP grouping (`$ip_1`, `$ip_2`, etc.)
   - Test import/export grouping
   - Test mixed tagged and untagged strings

6. **Condition generation tests**:
   - Test simple condition for ungrouped strings
   - Test compound condition with multiple tag groups
   - Test condition with score filtering

7. **Integration tests**:
   - Create synthetic `FoundString` vectors mimicking real extraction results
   - Test complete rule generation with realistic data
   - Verify generated rules are syntactically valid YARA (can be parsed)

Use helper functions to create test `FoundString` instances with various properties. Consider using `insta` for snapshot testing of complete rule output similar to patterns in `tests/integration_pe.rs`.

### tests/yara_formatter_integration.rs(NEW)

References: 

- tests/integration_pe.rs
- src/extraction/mod.rs

Create integration tests that combine real extraction with YARA formatting:

1. **Test with PE fixture**:
   - Use `tests/fixtures/test_binary_with_resources.exe`
   - Extract strings using `stringy::extraction::extract_resource_strings`
   - Format as YARA rule using `YaraFormatter`
   - Verify rule structure with snapshot testing
   - Check that version strings, string table entries are properly formatted

2. **Test with ELF fixture**:
   - Use `tests/fixtures/test_binary_elf`
   - Parse with `ElfParser` to get `ContainerInfo`
   - Create sample `FoundString` entries (extraction not yet implemented)
   - Format as YARA rule
   - Verify ELF-specific metadata in rule

3. **Test with Mach-O fixture**:
   - Use `tests/fixtures/test_binary_macho`
   - Extract load command strings using `extract_load_command_strings`
   - Format as YARA rule
   - Verify framework paths and dylib paths are properly formatted

4. **Snapshot tests**:
   - Use `insta::assert_snapshot!` to capture complete YARA rule output
   - Create snapshots for different binary formats
   - Verify rule syntax remains stable across changes

5. **Edge case tests**:
   - Test with empty extraction results
   - Test with very large string sets (100+ strings)
   - Test with strings containing only special characters
   - Test with mixed encodings in single binary

Follow the pattern established in `tests/integration_pe.rs` for fixture loading and test structure. Use the `get_fixture_path` helper pattern for consistent fixture access.

### src/lib.rs(MODIFY)

References: 

- src/output/mod.rs(MODIFY)

Update the library documentation and re-exports:

1. Update the module documentation comment to reflect that the output module now has YARA formatter implementation
2. Add re-exports for the new output types:
   - `pub use output::{Formatter, OutputFormat, YaraFormatter, YaraConfig};`
3. Update the architecture section to mark output module as partially complete (YARA formatter done)
4. Add a usage example showing how to use the YARA formatter:
   ```rust
   use stringy::output::{YaraFormatter, YaraConfig};
   use stringy::extraction::extract_resource_strings;
   
   let data = std::fs::read("binary.exe")?;
   let strings = extract_resource_strings(&data);
   let config = YaraConfig::default().with_rule_name("binary_strings");
   let formatter = YaraFormatter::new(config);
   let yara_rule = formatter.format(&strings, &container_info)?;
   println!("{}", yara_rule);
   ```

This makes the YARA formatter easily accessible to library users and documents its usage pattern.

### README.md(MODIFY)

Update the README to reflect the new YARA output capability:

1. Add YARA formatter to the features list
2. Include a quick example showing YARA rule generation:
   ```rust
   use stringy::output::{YaraFormatter, YaraConfig};
   
   // Extract and format as YARA rule
   let formatter = YaraFormatter::default();
   let yara_rule = formatter.format(&strings, &info)?;
   ```
3. Update the roadmap or status section to mark YARA output as implemented
4. Add a note about YARA rule validation and usage in the security analysis workflow

Keep the changes minimal and focused on documenting the new capability without overhauling the entire README.
Developer Humor

🔍 From binary chaos to YARA zen,\n> We escape the quotes and backslash them.\n> Wide strings get wide, ASCII stays tight,\n> Truncate the long ones, keep rules just right!\n> 🎯 Now threat hunters can detect with delight! ✨

Execution Information

Branch: main
Commit: baf4526

Implemented: YARA rule formatter with escaping/modifiers in src/output/yara/ (--yara). Closing as completed.

Implement YARA Rule Output Formatter with String Escaping and Modifiers #28

Description

Overview

Context

Technical Requirements

1. String Escaping

2. Encoding Mapping

3. String Truncation

4. Semantic Tag Integration

5. Rule Structure

Proposed Implementation

File: src/output/yara.rs

Module Registration: src/output/mod.rs

Example Output

Test Scenarios

Dependencies

Acceptance Criteria

References

Related Issues

Activity

unclesp1d3r commented on Nov 10, 2025

traycerai commented on Nov 10, 2025

Plan

Observations

Approach

Reasoning

Proposed File Changes

Diagram

Import In IDE

Developer Humor

Supported Commands (Inside Comments)

Supported Commands (Inside Description)

Community

coderabbitai commented on Nov 10, 2025

Changes Made

📝 Updated Title

📋 Enhanced Issue Body

🏷️ Metadata Status

traycerai commented on Nov 12, 2025

Observations

Approach

Reasoning

Proposed File Changes

Diagram

Import In IDE

Developer Humor

unclesp1d3r commented on Jul 4, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

File: `src/output/yara.rs`

Module Registration: `src/output/mod.rs`