Enhance ELF Dynamic Symbol Extraction with Library Mapping and Comprehensive Symbol Classification

Summary

Enhance the existing ELF import/export extraction to comprehensively parse the dynamic section, extract library dependencies (DT_NEEDED entries), map symbols to their originating libraries, and improve symbol classification beyond just functions.

Context

The current ELF parser in src/container/elf.rs provides basic import/export extraction by analyzing the dynamic symbol table (.dynsym). However, it has several limitations:

No library mapping: Imports don't identify which shared library they come from (DT_NEEDED entries are not parsed)
Limited symbol types: Only extracts function symbols (STT_FUNC), missing data objects, TLS variables, etc.
Incomplete dynamic section parsing: The dynamic section contains additional metadata (version requirements, symbol versioning, etc.) that isn't currently extracted
Missing symbol visibility: Doesn't account for symbol visibility (STV_DEFAULT, STV_HIDDEN, etc.)

This enhancement extends the existing functionality to provide more complete import/export analysis, matching the comprehensiveness of the PE parser implementation.

Technical Background

ELF Dynamic Section Structure:

Contains DT_NEEDED entries that specify required shared libraries
Includes DT_SYMTAB, DT_STRTAB, and DT_HASH/DT_GNU_HASH for symbol resolution
May contain version information via DT_VERNEED, DT_VERDEF, and DT_VERSYM

Symbol Classification:

Imports: Undefined symbols (SHN_UNDEF) that need to be resolved at link/load time
Exports: Defined symbols with global/weak binding available for other modules
Symbol types: Functions (STT_FUNC), objects (STT_OBJECT), TLS (STT_TLS), IFuncs (STT_GNU_IFUNC)

Proposed Solution

Implementation Steps

Parse DT_NEEDED entries from dynamic section
- Extract all required library names
- Build a mapping of libraries for symbol attribution
Enhance import extraction (extract_imports)
- Keep current undefined symbol detection
- Extend to handle all symbol types (not just STT_FUNC):
  - STT_OBJECT (data objects)
  - STT_TLS (thread-local storage)
  - STT_GNU_IFUNC (indirect functions)
- Attempt to map symbols to libraries using version information when available
- Preserve symbol visibility and binding information
Enhance export extraction (extract_exports)
- Include all globally visible defined symbols
- Add support for weak symbols (STB_WEAK)
- Include symbol type information in ExportInfo
- Filter out hidden symbols (STV_HIDDEN, STV_INTERNAL)
Extend data structures if needed
- May need to extend ImportInfo to include symbol type and version
- May need to extend ExportInfo to include symbol type and binding
Add comprehensive unit tests
- Test DT_NEEDED extraction with mock ELF data
- Test symbol classification for various types
- Test edge cases (weak symbols, versioned symbols, hidden symbols)
- Use insta snapshots for symbol extraction results
Add integration tests with real binaries
- Test with sample ELF binaries (e.g., /bin/ls equivalent)
- Verify correct library mapping
- Validate symbol counts and types

Code Structure

// New helper method
fn extract_needed_libraries(&self, elf: &Elf) -> Vec<String> {
    // Parse DT_NEEDED entries from elf.dynamic
}

// Enhanced import extraction
fn extract_imports(&self, elf: &Elf, libraries: &[String]) -> Vec<ImportInfo> {
    // Current logic + handle all symbol types + library mapping
}

// Enhanced export extraction  
fn extract_exports(&self, elf: &Elf) -> Vec<ExportInfo> {
    // Current logic + handle weak symbols + filter hidden
}

Requirements

4.2, 4.3

Acceptance Criteria

✅ Parse DT_NEEDED entries and extract library dependencies
✅ Map imported symbols to their originating libraries where possible
✅ Extract all relevant symbol types (functions, objects, TLS, IFuncs)
✅ Properly classify symbols as imports vs exports based on definition and binding
✅ Handle weak symbols and symbol visibility correctly
✅ Add comprehensive unit tests with cargo test passing
✅ Add integration tests with real ELF binaries
✅ Use insta for snapshot testing of symbol extraction
✅ Ensure cargo clippy -- -D warnings passes
✅ Update documentation in docs/ explaining the enhanced extraction
✅ Add benchmarks with criterion if performance-sensitive paths are added
✅ Update justfile recipes if new test targets are added
✅ Ensure CI passes

Dependencies

Blocked by: Implement Intelligent ELF Section Classification for Targeted String Extraction #1 ELF Section Classification (appears to be already implemented based on codebase)

Task-ID

stringy-analyzer/elf-import-export-extraction

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

@unclesp1d3r

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch the details of issue #2
gh issue view 2 --json title,body,labels,assignees,state,url

# Also get repository information to understand the project context
gh repo view --json name,description,languages

# Get a sense of the codebase structure
fd -t f -e rb -e rake -e gemspec | head -20

Length of output: 2108

🏁 Script executed:

#!/bin/bash
# Look for existing ELF-related code to understand the current implementation
rg -n "ELF|elf" --type rust -C 2 | head -50

# Find any existing symbol or dynamic section handling
ast-grep --pattern 'struct $_ { $$$ }'  | rg -i "symbol|dynamic|elf" | head -20

# Check for any existing tests related to ELF
fd test -e rs --exec rg -l "elf|ELF" {} | head -10

# Look at the project structure
ls -la
find . -name "*.rs" -type f | head -20

Length of output: 3652

🏁 Script executed:

#!/bin/bash
# Examine the ELF parser implementation
cat src/container/elf.rs

# Look at the types to understand ImportInfo and ExportInfo structures
cat src/types.rs | grep -A 20 "struct ImportInfo\|struct ExportInfo\|struct SectionInfo"

# Check the container mod to understand the structure
cat src/container/mod.rs

Length of output: 10703

🏁 Script executed:

#!/bin/bash
# Update the issue with enhanced content
gh issue edit 2 \
  --title "Enhance ELF Dynamic Symbol Extraction with Library Mapping and Comprehensive Symbol Classification" \
  --body "## Summary

Enhance the existing ELF import/export extraction to comprehensively parse the dynamic section, extract library dependencies (DT_NEEDED entries), map symbols to their originating libraries, and improve symbol classification beyond just functions.

## Context

The current ELF parser in \`src/container/elf.rs\` provides basic import/export extraction by analyzing the dynamic symbol table (\`.dynsym\`). However, it has several limitations:

1. **No library mapping**: Imports don't identify which shared library they come from (DT_NEEDED entries are not parsed)
2. **Limited symbol types**: Only extracts function symbols (STT_FUNC), missing data objects, TLS variables, etc.
3. **Incomplete dynamic section parsing**: The dynamic section contains additional metadata (version requirements, symbol versioning, etc.) that isn't currently extracted
4. **Missing symbol visibility**: Doesn't account for symbol visibility (STV_DEFAULT, STV_HIDDEN, etc.)

This enhancement extends the existing functionality to provide more complete import/export analysis, matching the comprehensiveness of the PE parser implementation.

## Technical Background

**ELF Dynamic Section Structure:**
- Contains \`DT_NEEDED\` entries that specify required shared libraries
- Includes \`DT_SYMTAB\`, \`DT_STRTAB\`, and \`DT_HASH\`/\`DT_GNU_HASH\` for symbol resolution
- May contain version information via \`DT_VERNEED\`, \`DT_VERDEF\`, and \`DT_VERSYM\`

**Symbol Classification:**
- **Imports**: Undefined symbols (SHN_UNDEF) that need to be resolved at link/load time
- **Exports**: Defined symbols with global/weak binding available for other modules
- **Symbol types**: Functions (STT_FUNC), objects (STT_OBJECT), TLS (STT_TLS), IFuncs (STT_GNU_IFUNC)

## Proposed Solution

### Implementation Steps

1. **Parse DT_NEEDED entries from dynamic section**
   - Extract all required library names
   - Build a mapping of libraries for symbol attribution

2. **Enhance import extraction (\`extract_imports\`)**
   - Keep current undefined symbol detection
   - Extend to handle all symbol types (not just STT_FUNC):
     - STT_OBJECT (data objects)
     - STT_TLS (thread-local storage)
     - STT_GNU_IFUNC (indirect functions)
   - Attempt to map symbols to libraries using version information when available
   - Preserve symbol visibility and binding information

3. **Enhance export extraction (\`extract_exports\`)**
   - Include all globally visible defined symbols
   - Add support for weak symbols (STB_WEAK)
   - Include symbol type information in ExportInfo
   - Filter out hidden symbols (STV_HIDDEN, STV_INTERNAL)

4. **Extend data structures if needed**
   - May need to extend \`ImportInfo\` to include symbol type and version
   - May need to extend \`ExportInfo\` to include symbol type and binding

5. **Add comprehensive unit tests**
   - Test DT_NEEDED extraction with mock ELF data
   - Test symbol classification for various types
   - Test edge cases (weak symbols, versioned symbols, hidden symbols)
   - Use insta snapshots for symbol extraction results

6. **Add integration tests with real binaries**
   - Test with sample ELF binaries (e.g., \`/bin/ls\` equivalent)
   - Verify correct library mapping
   - Validate symbol counts and types

### Code Structure

\`\`\`rust
// New helper method
fn extract_needed_libraries(&self, elf: &Elf) -> Vec<String> {
    // Parse DT_NEEDED entries from elf.dynamic
}

// Enhanced import extraction
fn extract_imports(&self, elf: &Elf, libraries: &[String]) -> Vec<ImportInfo> {
    // Current logic + handle all symbol types + library mapping
}

// Enhanced export extraction  
fn extract_exports(&self, elf: &Elf) -> Vec<ExportInfo> {
    // Current logic + handle weak symbols + filter hidden
}
\`\`\`

## Requirements

4.2, 4.3

## Acceptance Criteria

- ✅ Parse DT_NEEDED entries and extract library dependencies
- ✅ Map imported symbols to their originating libraries where possible
- ✅ Extract all relevant symbol types (functions, objects, TLS, IFuncs)
- ✅ Properly classify symbols as imports vs exports based on definition and binding
- ✅ Handle weak symbols and symbol visibility correctly
- ✅ Add comprehensive unit tests with \`cargo test\` passing
- ✅ Add integration tests with real ELF binaries
- ✅ Use insta for snapshot testing of symbol extraction
- ✅ Ensure \`cargo clippy -- -D warnings\` passes
- ✅ Update documentation in \`docs/\` explaining the enhanced extraction
- ✅ Add benchmarks with criterion if performance-sensitive paths are added
- ✅ Update justfile recipes if new test targets are added
- ✅ Ensure CI passes

## Dependencies

- **Blocked by**: #1 ELF Section Classification (appears to be already implemented based on codebase)

## Task-ID

stringy-analyzer/elf-import-export-extraction"

# Add priority:medium label
gh issue edit 2 --add-label "priority:medium"

echo "Issue updated successfully!"

Length of output: 221

🏁 Script executed:

#!/bin/bash
# List all available labels in the repository
gh label list

# Check if there are any priority-related labels
gh label list | grep -i priority

Length of output: 834

🏁 Script executed:

#!/bin/bash
# Create priority:medium label with an orange color
gh label create "priority:medium" --description "Medium priority task" --color "ff9800"

# Now add the label to issue #2
gh issue edit 2 --add-label "priority:medium"

# Verify the issue was updated correctly
gh issue view 2 --json title,labels,url | jq '.labels[].name'

Length of output: 156

@unclesp1d3r I've successfully enhanced issue #2 with the following improvements:

Title Enhancement

New title: "Enhance ELF Dynamic Symbol Extraction with Library Mapping and Comprehensive Symbol Classification"

This is more descriptive and clearly communicates the scope beyond just basic extraction.

Body Enhancements

✅ Added detailed context explaining the current limitations in src/container/elf.rs:

No library mapping from DT_NEEDED entries
Limited to function symbols only (missing data objects, TLS variables, etc.)
Incomplete dynamic section parsing
Missing symbol visibility handling

✅ Added technical background covering:

ELF dynamic section structure and key entries (DT_NEEDED, DT_SYMTAB, etc.)
Symbol classification rules (imports vs exports)
Symbol types (functions, objects, TLS, IFuncs)

✅ Added comprehensive proposed solution with:

Step-by-step implementation plan
Code structure examples showing new helper methods
Specific symbol types to handle
Data structure extensions if needed

✅ Enhanced acceptance criteria with specific, measurable goals including:

DT_NEEDED parsing
Library-to-symbol mapping
All symbol type extraction
Proper handling of weak symbols and visibility
Comprehensive testing with insta snapshots

Metadata Updates

✅ Created and applied priority:medium label (orange color)
✅ Retained existing labels:

area:analyzer
type:enhancement
status:backlog
lang:rust
needs:tests

The issue is now ready for implementation with clear technical direction and acceptance criteria! You can view it at: #2

@coderabbitai Enhance this issue body with additional context and a proposed solution. Improve the issue title and assign the proper labels and other metadata.

@unclesp1d3r

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check the current state of issue #2
gh issue view 2 --json title,body,labels,assignees,state,url

# Also verify if priority:medium label exists
gh label list | grep -i priority

Length of output: 5925

@unclesp1d3r The enhancement of issue #2 has already been completed! Here's a summary of what was applied:

✅ Completed Enhancements

Title Updated:
"Enhance ELF Dynamic Symbol Extraction with Library Mapping and Comprehensive Symbol Classification"

Body Enhancements Applied:

✅ Summary - Clear overview of the enhancement goals
✅ Context - Detailed explanation of current ELF parser limitations in src/container/elf.rs
✅ Technical Background - ELF dynamic section structure and symbol classification details
✅ Proposed Solution - 6-step implementation plan with code structure examples
✅ Acceptance Criteria - Comprehensive checklist covering all aspects (DT_NEEDED parsing, symbol mapping, testing, etc.)
✅ Requirements - 4.2, 4.3
✅ Dependencies - Noted dependency on Implement Intelligent ELF Section Classification for Targeted String Extraction #1
✅ Task-ID - stringy-analyzer/elf-import-export-extraction

Labels Applied:

✅ area:analyzer - Binary analyzer functionality
✅ type:enhancement - New feature or request
✅ status:backlog - Task in backlog
✅ lang:rust - Rust implementation
✅ needs:tests - Needs test coverage
✅ priority:medium - Medium priority task (orange)

Assignment:

✅ Assigned to you (unclesp1d3r)

The issue is now ready for implementation with comprehensive technical direction! You can view it here: #2

Plan

Observations

The codebase has a well-structured container parsing system with ELF, PE, and Mach-O parsers. The ELF parser currently extracts imports/exports but has limitations:

Library mapping returns None: The extract_library_from_needed method exists but is a placeholder
Limited symbol types: Only handles STT_FUNC, STT_OBJECT, STT_NOTYPE
No visibility filtering: Doesn't filter hidden symbols from exports
No version information: Doesn't use ELF's versioning tables

Key findings from exploration:

Goblin provides elf.libraries (Vec<&str>) with all DT_NEEDED entries already parsed
Goblin exposes elf.versym, elf.verdef, elf.verneed for symbol versioning
Symbol type constants available: sym::STT_TLS, sym::STT_GNU_IFUNC
Visibility helpers: sym::st_visibility(), sym::STV_HIDDEN, sym::STV_INTERNAL
PE parser shows the pattern: library: Some(import.dll.to_string())
No existing benchmarks or snapshot tests yet
CI runs clippy with -D warnings, nextest for testing, and llvm-cov for coverage

Approach

Enhance the ELF parser to provide comprehensive symbol extraction matching PE parser capabilities:

Extract DT_NEEDED libraries using elf.libraries field
Map symbols to libraries using version information (best-effort via elf.verneed)
Add missing symbol types (STT_TLS, STT_GNU_IFUNC) to import/export extraction
Filter hidden symbols from exports using visibility checks
Optionally extend data structures to include symbol metadata (type, binding, visibility, version)
Add comprehensive tests with insta snapshots and real ELF binaries
Update documentation explaining the enhanced extraction capabilities

Reasoning

I explored the codebase structure, examined the existing ELF parser implementation, compared it with PE and Mach-O parsers, reviewed the data structures in types.rs, checked integration tests, and researched goblin's API for ELF dynamic section parsing and symbol versioning. I also investigated ELF symbol versioning mechanisms to understand how to map symbols to libraries.

Proposed File Changes

📄 src/container/elf.rs ^(MODIFY) 🔗

References

📄 src/types.rs ^(MODIFY)

Add library extraction method:

Replace the placeholder extract_library_from_needed method to return the list of DT_NEEDED libraries from elf.libraries

Add a new helper method extract_needed_libraries that returns Vec<String> from elf.libraries

Enhance import extraction:

Update extract_imports to accept a libraries parameter: fn extract_imports(&self, elf: &Elf, libraries: &[String])

Add support for STT_TLS (thread-local storage) and STT_GNU_IFUNC (indirect functions) symbol types using goblin::elf::sym::STT_TLS and goblin::elf::sym::STT_GNU_IFUNC constants

Implement best-effort library mapping using elf.verneed and elf.versym to match undefined symbols to their providing libraries

Add a helper method map_symbol_to_library that uses version information to determine which library provides a symbol

Keep the existing logic for both dynamic and static symbol tables

Enhance export extraction:

Update extract_exports to filter out hidden symbols using goblin::elf::sym::st_visibility() and checking against goblin::elf::sym::STV_HIDDEN and goblin::elf::sym::STV_INTERNAL

Add support for STT_TLS and STT_GNU_IFUNC in exports as well

Maintain the existing deduplication logic with HashSet

Update parse method:

Call extract_needed_libraries first to get the library list

Pass the libraries to extract_imports

Add helper methods for version information:

get_symbol_version_info(&self, elf: &Elf, sym_index: usize) -> Option<(String, String)> - returns (library_name, version_string) for undefined symbols using elf.versym and elf.verneed

is_symbol_hidden(&self, elf: &Elf, sym: &goblin::elf::Sym) -> bool - checks if a symbol has hidden visibility

Add comprehensive unit tests:

Test extract_needed_libraries with mock data

Test symbol type filtering for TLS and IFUNC

Test visibility filtering for hidden symbols

Test version information extraction

Use the existing test structure as a pattern

📄 src/types.rs ^(MODIFY) 🔗

Optionally extend ImportInfo and ExportInfo structures:

Evaluate whether to add optional fields for enhanced metadata:

symbol_type: Option<String> - to store "FUNC", "OBJECT", "TLS", "IFUNC", etc.

binding: Option<String> - to store "GLOBAL", "WEAK", etc.

visibility: Option<String> - to store "DEFAULT", "HIDDEN", etc.

version: Option<String> - to store version information like "GLIBC_2.28"

Decision criteria:

If these fields would be useful for downstream consumers (classification, ranking, output)

If PE and Mach-O parsers would also benefit from these fields

Keep fields optional to maintain backward compatibility

Consider that the current simple structure may be sufficient for the string extraction use case

Alternative approach:

Keep the current simple structures and document the enhanced extraction in comments

The library field in ImportInfo is already sufficient for the main requirement

Recommendation: Start without extending the structures, as the library mapping is the primary requirement. The symbol type and visibility information can be added later if needed for classification or ranking.

📄 tests/integration_elf.rs ^(MODIFY) 🔗

References

📄 src/container/elf.rs ^(MODIFY)

Add comprehensive integration tests:

Test DT_NEEDED extraction:

Add test test_elf_dt_needed_extraction that compiles a C program linking to multiple libraries (e.g., -lm -lpthread)

Verify that container_info.imports includes symbols from those libraries

Check that the library field is populated for at least some imports

Test symbol type extraction:

Add test test_elf_tls_symbol_extraction that uses __thread variables to create TLS symbols

Verify TLS symbols are extracted in imports/exports

Add test test_elf_ifunc_symbol_extraction if feasible (IFUNC requires more complex setup)

Test symbol visibility:

Add test test_elf_hidden_symbol_filtering that uses __attribute__((visibility("hidden"))) in C code

Verify hidden symbols are NOT in the exports list

Verify visible symbols ARE in the exports list

Add snapshot tests with insta:

Add test test_elf_import_export_snapshot that uses insta::assert_yaml_snapshot! to capture the full import/export structure

Use a deterministic test binary (compiled with fixed flags)

This will catch any regressions in symbol extraction

Enhance existing tests:

Update test_elf_import_export_extraction_dynamic to check for library field population

Add assertions for symbol counts and specific expected symbols

Add more detailed output on test failures

Test edge cases:

Test with stripped binaries (no symbol table)

Test with statically linked binaries (no dynamic symbols)

Test with binaries that have version information

Follow the existing test pattern of compiling C code with gcc/x86_64-linux-gnu-gcc and handling platform differences gracefully.

📁 tests/snapshots ^(NEW) 🔗

Create directory for insta snapshot files. Insta will automatically populate this directory with .snap files when snapshot tests run.

The directory structure will be:
tests/snapshots/
  integration_elf__elf_import_export_snapshot.snap
  integration_elf__elf_tls_symbol_snapshot.snap
  (other snapshot files as tests are added)
No manual file creation needed - insta handles this automatically.

📄 docs/src/binary-formats.md ^(MODIFY) 🔗

References

📄 src/container/elf.rs ^(MODIFY)

Update ELF section to document enhanced import/export extraction:

Add subsection "Import/Export Extraction" under "ELF-Specific Features":

Document that imports are extracted from both .dynsym and .symtab

Explain that library mapping uses DT_NEEDED entries and symbol versioning information

Note that the mapping is best-effort since ELF doesn't directly link symbols to libraries

List supported symbol types: FUNC, OBJECT, NOTYPE, TLS, GNU_IFUNC

Explain that hidden symbols (STV_HIDDEN, STV_INTERNAL) are filtered from exports

Add subsection "Library Dependency Mapping":

Explain how DT_NEEDED entries are parsed from the dynamic section

Describe the use of .gnu.version, .gnu.version_r (verneed), and .gnu.version_d (verdef) for version-based mapping

Note that unversioned symbols may not have library attribution

Provide example showing how a symbol like malloc@GLIBC_2.2.5 maps to libc.so.6

Add subsection "Symbol Types and Visibility":

Document the different symbol types extracted:

STT_FUNC: Function symbols

STT_OBJECT: Data object symbols

STT_TLS: Thread-local storage variables

STT_GNU_IFUNC: Indirect functions (runtime-resolved)

STT_NOTYPE: Untyped symbols

Explain visibility filtering:

STV_DEFAULT: Exported and visible

STV_HIDDEN: Not exported (filtered out)

STV_INTERNAL: Internal use only (filtered out)

Update the implementation example:

Show the enhanced extract_imports signature with libraries parameter

Add example of library mapping logic

Show visibility filtering in extract_exports

Reference the existing PE parser documentation as a comparison point for library mapping.

📁 benches ^(NEW) 🔗

Create benchmarks directory for criterion benchmarks.

Decision: Only add benchmarks if performance-sensitive paths are identified during implementation. The issue mentions "Add benchmarks with criterion if performance-sensitive paths are added."

Potential benchmarks to consider:

Parsing large ELF binaries with many symbols

Version information lookup for symbols

Symbol filtering and deduplication

For now, create the directory structure but defer actual benchmark implementation until performance concerns are identified. The existing parsing logic is likely fast enough for typical binaries.

📄 benches/elf_parsing.rs ^(NEW) 🔗

References

📄 src/container/elf.rs ^(MODIFY)

Create benchmark file for ELF parsing performance (if needed):

Use criterion to benchmark:

parse() method on various ELF binary sizes

extract_imports() with different symbol counts

extract_exports() with different symbol counts

Version information lookup performance

Structure:
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use stringy::container::{ContainerParser, ElfParser};

fn benchmark_elf_parsing(c: &mut Criterion) {
    // Load test binary
    let data = std::fs::read("path/to/test/binary").unwrap();
    let parser = ElfParser::new();
    
    c.bench_function("elf_parse", |b| {
        b.iter(|| parser.parse(black_box(&data)))
    });
}

criterion_group!(benches, benchmark_elf_parsing);
criterion_main!(benches);
Note: Only implement if performance testing reveals bottlenecks. The current implementation is likely efficient enough for typical use cases.

📄 Cargo.toml ^(MODIFY) 🔗

Add benchmark configuration if benchmarks are implemented:

Add to the file:
[[bench]]
name = "elf_parsing"
harness = false
This is only needed if the benchmark file in benches/elf_parsing.rs is created. The criterion dependency is already in [dev-dependencies], so no additional dependencies are needed.

Note: This change is optional and should only be made if performance benchmarks are actually implemented.

📄 justfile ^(MODIFY) 🔗

Verify benchmark recipe exists:

The justfile already has a bench recipe at line 252:
bench:
    @cargo bench --workspace
No changes needed. This recipe will automatically run any benchmarks in the benches/ directory when they are added.

Verify test recipe:
The existing test recipe at line 210 uses cargo nextest run --workspace --no-capture, which will run all tests including the new integration tests.

No changes needed to the justfile.

Diagram

sequenceDiagram
    participant Parser as ElfParser
    participant Goblin as goblin::elf::Elf
    participant Types as ImportInfo/ExportInfo
    
    Note over Parser: parse() called with ELF data
    
    Parser->>Goblin: Parse ELF binary
    Goblin-->>Parser: Elf struct with sections, symbols, dynamic info
    
    Parser->>Parser: extract_needed_libraries(elf)
    Parser->>Goblin: Access elf.libraries (DT_NEEDED)
    Goblin-->>Parser: Vec<String> of library names
    
    Parser->>Parser: extract_imports(elf, libraries)
    
    loop For each symbol in dynsyms
        Parser->>Goblin: Check sym.st_shndx == SHN_UNDEF
        Parser->>Goblin: Check sym.st_type() (FUNC, OBJECT, TLS, IFUNC, NOTYPE)
        Parser->>Goblin: Check sym.st_bind() (GLOBAL, WEAK)
        
        alt Symbol is undefined import
            Parser->>Parser: map_symbol_to_library(elf, sym_index)
            Parser->>Goblin: Access elf.versym[sym_index]
            Parser->>Goblin: Access elf.verneed (match version index)
            Goblin-->>Parser: Library name from version info
            Parser->>Types: Create ImportInfo with library
        end
    end
    
    Parser->>Parser: extract_exports(elf)
    
    loop For each symbol in dynsyms
        Parser->>Goblin: Check sym.st_shndx != SHN_UNDEF
        Parser->>Goblin: Check st_visibility(sym.st_other)
        
        alt Symbol is visible (not HIDDEN/INTERNAL)
            Parser->>Goblin: Check symbol type (FUNC, OBJECT, TLS, IFUNC)
            Parser->>Types: Create ExportInfo
        end
    end
    
    Parser-->>Types: Return ContainerInfo with enhanced imports/exports

Import In IDE

🤖 Prompt for AI Agents

I have created the following plan after thorough exploration and analysis of the codebase. Follow the below plan verbatim. Trust the files and references. Do not re-verify what's written in the plan. Explore only when absolutely necessary. First implement all the proposed file changes and then I'll review all the changes together at the end.

### Observations

The codebase has a well-structured container parsing system with ELF, PE, and Mach-O parsers. The ELF parser currently extracts imports/exports but has limitations:

1. **Library mapping returns None**: The `extract_library_from_needed` method exists but is a placeholder
2. **Limited symbol types**: Only handles STT_FUNC, STT_OBJECT, STT_NOTYPE
3. **No visibility filtering**: Doesn't filter hidden symbols from exports
4. **No version information**: Doesn't use ELF's versioning tables

**Key findings from exploration:**
- Goblin provides `elf.libraries` (Vec<&str>) with all DT_NEEDED entries already parsed
- Goblin exposes `elf.versym`, `elf.verdef`, `elf.verneed` for symbol versioning
- Symbol type constants available: `sym::STT_TLS`, `sym::STT_GNU_IFUNC`
- Visibility helpers: `sym::st_visibility()`, `sym::STV_HIDDEN`, `sym::STV_INTERNAL`
- PE parser shows the pattern: `library: Some(import.dll.to_string())`
- No existing benchmarks or snapshot tests yet
- CI runs clippy with `-D warnings`, nextest for testing, and llvm-cov for coverage

### Approach

Enhance the ELF parser to provide comprehensive symbol extraction matching PE parser capabilities:

1. **Extract DT_NEEDED libraries** using `elf.libraries` field
2. **Map symbols to libraries** using version information (best-effort via `elf.verneed`)
3. **Add missing symbol types** (STT_TLS, STT_GNU_IFUNC) to import/export extraction
4. **Filter hidden symbols** from exports using visibility checks
5. **Optionally extend data structures** to include symbol metadata (type, binding, visibility, version)
6. **Add comprehensive tests** with insta snapshots and real ELF binaries
7. **Update documentation** explaining the enhanced extraction capabilities

### Reasoning

I explored the codebase structure, examined the existing ELF parser implementation, compared it with PE and Mach-O parsers, reviewed the data structures in types.rs, checked integration tests, and researched goblin's API for ELF dynamic section parsing and symbol versioning. I also investigated ELF symbol versioning mechanisms to understand how to map symbols to libraries.

## Mermaid Diagram

sequenceDiagram
    participant Parser as ElfParser
    participant Goblin as goblin::elf::Elf
    participant Types as ImportInfo/ExportInfo
    
    Note over Parser: parse() called with ELF data
    
    Parser->>Goblin: Parse ELF binary
    Goblin-->>Parser: Elf struct with sections, symbols, dynamic info
    
    Parser->>Parser: extract_needed_libraries(elf)
    Parser->>Goblin: Access elf.libraries (DT_NEEDED)
    Goblin-->>Parser: Vec<String> of library names
    
    Parser->>Parser: extract_imports(elf, libraries)
    
    loop For each symbol in dynsyms
        Parser->>Goblin: Check sym.st_shndx == SHN_UNDEF
        Parser->>Goblin: Check sym.st_type() (FUNC, OBJECT, TLS, IFUNC, NOTYPE)
        Parser->>Goblin: Check sym.st_bind() (GLOBAL, WEAK)
        
        alt Symbol is undefined import
            Parser->>Parser: map_symbol_to_library(elf, sym_index)
            Parser->>Goblin: Access elf.versym[sym_index]
            Parser->>Goblin: Access elf.verneed (match version index)
            Goblin-->>Parser: Library name from version info
            Parser->>Types: Create ImportInfo with library
        end
    end
    
    Parser->>Parser: extract_exports(elf)
    
    loop For each symbol in dynsyms
        Parser->>Goblin: Check sym.st_shndx != SHN_UNDEF
        Parser->>Goblin: Check st_visibility(sym.st_other)
        
        alt Symbol is visible (not HIDDEN/INTERNAL)
            Parser->>Goblin: Check symbol type (FUNC, OBJECT, TLS, IFUNC)
            Parser->>Types: Create ExportInfo
        end
    end
    
    Parser-->>Types: Return ContainerInfo with enhanced imports/exports

## Proposed File Changes

### src/container/elf.rs(MODIFY)

References: 

- src/types.rs(MODIFY)

**Add library extraction method:**
- Replace the placeholder `extract_library_from_needed` method to return the list of DT_NEEDED libraries from `elf.libraries`
- Add a new helper method `extract_needed_libraries` that returns `Vec<String>` from `elf.libraries`

**Enhance import extraction:**
- Update `extract_imports` to accept a libraries parameter: `fn extract_imports(&self, elf: &Elf, libraries: &[String])`
- Add support for `STT_TLS` (thread-local storage) and `STT_GNU_IFUNC` (indirect functions) symbol types using `goblin::elf::sym::STT_TLS` and `goblin::elf::sym::STT_GNU_IFUNC` constants
- Implement best-effort library mapping using `elf.verneed` and `elf.versym` to match undefined symbols to their providing libraries
- Add a helper method `map_symbol_to_library` that uses version information to determine which library provides a symbol
- Keep the existing logic for both dynamic and static symbol tables

**Enhance export extraction:**
- Update `extract_exports` to filter out hidden symbols using `goblin::elf::sym::st_visibility()` and checking against `goblin::elf::sym::STV_HIDDEN` and `goblin::elf::sym::STV_INTERNAL`
- Add support for STT_TLS and STT_GNU_IFUNC in exports as well
- Maintain the existing deduplication logic with `HashSet`

**Update parse method:**
- Call `extract_needed_libraries` first to get the library list
- Pass the libraries to `extract_imports`

**Add helper methods for version information:**
- `get_symbol_version_info(&self, elf: &Elf, sym_index: usize) -> Option<(String, String)>` - returns (library_name, version_string) for undefined symbols using `elf.versym` and `elf.verneed`
- `is_symbol_hidden(&self, elf: &Elf, sym: &goblin::elf::Sym) -> bool` - checks if a symbol has hidden visibility

**Add comprehensive unit tests:**
- Test `extract_needed_libraries` with mock data
- Test symbol type filtering for TLS and IFUNC
- Test visibility filtering for hidden symbols
- Test version information extraction
- Use the existing test structure as a pattern

### src/types.rs(MODIFY)

**Optionally extend ImportInfo and ExportInfo structures:**

Evaluate whether to add optional fields for enhanced metadata:
- `symbol_type: Option<String>` - to store "FUNC", "OBJECT", "TLS", "IFUNC", etc.
- `binding: Option<String>` - to store "GLOBAL", "WEAK", etc.
- `visibility: Option<String>` - to store "DEFAULT", "HIDDEN", etc.
- `version: Option<String>` - to store version information like "GLIBC_2.28"

**Decision criteria:**
- If these fields would be useful for downstream consumers (classification, ranking, output)
- If PE and Mach-O parsers would also benefit from these fields
- Keep fields optional to maintain backward compatibility
- Consider that the current simple structure may be sufficient for the string extraction use case

**Alternative approach:**
- Keep the current simple structures and document the enhanced extraction in comments
- The library field in ImportInfo is already sufficient for the main requirement

**Recommendation:** Start without extending the structures, as the library mapping is the primary requirement. The symbol type and visibility information can be added later if needed for classification or ranking.

### tests/integration_elf.rs(MODIFY)

References: 

- src/container/elf.rs(MODIFY)

**Add comprehensive integration tests:**

**Test DT_NEEDED extraction:**
- Add test `test_elf_dt_needed_extraction` that compiles a C program linking to multiple libraries (e.g., `-lm -lpthread`)
- Verify that `container_info.imports` includes symbols from those libraries
- Check that the library field is populated for at least some imports

**Test symbol type extraction:**
- Add test `test_elf_tls_symbol_extraction` that uses `__thread` variables to create TLS symbols
- Verify TLS symbols are extracted in imports/exports
- Add test `test_elf_ifunc_symbol_extraction` if feasible (IFUNC requires more complex setup)

**Test symbol visibility:**
- Add test `test_elf_hidden_symbol_filtering` that uses `__attribute__((visibility("hidden")))` in C code
- Verify hidden symbols are NOT in the exports list
- Verify visible symbols ARE in the exports list

**Add snapshot tests with insta:**
- Add test `test_elf_import_export_snapshot` that uses `insta::assert_yaml_snapshot!` to capture the full import/export structure
- Use a deterministic test binary (compiled with fixed flags)
- This will catch any regressions in symbol extraction

**Enhance existing tests:**
- Update `test_elf_import_export_extraction_dynamic` to check for library field population
- Add assertions for symbol counts and specific expected symbols
- Add more detailed output on test failures

**Test edge cases:**
- Test with stripped binaries (no symbol table)
- Test with statically linked binaries (no dynamic symbols)
- Test with binaries that have version information

Follow the existing test pattern of compiling C code with gcc/x86_64-linux-gnu-gcc and handling platform differences gracefully.

### tests/snapshots(NEW)

Create directory for insta snapshot files. Insta will automatically populate this directory with `.snap` files when snapshot tests run.

The directory structure will be:
```
tests/snapshots/
  integration_elf__elf_import_export_snapshot.snap
  integration_elf__elf_tls_symbol_snapshot.snap
  (other snapshot files as tests are added)
```

No manual file creation needed - insta handles this automatically.

### docs/src/binary-formats.md(MODIFY)

References: 

- src/container/elf.rs(MODIFY)

**Update ELF section to document enhanced import/export extraction:**

**Add subsection "Import/Export Extraction" under "ELF-Specific Features":**
- Document that imports are extracted from both `.dynsym` and `.symtab`
- Explain that library mapping uses DT_NEEDED entries and symbol versioning information
- Note that the mapping is best-effort since ELF doesn't directly link symbols to libraries
- List supported symbol types: FUNC, OBJECT, NOTYPE, TLS, GNU_IFUNC
- Explain that hidden symbols (STV_HIDDEN, STV_INTERNAL) are filtered from exports

**Add subsection "Library Dependency Mapping":**
- Explain how DT_NEEDED entries are parsed from the dynamic section
- Describe the use of `.gnu.version`, `.gnu.version_r` (verneed), and `.gnu.version_d` (verdef) for version-based mapping
- Note that unversioned symbols may not have library attribution
- Provide example showing how a symbol like `malloc@GLIBC_2.2.5` maps to `libc.so.6`

**Add subsection "Symbol Types and Visibility":**
- Document the different symbol types extracted:
  - STT_FUNC: Function symbols
  - STT_OBJECT: Data object symbols
  - STT_TLS: Thread-local storage variables
  - STT_GNU_IFUNC: Indirect functions (runtime-resolved)
  - STT_NOTYPE: Untyped symbols
- Explain visibility filtering:
  - STV_DEFAULT: Exported and visible
  - STV_HIDDEN: Not exported (filtered out)
  - STV_INTERNAL: Internal use only (filtered out)

**Update the implementation example:**
- Show the enhanced `extract_imports` signature with libraries parameter
- Add example of library mapping logic
- Show visibility filtering in `extract_exports`

Reference the existing PE parser documentation as a comparison point for library mapping.

### benches(NEW)

Create benchmarks directory for criterion benchmarks.

**Decision:** Only add benchmarks if performance-sensitive paths are identified during implementation. The issue mentions "Add benchmarks with criterion if performance-sensitive paths are added."

**Potential benchmarks to consider:**
- Parsing large ELF binaries with many symbols
- Version information lookup for symbols
- Symbol filtering and deduplication

For now, create the directory structure but defer actual benchmark implementation until performance concerns are identified. The existing parsing logic is likely fast enough for typical binaries.

### benches/elf_parsing.rs(NEW)

References: 

- src/container/elf.rs(MODIFY)

**Create benchmark file for ELF parsing performance (if needed):**

Use criterion to benchmark:
- `parse()` method on various ELF binary sizes
- `extract_imports()` with different symbol counts
- `extract_exports()` with different symbol counts
- Version information lookup performance

Structure:
```rust
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use stringy::container::{ContainerParser, ElfParser};

fn benchmark_elf_parsing(c: &mut Criterion) {
    // Load test binary
    let data = std::fs::read("path/to/test/binary").unwrap();
    let parser = ElfParser::new();
    
    c.bench_function("elf_parse", |b| {
        b.iter(|| parser.parse(black_box(&data)))
    });
}

criterion_group!(benches, benchmark_elf_parsing);
criterion_main!(benches);
```

**Note:** Only implement if performance testing reveals bottlenecks. The current implementation is likely efficient enough for typical use cases.

### Cargo.toml(MODIFY)

**Add benchmark configuration if benchmarks are implemented:**

Add to the file:
```toml
[[bench]]
name = "elf_parsing"
harness = false
```

This is only needed if the benchmark file in `benches/elf_parsing.rs` is created. The criterion dependency is already in `[dev-dependencies]`, so no additional dependencies are needed.

**Note:** This change is optional and should only be made if performance benchmarks are actually implemented.

### justfile(MODIFY)

**Verify benchmark recipe exists:**

The justfile already has a `bench` recipe at line 252:
```just
bench:
    @cargo bench --workspace
```

No changes needed. This recipe will automatically run any benchmarks in the `benches/` directory when they are added.

**Verify test recipe:**
The existing `test` recipe at line 210 uses `cargo nextest run --workspace --no-capture`, which will run all tests including the new integration tests.

No changes needed to the justfile.

Developer Humor

🔍 The ELF's New Clothes\n>\n> An ELF walked into a bar,\n> Said \My symbols come from afar!\n> With DT_NEEDED in hand,\n> And versions so grand,\n> Now we know which .so they are! ⭐\n>\n> 🧵 TLS threads weave through the night,\n> 🎭 Hidden symbols stay out of sight,\n> 📚 Libraries mapped with care,\n> 🔧 IFUNC resolves with flair,\n> The ELF parser shines so bright! ✨

Execution Information

Branch: main
Commit: 6402946

💡 Tips

Supported Commands (Inside Comments)

Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

Navigation Menu

Repository navigation

Enhance ELF Dynamic Symbol Extraction with Library Mapping and Comprehensive Symbol Classification #2

Summary

Context

Technical Background

Proposed Solution

Implementation Steps

Code Structure

Requirements

Acceptance Criteria

Dependencies

Task-ID

Title Enhancement

Body Enhancements

Metadata Updates

✅ Completed Enhancements

Plan

Observations

Approach

Reasoning

Proposed File Changes

Diagram

Import In IDE

Developer Humor

Supported Commands (Inside Comments)

Supported Commands (Inside Description)

Community

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Enhance ELF Dynamic Symbol Extraction with Library Mapping and Comprehensive Symbol Classification #2

Description

Summary

Context

Technical Background

Proposed Solution

Implementation Steps

Code Structure

Requirements

Acceptance Criteria

Dependencies

Task-ID

Activity

unclesp1d3r commented on Sep 30, 2025

coderabbitai commented on Sep 30, 2025

Title Enhancement

Body Enhancements

Metadata Updates

unclesp1d3r commented on Nov 10, 2025

coderabbitai commented on Nov 10, 2025

✅ Completed Enhancements

traycerai commented on Nov 10, 2025

Plan

Observations

Approach

Reasoning

Proposed File Changes

Diagram

Import In IDE

Developer Humor

Supported Commands (Inside Comments)

Supported Commands (Inside Description)

Community

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions