Changelog: Firm-Based IO Organization System (WIP)

Date: 2025-11-30 Type: Architecture Refactor Scope: Directory structure, versioning, data organization Status: Work in Progress (uncommitted)

Summary

Began major refactor to reorganize the flat data/ and output/ directories into a hierarchical io/ directory organized by firm and deal. This enables private git submodules for firm-specific data while keeping the orchestrator open-source.

Problem

The previous flat structure had several limitations:

investment-memo-orchestrator/
├── data/                    # All firms' input data mixed together
│   ├── Aito.json
│   ├── Avalanche.json
│   └── TheoryForge.json
├── output/                  # All firms' outputs mixed together
│   ├── Aito-v0.0.1/
│   ├── Avalanche-v0.0.3/
│   └── TheoryForge-v0.0.2/
└── versions.json            # Single global version tracking

Issues:

• All firms' proprietary deal data in one public repository

• No clear ownership boundaries between firms

• Cannot gitignore or submodule firm-specific data

• Difficult to navigate at scale (50+ deals across multiple firms)

• No separation between inputs (decks, datarooms) and outputs (memos, scorecards)

Solution

New Directory Structure

investment-memo-orchestrator/
├── io/                              # All firm IO in one place
│   ├── .gitignore                   # Ignores firm dirs by default
│   └── hypernova/                   # Git submodule (private repo)
│       ├── configs/                 # Firm-specific configs
│       ├── deals/                   # Deal directories
│       │   ├── Aalo/
│       │   │   ├── inputs/          # deal.json, deck.pdf, dataroom/
│       │   │   ├── outputs/         # versioned artifacts
│       │   │   └── exports/         # HTML/PDF exports
│       │   ├── Avalanche/
│       │   ├── TheoryForge/
│       │   └── ... (17 deals total)
│       └── versions.json            # Firm-scoped version tracking

Work Completed

1. Git Submodule Setup (Committed: 9216a84)

git submodule add <private-repo-url> io/hypernova

• Created io/hypernova as a git submodule pointing to private repository

• Enables firm data to have its own commit history, access control, and branches

2. Versioning System Refactor (src/versioning.py)

Enhanced VersionManager class to support both firm-scoped and legacy modes:

class VersionManager:
    """
    Supports two modes:
    1. Firm-scoped: io/{firm}/versions.json
    2. Legacy: output/versions.json
    """

    def __init__(self, output_dir=None, firm=None, io_root=None):
        if firm:
            # Firm-scoped mode
            self.versions_file = io_root / firm / "versions.json"
        else:
            # Legacy mode (backward compatible)
            self.versions_file = output_dir / "versions.json"

    @classmethod
    def from_deal_path(cls, deal_path) -> "VersionManager":
        """Auto-detect mode from path structure."""

    @classmethod
    def for_firm(cls, firm: str) -> "VersionManager":
        """Create for specific firm."""

    @classmethod
    def legacy(cls) -> "VersionManager":
        """Create in legacy mode."""

New methods added:

• get_deal_output_dir(deal_name, version) - Returns correct path based on mode

• get_relative_file_path(deal_name, version, filename) - For version history storage

3. Migration Functions (src/versioning.py)

def migrate_versions_to_firm(
    legacy_versions_file: Path,
    firm: str,
    deals_to_migrate: list[str],
    io_root: Path = DEFAULT_IO_ROOT,
    dry_run: bool = False
) -> Dict[str, Any]:
    """Migrate deals from legacy versions.json to firm-scoped versions.json."""

def get_firm_deals(io_root: Path, firm: str) -> list[str]:
    """Get list of deal names from io/{firm}/deals/."""

4. Migration CLI Script (cli/migrate_versions.py)

Standalone script for migrating version history:

# Dry run - preview what would be migrated
python cli/migrate_versions.py --firm hypernova --dry-run

# Actually migrate
python cli/migrate_versions.py --firm hypernova

# Migrate specific deals only
python cli/migrate_versions.py --firm hypernova --deals Aalo,Ontra,Avalanche

Features:

• Auto-detects deals from io/{firm}/deals/ directory

• Handles name mappings (e.g., Aalo-Atomics → Aalo)

• Updates file paths in version history to new structure

• Supports dry-run mode

5. Data Migration (Uncommitted)

Moved 14 deal JSON files from data/ to io/hypernova/deals/{deal}/inputs/:

Also deleted: Avalanche-corrections.yaml (moved to Avalanche deal)

6. Firm Versions Created

io/hypernova/versions.json now contains version history for 15 deals:

• Aalo (v0.0.5), Aito (v0.0.2), Avalanche (v0.0.5), Bruin (v0.0.1)

• Class5-Global (v0.0.2), DontQuitVentures (v0.0.2), Harmonic (v0.0.3)

• KearnyJackson (v0.0.2), Ontra (v0.0.2), Star-Catcher (v0.0.1)

• TheoryForge (v0.0.3), Thinking-Machines (v0.0.1), Trela (v0.0.2)

• WatershedVC (v0.0.1), Yassir (v0.0.1)

Files Changed Summary

New Files

Modified Files

Deleted Files (14 data files moved to submodule)

• data/Aito.json, data/Andela.json, data/Avalanche.json

• data/Avalanche-corrections.yaml, data/BlueLayer.json

• data/Class5-Global.json, data/DontQuitVentures.json

• data/KearnyJackson.json, data/Ontra.json, data/TheoryForge.json

• data/Thinking-Machines.json, data/Trela.json, data/WatershedVC.json

Remaining Work

This refactor is incomplete. The following still needs to be done:

Phase 1: Path Resolution (High Priority)

1. Create src/paths.py - Centralized path resolution utilities:

   PYTHON

   def resolve_deal_path(firm: str, deal: str) -> Path
   def get_deal_inputs(deal_path: Path) -> Path
   def get_deal_outputs(deal_path: Path) -> Path
   def get_deal_exports(deal_path: Path) -> Path

2. Update src/main.py - Add --firm and --deal CLI flags

3. Update src/workflow.py - Use new path resolution

4. Update src/artifacts.py - Save to deal-specific outputs/

Phase 2: CLI Updates

Update all CLI scripts to support --firm and --deal:

• cli/improve-section.py

• cli/export_branded.py (new name from export-branded.py)

• cli/score_memo.py

• cli/refocus_section.py

• cli/recompile_memo.py

Phase 3: Dual-Mode Operation

1. Check io/{firm}/deals/{deal} first (new structure)

2. Fall back to data/{deal}.json + output/{deal}-v*/ (legacy)

3. Log deprecation warning when using legacy paths

4. Support --legacy flag to force old behavior

Phase 4: Environment & Documentation

1. Add environment variables:

   • MEMO_DEFAULT_FIRM - Default firm when not specified

   • MEMO_IO_ROOT - Override IO root directory

2. Update documentation:

   • README.md - New directory structure

   • io/README.md - Setup instructions for contributors

   • CLAUDE.md - Update all path references

Usage (Current State)

Using the New Structure

The system currently still expects legacy paths. To use the new structure:

1. Run memo generation (outputs still go to output/)

2. Manually copy outputs to io/hypernova/deals/{deal}/outputs/

3. Use migration script to update versions.json

Migration Script

# See what would be migrated
python cli/migrate_versions.py --firm hypernova --dry-run

# Run migration
python cli/migrate_versions.py --firm hypernova

Design Decisions

Per-Firm vs Per-Deal Versioning

Decision: Per-firm versions.json with deal name as key.

Rationale:

• Reduces file count (1 file per firm vs 1 per deal)

• Easier to backup/restore firm data

• Natural grouping for version comparisons within firm

Brand Configs Location

Decision: Stay in templates/brand-configs/, not io/{firm}/.

Rationale:

• Brand configs are visual styling, not sensitive data

• Can be shared across open-source users

• Firm-config.yaml references brand by name

Backward Compatibility

Decision: Dual-mode operation during migration.

Rationale:

• Non-breaking change for existing workflows

• Gradual migration path

• Deprecation warnings guide users to new structure

Contributors

Architecture design and implementation by Claude Code (Anthropic) with user guidance.