Changelog - 2025-11-23 (#2)

Resume Workflow Implementation

Overview

Implemented checkpoint-based resume functionality to recover from workflow interruptions without losing progress or re-running expensive API calls. System now detects completed agents from saved artifacts and resumes execution from the last successful checkpoint.

New Features

Resume Script (resume-from-last-interruption.py)

• NEW: Standalone script for resuming interrupted workflows

• NEW: Automatic checkpoint detection from artifact analysis

• NEW: State reconstruction from saved files (0-deck-analysis.json, 1-research.json, section files, etc.)

• NEW: Agent execution from detected checkpoint

• NEW: Support for version-specific resume (--version v0.0.3)

• NEW: Graceful error handling for missing/corrupted artifacts

Usage:

python resume-from-last-interruption.py "CompanyName"
python resume-from-last-interruption.py "CompanyName" --version v0.0.3

CLI Integration (src/main.py)

• NEW: Added --resume flag to main CLI

• NEW: Added --version parameter for resume-specific version selection

• NEW: Automatic delegation to resume script when --resume enabled

• NEW: Validation and error handling for non-existent artifacts

Usage:

python -m src.main "CompanyName" --resume
python -m src.main "CompanyName" --resume --version v0.0.3

Technical Implementation

Checkpoint Detection (detect_resume_point())

Analyzes existing artifacts to determine where to resume:

Detection Logic:

• Checks artifacts in reverse order (latest first)

• Validates JSON integrity before treating as checkpoint

• Examines file content to verify completion (e.g., LinkedIn links in Team section)

• Falls back gracefully if artifacts corrupted

State Reconstruction (reconstruct_state_from_artifacts())

Rebuilds MemoState from saved files:

1. Load company data: Calls create_initial_state() with company name

2. Override output directory: Uses existing artifact directory

3. Load deck analysis: From 0-deck-analysis.json if exists

4. Load research: From 1-research.json if exists

5. Prepare draft sections: Sets empty dict (sections live in files per new architecture)

6. Load validation: From 3-validation.json if exists

7. Load final memo: From 4-final-draft.md if exists

Key Insight: Sections stored in files (2-sections/*.md), not in state dict, per 2025-11-20 architecture refactor.

Agent Execution (execute_from_checkpoint())

Executes remaining agents from checkpoint:

Agent Sequence:

[
    ("research", research_agent_enhanced),
    ("draft", writer_agent),
    ("enrich_trademark", trademark_enrichment_agent),
    ("enrich_socials", socials_enrichment_agent),
    ("enrich_links", link_enrichment_agent),
    ("enrich_visualizations", visualization_enrichment_agent),
    ("cite", citation_enrichment_agent),
    ("validate_citations", citation_validator_agent),
    ("validate", validator_agent),
    ("finalize", finalize_memo),
]

Execution:

• Finds starting index based on resume point

• Runs agents sequentially from that index

• Updates state after each agent

• Handles validation threshold logic (score < 8.0 → human review)

• Catches and reports errors per agent

Benefits

Time Savings

• Skip completed agents (each 2-5 minutes for API-heavy operations)

• Resume from 80%+ complete workflows instead of restarting

• Example: Terrantic resumed from citation enrichment (agent 7/10), saved ~15 minutes

Cost Savings

Avoid redundant API calls:

• Deck analysis: ~$0.10 (PyMuPDF + Claude Sonnet)

• Research: ~$0.50 (Tavily/Perplexity searches)

• Writer: ~$2.00 (10 section drafts via Claude Sonnet)

• Citation enrichment: ~$1.50 (Perplexity Sonar Pro per section)

• Total savings per resume: ~$4.00+

User Experience

• No frustration from lost progress after interruptions

• Encourages experimentation (can stop/resume freely)

• Enables manual workflow editing (improve 1-research.json → resume → system uses edited data)

• Transparent progress tracking (shows completed artifacts before resuming)

Real-World Testing

Terrantic Case Study

Problem: Original workflow stuck for 3+ hours during enrichment phase

Solution: Resume workflow implementation

Results:

• Detected checkpoint: cite (citation enrichment)

• Resumed from partial completion (10 sections written, 0 citations)

• Successfully completed:

  • Citation enrichment: 367 citations across 10 sections

  • Global renumbering: Consolidated to 69 unique citations

  • Citation validation: Identified 69 errors, 3 warnings

  • Quality validation: Score 6.5/10

  • Final draft: output/Terrantic-v0.0.1/4-final-draft.md (55KB)

Time to complete: ~5 minutes from resume (vs. 20-30 minutes for full workflow)

Edge Cases Handled

1. Artifact Corruption

Problem: JSON file incomplete/corrupted (interrupted mid-write)

Solution: _is_valid_json() validates before treating as checkpoint

• Checks file size (> 10 bytes minimum)

• Attempts JSON parsing

• Returns False if corrupted → falls back to earlier checkpoint

2. Missing Artifacts

Problem: User requests resume but no artifacts exist

Solution: Graceful error with helpful message:

Error: No artifacts found for 'CompanyName'
Searched in: output/CompanyName-*
Run the normal workflow first:
  python -m src.main "CompanyName"

3. Partial Section Completion

Problem: Writer created 7/10 sections before crashing

Solution: Checkpoint detection requires 10 sections (len(sections) >= 10)

• If fewer than 10: Resume at draft agent (regenerates all sections)

• Alternative approach (future): Smart resume that only writes missing sections

4. Multi-Version Scenarios

Problem: Multiple incomplete versions exist (v0.0.1, v0.0.2, v0.0.3)

Solution:

• Default: Resume latest version via get_latest_output_dir()

• Explicit: --version v0.0.2 flag to specify version

• No ambiguity (always deterministic)

5. Already Complete

Problem: User runs resume on finished memo

Solution: Detects completion from state.json with final_memo:

✓ Resuming from checkpoint: finalize
... (runs finalize agent to save state)
✅ Memo generation complete!

Implementation Files

Core Resume Logic

• resume-from-last-interruption.py - Standalone resume script (377 lines)

  • detect_resume_point() - Artifact analysis and checkpoint detection

  • reconstruct_state_from_artifacts() - State rebuilding from files

  • execute_from_checkpoint() - Agent execution from resume point

  • _is_valid_json() - JSON integrity validation

  • main() - CLI interface and error handling

CLI Integration

• src/main.py - Main entry point modifications

  • Added --resume flag (lines 59-63)

  • Added --version parameter for resume (lines 64-69)

  • Added resume mode delegation logic (lines 85-113)

  • Imported get_latest_output_dir, sanitize_filename (lines 21-22)

Architecture Compatibility

Integration with Section-by-Section Processing

Resume workflow fully compatible with 2025-11-20 refactor:

Writer Agent:

• Sections saved as individual files (2-sections/*.md)

• State reconstruction leaves draft_sections empty (correct per new architecture)

• Enrichment agents load from files, not state

Enrichment Agents:

• Resume detects completion by examining file contents (LinkedIn links, markdown links, etc.)

• No reliance on state dict for section data

• Each agent can resume independently without breaking others

Citation Enrichment:

• Detects if citations already added by checking for [^1] markers

• Can re-run citation enrichment on improved sections without issues

• Global renumbering handles both fresh and existing citations

Future Enhancements

Phase 2: Validation & Testing (Planned)

• Add comprehensive test suite for each checkpoint

• Implement artifact integrity checksums (MD5/SHA256)

• Add state consistency validation

• Enhanced logging to track resume operations

Phase 3: Enhanced Features (Planned)

• Auto-detect interruption and offer resume on next run

• Add --force-restart flag to override resume behavior

• Visual progress indicator showing completed/pending agents

• Support resume after manual artifact edits with change detection

Phase 4: Robustness (Planned)

• Checkpoint versioning (handle schema changes between versions)

• Partial rollback capability (e.g., "redo citation enrichment only")

• Agent-level retry logic (retry failed agent N times before full restart)

• Incremental section writing (resume incomplete section sets)

Migration Notes

For Users

• No breaking changes to existing workflows

• Resume is opt-in via --resume flag

• Normal workflow continues to work unchanged

• Existing artifacts automatically resumable

For Developers

• No changes required to existing agents

• Agents must save artifacts properly (already implemented)

• State reconstruction relies on artifact files, not in-memory state

• New agents auto-supported if they follow artifact patterns

Dependencies

New Imports

• src/utils.get_latest_output_dir - Dynamic version resolution

• src/artifacts.sanitize_filename - Filename sanitization

• src/state.create_initial_state - State initialization

• subprocess - CLI delegation to resume script

Agent Imports (in resume script)

from src.agents.research_enhanced import research_agent_enhanced
from src.agents.writer import writer_agent
from src.agents.trademark_enrichment import trademark_enrichment_agent
from src.agents.socials_enrichment import socials_enrichment_agent
from src.agents.link_enrichment import link_enrichment_agent
from src.agents.visualization_enrichment import visualization_enrichment_agent
from src.agents.citation_enrichment import citation_enrichment_agent
from src.agents.citation_validator import citation_validator_agent
from src.agents.validator import validator_agent
from src.workflow import finalize_memo, human_review

Related Documentation

• context-vigilance/issue-resolution/Resume-Workflow-After-Interruption.md - Original design doc

• changelog/2025-11-20_01.md - Section-by-section processing refactor

• CLAUDE.md - Updated with resume usage examples (pending)

• README.md - User-facing documentation (pending update)

Issue Resolution

Addressed Issues

• Issue: Workflow interruptions waste time and API credits

• Issue: Long-running workflows (20-30 min) vulnerable to network/timeout issues

• Issue: No way to recover from crashes during expensive operations

• Issue: Users hesitant to experiment due to restart cost

Remaining Issues

• TODO: Add resume documentation to CLAUDE.md Essential Commands section

• TODO: Create --force-restart flag for explicit fresh start

• TODO: Implement progress indicator showing checkpoint status

• TODO: Add resume capability to improve-section.py tool

Testing Summary

Manual Testing

• ✅ Resume from citation enrichment (Terrantic)

• ✅ Resume on completed memo (detects complete)

• ✅ Resume with non-existent company (proper error)

• ✅ Resume with specific version (--version v0.0.1)

• ✅ CLI integration (--resume flag)

• ✅ State reconstruction (loads research, sections, validation)

• ✅ Checkpoint detection (detects correct agent from artifacts)

Automated Testing

• ⏳ Pending: Unit tests for checkpoint detection logic

• ⏳ Pending: Integration tests for state reconstruction

• ⏳ Pending: End-to-end tests for each checkpoint scenario

Performance Impact

Memory

• No impact: State reconstruction minimal overhead (loads existing data)

• No impact: Agent execution same as normal workflow

Disk I/O

• Minimal increase: Extra reads during artifact analysis

• Benefit: Fewer writes overall (skips completed agent artifacts)

API Costs

• Significant reduction: Only calls APIs for remaining agents

• Example: Resume from citation = skip research ($0.50) + drafting ($2.00) = $2.50 saved

Breaking Changes

None. Resume workflow is fully backward compatible:

• Existing workflows continue unchanged

• Artifacts from pre-resume workflows compatible

• Opt-in via --resume flag only

Commit References

Related commits:

• Fixed PyMuPDF dependency (pyproject.toml update)

• Implemented resume detection logic

• Implemented state reconstruction

• Implemented checkpoint execution

• Added CLI integration

Contributors

Implementation by Claude Code (Anthropic) with user guidance.

Next Steps

1. ✅ Complete: Core resume functionality

2. ✅ Complete: CLI integration

3. ✅ Complete: Real-world testing (Terrantic)

4. ⏳ Pending: Update CLAUDE.md with resume examples

5. ⏳ Pending: Add unit tests for resume logic

6. ⏳ Pending: Implement --force-restart flag

7. ⏳ Pending: Add progress indicator UI