Claude: VSOM Training Implementation - Making Self-Organizing Maps Useful

Date: October 2, 2025 Session: VSOM Training Feature Development Status: Implemented & Ready for Testing

Overview

Implemented actual self-organizing map (SOM) training for the VSOM visualization interface, transforming it from a simple grid layout into a semantically meaningful spatial organization tool. The user's insight was spot-on: "A Train button might be a good starting point?"

Problem Identified

During investigation of the VSOM codebase, discovered a revealing comment in DataProcessor.js line 449:

// In a real VSOM, this would involve training and similarity calculations

The VSOM visualization was using:

Key Discovery: Existing Infrastructure

Found comprehensive VSOM infrastructure already in place:

This changed the implementation strategy from "build SOM from scratch" to "wire existing backend to frontend."

Implementation

1. Backend Integration

Created TrainVSOMCommand.js (src/mcp/tools/verbs/commands/):

Added Training Endpoint (src/mcp/http-server.js:540-565):

app.post('/train-vsom', async (req, res) => {
  const { epochs = 100, learningRate = 0.1, gridSize = 20 } = req.body;
  const trainingResult = await simpleVerbsService.execute('train-vsom', {
    epochs, learningRate, gridSize
  });
  res.json(trainingResult);
});

Registry Updates:

2. Frontend Integration

UI Enhancement (src/frontend/vsom-standalone/public/index.html):

<button class="control-button" id="train-vsom">
    <span class="button-icon">🧠</span>
    Train Map
</button>

API Service Method (VSOMApiService.js:232-271):

async trainVSOM(options = {}) {
  const result = await this.makeRequest('/train-vsom', {
    method: 'POST',
    body: JSON.stringify({
      epochs, learningRate, gridSize
    })
  });
  return result;
}

Event Handler (vsom-standalone.js:728-779):

async handleTrainVSOM() {
  this.showToast('Starting VSOM training...', 'info');
  const trainingResult = await this.services.api.trainVSOM({
    epochs: 100, learningRate: 0.1, gridSize: 20
  });

  if (trainingResult.success) {
    // Convert mappings to positioned nodes
    const trainedNodes = trainingResult.mappings.map(mapping => ({
      ...mapping.entity,
      x: mapping.mapPosition[0],
      y: mapping.mapPosition[1],
      trained: true
    }));

    this.components.grid.updateNodes(trainedNodes);
    this.showToast(
      `Training complete! ${trainingResult.metadata.entitiesCount} nodes organized`,
      'success'
    );
  }
}

Architecture Flow

  1. User clicks "Train Map" button
  2. Frontend → trainVSOM() → POST /train-vsom
  3. MCP Server → SimpleVerbsService.execute('train-vsom')
  4. TrainVSOMCommand:
    • Queries SPARQL for entities with embeddings
    • Creates VSOMService instance (20×20 grid, 1536-dim embeddings)
    • Loads entities into VSOM
    • Trains with Kohonen algorithm (100 epochs, learning rate 0.1→0.01)
    • Returns grid positions and cluster info
  5. Frontend ← Receives trained positions
  6. VSOMGrid ← Updates with spatially-organized node positions

Technical Details

Training Parameters

Data Flow

Benefits for End Users

Before Training:

After Training:

Files Modified

  1. /src/mcp/tools/verbs/commands/TrainVSOMCommand.js - Created (305 lines)
  2. /src/mcp/tools/VerbSchemas.js - Added TrainVSOMSchema
  3. /src/mcp/tools/verbs/VerbCommandRegistry.js - Registered command
  4. /src/mcp/tools/SimpleVerbsService.js - Added to core tool names
  5. /src/mcp/http-server.js - Added /train-vsom endpoint
  6. /src/frontend/vsom-standalone/public/index.html - Added Train button
  7. /src/frontend/vsom-standalone/public/js/services/VSOMApiService.js - Added trainVSOM()
  8. /src/frontend/vsom-standalone/public/js/vsom-standalone.js - Added handleTrainVSOM()

Code Reuse

Successfully leveraged existing infrastructure:

No duplication - clean integration with existing architecture.

Next Steps

  1. User Testing: Click Train Map button with real knowledge graph data
  2. Performance Tuning: Optimize for 4739+ nodes
  3. Progress Indicator: Add real-time training progress updates (SSE/polling)
  4. Training Options: Expose parameters in UI (epochs, learning rate, grid size)
  5. Model Persistence: Cache trained positions to avoid retraining
  6. Quality Metrics: Display quantization/topographic errors in UI
  7. Incremental Training: Update positions when new nodes added

Observations

User's Question Was Key: "I would like you to think hard about how to make the vsom view useful for the end user. I think a Train button might be a good starting point?"

This simple question revealed:

Code Comment Gold: The // In a real VSOM... comment was the Rosetta Stone that confirmed the current implementation was placeholder code.

Architecture Surprise: Discovering comprehensive VSOM infrastructure already implemented was a pleasant surprise. The task transformed from "implement SOM algorithm" to "connect the dots."

Status

✅ All implementation complete ✅ Servers running (MCP: 4101, VSOM: 4103) ✅ End-user testing SUCCESSFUL

Test Results

Training Execution:

Data Statistics:

User Experience:

  1. Clicked "🧠 Train Map" button
  2. Toast notification: "Starting VSOM training..."
  3. Training completed in ~4 seconds
  4. Visualization updated with trained spatial positions
  5. Console confirmed: ✅ [VSOM] Training completed: {success: true}

Visual Result: The map now displays nodes in semantically meaningful positions where similar concepts cluster together. Pink/magenta clusters visible at bottom of grid show entity groupings. The transformation from arbitrary grid layout to trained semantic space is complete.

Critical Fixes Applied

Fix #1: Correct RDF Property Path

Problem: Initial query used semem:hasEmbedding with intermediate node structure. Reality: Embeddings stored directly on semem:embedding property as JSON array literals. Solution: Updated SPARQL query in TrainVSOMCommand.js:153-168.

Fix #2: VSOMService API Mismatch

Problem: VSOMService.loadData() calls non-existent vsom.loadEntities() method. Reality: VSOM.js only provides loadFromEntities() requiring embeddingHandler. Solution: Bypassed VSOMService entirely, used VSOM class directly with pre-loaded embeddings.

Fix #3: Direct VSOM Population

Since embeddings are pre-loaded from SPARQL, directly populate VSOM internal arrays:

vsom.embeddings = validNodes.map(node => node.embedding);
vsom.entities = validNodes.map((node, index) => ({ id: node.id, index }));
vsom.entityMetadata = validNodes.map(node => ({...}));

Conclusion

The Train Map button is now fully functional and tested. It successfully transforms the VSOM visualization from a simple grid into a semantically meaningful knowledge space where similar concepts cluster together based on their 1536-dimensional embeddings.