Rawer

Under Construction

Next Steps

Semem

MCP server has stopped exposing tools

Next Steps

Semem on the server

I just went to do something unrelated with Claude chat in the browser and noticed that I'd already added a pointer to Semem MCP running on localhost. Claude can't see it, so I guess the connection must be made server-side. Ok, let me run Semem MCP on a server. It should be doable using npx semem-mcp, but I want the whole shebang on the server anyway, so I'll go the long way round.

It should just take :

cd ~/hyperdata
git clone https://github.com/danja/semem.git
cd semem
npm install
npm run mcp:http
# or node mcp/http-server.js

But the version of node I have on the server is a bit old, so:

sudo npm install n -g
n stable
node --version
# v22.16.0

I'd better set it up through the nginx proxy to get https:

nano /etc/nginx/sites-available/semem.conf

server {
    server_name semem.tensegrity.it;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
    }
    listen 80;
}

Add a DNS entry :

semem 10800 IN CNAME tensegrity.it.

ln -s /etc/nginx/sites-available/semem.conf /etc/nginx/sites-enabled/semem.conf
nginx -t
systemctl restart nginx
certbot

Hmm. Bad gateway. Let me try it on port 4600 and make it 127.0.0.1 rather than localhost just in case:

MCP_PORT=4600 MCP_HOST=127.0.0.1 node mcp/http-server.js

Locally :

curl https://semem.tensegrity.it/health

{"status":"healthy","timestamp":"2025-06-22T10:17:25.243Z","services":{"memoryManager":true,"config":true},"sessions":0}

Yay!

Oh.

I forgot that it's currently relying on Ollama locally.

I'd already got support for different chat providers, but not embeddings providers, that was quasi-hardcoded to Ollama. So I've added a bit to hyperdata-clients and updated the relevant bits of Semem. Which is bound to break things.

PS. Things might be fixed enough.

I've made a script containing just :

#!/bin/bash

MCP_PORT=4600 MCP_HOST=127.0.0.1 node mcp/http-server.js

so then -

 pm2 start start-mcp.sh --name semem-mcp --watch

and it's visible from MCP Inspector locally,

But broken. Hey ho. Next Claude Code cycle.

Semem on the server

HyDE and Seek

I have been developing recently by doing feature spikes then consolidating. It's dawned on me, quite a way down the road, that in Semem I did too much of the spike before the consolidate.

Yesterday I wanted to try Semem MCP with the online Claude chat ('Pro'). For this I needed to have the MCP server live online. But hit a snag - the default config uses Ollama locally for chat & embeddings. My server is weedy, Ollama would be too much. So I signed up for a Nomic account to use their free tier for embeddedings, flipped the priority completion LLM to Mistral's API.

I should have anticipated issues in this part of the config. I'd got the embeddings provider pretty much hardcoded. So I cycled back through hyperdata-clients to add a common interface, using a factory. (in the same way the chat LLM interface works).

So far so good. I was able to get embeddings using the Nomic API working in one of the examples.

But then as I went to an example that needed chat, it threw up loads of issues around the config setup. It tried to load the Mistral model through Ollama. Oops.

So to resolve this I went to an example that I knew had worked, HyDE, that uses responses from prompts as hypotheticals in subsequent similarity search. Long story short, it took a lot of cycles with Claude Code to get this working again properly.

But this involved changes to core code - Config, MemoryManager... and now there are several breaking tests. Time to cycle back through them.

Ok, tests fixed - they had a lot of hardcoding of strings. Next I need to cycle through the other examples to check. I need to do that anyway to see what each is actually doing. But there are 84 examples in total, I may leave it for today.

HyDE and Seek

Semem/Claude MCP Workflows

At least some of the Semem tools are now exposed over MCP. They can be made available in Claude Code by using :

claude mcp add semem npx semem-mcp

Right now I really need to take a step back. I've got a lot of example files that have run at least once:

find ~/hyperdata/semem/examples -type f | wc -l
79

There's a good chance many of those no longer work following recent changes in places like the config. I got a bit impatient with tests.

Test Files  84 failed | 72 passed | 8 skipped (164)
Tests  589 failed | 1001 passed | 98 skipped (1688)
Errors  3 errors

Hmm, I see some are trying to call the Semem HTTP server. Ok, with it running :

Test Files  84 failed | 72 passed | 8 skipped (164)
Tests  590 failed | 1000 passed | 98 skipped (1688)
Errors  2 errors

Claude had a go at orchestrating workflows. I was curious to see if this might be a good approach because issuing a simple(ish) command like :

run the enhanced-research-workflow tool on the content of brain-gpt-paper.md

- would be a neat kind of interaction. But Claude got stuck at this level. After trying and failing to run the workflow directly, it went on to run the individual tools it uses. I asked for the list, below. This is informative. A little confusing because the naming has got a bit obfuscated that to some (unnecessary) name mapping Claude inserted in there. But I should be able to pick out the pieces, look at doing the same workflow using Semem's HTTP API, orchestrated by Transmissions.

Tool List in the Workflow

According to the search results, the enhanced-research-workflow tries to execute these tools in sequence:

  1. semem_switch_storage_backend - Initialize SPARQL storage
  2. research_ingest_documents - Enhanced document ingestion (custom tool)
  3. semem_extract_concepts - Extract domain-focused concepts
  4. ragno_decompose_corpus - Build knowledge graph with embeddings
  5. ragno_analyze_graph - Comprehensive graph analysis
  6. research_generate_insights - Generate insights (custom tool)
  7. hybrid_search - Setup hybrid search capabilities (custom tool)
  8. semem_generate_response - Generate executive summary
  9. capture_user_feedback - Initialize feedback learning (custom tool)

Tool Mapping System

The workflow uses a tool mapping system located in: File: /flow/hyperdata/semem/mcp/lib/workflow-orchestrator.js

This orchestrator maps workflow tool names to actual MCP tool implementations using the TOOL_MAPPING object and adds the mcp__semem__ prefix for standard tools.

Custom Tool Implementations

The custom tools (like research_ingest_documents, research_generate_insights, etc.) are implemented in: File: /flow/hyperdata/semem/mcp/tools/research-workflow-tools.js

The error you encountered ("Required tool not available") occurs because the workflow orchestrator expects these custom tools to be available, but they may not be properly registered or implemented in the current MCP server setup.

There's also a log on my desktop under : ``` ~/.cache/claude-cli-nodejs/-flow-hyperdata-semem/mcp-logs-semem ```

Semem/Claude MCP Workflows

Tensegrity Suite

Main components

  • Transmissions - pipeline composer, nuts & bolts layer
  • Semem - semantic memory, knowledgebase interface, system layer
  • Ontologies - several, abstract layer
  • TIA Intelligence Agency - the goal is communities of autonomous agents, app layer
  • Client stuff - assorted tools, app layer
  • TBox - a container for experimenting with the above, quasi-production layer

I gave myself a deadline of 2025-06-21 to get this bunch of things to some kind of minimum viable pseudo-products. If I'm happy with what I've got to show at that point, I'll be doing announcements (haranguing the lists, socials and maybe even ex-girlfriends) on 2025-07-03. I'm more confident than usual that I'll have at least something to show, I have got the lion's share of the hard parts done already. We'll see.

My dev process is to cycle through the subprojects, spending a day or two on each at time. Not a regular cycle : cross-dependencies often force a different direction; things always take longer than expected; distractions (often trying dev spikes, especially now with vibes).

Goals

Have fun. Hopefully produce useful things as a side effect.

Philosophy

This comes from wondering about cognition, what can computers, AI achieve? What can I achieve? I'll use problem solving as a catch-all for intentional state change.

  1. Systems can be described in terms of concepts and relationships between those concepts. The human way.
  2. Problem solving is mostly about pattern matching. Fitting unknown systems to ones where their behaviour is known.
  3. All known agents, animal or artificial, at any given moment in time, have a finite amount of working memory. This acts as boundary on what they can process.
  4. Concepts can often be decomposed into smaller units.
  5. Groups of associated concepts can be abstracted into meta-concepts.
  6. Scale-free thinking for the win!

Filthy Lucre

I can see commercial potential in this stuff (nb. a ground rule is that it's all open source), but the motivation is more an Everest-ian because it's there challenge, my wanting to get my head around these things and, yeah, better tools for problem-solving would be nice to have. Whatever, this lot maybe best considered an art/school project.

Bag of Stuff

I have been building these things as quasi-independent projects, each with their own repo. But to start seeing the utility I'm after, they need joining up, which makes a big dependency graph : a tensegrity structure. However many of the dependencies are soft, X helps Y, rather than critical, X needs Y (cf. non-functional requirements). Some parts I've been working on for a long time (25 years is a long time in tech, right..?), but the arrival of Deep Learning, especially that around LLMs, forced a reboot. And obviously everything else is continually evolving. Sitting still is not an option.

It may seem like there's a lot of wheel reinvention, but that's mostly because I'm fed up of (other people's) frameworks. My own very limited context window needs to be used on the tasty bits, not conventions. I'm favouring ES module style nodejs and/or vanilla browser JS etc. so I don't have to flip languages so often.

All this stuff is riddled with RDF. I'll write a Why RDF? note (again) some time. But the tl;dr is twofold : the Web is the best graph knowledgebase in human history; some level of formal ontology definition helps considerably when describing things (especially knowledge).

Sites

Hyperdata is an arbitrary umbrella name I'm using for all this stuff. I've got the domain

hyperdata.it and associated hosting for any quasi-reliable service fixtures, docs etc. For experiments, maybe even useful things, I have strandz.it.

danny.ayers.name is the dumping ground for everything else.

TIA

I've many, mostly fuzzy forms which require a lot of hand-waving to describe. The main high-level target is to create communities of intelligent agents which improve on the state of the art in solving arbitrary problems. This aspect I've labeled TIA Intelligence Agency (it has a repo but there's nothing to see yet). Some example problems below.

For pragmatic reasons, and to keep things conceptually simple, the environment will be multi-user chat over XMPP. A containerized environment will enable repeatable experimentation :

tbox

A docker-compose setup with :

  • SPARQL Store (Fuseki)
  • XMPP IM Server (Prosody)
  • Support for various subprojects (agent boilerplate, all this hyperdata stuff)
  • Dev tooling (ssh etc, CI/CD, observability/metrics...)

Status 2025-05-04 : it's largely in place, needs a lot of tidying up, hardening. Only just started with telemetry

The subproject with the most novelty is :

Semem

Semantic memory for agents. I want this mostly defined in terms of interfaces (see Vocabs), keep it flexible. Running code essentially helpers, just lowish-level things like concept extraction etc.

Core components :

  • SPARQL Store, RDF support (rdfjs & RDF-Ext libs)
  • Embeddings vector store & tools (FAISS, Nomic embedding model(s))
  • Resource Store (quick & dirty old school REST)
  • LLM Clients
  • Vocabularies

The first 4 components are fairly self-explanatory, kind-of off the shelf. What sets it apart is an ontology-backed approach to composition & choreography, see Vocabs...

The background to this is probably worth noting. A while back I had a play with Graph RAG using SPARQL, working with LlamaIndex. Crude proof of context. I wanted more, but not dependent on the whims of a framework. I stumbled on memoripy which had most of the skeleton of what I wanted. So my starting point for #:semem was a kind of rewrite that using JS & SPARQL. At that point I shifted my energies back to #:transmissions, meanwhile working on the vocabulary support - very significant, see Vocabs...

Status 2025-05-04 : the basics are pretty much in place, although in need of a lot of tweaking, #:transmissions hooking in. Dependencies around here :

  • #:semem needs #:transmissions for proper wiring.
  • #:semem isHelpedBy vocabs - to enable to good bits
  • #:semem is helpedBy the utility stuff - docs, visualizations etc.

Transmissions

This is a pipeliney thing. In fancy speak, a workflow compositor.

Postcraft

TIA Information Agency

Vocabs

These are key. I've determined the main things I need are : a knowledge representation language (Ragno), a knowledge navigation language (ZPT), a knowledge communication language (Lingue) and a methodology language (UM).

The primary vocab links are placeholders, they'll hopefully resolve to something useful by 2025-06-21.

Ragno

The Ragno ontology is used to describe a knowledgebase. General-purpose terms are pretty well covered by SKOS already, what was demanded here were terms for specialisms that would fit well with RAG applications. I'd got a lot sketched out when I stumbled on the paper NodeRAG: Structuring Graph-based RAG with Heterogenous Nodes. The information model in that has nice intersections with what I was after, so I merged the ideas in. A significant bonus is that the paper describes algorithms for knowledge augmentation and retrieval that are a very good fit for what I was planning with Semem.

Zoom, Pan, Tilt (ZPT)

The ZPT ontology is used to describe a view of (part of) a knowledgebase. The primary purpose is to navigate the knowledge Universe and scope the current context of interest. The core terms are lifted from the world of cinema, with increasing levels of analogy-stretching :

  • Zoom : the layer of abstraction/detail
  • Pan : the (sub)domain, topics of interest
  • Tilt : the parametric projection

Lingue

The Lingue ontology is designed to facilitate communication between agents in a way that is flexible in regard to agent capabilities. It subsumes much of MCP and A2A. A key notion is negotiation, such that an optimal local lingue franca may be determined. Agents will have self-descriptions (? in MCP cards in A2A) that may be compared at a fairly mechanical level. Any commonalities can allow promotion to more sophisticated or domain-specific language(s).

Avalanche Development Methodology

It seems clear that recent developments around the use of AI in dev call for a new paradigm in the whole process. I've been trying to capture ideas on what works and what doesn't : avalanche methodology.

Btw, I have a sandpile avalanche simulator

Client Stuff

Event Bus

I've been hopping between Node.js and vanilla browser JS (with npm, webpack, vitest). A mechanism that seemed to make sense is an event bus I could use in either. So I made a quick & dirty lib - "evb" [4]. Even though there's very little to it, I've published it as an npm package to try and keep some consistency across the things I make.

farelo postcraft sheltopusik squirt trestle wstore foaf-retro mcp-o trans-apps a2a-o elfquake hyperdata thiki uve atuin evb hyperdata-clients node-flow tbox tia viz

Tensegrity Suite