🕸️ Semantic Networks in the LLM World

Welcome to the recap of my talk at the Vegas Tech Alley AI Meetup, where we explored the powerful, long-standing concept of Semantic Networks and how modern LLMs are finally making them practical for real-world application.

A Semantic Network is a way of representing knowledge by drawing circles (entities) and connecting them with lines (relationships) to overcome the ambiguity of pure text.

Video: Tech Alley Vegas | Las Vegas AI Meetup - March 2024

The Challenge of Knowledge Retrieval

The AI industry has been pursuing semantic network concepts since the 1950s (Marvin Minsky’s Frame System). However, the core challenge was always: How do you get the knowledge into the system?

Now, the Large Language Model (LLM) has solved this problem:

• Old Way: Engineers used complex, rigid programming to define relationships from text.
• New Way: A simple, well-crafted LLM prompt can digest a chunk of text (like an email or legal document) and instantly return a set of relationships as a structured data format (CSV or JSON).

This process is critical for building a complete, holistic view of knowledge by connecting discrete facts scattered across thousands of documents, such as linking two individuals who were mentioned in different documents but lived in the same city.

🏗️ The Data Architecture for Structured Knowledge

At Kusog AI, we use a three-pronged architecture to ensure data is structured, reliable, and available for LLM queries:

1. The Source of Truth: PostgreSQL

All raw data is ingested and stored in PostgreSQL.

• Reliability: PostgreSQL serves as the Source of Truth because it is robust and easy to back up.
• Efficiency: This eliminates the need to back up less reliable databases like Elastic Search or Neo4j. If an index is corrupted, the system can simply re-index the data directly from PostgreSQL.

2. The Relationship Engine: Neo4j

We use a Graph Database (like Neo4j) to store the Subject-Predicate-Object triplets extracted by the LLM prompt.

• Granularity: Relationships are extracted at a highly detailed level, mapping things like “Matt Denman started a project” or “John Smith was married to Jane Doe”.
• Vocabulary Tracking: The system tracks every word seen in every document, storing the five words that appeared before and after it. This deep vocabulary knowledge could eventually serve as a foundation for training custom language models.

3. The Retrieval Engine: Elastic Search (Embeddings)

The standard vector embeddings (the “magic numbers”) are stored in Elastic Search.

• Multi-Embedding: The system allows for multiple embedding models (e.g., comparing BERT, GPT-3.5, etc.) to be stored simultaneously, enabling comparison tests to ensure the highest quality matches are used for retrieval.

🧠 The Power of Personalized Context

This combined structure allows the system to build an incredibly detailed dossier on any entity (a person, a company, a project) it has ingested data on.

• Personalized Context: By ingesting decades of personal data (emails, voicemails, documents), the system builds a “footprint in the snow” of your life, enabling hyper-personalized interactions, even for something as complex as a counseling session.
• Entity Resolution: Since a name like “Matt Denman” might refer to multiple people online, the system focuses on one entity linked to all the ingested documents, ensuring the data is structured and cohesive.
• Exploration: Unlike raw LLM chat, which only answers specific queries, this structured knowledge can be explored—users can visually navigate the connections, categories, and relationships discovered by the AI, which is useful when a user “doesn’t know what to ask”.

📈 Improving LLM Efficiency

Semantic networks help reduce the cost and latency of LLM interactions by reducing the amount of context (tokens) sent per request.

• Temporal and Z-Index Ranking: When retrieving information, the system uses more than just cosine similarity:
  • Date: It prefers the most recent version of a document or class.
  • Z-Index (Summarization Level): It prioritizes chunks of knowledge that are already summarized (higher Z-index) over raw, long-form documents (Z-index 0).
• The Summarization Loop: When a conversation builds new knowledge (e.g., listing all of Matt Denman’s children from 12 separate source chunks), the AI summarizes that list into one high-level chunk (higher Z-index). Future queries for that topic retrieve only the single summarized chunk, saving thousands of tokens. This is essential for preventing huge bills when engaging in long, complex conversations.