Graph based LLMs architectures

[wassfila]

Scope

investigation about Large Language Models which inference time can assimilated to a graph traversal process similar to reasoning through a graph traversal sequence.
Although a deep neural net is also a propagation computation of neural weights, a graph based network would emphasize the conditional path taken for each different reasoning sequence.

Not in Scope

• graph learning and prediction by a transformer architecture
• graph inference outside an LLM by function calling
• RAG of graph by transformation in text and insertion in context

Architectures

• Graph Neural Networks (GNNs)
• Neuro-Symbolic AI https://wordlift.io/blog/en/neuro-symbolic-ai/
• Differentiable Reasoning Networks
• Transformers with Structured Memory

[wassfila]

this article propose a method how a so called Graph LM can assimilate a graph as input
https://aclanthology.org/2024.acl-long.245/
https://github.com/Heidelberg-NLP/GraphLanguageModels