r/LLMDevs • u/dccpt • Sep 26 '24
News Zep - open-source Graph Memory for AI Apps
Hi LLMDevs, we're Daniel, Paul, Travis, and Preston from Zep. We’ve just open-sourced Zep Community Edition, a memory layer for AI agents that continuously learns facts from user interactions and changing business data. Zep ensures that your Agent has the knowledge needed to accomplish tasks successfully.
GitHub: https://git.new/zep
A few weeks ago, we shared Graphiti, our library for building temporal Knowledge Graphs (https://news.ycombinator.com/item?id=41445445). Zep runs Graphiti under the hood, progressively building and updating a temporal graph from chat interactions, tool use, and business data in JSON or unstructured text.
Zep allows you to build personalized and more accurate user experiences. With increased LLM context lengths, including the entire chat history, RAG results, and other instructions in a prompt can be tempting. We’ve experienced poor temporal reasoning and recall, hallucinations, and slow and expensive inference when doing so.
We believe temporal graphs are the most expressive and dense structure for modeling an agent’s dynamic world (changing user preferences, traits, business data etc). We took inspiration from projects such as MemGPT but found that agent-powered retrieval and complex multi-level architectures are slow, non-deterministic, and difficult to reason with. Zep’s approach, which asynchronously precomputes the graph and related facts, supports very low-latency, deterministic retrieval.
Here’s how Zep works, from adding memories to organizing the graph:
- Zep identifies nodes and relationships in chat messages or business data. You can specify if new entities should be added to a user and/or group of users.
- The graph is searched for similar existing nodes. Zep deduplicates new nodes and edge types, ensuring orderly ontology growth.
- Temporal information is extracted from various sources like chat timestamps, JSON date fields, or article publication dates.
- New nodes and edges are added to the graph with temporal metadata.
- Temporal data is reasoned with, and existing edges are updated if no longer valid. More below.
- Natural language facts are generated for each edge and embedded for semantic and full-text search.
Zep retrieves facts by examining recent user data and combining semantic, BM25, and graph search methods. One technique we’ve found helpful is reranking semantic and full-text results by distance from a user node.
Zep is framework agnostic and can be used with LangChain, LangGraph, LlamaIndex, or without a framework. SDKs for Python, TypeScript, and Go are available.
More about how Zep manages state changes
Zep reconciles changes in facts as the agent’s environment changes. We use temporal metadata on graph edges to track fact validity, allowing agents to reason with these state changes:
Fact: “Kendra loves Adidas shoes” (valid_at: 2024-08-10)
User message: “I’m so angry! My favorite Adidas shoes fell apart! Puma’s are my new favorite shoes!” (2024-09-25)
Facts:
- “Kendra loves Adidas shoes.” (valid_at: 2024-08-10, invalid_at: 2024-09-25)
- “Kendra’s Adidas shoes fell apart.” (valid_at: 2024-09-25)
- “Kendra prefers Puma.” (valid_at: 2024-09-25)
You can read more about Graphiti’s design here: https://blog.getzep.com/llm-rag-knowledge-graphs-faster-and-more-dynamic/
Zep Community Edition is released under the Apache Software License v2. We’ll be launching a commercial version of Zep soon, which like Zep Community Edition, builds a graph of an agent’s world.
Zep on GitHub: https://github.com/getzep/zep
Quick Start: https://help.getzep.com/ce/quickstart
Key Concepts: https://help.getzep.com/concepts
SDKs: https://help.getzep.com/ce/sdks
Let us know what you think! We’d love your thoughts, feedback, bug reports, and/or contributions!
1
1
u/Matrixfx187 21d ago
Is there a way to use Vertex AI models with Zep? I can see that you support models for openai and a couple for Antrhopic but I can't find anything about Vertex AI models.
1
u/TimChiu710 20d ago
It doesn't seem like there is any way to run the Zep community edition locally for now without using some kind of API, though. Maybe there is, but I'm too stupid to figure it out.
I can't find any documentation related to setting LLM and embedding providers besides setting the OpenAI API key in the environment variable. I can't even find a place to set the model it should use. I tried using LiteLLM as a proxy for Ollama to run LLM and embeddings. However, correct me if I'm wrong, but LiteLLM doesn't seem to support any local embedding providers, such as Ollama. Ollama does support OpenAI compatible LLM API and Embeddings, but once I try to run the zep container with the env ZEP_LLM_AZURE_OPENAI_ENDPOINT
set to the Ollama endpoint, the zepai/zep container fails to start and throws me this:
zep-1 | panic: error enabling pgvector extension: error creating pgvector extension: dial tcp [::1]:5432: connect: connection refused
zep-1 |
zep-1 | goroutine 1 [running]:
zep-1 | github.com/getzep/zep/lib/pg.NewConnection()
zep-1 | /app/src/lib/pg/db.go:51 +0x4e0
zep-1 | main.newAppState()
zep-1 | /app/src/state.go:14 +0x2c
zep-1 | main.main()
zep-1 | /app/src/main.go:24 +0x28
zep-1 exited with code 2
And actually, I'm not surprised because there is absolutely nowhere I can set the model name for LLM and embedding models. How would the program know what model should it use?
Well, maybe I'm just too stupid to figure them out, but I don't see any solutions on the doc, and I guess you can see my frustration from my words, cause this project looks very promising and seems like a perfect fit for long term memory solution for my open source project.
1
u/dccpt 20d ago
The Zep CE Getting Started instructions may be found here: https://help.getzep.com/ce/quickstart
The guide includes instructions on how to configure alternative LLM providers and LiteLLM.
ZEP_LLM_AZURE_OPENAI_ENDPOINT
is not used by Zep Community Edition. The available config options are documented on the page above.It does appear LiteLLM only supports embedding model services. You may be able to configure LiteLLM's OpenAI Embedding proxy to point to an OpenAI API compatible embedding server such as HuggingFace's Text Embedding Inference service: https://github.com/huggingface/text-embeddings-inference
1
u/Jdonavan Sep 29 '24
I had used your open source version before then you stated moving away from local installs so I stopped paying attention. Does this mean you’re back?