Wrap the agent in a FastAPI Server and make it a standalone service

[Ar-temis]

What will be the goal

The agent and the backend have dependency correlations and because of that it really hard to update development environments. If we separate the environments, we can decouple a lot of stuff and make life easy for both backend and agent teams.

What APIs should be available

• /generate (you can rename this) which is a POST route:
  • Should be able to specify tools to use with specific parameters such as user_id
  • Should be able to generate an answer to a given prompt. Should yield the answer.
  • Should call the PersistentMemory module after the last token
• More to be suggested

Breaking changes

The agent will be on a fastAPI server so the backend will be doing http requests to it.

Any other breaking changes we should consider? Let's discuss the benefit to development energy ratio.

[Ar-temis]

@munishlohani What do you think?

[Ar-temis]

@kkwan0 You have two weeks to do this. Does that sound okay? We can discuss the details with @munishlohani also.

[kkwan0]

Yeah I think that sounds okay

[munishlohani]

I was also thinking of using redis to broadcast the stream to django backend and frontend so that we can store the response since the authentication and all is managed my the django backend. What do you guys think about it?

[Ar-temis]

yea, sounds good. This means that redis will work like kafka right?

[kkwan0]

@Ar-temis Are there any other implementations aside from memorytools that the FastAPI server would be handling? I'm asking because on mem0 documentation they have their own async memory feature so that it wouldn't block other worker threads. If there aren't any other plans for FastAPI, I will skip that

[Ar-temis]

It should handle everything the current agent is doing.

[Ar-temis]

@kkwan0 updates? Have you looked into fastAPI docs?

[kkwan0]

I am looking at the fastAPI docs. Right now I'm looking at the pydantic classes to implement the methods. Other than that no issues so far.