Design a Memory System for an AI Product

A Product Design question for PM interviews at AI-native companies, consumer tech, and enterprise AI startups

This question tests whether you understand that memory in an AI product is not a feature, it is a relationship contract between the product and its user. The interviewer is not looking for an architecture diagram. They are stress-testing your ability to reason about user trust, AI-specific risks, and which tradeoffs to surface versus which to quietly resolve.

Step 1: Ask Clarifying Questions

Before jumping into the design, I want to make sure I understand the scope and context of this question.

Q: What type of AI product are we designing memory for? A general-purpose assistant, a health companion, a coding tool, or something else?

Let us assume we are designing for a general-purpose AI assistant, similar to ChatGPT or Claude, that users interact with across personal and professional contexts.

Q: What is the primary platform? Web, mobile, API, or embedded in another product?

Let us focus on a consumer-facing web and mobile app where users have personal accounts.

Q: Are we optimizing for personalization depth, user trust, or retention?

Let us say our north-star is long-term user trust, with personalization as the mechanism and retention as the downstream outcome.

Q: Which user segment should we prioritize? New users, casual users, or power users?

Let us focus on power users who interact with the assistant daily and are most sensitive to personalization quality and privacy.

Q: Should I consider existing implementations like ChatGPT’s memory feature (launched February 2024, expanded April 2025), Claude’s memory system, or Gemini’s personalization?

Yes. The interviewer expects you to be current. Build on top of what exists in the market.

Q: Are there specific regulatory constraints I should design around? GDPR, EU AI Act, India’s DPDP Act?

Yes. Design for global compliance from the start, with GDPR’s right to erasure and the EU AI Act’s provisions on sensitive data inference as hard constraints.

Interview Tip: That question about existing implementations is critical. If you pitch “let the AI remember things between conversations” as your headline idea, you are describing a feature that ChatGPT shipped in February 2024 and expanded significantly in April 2025 with chat history referencing. Anthropic’s Claude rolled out automatic memory in 2025 as well. Always do your product research before the interview.