This local-first memory approach is exactly what I landed on after the Limitless EU ban hit.
After losing access to 6 months of voice data overnight, I built a similar setup: Claude reads a structured markdown context file at each session start. The difference from Limitless: the files are mine, portable, no vendor risk, no privacy theater.
The 'Claude updates notes automatically' framing undersells the harder problem though. Of 10GB of voice data I had, maybe 10% was genuinely useful context. Signal-to-noise is the real design challenge.