Parametric Memory
Parametric memory is the knowledge a language model encodes directly into its weights during training, which it draws on to generate answers without consulting any external source at the moment it’s asked a question. Parametric Memory in Practice Ask a model something well-established and older than its training cutoff — “what is the capital of…
