• 3 Posts
  • 31 Comments
Joined 1 year ago
cake
Cake day: July 2nd, 2023

help-circle




  • I had a similar relationship with my father. He was an alcoholic. These days I don’t have much of a relationship with him. I recognize that he’s a better person now that he’s older, but I don’t really see him as “dad”. Just “father”.

    My mom sometimes asks if I will regret not spending more time with him. Honestly, I’m not sure I would. I don’t have many fond memories with him at all. It’s weird saying this knowing that I have a father who loves me in his own way when others might not have one at all.
















  • We’ve reached far beyond practical necessity in model sizes for Moore’s Law to apply there. That is, model sizes have become so huge that they are performing at 99% of the capability they ever will be able to.

    Context size however, has a lot farther to go. You can think of context size as “working memory” where model sizes are more akin to “long term memory”. The larger the context size, the more a model is able to understand beyond the scope of it’s original model training in one go.