{"@attributes":{"version":"2.0"},"channel":{"title":"Metal Whale's blog","link":"https:\/\/metalwhale.github.io\/","description":"Recent content on Metal Whale's blog","generator":"Hugo -- gohugo.io","language":"en","managingEditor":"metalwhale.dev@gmail.com (Metal Whale)","webMaster":"metalwhale.dev@gmail.com (Metal Whale)","lastBuildDate":"Wed, 07 Feb 2024 14:08:00 +0000","item":[{"title":"Are LLMs not truly \"generative\"?","link":"https:\/\/metalwhale.github.io\/posts\/are-llms-truly-generative\/","pubDate":"Wed, 07 Feb 2024 14:08:00 +0000","author":"metalwhale.dev@gmail.com (Metal Whale)","guid":"https:\/\/metalwhale.github.io\/posts\/are-llms-truly-generative\/","description":"TL;DR\nIMHO, LLMs (Large Language Models) trained with autoregressive process are not “truly” generative, especially those that don’t use BOS (beginning-of-sentence) token.\nI’ve asked about this on nanoGPT repo, transformers repo and HN but haven’t yet found the final answer. This post shows my thoughts on the generative capability of LLMs. Feel free to discuss with me (:\nWhat is a generative model? There are two types of models in machine learning: discriminative and generative.","category":["LLM","Generative AI","Machine Learning"]},{"title":"My Will","link":"https:\/\/metalwhale.github.io\/posts\/my-will\/","pubDate":"Wed, 24 Jun 2020 11:36:01 +0000","author":"metalwhale.dev@gmail.com (Metal Whale)","guid":"https:\/\/metalwhale.github.io\/posts\/my-will\/","description":"Adelitas Way  Still Hungry  AKB48  Love Trip \u685c\u306e\u681e  Alisa Takigawa  \u590f\u306e\u82b1  All That Remains  What If I Was Nothing  Amon Amarth  Raven’s Flight War of the Gods  Architects  Doomsday  Ariel Tsai  \u525b\u597d\u9047\u898b\u4f60 (x)  Art Of Dying  Tear Down the Wall  As I Lay Dying  Blinded  Ashes Remain  End of Me  Asking Alexandria  Into The Fire  Back Number  \u30af\u30ea\u30b9\u30de\u30b9\u30bd\u30f3\u30b0  \u805e\u3053\u3048\u308b\u307e\u3067\u4f55\u5ea6\u3060\u3063\u3066\u8a00\u3046\u3088\u3002\u541b\u304c\u597d\u304d\u3060\u3002"}]}}