foundation model
Table of Contents
Foundation model is typically large(parameter numbers) model pre-trained on large(internet-scale) data, and usually fine-tuned in use to downstream(specific) task.
Backlinks
with a foundation model embedding map (download from a open source large model):
- map titles of (perhaps abstract?) literatures into embeddings
- map query string (“foundation model and …”) into embedding
- search with vector similarity between query string’s embedding and embeddings of literature titles.
Brought up by John Kitchen’s proof of concept on embedding with literature titles.
highlight + comment method is slow
(expectation)
- this may be due to density of novel ideas, which require more typing work in zettelkasten, as well as setting up basic nodes such as foundation model to talk about it in foundation model contains generic human behaviours.
- when density of novel ideas in a text is lower for familiarity of the topic or just nature of text, (current test are done with introduction, where bird eye’s view of lots of ideas are mentioned), the time is expected to lower.
- yet still, this method is going to be slower and suitable for close read.
foundation model contains generic human behaviours
as foundation model are pre-trained on large amount of human behaviours (language, language-visual association, etc.), it contains a comprehensive model of generic human behaviours.
when used in specific task, this model of generic human behaviours could help, if the task would largely comply to, or understandable with generic human behaviours, which is trained, with lots of design works and computation resources, into the foundation model.
foundation model for sensor fusion
As foundation model may represent language-visual, language-audio associations as seen in human behaviours, and output text/embeddings of homogeneous format, it could used to fuse sensor readings(perhaps multimodal) to one representation(the embedding)
Definition are usually aiming for abstractness and ability to be futher tuned/plugged-in with user-specified functions.
In this way, it functions like foundation model, where concrete/downstream task performs on