Embeddings as Dirichlet counts: Attention is the tip of the iceberg
{{output}}
Despite the overtly discrete nature of language, the use of semantic embedding spaces is pervasive in modern computational linguistics and machine learning for natural language. I argue that this is intelligible if language is viewed as an interface into a gen... ...
