Wiki › Project

Language model study

Project · study · 2025

Language model study is code from his study of language-model fundamentals in spring 2025, following material from Carnegie Mellon University's Advanced NLP course (spring 2025). It starts where language modelling starts: an autoregressive bigram model over characters, written in Python.

The bigram model

The model reads a list of names, wraps each in start and end markers, and counts how often each character follows each other one. Normalised, those counts give the probability of the next character given the current one, and sampling from them one character at a time produces new name-like words. It is the simplest possible autoregressive model, and the starting point for neural ones. A test folder holds experiments for running on a GPU.

See also

In the knowledge graph

Language model study and the 5 things it is linked to. Open any dot for its page, or explore it in the full graph.

Python · skill · Built withPythonNatural language processing · skill · Built withNLPBigram language model · technique · ImplementsBigram language m…Machine learning and data · theme · ThemesMachine learning …Open source on GitHub · person, role or place · ContextOpen source Languagemodel study
5 linked in the graph: 2 skills1 technique1 theme1 person, role or place
When
2023202420252026
Apr 2025 · 4 commits

As data

Everything on this page is read from triples in the knowledge graph. These are the 17 it states about Language model study, and 2 more its ontology implies; the console runs SPARQL over the whole graph, here in your browser.

Download
@prefix id: <https://ask.chaipat.cc/id/> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix cj: <https://ask.chaipat.cc/ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .

id:LearnLM
    a cj:Project ;
    rdfs:label "Language model study" ;
    cj:description "Language-model study code." ;
    cj:context id:OpenSource ;
    cj:repository "https://github.com/AppleBoiy/LM"^^xsd:anyURI ;
    cj:firstCommit "2025-04-21"^^xsd:date ;
    cj:lastCommit "2025-04-21"^^xsd:date ;
    cj:commitCount 4 ;
    cj:kind "study" ;
    cj:theme id:ThemeML ;
    cj:uses id:Python , id:NLP ;
    cj:tier 3 ;
    cj:implements id:BigramModel ;
    cj:article "https://ask.chaipat.cc/wiki/LearnLM"^^xsd:anyURI ;
    cj:activeIn id:Y2025 ;
    cj:activity _:b91 .