Apple releases OpenELM, a slightly more accurate LLM

Apple, not normally known for its openness, has released a generative AI model called OpenELM which apparently outperforms a set of other language models trained on public data sets.

It’s not by much – compared to OLMo, which debuted in February, OpenELM is 2.36 percent more accurate while using 2x fewer pretraining tokens. But it’s perhaps enough to remind people that Apple is no longer content to be the wallflower at the industry AI rave.

Apple’s claim to openness comes from its decision to release not just the model, but its training and evaluation framework.

“Diverging from prior practices that only provide model weights and inference code, and pre-train on private datasets, our release includes the complete framework for training and evaluation of the language model on publicly available datasets, including training logs, multiple checkpoints, and pre-training configurations,” explain eleven Apple researchers in the associated technical paper.