cover

How an Open Model and a Pile of Data are Changing Time Series Analysis

30 Jun 2025

MOMENT delivers an open-source foundation model and the "Time Series Pile," advancing low-supervision analysis and promoting transparent, open science.

cover

When a Specialized Time Series Model Outshines General LLMs

30 Jun 2025

MOMENT excels in low-supervision tasks like forecasting and anomaly detection, often outperforming LLM-based models and showing strong scaling properties

cover

How Do You Train an AI to Understand Time? With a Giant Pile of Data.

30 Jun 2025

Built on the "Time Series Pile," MOMENT uses masked patch prediction to pre-train a versatile Transformer, ready for fine-tuning on diverse tasks.

cover

Why Training on Time Series Beats Fine-Tuning LLMs for Time Series Tasks

30 Jun 2025

MOMENT uses masked patch pre-training on diverse time series, moving beyond single-dataset models and LLM to explore large-scale, low-supervision learning.

cover

How a New AI Model is Taming the Chaos of Time Series Data

30 Jun 2025

MOMENT: an open-source foundation model for time series, pre-trained on a massive "Time Series Pile" to excel at diverse tasks with limited supervision.

cover

Transformer Theory & LLM References: Here's What You Should Check Out

25 Jun 2025

A concise list of key academic works informing our research on Transformer model dynamics, cross-entropy loss, and theoretical connections to Hopfield networks.

cover

GPT-2 Architecture and Training Details: Parameters & Cross-Entropy Loss

24 Jun 2025

Explore the original GPT-2 model's architecture, including its training on WebText, BPE tokenizer, hidden dimensions, and layer parameters

cover

Theoretical Derivations: Cross-Entropy Loss and Energy Functions in LLMs

24 Jun 2025

Explore rigorous mathematical proofs, including properties of incomplete gamma functions, Stirling's approximation, and derivations of loss functions

cover

LogSumExp Function Properties: Lemmas for Energy Functions

24 Jun 2025

Explore key mathematical properties of the LogSumExp function, including bounds and continuity, which are crucial for understanding energy functions