Bahrad A. Sokhansanj

Context-Aware Regularization with Markovian Integration for Attention-Based Nucleotide Analysis

Transformers have revolutionized nucleotide sequence analysis, yet capturing long‑range dependencies remains challenging. Recent studies show that autoregressive transformers often exhibit Markovian behavior by relying on fixed-length context windows for next-token prediction. However, standard self…