Siyue Liang — accepted AI-conference papers

2025

ExLM: Rethinking the Impact of $\texttt{[MASK]}$ Tokens in Masked Language Models

Kangjie Zheng, Junwei Yang, Siyue Liang, Bin Feng, Zequn Liu, Wei Ju +2

ICML 2025poster

Masked Language Models (MLMs) have achieved remarkable success in many self-supervised representation learning tasks. MLMs are trained by randomly masking portions of the input sequences with $\texttt{[MASK]}$ tokens and learning to reconstruct the original content based on the remaining context. Th…

Cited by 0 Source ↗PDF ↗Scholar ↗

2025

SMI-Editor: Edit-based SMILES Language Model with Fragment-level Supervision

Kangjie Zheng, Siyue Liang, Junwei Yang, Bin Feng, Zequn Liu, Wei Ju +2

ICLR 2025poster

SMILES, a crucial textual representation of molecular structures, has garnered significant attention as a foundation for pre-trained language models (LMs). However, most existing pre-trained SMILES LMs focus solely on the single-token level supervision during pre-training, failing to fully leverage…

Cited by 1 Source ↗PDF ↗Scholar ↗