// Hacker Noon · 27 February 2026

Analyzing ReLUfication Limitations: Enhancing LLM Sparsity via Up Projection

Explore the limitations of existing ReLUfication methods, which only improve sparsity from 40% to 67%. Learn why modifying the up projection component is key to achieving higher LLM activation sparsity.

Hacker Noon

@hacker-noon · Language Models (dot tech)

hackernoon.com

Hacker Noon@hacker-noon