Vertex-Softmax: Tight Transformer Verification via Exact Softmax Optimization

مقتطف من المصدر

تنظم هذه الصفحة محتوى عن الذكاء الاصطناعي والبحث لتسهيل قراءته والاستشهاد به والتحقق منه من قبل الباحثين والطلاب وأدوات الذكاء الاصطناعي.

arXiv:2605.10974v1 Announce Type: new Abstract: Certified verification of transformer attention requires bounding the softmax function over interval constraints on the pre-softmax scores. Existing verifiers relax softmax ndependently of the downstream objective, leaving avoidable slack. We prove that the exact optimum of this score-box problem is attained at a vertex of the constraint box, and establish a threshold structure theorem showing that, after sorting the objective coefficients, the optimum lies among only linearly many candidates, yielding the Vertex-Softmax primitive with log-linear complexity in the sequence length. We further prove a formal optimality result showing that Vertex-Softmax is the tightest sound bound obtainable from score intervals alone, characterizing precisely what additional structure (score correlations, score-value coupling) is needed for further improvement. Integrated into a CROWN Convex Relaxation based Optimization for Worst-case Neurons)-style verifier with a formal soundness guarantee, Vertex-Softmax significantly improves certified rates and substantially tightens lower bounds across MNIST, Fashion-MNIST, and CIFAR-10 attention models, while consistently matching or outperforming alpha-CROWN and branch-and-bound baselines at a fraction of their cost.

عرض المصدر الأصلي

المصدر الرسمي أو الأصلي: arXiv Machine Learning. تحقق دائما من التفاصيل في المصدر الأساسي.

العودة إلى الذكاء الاصطناعي والبحث