comparison src/ex_cmds.c @ 34309:d7cfd8fb1d75 v9.1.0089

patch 9.1.0089: qsort() comparison functions should be transitive Commit: https://github.com/vim/vim/commit/e06e43766500ecb4cd1031fa16cf9cbebdb222c1 Author: Christian Brabandt <cb@256bit.org> Date: Fri Feb 9 19:39:14 2024 +0100 patch 9.1.0089: qsort() comparison functions should be transitive Problem: qsort() comparison functions should be transitive Solution: Do not subtract values, but rather use explicit comparisons Improve qsort() comparison functions There has been a recent report on qsort() causing out-of-bounds read & write in glibc for non transitive comparison functions https://www.qualys.com/2024/01/30/qsort.txt Even so the bug is in glibc's implementation of the qsort() algorithm, it's bad style to just use substraction for the comparison functions, which may cause overflow issues and as hinted at in OpenBSD's manual page for qsort(): "It is almost always an error to use subtraction to compute the return value of the comparison function." So check the qsort() comparison functions and change them to be safe. closes: #13980 Signed-off-by: Christian Brabandt <cb@256bit.org>
author Christian Brabandt <cb@256bit.org>
date Fri, 09 Feb 2024 19:45:06 +0100
parents a522c6c0127b
children dd8f5311cee5
comparison
equal deleted inserted replaced
34308:781c39b50ce9 34309:d7cfd8fb1d75
321 sort_abort = TRUE; 321 sort_abort = TRUE;
322 322
323 if (sort_nr) 323 if (sort_nr)
324 { 324 {
325 if (l1.st_u.num.is_number != l2.st_u.num.is_number) 325 if (l1.st_u.num.is_number != l2.st_u.num.is_number)
326 result = l1.st_u.num.is_number - l2.st_u.num.is_number; 326 result = l1.st_u.num.is_number > l2.st_u.num.is_number ? 1 : -1;
327 else 327 else
328 result = l1.st_u.num.value == l2.st_u.num.value ? 0 328 result = l1.st_u.num.value == l2.st_u.num.value ? 0
329 : l1.st_u.num.value > l2.st_u.num.value ? 1 : -1; 329 : l1.st_u.num.value > l2.st_u.num.value ? 1 : -1;
330 } 330 }
331 else if (sort_flt) 331 else if (sort_flt)