Mercurial > vim
changeset 15040:da8a8cdcb8b0 v8.1.0531
patch 8.1.0531: flaky tests often fail with a common error message
commit https://github.com/vim/vim/commit/dbc0d2163aa5e090d5a0c83aea448803ddbab664
Author: Bram Moolenaar <Bram@vim.org>
Date: Fri Nov 16 18:22:45 2018 +0100
patch 8.1.0531: flaky tests often fail with a common error message
Problem: Flaky tests often fail with a common error message.
Solution: Add a pattern to match an error message indicating a flaky test.
| author | Bram Moolenaar <Bram@vim.org> |
|---|---|
| date | Fri, 16 Nov 2018 18:30:05 +0100 |
| parents | 8296f67e651c |
| children | f51dd8405b99 |
| files | src/testdir/runtest.vim src/version.c |
| diffstat | 2 files changed, 9 insertions(+), 2 deletions(-) [+] |
line wrap: on
line diff
--- a/src/testdir/runtest.vim +++ b/src/testdir/runtest.vim @@ -273,7 +273,7 @@ else endif " Names of flaky tests. -let s:flaky = [ +let s:flaky_tests = [ \ 'Test_call()', \ 'Test_channel_handler()', \ 'Test_client_server()', @@ -329,6 +329,9 @@ let s:flaky = [ \ 'Test_zz1_terminal_in_gui()', \ ] +" Pattern indicating a common flaky test failure. +let s:flaky_errors_re = 'StopVimInTerminal' + " Locate Test_ functions and execute them. redir @q silent function /^Test_ @@ -353,7 +356,9 @@ for s:test in sort(s:tests) " Repeat a flaky test. Give up when: " - it fails again with the same message " - it fails five times (with a different mesage) - if len(v:errors) > 0 && index(s:flaky, s:test) >= 0 + if len(v:errors) > 0 + \ && (index(s:flaky_tests, s:test) >= 0 + \ || v:errors[0] =~ s:flaky_errors_re) while 1 call add(s:messages, 'Found errors in ' . s:test . ':') call extend(s:messages, v:errors)