Mercurial > vim

annotate src/testdir/runtest.vim @ 30332:462d122636b3

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Added tag v9.0.0501 for changeset ddbac2551dacc3dce403cc1c00c1c354434bc44e
author Bram Moolenaar <Bram@vim.org>
date Mon, 19 Sep 2022 12:45:05 +0200
parents 029c59bf78f1
children 2f295612c46a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
1 " This script is sourced while editing the .vim file with the tests.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
2 " When the script is successful the .res file will be created.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
3 " Errors are appended to the test.log file.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
4 "
8142
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
5 " To execute only specific test functions, add a second argument. It will be
18748
c7b16393c248 patch 8.1.2364: termwinscroll test is flaky on FreeBSD
Bram Moolenaar <Bram@vim.org>
parents: 18691
diff changeset
6 " matched against the names of the Test_ function. E.g.:
8142
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
7 " ../vim -u NONE -S runtest.vim test_channel.vim open_delay
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
8 " The output can be found in the "messages" file.
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
9 "
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
10 " If the environment variable $TEST_FILTER is set then only test functions
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
11 " matching this pattern are executed. E.g. for sh/bash:
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
12 " export TEST_FILTER=Test_channel
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
13 " For csh:
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
14 " setenv TEST_FILTER Test_channel
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
15 "
25549
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
16 " If the environment variable $TEST_SKIP_PAT is set then test functions
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
17 " matching this pattern will be skipped. It's the opposite of $TEST_FILTER.
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
18 "
21508
6b2a5418cbc3 patch 8.2.1304: debug backtrace isn't tested much
Bram Moolenaar <Bram@vim.org>
parents: 21433
diff changeset
19 " While working on a test you can make $TEST_NO_RETRY non-empty to not retry:
6b2a5418cbc3 patch 8.2.1304: debug backtrace isn't tested much
Bram Moolenaar <Bram@vim.org>
parents: 21433
diff changeset
20 " export TEST_NO_RETRY=yes
6b2a5418cbc3 patch 8.2.1304: debug backtrace isn't tested much
Bram Moolenaar <Bram@vim.org>
parents: 21433
diff changeset
21 "
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
22 " To ignore failure for tests that are known to fail in a certain environment,
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
23 " set $TEST_MAY_FAIL to a comma separated list of function names. E.g. for
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
24 " sh/bash:
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
25 " export TEST_MAY_FAIL=Test_channel_one,Test_channel_other
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
26 " The failure report will then not be included in the test.log file and
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
27 " "make test" will not fail.
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
28 "
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
29 " The test script may contain anything, only functions that start with
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
30 " "Test_" are special. These will be invoked and should contain assert
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
31 " functions. See test_assert.vim for an example.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
32 "
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
33 " It is possible to source other files that contain "Test_" functions. This
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
34 " can speed up testing, since Vim does not need to restart. But be careful
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
35 " that the tests do not interfere with each other.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
36 "
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
37 " If an error cannot be detected properly with an assert function add the
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
38 " error to the v:errors list:
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
39 " call add(v:errors, 'test foo failed: Cannot find xyz')
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
40 "
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
41 " If preparation for each Test_ function is needed, define a SetUp function.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
42 " It will be called before each Test_ function.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
43 "
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
44 " If cleanup after each Test_ function is needed, define a TearDown function.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
45 " It will be called after each Test_ function.
8029
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
46 "
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
47 " When debugging a test it can be useful to add messages to v:errors:
15377
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset
Bram Moolenaar <Bram@vim.org>
parents: 15180
diff changeset
48 " call add(v:errors, "this happened")
8029
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
49
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
50
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
51 " Without the +eval feature we can't run these tests, bail out.
21779
30bdd2e4a6f9 patch 8.2.1439: tiny and small builds have no test coverage
Bram Moolenaar <Bram@vim.org>
parents: 21668
diff changeset
52 silent! while 0
30bdd2e4a6f9 patch 8.2.1439: tiny and small builds have no test coverage
Bram Moolenaar <Bram@vim.org>
parents: 21668
diff changeset
53 qa!
30bdd2e4a6f9 patch 8.2.1439: tiny and small builds have no test coverage
Bram Moolenaar <Bram@vim.org>
parents: 21668
diff changeset
54 silent! endwhile
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
55
21253
ffa6b82fa993 patch 8.2.1177: terminal2 test sometimes hangs in the GUI
Bram Moolenaar <Bram@vim.org>
parents: 20963
diff changeset
56 " In the GUI we can always change the screen size.
ffa6b82fa993 patch 8.2.1177: terminal2 test sometimes hangs in the GUI
Bram Moolenaar <Bram@vim.org>
parents: 20963
diff changeset
57 if has('gui_running')
ffa6b82fa993 patch 8.2.1177: terminal2 test sometimes hangs in the GUI
Bram Moolenaar <Bram@vim.org>
parents: 20963
diff changeset
58 set columns=80 lines=25
ffa6b82fa993 patch 8.2.1177: terminal2 test sometimes hangs in the GUI
Bram Moolenaar <Bram@vim.org>
parents: 20963
diff changeset
59 endif
ffa6b82fa993 patch 8.2.1177: terminal2 test sometimes hangs in the GUI
Bram Moolenaar <Bram@vim.org>
parents: 20963
diff changeset
60
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
61 " Check that the screen size is at least 24 x 80 characters.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
62 if &lines < 24 || &columns < 80
17684
e2bfdd281578 patch 8.1.1839: insufficient info when test fails because of screen size
Bram Moolenaar <Bram@vim.org>
parents: 17356
diff changeset
63 let error = 'Screen size too small! Tests require at least 24 lines with 80 characters, got ' .. &lines .. ' lines with ' .. &columns .. ' characters'
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
64 echoerr error
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
65 split test.log
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
66 $put =error
17091
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that
Bram Moolenaar <Bram@vim.org>
parents: 17032
diff changeset
67 write
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that
Bram Moolenaar <Bram@vim.org>
parents: 17032
diff changeset
68 split messages
17684
e2bfdd281578 patch 8.1.1839: insufficient info when test fails because of screen size
Bram Moolenaar <Bram@vim.org>
parents: 17356
diff changeset
69 call append(line('$'), '')
e2bfdd281578 patch 8.1.1839: insufficient info when test fails because of screen size
Bram Moolenaar <Bram@vim.org>
parents: 17356
diff changeset
70 call append(line('$'), 'From ' . expand('%') . ':')
17091
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that
Bram Moolenaar <Bram@vim.org>
parents: 17032
diff changeset
71 call append(line('$'), error)
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that
Bram Moolenaar <Bram@vim.org>
parents: 17032
diff changeset
72 write
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that
Bram Moolenaar <Bram@vim.org>
parents: 17032
diff changeset
73 qa!
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
74 endif
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
75
16967
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
76 if has('reltime')
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
77 let s:start_time = reltime()
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
78 endif
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
79
26500
8088fa133b93 patch 8.2.3780: ":cd" works differently on MS-Windows
Bram Moolenaar <Bram@vim.org>
parents: 26211
diff changeset
80 " Always use forward slashes.
8088fa133b93 patch 8.2.3780: ":cd" works differently on MS-Windows
Bram Moolenaar <Bram@vim.org>
parents: 26211
diff changeset
81 set shellslash
8088fa133b93 patch 8.2.3780: ":cd" works differently on MS-Windows
Bram Moolenaar <Bram@vim.org>
parents: 26211
diff changeset
82
9501
f0201deac2a2 commit https://github.com/vim/vim/commit/89b10421ca1aea55acbafac1df93be4e20210ceb
Christian Brabandt <cb@256bit.org>
parents: 9452
diff changeset
83 " Common with all tests on all systems.
f0201deac2a2 commit https://github.com/vim/vim/commit/89b10421ca1aea55acbafac1df93be4e20210ceb
Christian Brabandt <cb@256bit.org>
parents: 9452
diff changeset
84 source setup.vim
f0201deac2a2 commit https://github.com/vim/vim/commit/89b10421ca1aea55acbafac1df93be4e20210ceb
Christian Brabandt <cb@256bit.org>
parents: 9452
diff changeset
85
7391
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
86 " For consistency run all tests with 'nocompatible' set.
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
87 " This also enables use of line continuation.
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
88 set nocp viminfo+=nviminfo
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
89
15607
2dcaa860e3fc patch 8.1.0811: too many #ifdefs
Bram Moolenaar <Bram@vim.org>
parents: 15377
diff changeset
90 " Use utf-8 by default, instead of whatever the system default happens to be.
15928
857ce36c8412 patch 8.1.0970: text properties test fails when 'encoding' is not utf-8
Bram Moolenaar <Bram@vim.org>
parents: 15920
diff changeset
91 " Individual tests can overrule this at the top of the file and use
857ce36c8412 patch 8.1.0970: text properties test fails when 'encoding' is not utf-8
Bram Moolenaar <Bram@vim.org>
parents: 15920
diff changeset
92 " g:orig_encoding if needed.
857ce36c8412 patch 8.1.0970: text properties test fails when 'encoding' is not utf-8
Bram Moolenaar <Bram@vim.org>
parents: 15920
diff changeset
93 let g:orig_encoding = &encoding
15607
2dcaa860e3fc patch 8.1.0811: too many #ifdefs
Bram Moolenaar <Bram@vim.org>
parents: 15377
diff changeset
94 set encoding=utf-8
9618
81ba6e4eb72b commit https://github.com/vim/vim/commit/ac105ed3c420660ddbddc501c97875c48220817e
Christian Brabandt <cb@256bit.org>
parents: 9501
diff changeset
95
14889
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
96 " REDIR_TEST_TO_NULL has a very permissive SwapExists autocommand which is for
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
97 " the test_name.vim file itself. Replace it here with a more restrictive one,
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
98 " so we still catch mistakes.
26092
6a0f603c52a9 patch 8.2.3579: CI sometimes fails for MinGW
Bram Moolenaar <Bram@vim.org>
parents: 25969
diff changeset
99 if has("win32")
6a0f603c52a9 patch 8.2.3579: CI sometimes fails for MinGW
Bram Moolenaar <Bram@vim.org>
parents: 25969
diff changeset
100 " replace any '/' directory separators by '\\'
6a0f603c52a9 patch 8.2.3579: CI sometimes fails for MinGW
Bram Moolenaar <Bram@vim.org>
parents: 25969
diff changeset
101 let s:test_script_fname = substitute(expand('%'), '/', '\\', 'g')
6a0f603c52a9 patch 8.2.3579: CI sometimes fails for MinGW
Bram Moolenaar <Bram@vim.org>
parents: 25969
diff changeset
102 else
6a0f603c52a9 patch 8.2.3579: CI sometimes fails for MinGW
Bram Moolenaar <Bram@vim.org>
parents: 25969
diff changeset
103 let s:test_script_fname = expand('%')
6a0f603c52a9 patch 8.2.3579: CI sometimes fails for MinGW
Bram Moolenaar <Bram@vim.org>
parents: 25969
diff changeset
104 endif
14889
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
105 au! SwapExists * call HandleSwapExists()
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
106 func HandleSwapExists()
21668
0843f244e547 patch 8.2.1384: no ATTENTION prompt for :vimgrep first match file
Bram Moolenaar <Bram@vim.org>
parents: 21530
diff changeset
107 if exists('g:ignoreSwapExists')
0843f244e547 patch 8.2.1384: no ATTENTION prompt for :vimgrep first match file
Bram Moolenaar <Bram@vim.org>
parents: 21530
diff changeset
108 return
0843f244e547 patch 8.2.1384: no ATTENTION prompt for :vimgrep first match file
Bram Moolenaar <Bram@vim.org>
parents: 21530
diff changeset
109 endif
17356
4fe44a3b732f patch 8.1.1677: tests get stuck when running into an existing swapfile
Bram Moolenaar <Bram@vim.org>
parents: 17123
diff changeset
110 " Ignore finding a swap file for the test script (the user might be
14889
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
111 " editing it and do ":make test_name") and the output file.
17356
4fe44a3b732f patch 8.1.1677: tests get stuck when running into an existing swapfile
Bram Moolenaar <Bram@vim.org>
parents: 17123
diff changeset
112 " Report finding another swap file and chose 'q' to avoid getting stuck.
14889
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
113 if expand('<afile>') == 'messages' || expand('<afile>') =~ s:test_script_fname
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
114 let v:swapchoice = 'e'
17356
4fe44a3b732f patch 8.1.1677: tests get stuck when running into an existing swapfile
Bram Moolenaar <Bram@vim.org>
parents: 17123
diff changeset
115 else
4fe44a3b732f patch 8.1.1677: tests get stuck when running into an existing swapfile
Bram Moolenaar <Bram@vim.org>
parents: 17123
diff changeset
116 call assert_report('Unexpected swap file: ' .. v:swapname)
4fe44a3b732f patch 8.1.1677: tests get stuck when running into an existing swapfile
Bram Moolenaar <Bram@vim.org>
parents: 17123
diff changeset
117 let v:swapchoice = 'q'
14889
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
118 endif
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
119 endfunc
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
120
10811
4cb74eab1733 patch 8.0.0295: test_viml hangs
Christian Brabandt <cb@256bit.org>
parents: 10807
diff changeset
121 " Avoid stopping at the "hit enter" prompt
4cb74eab1733 patch 8.0.0295: test_viml hangs
Christian Brabandt <cb@256bit.org>
parents: 10807
diff changeset
122 set nomore
4cb74eab1733 patch 8.0.0295: test_viml hangs
Christian Brabandt <cb@256bit.org>
parents: 10807
diff changeset
123
7391
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
124 " Output all messages in English.
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
125 lang mess C
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
126
21379
20dad734a2cb patch 8.2.1240: GUI tests sometimes fail because of translations
Bram Moolenaar <Bram@vim.org>
parents: 21253
diff changeset
127 " suppress menu translation
20dad734a2cb patch 8.2.1240: GUI tests sometimes fail because of translations
Bram Moolenaar <Bram@vim.org>
parents: 21253
diff changeset
128 if has('gui_running') && exists('did_install_default_menus')
20dad734a2cb patch 8.2.1240: GUI tests sometimes fail because of translations
Bram Moolenaar <Bram@vim.org>
parents: 21253
diff changeset
129 source $VIMRUNTIME/delmenu.vim
20dad734a2cb patch 8.2.1240: GUI tests sometimes fail because of translations
Bram Moolenaar <Bram@vim.org>
parents: 21253
diff changeset
130 set langmenu=none
20dad734a2cb patch 8.2.1240: GUI tests sometimes fail because of translations
Bram Moolenaar <Bram@vim.org>
parents: 21253
diff changeset
131 source $VIMRUNTIME/menu.vim
20dad734a2cb patch 8.2.1240: GUI tests sometimes fail because of translations
Bram Moolenaar <Bram@vim.org>
parents: 21253
diff changeset
132 endif
20dad734a2cb patch 8.2.1240: GUI tests sometimes fail because of translations
Bram Moolenaar <Bram@vim.org>
parents: 21253
diff changeset
133
7545
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
134 let s:srcdir = expand('%:p:h:h')
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
135
21433
ad7ffbd3f02f patch 8.2.1267: MS-Windows: tests may fail due to $PROMPT value
Bram Moolenaar <Bram@vim.org>
parents: 21379
diff changeset
136 if has('win32')
ad7ffbd3f02f patch 8.2.1267: MS-Windows: tests may fail due to $PROMPT value
Bram Moolenaar <Bram@vim.org>
parents: 21379
diff changeset
137 " avoid prompt that is long or contains a line break
ad7ffbd3f02f patch 8.2.1267: MS-Windows: tests may fail due to $PROMPT value
Bram Moolenaar <Bram@vim.org>
parents: 21379
diff changeset
138 let $PROMPT = '$P$G'
21530
66f12838071d patch 8.2.1315: MS-Windows: test log contains escape sequences
Bram Moolenaar <Bram@vim.org>
parents: 21510
diff changeset
139 " On MS-Windows t_md and t_me are Vim specific escape sequences.
66f12838071d patch 8.2.1315: MS-Windows: test log contains escape sequences
Bram Moolenaar <Bram@vim.org>
parents: 21510
diff changeset
140 let s:t_bold = "\x1b[1m"
66f12838071d patch 8.2.1315: MS-Windows: test log contains escape sequences
Bram Moolenaar <Bram@vim.org>
parents: 21510
diff changeset
141 let s:t_normal = "\x1b[m"
66f12838071d patch 8.2.1315: MS-Windows: test log contains escape sequences
Bram Moolenaar <Bram@vim.org>
parents: 21510
diff changeset
142 else
66f12838071d patch 8.2.1315: MS-Windows: test log contains escape sequences
Bram Moolenaar <Bram@vim.org>
parents: 21510
diff changeset
143 let s:t_bold = &t_md
66f12838071d patch 8.2.1315: MS-Windows: test log contains escape sequences
Bram Moolenaar <Bram@vim.org>
parents: 21510
diff changeset
144 let s:t_normal = &t_me
21433
ad7ffbd3f02f patch 8.2.1267: MS-Windows: tests may fail due to $PROMPT value
Bram Moolenaar <Bram@vim.org>
parents: 21379
diff changeset
145 endif
ad7ffbd3f02f patch 8.2.1267: MS-Windows: tests may fail due to $PROMPT value
Bram Moolenaar <Bram@vim.org>
parents: 21379
diff changeset
146
22967
47a5349168ea patch 8.2.2030: some tests fail on Mac
Bram Moolenaar <Bram@vim.org>
parents: 22480
diff changeset
147 if has('mac')
26211
485c7c4afeb7 patch 8.2.3637: typos in test files
Bram Moolenaar <Bram@vim.org>
parents: 26092
diff changeset
148 " In macOS, when starting a shell in a terminal, a bash deprecation warning
22967
47a5349168ea patch 8.2.2030: some tests fail on Mac
Bram Moolenaar <Bram@vim.org>
parents: 22480
diff changeset
149 " message is displayed. This breaks the terminal test. Disable the warning
47a5349168ea patch 8.2.2030: some tests fail on Mac
Bram Moolenaar <Bram@vim.org>
parents: 22480
diff changeset
150 " message.
47a5349168ea patch 8.2.2030: some tests fail on Mac
Bram Moolenaar <Bram@vim.org>
parents: 22480
diff changeset
151 let $BASH_SILENCE_DEPRECATION_WARNING = 1
47a5349168ea patch 8.2.2030: some tests fail on Mac
Bram Moolenaar <Bram@vim.org>
parents: 22480
diff changeset
152 endif
47a5349168ea patch 8.2.2030: some tests fail on Mac
Bram Moolenaar <Bram@vim.org>
parents: 22480
diff changeset
153
9121
7350959e53c3 commit https://github.com/vim/vim/commit/8e8df251bf2505e5decf258397c6069fbe5e2e01
Christian Brabandt <cb@256bit.org>
parents: 8881
diff changeset
154 " Prepare for calling test_garbagecollect_now().
8881
ed0b39dd7fd6 commit https://github.com/vim/vim/commit/ebf7dfa6f121c82f97d2adca3d45fbaba9ad8f7e
Christian Brabandt <cb@256bit.org>
parents: 8796
diff changeset
155 let v:testing = 1
ed0b39dd7fd6 commit https://github.com/vim/vim/commit/ebf7dfa6f121c82f97d2adca3d45fbaba9ad8f7e
Christian Brabandt <cb@256bit.org>
parents: 8796
diff changeset
156
29340
fba9e366ced4 patch 9.0.0013: reproducing memory access errors can be difficult
Bram Moolenaar <Bram@vim.org>
parents: 29165
diff changeset
157 " By default, copy each buffer line into allocated memory, so that valgrind can
fba9e366ced4 patch 9.0.0013: reproducing memory access errors can be difficult
Bram Moolenaar <Bram@vim.org>
parents: 29165
diff changeset
158 " detect accessing memory before and after it.
fba9e366ced4 patch 9.0.0013: reproducing memory access errors can be difficult
Bram Moolenaar <Bram@vim.org>
parents: 29165
diff changeset
159 call test_override('alloc_lines', 1)
fba9e366ced4 patch 9.0.0013: reproducing memory access errors can be difficult
Bram Moolenaar <Bram@vim.org>
parents: 29165
diff changeset
160
7545
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
161 " Support function: get the alloc ID by name.
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
162 function GetAllocId(name)
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
163 exe 'split ' . s:srcdir . '/alloc.h'
7595
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0
Christian Brabandt <cb@256bit.org>
parents: 7545
diff changeset
164 let top = search('typedef enum')
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0
Christian Brabandt <cb@256bit.org>
parents: 7545
diff changeset
165 if top == 0
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0
Christian Brabandt <cb@256bit.org>
parents: 7545
diff changeset
166 call add(v:errors, 'typedef not found in alloc.h')
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0
Christian Brabandt <cb@256bit.org>
parents: 7545
diff changeset
167 endif
7545
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
168 let lnum = search('aid_' . a:name . ',')
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
169 if lnum == 0
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
170 call add(v:errors, 'Alloc ID ' . a:name . ' not defined')
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
171 endif
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
172 close
7595
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0
Christian Brabandt <cb@256bit.org>
parents: 7545
diff changeset
173 return lnum - top - 1
7545
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
174 endfunc
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
175
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
176 func RunTheTest(test)
24194
9f64c420f280 patch 8.2.2638: cannot write a message to the terminal from the GUI
Bram Moolenaar <Bram@vim.org>
parents: 24095
diff changeset
177 echoconsole 'Executing ' . a:test
16967
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
178 if has('reltime')
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
179 let func_start = reltime()
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
180 endif
10807
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
181
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
182 " Avoid stopping at the "hit enter" prompt
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
183 set nomore
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
184
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
185 " Avoid a three second wait when a message is about to be overwritten by the
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
186 " mode message.
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
187 set noshowmode
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
188
29340
fba9e366ced4 patch 9.0.0013: reproducing memory access errors can be difficult
Bram Moolenaar <Bram@vim.org>
parents: 29165
diff changeset
189 " Clear any overrides, except "alloc_lines".
11105
7c7e496e625d patch 8.0.0440: not enough test coverage in Insert mode
Christian Brabandt <cb@256bit.org>
parents: 11056
diff changeset
190 call test_override('ALL', 0)
7c7e496e625d patch 8.0.0440: not enough test coverage in Insert mode
Christian Brabandt <cb@256bit.org>
parents: 11056
diff changeset
191
12692
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
192 " Some tests wipe out buffers. To be consistent, always wipe out all
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
193 " buffers.
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
194 %bwipe!
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
195
12851
90aaa974594e patch 8.0.1302: still too many old style tests
Christian Brabandt <cb@256bit.org>
parents: 12769
diff changeset
196 " The test may change the current directory. Save and restore the
90aaa974594e patch 8.0.1302: still too many old style tests
Christian Brabandt <cb@256bit.org>
parents: 12769
diff changeset
197 " directory after executing the test.
90aaa974594e patch 8.0.1302: still too many old style tests
Christian Brabandt <cb@256bit.org>
parents: 12769
diff changeset
198 let save_cwd = getcwd()
90aaa974594e patch 8.0.1302: still too many old style tests
Christian Brabandt <cb@256bit.org>
parents: 12769
diff changeset
199
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
200 if exists("*SetUp")
10388
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
201 try
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
202 call SetUp()
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
203 catch
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
204 call add(v:errors, 'Caught exception in SetUp() before ' . a:test . ': ' . v:exception . ' @ ' . v:throwpoint)
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
205 endtry
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
206 endif
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
207
29165
0e0e298e05c1 patch 8.2.5102: interrupt not caught in test
Bram Moolenaar <Bram@vim.org>
parents: 29148
diff changeset
208 au VimLeavePre * call EarlyExit(g:testfunc)
12682
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
209 if a:test =~ 'Test_nocatch_'
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
210 " Function handles errors itself. This avoids skipping commands after the
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
211 " error.
24095
467c3022fba6 patch 8.2.2589: recover test hangs in the GUI
Bram Moolenaar <Bram@vim.org>
parents: 22967
diff changeset
212 let g:skipped_reason = ''
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
213 exe 'call ' . a:test
24095
467c3022fba6 patch 8.2.2589: recover test hangs in the GUI
Bram Moolenaar <Bram@vim.org>
parents: 22967
diff changeset
214 if g:skipped_reason != ''
467c3022fba6 patch 8.2.2589: recover test hangs in the GUI
Bram Moolenaar <Bram@vim.org>
parents: 22967
diff changeset
215 call add(s:messages, ' Skipped')
467c3022fba6 patch 8.2.2589: recover test hangs in the GUI
Bram Moolenaar <Bram@vim.org>
parents: 22967
diff changeset
216 call add(s:skipped, 'SKIPPED ' . a:test . ': ' . g:skipped_reason)
467c3022fba6 patch 8.2.2589: recover test hangs in the GUI
Bram Moolenaar <Bram@vim.org>
parents: 22967
diff changeset
217 endif
12682
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
218 else
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
219 try
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
220 exe 'call ' . a:test
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
221 catch /^\cskipped/
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
222 call add(s:messages, ' Skipped')
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
223 call add(s:skipped, 'SKIPPED ' . a:test . ': ' . substitute(v:exception, '^\S*\s\+', '', ''))
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
224 catch
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
225 call add(v:errors, 'Caught exception in ' . a:test . ': ' . v:exception . ' @ ' . v:throwpoint)
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
226 endtry
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
227 endif
29165
0e0e298e05c1 patch 8.2.5102: interrupt not caught in test
Bram Moolenaar <Bram@vim.org>
parents: 29148
diff changeset
228 au! VimLeavePre
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
229
15377
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset
Bram Moolenaar <Bram@vim.org>
parents: 15180
diff changeset
230 " In case 'insertmode' was set and something went wrong, make sure it is
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset
Bram Moolenaar <Bram@vim.org>
parents: 15180
diff changeset
231 " reset to avoid trouble with anything else.
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset
Bram Moolenaar <Bram@vim.org>
parents: 15180
diff changeset
232 set noinsertmode
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset
Bram Moolenaar <Bram@vim.org>
parents: 15180
diff changeset
233
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
234 if exists("*TearDown")
10388
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
235 try
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
236 call TearDown()
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
237 catch
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
238 call add(v:errors, 'Caught exception in TearDown() after ' . a:test . ': ' . v:exception . ' @ ' . v:throwpoint)
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
239 endtry
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
240 endif
9748
5440f3442841 commit https://github.com/vim/vim/commit/7cba71d7e3576639679b6a3aedeeb1ac07f7f2f5
Christian Brabandt <cb@256bit.org>
parents: 9632
diff changeset
241
17684
e2bfdd281578 patch 8.1.1839: insufficient info when test fails because of screen size
Bram Moolenaar <Bram@vim.org>
parents: 17356
diff changeset
242 " Clear any autocommands and put back the catch-all for SwapExists.
12692
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
243 au!
14889
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
244 au SwapExists * call HandleSwapExists()
12692
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
245
20386
3b3589275ab9 patch 8.2.0748: cannot get a list of all popups
Bram Moolenaar <Bram@vim.org>
parents: 20384
diff changeset
246 " Check for and close any stray popup windows.
18763
49b78d6465e5 patch 8.1.2371: FEAT_TEXT_PROP is a confusing name
Bram Moolenaar <Bram@vim.org>
parents: 18748
diff changeset
247 if has('popupwin')
29587
d70f588baaa1 patch 9.0.0134: no test for text property with column zero
Bram Moolenaar <Bram@vim.org>
parents: 29387
diff changeset
248 call assert_equal([], popup_list(), 'Popup is still present')
20384
42ab4d40e78f patch 8.2.0747: cannot forcefully close all popups
Bram Moolenaar <Bram@vim.org>
parents: 20087
diff changeset
249 call popup_clear(1)
17123
efc6f5e3b543 patch 8.1.1561: popup_setoptions() is not implemented yet
Bram Moolenaar <Bram@vim.org>
parents: 17091
diff changeset
250 endif
efc6f5e3b543 patch 8.1.1561: popup_setoptions() is not implemented yet
Bram Moolenaar <Bram@vim.org>
parents: 17091
diff changeset
251
29128
d8a962d7b008 patch 8.2.5084: when the GUI shows a dialog tests get stuck
Bram Moolenaar <Bram@vim.org>
parents: 28148
diff changeset
252 if filereadable('guidialogfile')
29148
8e87a5d2284c patch 8.2.5094: MS-Windows GUI: empty command may cause a dialog
Bram Moolenaar <Bram@vim.org>
parents: 29128
diff changeset
253 call add(v:errors, "Unexpected dialog: " .. readfile('guidialogfile')->join('<NL>'))
29128
d8a962d7b008 patch 8.2.5084: when the GUI shows a dialog tests get stuck
Bram Moolenaar <Bram@vim.org>
parents: 28148
diff changeset
254 call delete('guidialogfile')
d8a962d7b008 patch 8.2.5084: when the GUI shows a dialog tests get stuck
Bram Moolenaar <Bram@vim.org>
parents: 28148
diff changeset
255 endif
d8a962d7b008 patch 8.2.5084: when the GUI shows a dialog tests get stuck
Bram Moolenaar <Bram@vim.org>
parents: 28148
diff changeset
256
12688
a6a935b3270e patch 8.0.1222: test functions interfere with each other
Christian Brabandt <cb@256bit.org>
parents: 12682
diff changeset
257 " Close any extra tab pages and windows and make the current one not modified.
a6a935b3270e patch 8.0.1222: test functions interfere with each other
Christian Brabandt <cb@256bit.org>
parents: 12682
diff changeset
258 while tabpagenr('$') > 1
22470
f7471450243c patch 8.2.1783: try-catch test fails
Bram Moolenaar <Bram@vim.org>
parents: 22435
diff changeset
259 let winid = win_getid()
12692
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
260 quit!
22470
f7471450243c patch 8.2.1783: try-catch test fails
Bram Moolenaar <Bram@vim.org>
parents: 22435
diff changeset
261 if winid == win_getid()
f7471450243c patch 8.2.1783: try-catch test fails
Bram Moolenaar <Bram@vim.org>
parents: 22435
diff changeset
262 echoerr 'Could not quit window'
f7471450243c patch 8.2.1783: try-catch test fails
Bram Moolenaar <Bram@vim.org>
parents: 22435
diff changeset
263 break
f7471450243c patch 8.2.1783: try-catch test fails
Bram Moolenaar <Bram@vim.org>
parents: 22435
diff changeset
264 endif
12688
a6a935b3270e patch 8.0.1222: test functions interfere with each other
Christian Brabandt <cb@256bit.org>
parents: 12682
diff changeset
265 endwhile
a6a935b3270e patch 8.0.1222: test functions interfere with each other
Christian Brabandt <cb@256bit.org>
parents: 12682
diff changeset
266
9949
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
267 while 1
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
268 let wincount = winnr('$')
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
269 if wincount == 1
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
270 break
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
271 endif
9748
5440f3442841 commit https://github.com/vim/vim/commit/7cba71d7e3576639679b6a3aedeeb1ac07f7f2f5
Christian Brabandt <cb@256bit.org>
parents: 9632
diff changeset
272 bwipe!
9949
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
273 if wincount == winnr('$')
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
274 " Did not manage to close a window.
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
275 only!
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
276 break
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
277 endif
9748
5440f3442841 commit https://github.com/vim/vim/commit/7cba71d7e3576639679b6a3aedeeb1ac07f7f2f5
Christian Brabandt <cb@256bit.org>
parents: 9632
diff changeset
278 endwhile
12851
90aaa974594e patch 8.0.1302: still too many old style tests
Christian Brabandt <cb@256bit.org>
parents: 12769
diff changeset
279
90aaa974594e patch 8.0.1302: still too many old style tests
Christian Brabandt <cb@256bit.org>
parents: 12769
diff changeset
280 exe 'cd ' . save_cwd
17032
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
281
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
282 let message = 'Executed ' . a:test
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
283 if has('reltime')
20963
ed35c81bce77 patch 8.2.1033: not easy to read the test time in the test output
Bram Moolenaar <Bram@vim.org>
parents: 20386
diff changeset
284 let message ..= repeat(' ', 50 - len(message))
ed35c81bce77 patch 8.2.1033: not easy to read the test time in the test output
Bram Moolenaar <Bram@vim.org>
parents: 20386
diff changeset
285 let time = reltime(func_start)
30310
029c59bf78f1 patch 9.0.0491: no good reason to build without the float feature
Bram Moolenaar <Bram@vim.org>
parents: 30180
diff changeset
286 if reltimefloat(time) > 0.1
21530
66f12838071d patch 8.2.1315: MS-Windows: test log contains escape sequences
Bram Moolenaar <Bram@vim.org>
parents: 21510
diff changeset
287 let message = s:t_bold .. message
20963
ed35c81bce77 patch 8.2.1033: not easy to read the test time in the test output
Bram Moolenaar <Bram@vim.org>
parents: 20386
diff changeset
288 endif
ed35c81bce77 patch 8.2.1033: not easy to read the test time in the test output
Bram Moolenaar <Bram@vim.org>
parents: 20386
diff changeset
289 let message ..= ' in ' .. reltimestr(time) .. ' seconds'
30310
029c59bf78f1 patch 9.0.0491: no good reason to build without the float feature
Bram Moolenaar <Bram@vim.org>
parents: 30180
diff changeset
290 if reltimefloat(time) > 0.1
21530
66f12838071d patch 8.2.1315: MS-Windows: test log contains escape sequences
Bram Moolenaar <Bram@vim.org>
parents: 21510
diff changeset
291 let message ..= s:t_normal
20963
ed35c81bce77 patch 8.2.1033: not easy to read the test time in the test output
Bram Moolenaar <Bram@vim.org>
parents: 20386
diff changeset
292 endif
17032
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
293 endif
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
294 call add(s:messages, message)
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
295 let s:done += 1
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
296 endfunc
7545
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
297
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
298 func AfterTheTest(func_name)
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
299 if len(v:errors) > 0
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
300 if match(s:may_fail_list, '^' .. a:func_name) >= 0
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
301 let s:fail_expected += 1
20003
e373843e2980 patch 8.2.0557: no IPv6 support for channels
Bram Moolenaar <Bram@vim.org>
parents: 19795
diff changeset
302 call add(s:errors_expected, 'Found errors in ' . g:testfunc . ':')
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
303 call extend(s:errors_expected, v:errors)
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
304 else
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
305 let s:fail += 1
20003
e373843e2980 patch 8.2.0557: no IPv6 support for channels
Bram Moolenaar <Bram@vim.org>
parents: 19795
diff changeset
306 call add(s:errors, 'Found errors in ' . g:testfunc . ':')
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
307 call extend(s:errors, v:errors)
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
308 endif
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
309 let v:errors = []
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
310 endif
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
311 endfunc
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
312
14053
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
313 func EarlyExit(test)
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
314 " It's OK for the test we use to test the quit detection.
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
315 if a:test != 'Test_zz_quit_detected()'
26835
d9ced5bca6d6 patch 8.2.3946: when an internal error makes Vim exit the error is not seen
Bram Moolenaar <Bram@vim.org>
parents: 26500
diff changeset
316 call add(v:errors, v:errmsg)
14053
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
317 call add(v:errors, 'Test caused Vim to exit: ' . a:test)
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
318 endif
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
319
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
320 call FinishTesting()
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
321 endfunc
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
322
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
323 " This function can be called by a test if it wants to abort testing.
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
324 func FinishTesting()
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
325 call AfterTheTest('')
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
326
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
327 " Don't write viminfo on exit.
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
328 set viminfo=
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
329
11850
eb339f9af281 patch 8.0.0805: GUI test fails with gnome2
Christian Brabandt <cb@256bit.org>
parents: 11205
diff changeset
330 " Clean up files created by setup.vim
eb339f9af281 patch 8.0.0805: GUI test fails with gnome2
Christian Brabandt <cb@256bit.org>
parents: 11205
diff changeset
331 call delete('XfakeHOME', 'rf')
eb339f9af281 patch 8.0.0805: GUI test fails with gnome2
Christian Brabandt <cb@256bit.org>
parents: 11205
diff changeset
332
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
333 if s:fail == 0 && s:fail_expected == 0
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
334 " Success, create the .res file so that make knows it's done.
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
335 exe 'split ' . fnamemodify(g:testname, ':r') . '.res'
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
336 write
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
337 endif
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
338
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
339 if len(s:errors) > 0
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
340 " Append errors to test.log
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
341 split test.log
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
342 call append(line('$'), '')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
343 call append(line('$'), 'From ' . g:testname . ':')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
344 call append(line('$'), s:errors)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
345 write
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
346 endif
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
347
13634
2678e38e1de6 patch 8.0.1689: no tests for xxd
Christian Brabandt <cb@256bit.org>
parents: 13533
diff changeset
348 if s:done == 0
18189
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active
Bram Moolenaar <Bram@vim.org>
parents: 18184
diff changeset
349 if s:filtered > 0
25549
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
350 if $TEST_FILTER != ''
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
351 let message = "NO tests match $TEST_FILTER: '" .. $TEST_FILTER .. "'"
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
352 else
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
353 let message = "ALL tests match $TEST_SKIP_PAT: '" .. $TEST_SKIP_PAT .. "'"
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
354 endif
18189
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active
Bram Moolenaar <Bram@vim.org>
parents: 18184
diff changeset
355 else
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active
Bram Moolenaar <Bram@vim.org>
parents: 18184
diff changeset
356 let message = 'NO tests executed'
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active
Bram Moolenaar <Bram@vim.org>
parents: 18184
diff changeset
357 endif
13634
2678e38e1de6 patch 8.0.1689: no tests for xxd
Christian Brabandt <cb@256bit.org>
parents: 13533
diff changeset
358 else
18189
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active
Bram Moolenaar <Bram@vim.org>
parents: 18184
diff changeset
359 if s:filtered > 0
25549
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
360 call add(s:messages, "Filtered " .. s:filtered .. " tests with $TEST_FILTER and $TEST_SKIP_PAT")
18189
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active
Bram Moolenaar <Bram@vim.org>
parents: 18184
diff changeset
361 endif
13634
2678e38e1de6 patch 8.0.1689: no tests for xxd
Christian Brabandt <cb@256bit.org>
parents: 13533
diff changeset
362 let message = 'Executed ' . s:done . (s:done > 1 ? ' tests' : ' test')
2678e38e1de6 patch 8.0.1689: no tests for xxd
Christian Brabandt <cb@256bit.org>
parents: 13533
diff changeset
363 endif
18189
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active
Bram Moolenaar <Bram@vim.org>
parents: 18184
diff changeset
364 if s:done > 0 && has('reltime')
21530
66f12838071d patch 8.2.1315: MS-Windows: test log contains escape sequences
Bram Moolenaar <Bram@vim.org>
parents: 21510
diff changeset
365 let message = s:t_bold .. message .. repeat(' ', 40 - len(message))
16967
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
366 let message ..= ' in ' .. reltimestr(reltime(s:start_time)) .. ' seconds'
21530
66f12838071d patch 8.2.1315: MS-Windows: test log contains escape sequences
Bram Moolenaar <Bram@vim.org>
parents: 21510
diff changeset
367 let message ..= s:t_normal
16967
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
368 endif
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
369 echo message
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
370 call add(s:messages, message)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
371 if s:fail > 0
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
372 let message = s:fail . ' FAILED:'
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
373 echo message
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
374 call add(s:messages, message)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
375 call extend(s:messages, s:errors)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
376 endif
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
377 if s:fail_expected > 0
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
378 let message = s:fail_expected . ' FAILED (matching $TEST_MAY_FAIL):'
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
379 echo message
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
380 call add(s:messages, message)
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
381 call extend(s:messages, s:errors_expected)
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
382 endif
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
383
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
384 " Add SKIPPED messages
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
385 call extend(s:messages, s:skipped)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
386
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
387 " Append messages to the file "messages"
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
388 split messages
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
389 call append(line('$'), '')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
390 call append(line('$'), 'From ' . g:testname . ':')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
391 call append(line('$'), s:messages)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
392 write
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
393
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
394 qall!
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
395 endfunc
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
396
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
397 " Source the test script. First grab the file name, in case the script
8029
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
398 " navigates away. g:testname can be used by the tests.
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
399 let g:testname = expand('%')
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
400 let s:done = 0
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
401 let s:fail = 0
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
402 let s:fail_expected = 0
8029
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
403 let s:errors = []
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
404 let s:errors_expected = []
8029
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
405 let s:messages = []
10100
26441931dad7 commit https://github.com/vim/vim/commit/dac1947bb366ef43cd6da95acc730554e76d8b84
Christian Brabandt <cb@256bit.org>
parents: 9949
diff changeset
406 let s:skipped = []
10942
e05695e59f6d patch 8.0.0360: sometimes VimL is used instead of "Vim script"
Christian Brabandt <cb@256bit.org>
parents: 10811
diff changeset
407 if expand('%') =~ 'test_vimscript.vim'
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
408 " this test has intentional errors, don't use try/catch.
7352
ddab7ae8796d commit https://github.com/vim/vim/commit/4686b323e4bc0f466500b018959f6c8965f010f9
Christian Brabandt <cb@256bit.org>
parents: 7295
diff changeset
409 source %
7454
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1
Christian Brabandt <cb@256bit.org>
parents: 7391
diff changeset
410 else
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1
Christian Brabandt <cb@256bit.org>
parents: 7391
diff changeset
411 try
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1
Christian Brabandt <cb@256bit.org>
parents: 7391
diff changeset
412 source %
16951
ebdf6cd89910 patch 8.1.1476: no statistics displayed after running tests
Bram Moolenaar <Bram@vim.org>
parents: 15928
diff changeset
413 catch /^\cskipped/
ebdf6cd89910 patch 8.1.1476: no statistics displayed after running tests
Bram Moolenaar <Bram@vim.org>
parents: 15928
diff changeset
414 call add(s:messages, ' Skipped')
ebdf6cd89910 patch 8.1.1476: no statistics displayed after running tests
Bram Moolenaar <Bram@vim.org>
parents: 15928
diff changeset
415 call add(s:skipped, 'SKIPPED ' . expand('%') . ': ' . substitute(v:exception, '^\S*\s\+', '', ''))
7454
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1
Christian Brabandt <cb@256bit.org>
parents: 7391
diff changeset
416 catch
8029
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
417 let s:fail += 1
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
418 call add(s:errors, 'Caught exception: ' . v:exception . ' @ ' . v:throwpoint)
7454
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1
Christian Brabandt <cb@256bit.org>
parents: 7391
diff changeset
419 endtry
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1
Christian Brabandt <cb@256bit.org>
parents: 7391
diff changeset
420 endif
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
421
28148
fecc185fcf69 patch 8.2.4598: profile completion test sometimes fails
Bram Moolenaar <Bram@vim.org>
parents: 26835
diff changeset
422 " Delete the .res file, it may change behavior for completion
fecc185fcf69 patch 8.2.4598: profile completion test sometimes fails
Bram Moolenaar <Bram@vim.org>
parents: 26835
diff changeset
423 call delete(fnamemodify(g:testname, ':r') .. '.res')
fecc185fcf69 patch 8.2.4598: profile completion test sometimes fails
Bram Moolenaar <Bram@vim.org>
parents: 26835
diff changeset
424
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
425 " Locate Test_ functions and execute them.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
426 redir @q
8116
3c1bdf20c8b9 commit https://github.com/vim/vim/commit/93bf558caef2d507ef6baf56eaf6025b63da1e34
Christian Brabandt <cb@256bit.org>
parents: 8029
diff changeset
427 silent function /^Test_
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
428 redir END
19579
aae19dd172c0 patch 8.2.0346: Vim9: finding common list type not tested
Bram Moolenaar <Bram@vim.org>
parents: 18763
diff changeset
429 let s:tests = split(substitute(@q, '\(function\|def\) \(\k*()\)', '\2', 'g'))
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
430
8142
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
431 " If there is an extra argument filter the function names against it.
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
432 if argc() > 1
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
433 let s:tests = filter(s:tests, 'v:val =~ argv(1)')
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
434 endif
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
435
18184
ef35a3a70c24 patch 8.1.2087: cannot easily select one test function to execute
Bram Moolenaar <Bram@vim.org>
parents: 18158
diff changeset
436 " If the environment variable $TEST_FILTER is set then filter the function
ef35a3a70c24 patch 8.1.2087: cannot easily select one test function to execute
Bram Moolenaar <Bram@vim.org>
parents: 18158
diff changeset
437 " names against it.
18189
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active
Bram Moolenaar <Bram@vim.org>
parents: 18184
diff changeset
438 let s:filtered = 0
18184
ef35a3a70c24 patch 8.1.2087: cannot easily select one test function to execute
Bram Moolenaar <Bram@vim.org>
parents: 18158
diff changeset
439 if $TEST_FILTER != ''
18189
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active
Bram Moolenaar <Bram@vim.org>
parents: 18184
diff changeset
440 let s:filtered = len(s:tests)
18184
ef35a3a70c24 patch 8.1.2087: cannot easily select one test function to execute
Bram Moolenaar <Bram@vim.org>
parents: 18158
diff changeset
441 let s:tests = filter(s:tests, 'v:val =~ $TEST_FILTER')
18189
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active
Bram Moolenaar <Bram@vim.org>
parents: 18184
diff changeset
442 let s:filtered -= len(s:tests)
18184
ef35a3a70c24 patch 8.1.2087: cannot easily select one test function to execute
Bram Moolenaar <Bram@vim.org>
parents: 18158
diff changeset
443 endif
ef35a3a70c24 patch 8.1.2087: cannot easily select one test function to execute
Bram Moolenaar <Bram@vim.org>
parents: 18158
diff changeset
444
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
445 let s:may_fail_list = []
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
446 if $TEST_MAY_FAIL != ''
20003
e373843e2980 patch 8.2.0557: no IPv6 support for channels
Bram Moolenaar <Bram@vim.org>
parents: 19795
diff changeset
447 " Split the list at commas and add () to make it match g:testfunc.
19734
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
448 let s:may_fail_list = split($TEST_MAY_FAIL, ',')->map({i, v -> v .. '()'})
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
449 endif
2bd03926397a patch 8.2.0423: in some environments a few tests are expected to fail
Bram Moolenaar <Bram@vim.org>
parents: 19715
diff changeset
450
7540
b910bb01832a commit https://github.com/vim/vim/commit/cfc0a350a9fa04f1b0cfa1ba31fbd2847376513f
Christian Brabandt <cb@256bit.org>
parents: 7454
diff changeset
451 " Execute the tests in alphabetical order.
20003
e373843e2980 patch 8.2.0557: no IPv6 support for channels
Bram Moolenaar <Bram@vim.org>
parents: 19795
diff changeset
452 for g:testfunc in sort(s:tests)
25549
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
453 if $TEST_SKIP_PAT != '' && g:testfunc =~ $TEST_SKIP_PAT
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
454 call add(s:messages, g:testfunc .. ' matches $TEST_SKIP_PAT')
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
455 let s:filtered += 1
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
456 continue
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
457 endif
dc25589bfec0 patch 8.2.3311: Vim9: check for DO_NOT_FREE_CNT is very slow
Bram Moolenaar <Bram@vim.org>
parents: 24194
diff changeset
458
12616
4767939d10cc patch 8.0.1186: still quite a few old style tests
Christian Brabandt <cb@256bit.org>
parents: 12381
diff changeset
459 " Silence, please!
4767939d10cc patch 8.0.1186: still quite a few old style tests
Christian Brabandt <cb@256bit.org>
parents: 12381
diff changeset
460 set belloff=all
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
461 let prev_error = ''
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
462 let total_errors = []
19795
1c7f92a1100e patch 8.2.0454: some tests fail when the system is slow
Bram Moolenaar <Bram@vim.org>
parents: 19734
diff changeset
463 let g:run_nr = 1
12616
4767939d10cc patch 8.0.1186: still quite a few old style tests
Christian Brabandt <cb@256bit.org>
parents: 12381
diff changeset
464
20003
e373843e2980 patch 8.2.0557: no IPv6 support for channels
Bram Moolenaar <Bram@vim.org>
parents: 19795
diff changeset
465 " A test can set g:test_is_flaky to retry running the test.
e373843e2980 patch 8.2.0557: no IPv6 support for channels
Bram Moolenaar <Bram@vim.org>
parents: 19795
diff changeset
466 let g:test_is_flaky = 0
19687
c3f506e24eab patch 8.2.0400: not all tests using a terminal are in the list of flaky tests
Bram Moolenaar <Bram@vim.org>
parents: 19579
diff changeset
467
30069
cb21745cbbab patch 9.0.0372: MS-Windows: "%T" time format does not appear to work
Bram Moolenaar <Bram@vim.org>
parents: 30063
diff changeset
468 let starttime = strftime("%H:%M:%S")
20003
e373843e2980 patch 8.2.0557: no IPv6 support for channels
Bram Moolenaar <Bram@vim.org>
parents: 19795
diff changeset
469 call RunTheTest(g:testfunc)
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
470
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
471 " Repeat a flaky test. Give up when:
21508
6b2a5418cbc3 patch 8.2.1304: debug backtrace isn't tested much
Bram Moolenaar <Bram@vim.org>
parents: 21433
diff changeset
472 " - $TEST_NO_RETRY is not empty
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
473 " - it fails again with the same message
17918
9606c0adc148 patch 8.1.1955: tests contain typos
Bram Moolenaar <Bram@vim.org>
parents: 17910
diff changeset
474 " - it fails five times (with a different message)
15040
da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message
Bram Moolenaar <Bram@vim.org>
parents: 15038
diff changeset
475 if len(v:errors) > 0
21510
31cb78014fe4 patch 8.2.1305: some tests are still old style
Bram Moolenaar <Bram@vim.org>
parents: 21508
diff changeset
476 \ && $TEST_NO_RETRY == ''
25969
a5a772dace5b patch 8.2.3518: Test_xrestore sometimes fails
Bram Moolenaar <Bram@vim.org>
parents: 25549
diff changeset
477 \ && g:test_is_flaky
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
478 while 1
30063
72c8fbb9e242 patch 9.0.0369: a failing flaky test doesn't mention the time
Bram Moolenaar <Bram@vim.org>
parents: 29587
diff changeset
479 call add(s:messages, 'Found errors in ' .. g:testfunc .. ':')
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
480 call extend(s:messages, v:errors)
11187
515db00c4676 patch 8.0.0480: the remote_peek() test fails on MS-Windows
Christian Brabandt <cb@256bit.org>
parents: 11185
diff changeset
481
30180
b01eaf122e5a patch 9.0.0426: failed flaky tests reports only start time
Bram Moolenaar <Bram@vim.org>
parents: 30069
diff changeset
482 let endtime = strftime("%H:%M:%S")
b01eaf122e5a patch 9.0.0426: failed flaky tests reports only start time
Bram Moolenaar <Bram@vim.org>
parents: 30069
diff changeset
483 call add(total_errors, $'Run {g:run_nr}, {starttime} - {endtime}:')
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
484 call extend(total_errors, v:errors)
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
485
29387
9dce192d1ac2 patch 9.0.0036: 'fillchars' cannot have window-local values
Bram Moolenaar <Bram@vim.org>
parents: 29340
diff changeset
486 if g:run_nr >= 5 || prev_error == v:errors[0]
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
487 call add(total_errors, 'Flaky test failed too often, giving up')
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
488 let v:errors = total_errors
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
489 break
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
490 endif
12905
30b0d784c870 patch 8.0.1329: when a flaky test fails it also often fails the second time
Christian Brabandt <cb@256bit.org>
parents: 12851
diff changeset
491
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
492 call add(s:messages, 'Flaky test failed, running it again')
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
493
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
494 " Flakiness is often caused by the system being very busy. Sleep a
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
495 " couple of seconds to have a higher chance of succeeding the second
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
496 " time.
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
497 sleep 2
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
498
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
499 let prev_error = v:errors[0]
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
500 let v:errors = []
19795
1c7f92a1100e patch 8.2.0454: some tests fail when the system is slow
Bram Moolenaar <Bram@vim.org>
parents: 19734
diff changeset
501 let g:run_nr += 1
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
502
30069
cb21745cbbab patch 9.0.0372: MS-Windows: "%T" time format does not appear to work
Bram Moolenaar <Bram@vim.org>
parents: 30063
diff changeset
503 let starttime = strftime("%H:%M:%S")
20003
e373843e2980 patch 8.2.0557: no IPv6 support for channels
Bram Moolenaar <Bram@vim.org>
parents: 19795
diff changeset
504 call RunTheTest(g:testfunc)
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
505
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
506 if len(v:errors) == 0
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
507 " Test passed on rerun.
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
508 break
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
509 endif
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
510 endwhile
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
511 endif
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
512
20003
e373843e2980 patch 8.2.0557: no IPv6 support for channels
Bram Moolenaar <Bram@vim.org>
parents: 19795
diff changeset
513 call AfterTheTest(g:testfunc)
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
514 endfor
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
515
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
516 call FinishTesting()
10388
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
517
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
518 " vim: shiftwidth=2 sts=2 expandtab