Mercurial > vim

annotate src/testdir/runtest.vim @ 17127:d244a9be99db v8.1.1563

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
patch 8.1.1563: crash when using closures commit https://github.com/vim/vim/commit/6e5000d493b4f385f901eb97f3ce0c8088373403 Author: Bram Moolenaar <Bram@vim.org> Date: Mon Jun 17 21:18:41 2019 +0200 patch 8.1.1563: crash when using closures Problem: Crash when using closures. Solution: Set reference in varlist of funccal when running the garbage collector. (Ozaki Kiichi, closes #4554, closes #4547)
author Bram Moolenaar <Bram@vim.org>
date Mon, 17 Jun 2019 21:30:08 +0200
parents efc6f5e3b543
children 4fe44a3b732f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
1 " This script is sourced while editing the .vim file with the tests.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
2 " When the script is successful the .res file will be created.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
3 " Errors are appended to the test.log file.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
4 "
8142
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
5 " To execute only specific test functions, add a second argument. It will be
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
6 " matched against the names of the Test_ funtion. E.g.:
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
7 " ../vim -u NONE -S runtest.vim test_channel.vim open_delay
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
8 " The output can be found in the "messages" file.
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
9 "
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
10 " The test script may contain anything, only functions that start with
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
11 " "Test_" are special. These will be invoked and should contain assert
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
12 " functions. See test_assert.vim for an example.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
13 "
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
14 " It is possible to source other files that contain "Test_" functions. This
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
15 " can speed up testing, since Vim does not need to restart. But be careful
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
16 " that the tests do not interfere with each other.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
17 "
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
18 " If an error cannot be detected properly with an assert function add the
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
19 " error to the v:errors list:
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
20 " call add(v:errors, 'test foo failed: Cannot find xyz')
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
21 "
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
22 " If preparation for each Test_ function is needed, define a SetUp function.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
23 " It will be called before each Test_ function.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
24 "
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
25 " If cleanup after each Test_ function is needed, define a TearDown function.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
26 " It will be called after each Test_ function.
8029
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
27 "
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
28 " When debugging a test it can be useful to add messages to v:errors:
15377
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset
Bram Moolenaar <Bram@vim.org>
parents: 15180
diff changeset
29 " call add(v:errors, "this happened")
8029
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
30
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
31
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
32 " Without the +eval feature we can't run these tests, bail out.
7352
ddab7ae8796d commit https://github.com/vim/vim/commit/4686b323e4bc0f466500b018959f6c8965f010f9
Christian Brabandt <cb@256bit.org>
parents: 7295
diff changeset
33 so small.vim
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
34
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
35 " Check that the screen size is at least 24 x 80 characters.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
36 if &lines < 24 || &columns < 80
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
37 let error = 'Screen size too small! Tests require at least 24 lines with 80 characters'
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
38 echoerr error
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
39 split test.log
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
40 $put =error
17091
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that
Bram Moolenaar <Bram@vim.org>
parents: 17032
diff changeset
41 write
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that
Bram Moolenaar <Bram@vim.org>
parents: 17032
diff changeset
42 split messages
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that
Bram Moolenaar <Bram@vim.org>
parents: 17032
diff changeset
43 call append(line('$'), error)
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that
Bram Moolenaar <Bram@vim.org>
parents: 17032
diff changeset
44 write
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that
Bram Moolenaar <Bram@vim.org>
parents: 17032
diff changeset
45 qa!
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
46 endif
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
47
16967
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
48 if has('reltime')
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
49 let s:start_time = reltime()
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
50 endif
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
51
9501
f0201deac2a2 commit https://github.com/vim/vim/commit/89b10421ca1aea55acbafac1df93be4e20210ceb
Christian Brabandt <cb@256bit.org>
parents: 9452
diff changeset
52 " Common with all tests on all systems.
f0201deac2a2 commit https://github.com/vim/vim/commit/89b10421ca1aea55acbafac1df93be4e20210ceb
Christian Brabandt <cb@256bit.org>
parents: 9452
diff changeset
53 source setup.vim
f0201deac2a2 commit https://github.com/vim/vim/commit/89b10421ca1aea55acbafac1df93be4e20210ceb
Christian Brabandt <cb@256bit.org>
parents: 9452
diff changeset
54
7391
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
55 " For consistency run all tests with 'nocompatible' set.
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
56 " This also enables use of line continuation.
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
57 set nocp viminfo+=nviminfo
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
58
15607
2dcaa860e3fc patch 8.1.0811: too many #ifdefs
Bram Moolenaar <Bram@vim.org>
parents: 15377
diff changeset
59 " Use utf-8 by default, instead of whatever the system default happens to be.
15928
857ce36c8412 patch 8.1.0970: text properties test fails when 'encoding' is not utf-8
Bram Moolenaar <Bram@vim.org>
parents: 15920
diff changeset
60 " Individual tests can overrule this at the top of the file and use
857ce36c8412 patch 8.1.0970: text properties test fails when 'encoding' is not utf-8
Bram Moolenaar <Bram@vim.org>
parents: 15920
diff changeset
61 " g:orig_encoding if needed.
857ce36c8412 patch 8.1.0970: text properties test fails when 'encoding' is not utf-8
Bram Moolenaar <Bram@vim.org>
parents: 15920
diff changeset
62 let g:orig_encoding = &encoding
15607
2dcaa860e3fc patch 8.1.0811: too many #ifdefs
Bram Moolenaar <Bram@vim.org>
parents: 15377
diff changeset
63 set encoding=utf-8
9618
81ba6e4eb72b commit https://github.com/vim/vim/commit/ac105ed3c420660ddbddc501c97875c48220817e
Christian Brabandt <cb@256bit.org>
parents: 9501
diff changeset
64
14889
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
65 " REDIR_TEST_TO_NULL has a very permissive SwapExists autocommand which is for
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
66 " the test_name.vim file itself. Replace it here with a more restrictive one,
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
67 " so we still catch mistakes.
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
68 let s:test_script_fname = expand('%')
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
69 au! SwapExists * call HandleSwapExists()
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
70 func HandleSwapExists()
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
71 " Only ignore finding a swap file for the test script (the user might be
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
72 " editing it and do ":make test_name") and the output file.
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
73 if expand('<afile>') == 'messages' || expand('<afile>') =~ s:test_script_fname
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
74 let v:swapchoice = 'e'
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
75 endif
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
76 endfunc
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
77
10811
4cb74eab1733 patch 8.0.0295: test_viml hangs
Christian Brabandt <cb@256bit.org>
parents: 10807
diff changeset
78 " Avoid stopping at the "hit enter" prompt
4cb74eab1733 patch 8.0.0295: test_viml hangs
Christian Brabandt <cb@256bit.org>
parents: 10807
diff changeset
79 set nomore
4cb74eab1733 patch 8.0.0295: test_viml hangs
Christian Brabandt <cb@256bit.org>
parents: 10807
diff changeset
80
7391
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
81 " Output all messages in English.
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
82 lang mess C
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17
Christian Brabandt <cb@256bit.org>
parents: 7372
diff changeset
83
7623
2720952e9acb commit https://github.com/vim/vim/commit/f60b796fa9870bdfc4cdeb91653bac041916077d
Christian Brabandt <cb@256bit.org>
parents: 7595
diff changeset
84 " Always use forward slashes.
2720952e9acb commit https://github.com/vim/vim/commit/f60b796fa9870bdfc4cdeb91653bac041916077d
Christian Brabandt <cb@256bit.org>
parents: 7595
diff changeset
85 set shellslash
2720952e9acb commit https://github.com/vim/vim/commit/f60b796fa9870bdfc4cdeb91653bac041916077d
Christian Brabandt <cb@256bit.org>
parents: 7595
diff changeset
86
7545
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
87 let s:srcdir = expand('%:p:h:h')
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
88
9121
7350959e53c3 commit https://github.com/vim/vim/commit/8e8df251bf2505e5decf258397c6069fbe5e2e01
Christian Brabandt <cb@256bit.org>
parents: 8881
diff changeset
89 " Prepare for calling test_garbagecollect_now().
8881
ed0b39dd7fd6 commit https://github.com/vim/vim/commit/ebf7dfa6f121c82f97d2adca3d45fbaba9ad8f7e
Christian Brabandt <cb@256bit.org>
parents: 8796
diff changeset
90 let v:testing = 1
ed0b39dd7fd6 commit https://github.com/vim/vim/commit/ebf7dfa6f121c82f97d2adca3d45fbaba9ad8f7e
Christian Brabandt <cb@256bit.org>
parents: 8796
diff changeset
91
7545
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
92 " Support function: get the alloc ID by name.
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
93 function GetAllocId(name)
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
94 exe 'split ' . s:srcdir . '/alloc.h'
7595
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0
Christian Brabandt <cb@256bit.org>
parents: 7545
diff changeset
95 let top = search('typedef enum')
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0
Christian Brabandt <cb@256bit.org>
parents: 7545
diff changeset
96 if top == 0
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0
Christian Brabandt <cb@256bit.org>
parents: 7545
diff changeset
97 call add(v:errors, 'typedef not found in alloc.h')
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0
Christian Brabandt <cb@256bit.org>
parents: 7545
diff changeset
98 endif
7545
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
99 let lnum = search('aid_' . a:name . ',')
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
100 if lnum == 0
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
101 call add(v:errors, 'Alloc ID ' . a:name . ' not defined')
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
102 endif
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
103 close
7595
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0
Christian Brabandt <cb@256bit.org>
parents: 7545
diff changeset
104 return lnum - top - 1
7545
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
105 endfunc
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
106
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
107 func RunTheTest(test)
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
108 echo 'Executing ' . a:test
16967
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
109 if has('reltime')
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
110 let func_start = reltime()
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
111 endif
10807
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
112
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
113 " Avoid stopping at the "hit enter" prompt
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
114 set nomore
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
115
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
116 " Avoid a three second wait when a message is about to be overwritten by the
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
117 " mode message.
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
118 set noshowmode
82c43bca568e patch 8.0.0293: some tests have a one or three second wait
Christian Brabandt <cb@256bit.org>
parents: 10797
diff changeset
119
11105
7c7e496e625d patch 8.0.0440: not enough test coverage in Insert mode
Christian Brabandt <cb@256bit.org>
parents: 11056
diff changeset
120 " Clear any overrides.
7c7e496e625d patch 8.0.0440: not enough test coverage in Insert mode
Christian Brabandt <cb@256bit.org>
parents: 11056
diff changeset
121 call test_override('ALL', 0)
7c7e496e625d patch 8.0.0440: not enough test coverage in Insert mode
Christian Brabandt <cb@256bit.org>
parents: 11056
diff changeset
122
12692
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
123 " Some tests wipe out buffers. To be consistent, always wipe out all
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
124 " buffers.
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
125 %bwipe!
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
126
12851
90aaa974594e patch 8.0.1302: still too many old style tests
Christian Brabandt <cb@256bit.org>
parents: 12769
diff changeset
127 " The test may change the current directory. Save and restore the
90aaa974594e patch 8.0.1302: still too many old style tests
Christian Brabandt <cb@256bit.org>
parents: 12769
diff changeset
128 " directory after executing the test.
90aaa974594e patch 8.0.1302: still too many old style tests
Christian Brabandt <cb@256bit.org>
parents: 12769
diff changeset
129 let save_cwd = getcwd()
90aaa974594e patch 8.0.1302: still too many old style tests
Christian Brabandt <cb@256bit.org>
parents: 12769
diff changeset
130
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
131 if exists("*SetUp")
10388
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
132 try
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
133 call SetUp()
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
134 catch
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
135 call add(v:errors, 'Caught exception in SetUp() before ' . a:test . ': ' . v:exception . ' @ ' . v:throwpoint)
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
136 endtry
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
137 endif
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
138
12682
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
139 if a:test =~ 'Test_nocatch_'
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
140 " Function handles errors itself. This avoids skipping commands after the
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
141 " error.
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
142 exe 'call ' . a:test
12682
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
143 else
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
144 try
14053
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
145 let s:test = a:test
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
146 au VimLeavePre * call EarlyExit(s:test)
12682
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
147 exe 'call ' . a:test
14053
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
148 au! VimLeavePre
12682
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
149 catch /^\cskipped/
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
150 call add(s:messages, ' Skipped')
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
151 call add(s:skipped, 'SKIPPED ' . a:test . ': ' . substitute(v:exception, '^\S*\s\+', '', ''))
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
152 catch
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
153 call add(v:errors, 'Caught exception in ' . a:test . ': ' . v:exception . ' @ ' . v:throwpoint)
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
154 endtry
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
155 endif
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
156
15377
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset
Bram Moolenaar <Bram@vim.org>
parents: 15180
diff changeset
157 " In case 'insertmode' was set and something went wrong, make sure it is
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset
Bram Moolenaar <Bram@vim.org>
parents: 15180
diff changeset
158 " reset to avoid trouble with anything else.
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset
Bram Moolenaar <Bram@vim.org>
parents: 15180
diff changeset
159 set noinsertmode
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset
Bram Moolenaar <Bram@vim.org>
parents: 15180
diff changeset
160
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
161 if exists("*TearDown")
10388
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
162 try
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
163 call TearDown()
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
164 catch
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
165 call add(v:errors, 'Caught exception in TearDown() after ' . a:test . ': ' . v:exception . ' @ ' . v:throwpoint)
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
166 endtry
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
167 endif
9748
5440f3442841 commit https://github.com/vim/vim/commit/7cba71d7e3576639679b6a3aedeeb1ac07f7f2f5
Christian Brabandt <cb@256bit.org>
parents: 9632
diff changeset
168
12692
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
169 " Clear any autocommands
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
170 au!
14889
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited
Bram Moolenaar <Bram@vim.org>
parents: 14571
diff changeset
171 au SwapExists * call HandleSwapExists()
12692
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
172
17123
efc6f5e3b543 patch 8.1.1561: popup_setoptions() is not implemented yet
Bram Moolenaar <Bram@vim.org>
parents: 17091
diff changeset
173 " Close any stray popup windows
efc6f5e3b543 patch 8.1.1561: popup_setoptions() is not implemented yet
Bram Moolenaar <Bram@vim.org>
parents: 17091
diff changeset
174 if has('textprop')
efc6f5e3b543 patch 8.1.1561: popup_setoptions() is not implemented yet
Bram Moolenaar <Bram@vim.org>
parents: 17091
diff changeset
175 call popup_clear()
efc6f5e3b543 patch 8.1.1561: popup_setoptions() is not implemented yet
Bram Moolenaar <Bram@vim.org>
parents: 17091
diff changeset
176 endif
efc6f5e3b543 patch 8.1.1561: popup_setoptions() is not implemented yet
Bram Moolenaar <Bram@vim.org>
parents: 17091
diff changeset
177
12688
a6a935b3270e patch 8.0.1222: test functions interfere with each other
Christian Brabandt <cb@256bit.org>
parents: 12682
diff changeset
178 " Close any extra tab pages and windows and make the current one not modified.
a6a935b3270e patch 8.0.1222: test functions interfere with each other
Christian Brabandt <cb@256bit.org>
parents: 12682
diff changeset
179 while tabpagenr('$') > 1
12692
03a6aeea2096 patch 8.0.1224: still interference between test functions
Christian Brabandt <cb@256bit.org>
parents: 12688
diff changeset
180 quit!
12688
a6a935b3270e patch 8.0.1222: test functions interfere with each other
Christian Brabandt <cb@256bit.org>
parents: 12682
diff changeset
181 endwhile
a6a935b3270e patch 8.0.1222: test functions interfere with each other
Christian Brabandt <cb@256bit.org>
parents: 12682
diff changeset
182
9949
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
183 while 1
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
184 let wincount = winnr('$')
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
185 if wincount == 1
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
186 break
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
187 endif
9748
5440f3442841 commit https://github.com/vim/vim/commit/7cba71d7e3576639679b6a3aedeeb1ac07f7f2f5
Christian Brabandt <cb@256bit.org>
parents: 9632
diff changeset
188 bwipe!
9949
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
189 if wincount == winnr('$')
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
190 " Did not manage to close a window.
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
191 only!
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
192 break
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe
Christian Brabandt <cb@256bit.org>
parents: 9758
diff changeset
193 endif
9748
5440f3442841 commit https://github.com/vim/vim/commit/7cba71d7e3576639679b6a3aedeeb1ac07f7f2f5
Christian Brabandt <cb@256bit.org>
parents: 9632
diff changeset
194 endwhile
12851
90aaa974594e patch 8.0.1302: still too many old style tests
Christian Brabandt <cb@256bit.org>
parents: 12769
diff changeset
195
90aaa974594e patch 8.0.1302: still too many old style tests
Christian Brabandt <cb@256bit.org>
parents: 12769
diff changeset
196 exe 'cd ' . save_cwd
17032
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
197
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
198 let message = 'Executed ' . a:test
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
199 if has('reltime')
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
200 let message ..= ' in ' .. reltimestr(reltime(func_start)) .. ' seconds'
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
201 endif
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
202 call add(s:messages, message)
84f308b126de patch 8.1.1516: time reported for a test measured wrong
Bram Moolenaar <Bram@vim.org>
parents: 16967
diff changeset
203 let s:done += 1
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
204 endfunc
7545
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15
Christian Brabandt <cb@256bit.org>
parents: 7540
diff changeset
205
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
206 func AfterTheTest()
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
207 if len(v:errors) > 0
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
208 let s:fail += 1
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
209 call add(s:errors, 'Found errors in ' . s:test . ':')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
210 call extend(s:errors, v:errors)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
211 let v:errors = []
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
212 endif
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
213 endfunc
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
214
14053
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
215 func EarlyExit(test)
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
216 " It's OK for the test we use to test the quit detection.
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
217 if a:test != 'Test_zz_quit_detected()'
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
218 call add(v:errors, 'Test caused Vim to exit: ' . a:test)
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
219 endif
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
220
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
221 call FinishTesting()
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
222 endfunc
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed
Christian Brabandt <cb@256bit.org>
parents: 13634
diff changeset
223
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
224 " This function can be called by a test if it wants to abort testing.
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
225 func FinishTesting()
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
226 call AfterTheTest()
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
227
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
228 " Don't write viminfo on exit.
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
229 set viminfo=
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
230
11850
eb339f9af281 patch 8.0.0805: GUI test fails with gnome2
Christian Brabandt <cb@256bit.org>
parents: 11205
diff changeset
231 " Clean up files created by setup.vim
eb339f9af281 patch 8.0.0805: GUI test fails with gnome2
Christian Brabandt <cb@256bit.org>
parents: 11205
diff changeset
232 call delete('XfakeHOME', 'rf')
eb339f9af281 patch 8.0.0805: GUI test fails with gnome2
Christian Brabandt <cb@256bit.org>
parents: 11205
diff changeset
233
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
234 if s:fail == 0
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
235 " Success, create the .res file so that make knows it's done.
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
236 exe 'split ' . fnamemodify(g:testname, ':r') . '.res'
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
237 write
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
238 endif
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
239
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
240 if len(s:errors) > 0
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
241 " Append errors to test.log
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
242 split test.log
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
243 call append(line('$'), '')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
244 call append(line('$'), 'From ' . g:testname . ':')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
245 call append(line('$'), s:errors)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
246 write
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
247 endif
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
248
13634
2678e38e1de6 patch 8.0.1689: no tests for xxd
Christian Brabandt <cb@256bit.org>
parents: 13533
diff changeset
249 if s:done == 0
2678e38e1de6 patch 8.0.1689: no tests for xxd
Christian Brabandt <cb@256bit.org>
parents: 13533
diff changeset
250 let message = 'NO tests executed'
2678e38e1de6 patch 8.0.1689: no tests for xxd
Christian Brabandt <cb@256bit.org>
parents: 13533
diff changeset
251 else
2678e38e1de6 patch 8.0.1689: no tests for xxd
Christian Brabandt <cb@256bit.org>
parents: 13533
diff changeset
252 let message = 'Executed ' . s:done . (s:done > 1 ? ' tests' : ' test')
2678e38e1de6 patch 8.0.1689: no tests for xxd
Christian Brabandt <cb@256bit.org>
parents: 13533
diff changeset
253 endif
16967
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
254 if has('reltime')
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
255 let message ..= ' in ' .. reltimestr(reltime(s:start_time)) .. ' seconds'
586d625e21b4 patch 8.1.1484: some tests are slow
Bram Moolenaar <Bram@vim.org>
parents: 16951
diff changeset
256 endif
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
257 echo message
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
258 call add(s:messages, message)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
259 if s:fail > 0
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
260 let message = s:fail . ' FAILED:'
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
261 echo message
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
262 call add(s:messages, message)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
263 call extend(s:messages, s:errors)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
264 endif
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
265
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
266 " Add SKIPPED messages
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
267 call extend(s:messages, s:skipped)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
268
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
269 " Append messages to the file "messages"
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
270 split messages
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
271 call append(line('$'), '')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
272 call append(line('$'), 'From ' . g:testname . ':')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
273 call append(line('$'), s:messages)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
274 write
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
275
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
276 qall!
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
277 endfunc
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
278
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
279 " Source the test script. First grab the file name, in case the script
8029
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
280 " navigates away. g:testname can be used by the tests.
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
281 let g:testname = expand('%')
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
282 let s:done = 0
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
283 let s:fail = 0
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
284 let s:errors = []
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
285 let s:messages = []
10100
26441931dad7 commit https://github.com/vim/vim/commit/dac1947bb366ef43cd6da95acc730554e76d8b84
Christian Brabandt <cb@256bit.org>
parents: 9949
diff changeset
286 let s:skipped = []
10942
e05695e59f6d patch 8.0.0360: sometimes VimL is used instead of "Vim script"
Christian Brabandt <cb@256bit.org>
parents: 10811
diff changeset
287 if expand('%') =~ 'test_vimscript.vim'
8029
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
288 " this test has intentional s:errors, don't use try/catch.
7352
ddab7ae8796d commit https://github.com/vim/vim/commit/4686b323e4bc0f466500b018959f6c8965f010f9
Christian Brabandt <cb@256bit.org>
parents: 7295
diff changeset
289 source %
7454
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1
Christian Brabandt <cb@256bit.org>
parents: 7391
diff changeset
290 else
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1
Christian Brabandt <cb@256bit.org>
parents: 7391
diff changeset
291 try
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1
Christian Brabandt <cb@256bit.org>
parents: 7391
diff changeset
292 source %
16951
ebdf6cd89910 patch 8.1.1476: no statistics displayed after running tests
Bram Moolenaar <Bram@vim.org>
parents: 15928
diff changeset
293 catch /^\cskipped/
ebdf6cd89910 patch 8.1.1476: no statistics displayed after running tests
Bram Moolenaar <Bram@vim.org>
parents: 15928
diff changeset
294 call add(s:messages, ' Skipped')
ebdf6cd89910 patch 8.1.1476: no statistics displayed after running tests
Bram Moolenaar <Bram@vim.org>
parents: 15928
diff changeset
295 call add(s:skipped, 'SKIPPED ' . expand('%') . ': ' . substitute(v:exception, '^\S*\s\+', '', ''))
7454
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1
Christian Brabandt <cb@256bit.org>
parents: 7391
diff changeset
296 catch
8029
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
297 let s:fail += 1
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
298 call add(s:errors, 'Caught exception: ' . v:exception . ' @ ' . v:throwpoint)
7454
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1
Christian Brabandt <cb@256bit.org>
parents: 7391
diff changeset
299 endtry
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1
Christian Brabandt <cb@256bit.org>
parents: 7391
diff changeset
300 endif
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
301
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
302 " Names of flaky tests.
15040
da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message
Bram Moolenaar <Bram@vim.org>
parents: 15038
diff changeset
303 let s:flaky_tests = [
15038
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
304 \ 'Test_call()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
305 \ 'Test_channel_handler()',
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
306 \ 'Test_client_server()',
10789
214417c527b5 patch 8.0.0284: collapse buffers test failed once
Christian Brabandt <cb@256bit.org>
parents: 10755
diff changeset
307 \ 'Test_close_and_exit_cb()',
15038
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
308 \ 'Test_close_callback()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
309 \ 'Test_close_handle()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
310 \ 'Test_close_lambda()',
15180
52e883259d99 patch 8.1.0600: channel test is flaky
Bram Moolenaar <Bram@vim.org>
parents: 15044
diff changeset
311 \ 'Test_close_output_buffer()',
15038
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
312 \ 'Test_close_partial()',
10789
214417c527b5 patch 8.0.0284: collapse buffers test failed once
Christian Brabandt <cb@256bit.org>
parents: 10755
diff changeset
313 \ 'Test_collapse_buffers()',
214417c527b5 patch 8.0.0284: collapse buffers test failed once
Christian Brabandt <cb@256bit.org>
parents: 10755
diff changeset
314 \ 'Test_communicate()',
13533
9e9e9f10e311 patch 8.0.1640: Test_cwd() is flaky
Christian Brabandt <cb@256bit.org>
parents: 13266
diff changeset
315 \ 'Test_cwd()',
15018
563ca9655b8f patch 8.1.0520: screen diff test sometimes fails
Bram Moolenaar <Bram@vim.org>
parents: 14889
diff changeset
316 \ 'Test_diff_screen()',
15038
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
317 \ 'Test_exit_callback()',
11169
f4971739ea47 patch 8.0.0471: exit callback test sometimes fails
Christian Brabandt <cb@256bit.org>
parents: 11105
diff changeset
318 \ 'Test_exit_callback_interval()',
10189
2392e065efea commit https://github.com/vim/vim/commit/e1c8c7a6742be6072290f9aa54ae358060d9c42f
Christian Brabandt <cb@256bit.org>
parents: 10100
diff changeset
319 \ 'Test_nb_basic()',
10990
f18a98681da1 patch 8.0.0384: timer test failed for no apparent reason
Christian Brabandt <cb@256bit.org>
parents: 10942
diff changeset
320 \ 'Test_oneshot()',
15038
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
321 \ 'Test_open_delay()',
12988
da725bc3fd2c patch 8.0.1370: channel test for callback is flaky
Christian Brabandt <cb@256bit.org>
parents: 12979
diff changeset
322 \ 'Test_out_cb()',
12979
6cf6138b6484 patch 8.0.1365: when one channel test fails others fail as well
Christian Brabandt <cb@256bit.org>
parents: 12905
diff changeset
323 \ 'Test_paused()',
10229
19ced29f773a commit https://github.com/vim/vim/commit/c79d6aa01d86cd98cffddc80825c1c600e833bbf
Christian Brabandt <cb@256bit.org>
parents: 10223
diff changeset
324 \ 'Test_pipe_through_sort_all()',
10797
e1bddbed43cc patch 8.0.0288: errors reported while running tests
Christian Brabandt <cb@256bit.org>
parents: 10789
diff changeset
325 \ 'Test_pipe_through_sort_some()',
14571
44df6e5c17c3 patch 8.1.0299: misplaced comment
Christian Brabandt <cb@256bit.org>
parents: 14569
diff changeset
326 \ 'Test_popup_and_window_resize()',
11056
679c12064dbc patch 8.0.0417: test for the clipboard fails sometimes
Christian Brabandt <cb@256bit.org>
parents: 10990
diff changeset
327 \ 'Test_quoteplus()',
11205
4db196820d3b patch 8.0.0489: clipboard and "* register is not tested
Christian Brabandt <cb@256bit.org>
parents: 11187
diff changeset
328 \ 'Test_quotestar()',
15038
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
329 \ 'Test_raw_one_time_callback()',
10789
214417c527b5 patch 8.0.0284: collapse buffers test failed once
Christian Brabandt <cb@256bit.org>
parents: 10755
diff changeset
330 \ 'Test_reltime()',
13266
d73ebdbd8e35 patch 8.0.1507: timer test is a bit flaky
Christian Brabandt <cb@256bit.org>
parents: 12988
diff changeset
331 \ 'Test_repeat_three()',
15038
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
332 \ 'Test_server_crash()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
333 \ 'Test_terminal_ansicolors_default()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
334 \ 'Test_terminal_ansicolors_func()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
335 \ 'Test_terminal_ansicolors_global()',
12682
47f253caa805 patch 8.0.1219: terminal test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12616
diff changeset
336 \ 'Test_terminal_composing_unicode()',
15920
82e3ffdc1c2b patch 8.1.0966: one terminal test is flaky
Bram Moolenaar <Bram@vim.org>
parents: 15772
diff changeset
337 \ 'Test_terminal_does_not_truncate_last_newlines()',
15038
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
338 \ 'Test_terminal_env()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
339 \ 'Test_terminal_hide_buffer()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
340 \ 'Test_terminal_make_change()',
15772
649b81669197 patch 8.1.0893: terminal test is a bit flaky
Bram Moolenaar <Bram@vim.org>
parents: 15607
diff changeset
341 \ 'Test_terminal_no_cmd()',
12381
d3f8f1c33a20 patch 8.0.1070: terminal test is flaky on Mac
Christian Brabandt <cb@256bit.org>
parents: 11884
diff changeset
342 \ 'Test_terminal_noblock()',
12769
083ae2cfcc5b patch 8.0.1262: terminal redir test is flaky
Christian Brabandt <cb@256bit.org>
parents: 12692
diff changeset
343 \ 'Test_terminal_redir_file()',
15038
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
344 \ 'Test_terminal_response_to_control_sequence()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
345 \ 'Test_terminal_scrollback()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
346 \ 'Test_terminal_split_quit()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
347 \ 'Test_terminal_termwinkey()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
348 \ 'Test_terminal_termwinsize_mininmum()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
349 \ 'Test_terminal_termwinsize_option_fixed()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
350 \ 'Test_terminal_termwinsize_option_zero()',
12979
6cf6138b6484 patch 8.0.1365: when one channel test fails others fail as well
Christian Brabandt <cb@256bit.org>
parents: 12905
diff changeset
351 \ 'Test_terminal_tmap()',
15038
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
352 \ 'Test_terminal_wall()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
353 \ 'Test_terminal_wipe_buffer()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
354 \ 'Test_terminal_wqall()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
355 \ 'Test_two_channels()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
356 \ 'Test_unlet_handle()',
11884
3cb7318223a4 patch 8.0.0822: Test_with_partial_callback is a tiny bit flaky
Christian Brabandt <cb@256bit.org>
parents: 11850
diff changeset
357 \ 'Test_with_partial_callback()',
15038
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
358 \ 'Test_zero_reply()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15036
diff changeset
359 \ 'Test_zz1_terminal_in_gui()',
10189
2392e065efea commit https://github.com/vim/vim/commit/e1c8c7a6742be6072290f9aa54ae358060d9c42f
Christian Brabandt <cb@256bit.org>
parents: 10100
diff changeset
360 \ ]
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
361
15040
da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message
Bram Moolenaar <Bram@vim.org>
parents: 15038
diff changeset
362 " Pattern indicating a common flaky test failure.
15044
f47b5dfe5c6d patch 8.1.0533: screendump tests can be flaky
Bram Moolenaar <Bram@vim.org>
parents: 15040
diff changeset
363 let s:flaky_errors_re = 'StopVimInTerminal\|VerifyScreenDump'
15040
da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message
Bram Moolenaar <Bram@vim.org>
parents: 15038
diff changeset
364
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
365 " Locate Test_ functions and execute them.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
366 redir @q
8116
3c1bdf20c8b9 commit https://github.com/vim/vim/commit/93bf558caef2d507ef6baf56eaf6025b63da1e34
Christian Brabandt <cb@256bit.org>
parents: 8029
diff changeset
367 silent function /^Test_
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
368 redir END
8029
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce
Christian Brabandt <cb@256bit.org>
parents: 7637
diff changeset
369 let s:tests = split(substitute(@q, 'function \(\k*()\)', '\1', 'g'))
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
370
8142
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
371 " If there is an extra argument filter the function names against it.
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
372 if argc() > 1
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
373 let s:tests = filter(s:tests, 'v:val =~ argv(1)')
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
374 endif
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa
Christian Brabandt <cb@256bit.org>
parents: 8116
diff changeset
375
7540
b910bb01832a commit https://github.com/vim/vim/commit/cfc0a350a9fa04f1b0cfa1ba31fbd2847376513f
Christian Brabandt <cb@256bit.org>
parents: 7454
diff changeset
376 " Execute the tests in alphabetical order.
8116
3c1bdf20c8b9 commit https://github.com/vim/vim/commit/93bf558caef2d507ef6baf56eaf6025b63da1e34
Christian Brabandt <cb@256bit.org>
parents: 8029
diff changeset
377 for s:test in sort(s:tests)
12616
4767939d10cc patch 8.0.1186: still quite a few old style tests
Christian Brabandt <cb@256bit.org>
parents: 12381
diff changeset
378 " Silence, please!
4767939d10cc patch 8.0.1186: still quite a few old style tests
Christian Brabandt <cb@256bit.org>
parents: 12381
diff changeset
379 set belloff=all
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
380 let prev_error = ''
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
381 let total_errors = []
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
382 let run_nr = 1
12616
4767939d10cc patch 8.0.1186: still quite a few old style tests
Christian Brabandt <cb@256bit.org>
parents: 12381
diff changeset
383
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
384 call RunTheTest(s:test)
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
385
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
386 " Repeat a flaky test. Give up when:
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
387 " - it fails again with the same message
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
388 " - it fails five times (with a different mesage)
15040
da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message
Bram Moolenaar <Bram@vim.org>
parents: 15038
diff changeset
389 if len(v:errors) > 0
da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message
Bram Moolenaar <Bram@vim.org>
parents: 15038
diff changeset
390 \ && (index(s:flaky_tests, s:test) >= 0
da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message
Bram Moolenaar <Bram@vim.org>
parents: 15038
diff changeset
391 \ || v:errors[0] =~ s:flaky_errors_re)
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
392 while 1
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
393 call add(s:messages, 'Found errors in ' . s:test . ':')
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
394 call extend(s:messages, v:errors)
11187
515db00c4676 patch 8.0.0480: the remote_peek() test fails on MS-Windows
Christian Brabandt <cb@256bit.org>
parents: 11185
diff changeset
395
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
396 call add(total_errors, 'Run ' . run_nr . ':')
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
397 call extend(total_errors, v:errors)
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
398
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
399 if run_nr == 5 || prev_error == v:errors[0]
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
400 call add(total_errors, 'Flaky test failed too often, giving up')
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
401 let v:errors = total_errors
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
402 break
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
403 endif
12905
30b0d784c870 patch 8.0.1329: when a flaky test fails it also often fails the second time
Christian Brabandt <cb@256bit.org>
parents: 12851
diff changeset
404
15036
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
405 call add(s:messages, 'Flaky test failed, running it again')
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
406
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
407 " Flakiness is often caused by the system being very busy. Sleep a
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
408 " couple of seconds to have a higher chance of succeeding the second
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
409 " time.
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
410 sleep 2
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
411
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
412 let prev_error = v:errors[0]
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
413 let v:errors = []
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
414 let run_nr += 1
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
415
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
416 call RunTheTest(s:test)
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
417
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
418 if len(v:errors) == 0
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
419 " Test passed on rerun.
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
420 break
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
421 endif
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways
Bram Moolenaar <Bram@vim.org>
parents: 15018
diff changeset
422 endwhile
8370
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e
Christian Brabandt <cb@256bit.org>
parents: 8142
diff changeset
423 endif
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
424
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
425 call AfterTheTest()
7277
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
426 endfor
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801
Christian Brabandt <cb@256bit.org>
parents:
diff changeset
427
11181
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing
Christian Brabandt <cb@256bit.org>
parents: 11169
diff changeset
428 call FinishTesting()
10388
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
429
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4
Christian Brabandt <cb@256bit.org>
parents: 10229
diff changeset
430 " vim: shiftwidth=2 sts=2 expandtab