vim: src/testdir/runtest.vim annotate

annotate src/testdir/runtest.vim @ 18825:70f1e352d599 v8.1.2400

patch 8.1.2400: test39 is old style Commit: https://github.com/vim/vim/commit/1f3e7d3bf0aa1e015a591ce8f7ee7ab56589b452 Author: Bram Moolenaar <Bram@vim.org> Date: Fri Dec 6 20:43:36 2019 +0100 patch 8.1.2400: test39 is old style Problem: Test39 is old style. Solution: Convert the test cases into new style. (Yegappan Lakshmanan, closes #5324)

author	Bram Moolenaar <Bram@vim.org>
date	Fri, 06 Dec 2019 20:45:05 +0100
parents	49b78d6465e5
children	aae19dd172c0

rev	line source
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	1 " This script is sourced while editing the .vim file with the tests.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	2 " When the script is successful the .res file will be created.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	3 " Errors are appended to the test.log file.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	4 "
8142 71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa Christian Brabandt <cb@256bit.org> parents: 8116 diff changeset	5 " To execute only specific test functions, add a second argument. It will be
18748 c7b16393c248 patch 8.1.2364: termwinscroll test is flaky on FreeBSD Bram Moolenaar <Bram@vim.org> parents: 18691 diff changeset	6 " matched against the names of the Test_ function. E.g.:
8142 71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa Christian Brabandt <cb@256bit.org> parents: 8116 diff changeset	7 " ../vim -u NONE -S runtest.vim test_channel.vim open_delay
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa Christian Brabandt <cb@256bit.org> parents: 8116 diff changeset	8 " The output can be found in the "messages" file.
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa Christian Brabandt <cb@256bit.org> parents: 8116 diff changeset	9 "
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	10 " The test script may contain anything, only functions that start with
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	11 " "Test_" are special. These will be invoked and should contain assert
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	12 " functions. See test_assert.vim for an example.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	13 "
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	14 " It is possible to source other files that contain "Test_" functions. This
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	15 " can speed up testing, since Vim does not need to restart. But be careful
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	16 " that the tests do not interfere with each other.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	17 "
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	18 " If an error cannot be detected properly with an assert function add the
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	19 " error to the v:errors list:
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	20 " call add(v:errors, 'test foo failed: Cannot find xyz')
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	21 "
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	22 " If preparation for each Test_ function is needed, define a SetUp function.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	23 " It will be called before each Test_ function.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	24 "
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	25 " If cleanup after each Test_ function is needed, define a TearDown function.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	26 " It will be called after each Test_ function.
8029 770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	27 "
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	28 " When debugging a test it can be useful to add messages to v:errors:
15377 88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset Bram Moolenaar <Bram@vim.org> parents: 15180 diff changeset	29 " call add(v:errors, "this happened")
8029 770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	30
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	31
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	32 " Without the +eval feature we can't run these tests, bail out.
7352 ddab7ae8796d commit https://github.com/vim/vim/commit/4686b323e4bc0f466500b018959f6c8965f010f9 Christian Brabandt <cb@256bit.org> parents: 7295 diff changeset	33 so small.vim
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	34
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	35 " Check that the screen size is at least 24 x 80 characters.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	36 if &lines < 24 \|\| &columns < 80
17684 e2bfdd281578 patch 8.1.1839: insufficient info when test fails because of screen size Bram Moolenaar <Bram@vim.org> parents: 17356 diff changeset	37 let error = 'Screen size too small! Tests require at least 24 lines with 80 characters, got ' .. &lines .. ' lines with ' .. &columns .. ' characters'
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	38 echoerr error
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	39 split test.log
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	40 $put =error
17091 d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that Bram Moolenaar <Bram@vim.org> parents: 17032 diff changeset	41 write
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that Bram Moolenaar <Bram@vim.org> parents: 17032 diff changeset	42 split messages
17684 e2bfdd281578 patch 8.1.1839: insufficient info when test fails because of screen size Bram Moolenaar <Bram@vim.org> parents: 17356 diff changeset	43 call append(line('$'), '')
e2bfdd281578 patch 8.1.1839: insufficient info when test fails because of screen size Bram Moolenaar <Bram@vim.org> parents: 17356 diff changeset	44 call append(line('$'), 'From ' . expand('%') . ':')
17091 d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that Bram Moolenaar <Bram@vim.org> parents: 17032 diff changeset	45 call append(line('$'), error)
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that Bram Moolenaar <Bram@vim.org> parents: 17032 diff changeset	46 write
d30d399139ab patch 8.1.1545: when the screen is to small there is no message about that Bram Moolenaar <Bram@vim.org> parents: 17032 diff changeset	47 qa!
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	48 endif
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	49
16967 586d625e21b4 patch 8.1.1484: some tests are slow Bram Moolenaar <Bram@vim.org> parents: 16951 diff changeset	50 if has('reltime')
586d625e21b4 patch 8.1.1484: some tests are slow Bram Moolenaar <Bram@vim.org> parents: 16951 diff changeset	51 let s:start_time = reltime()
586d625e21b4 patch 8.1.1484: some tests are slow Bram Moolenaar <Bram@vim.org> parents: 16951 diff changeset	52 endif
586d625e21b4 patch 8.1.1484: some tests are slow Bram Moolenaar <Bram@vim.org> parents: 16951 diff changeset	53
9501 f0201deac2a2 commit https://github.com/vim/vim/commit/89b10421ca1aea55acbafac1df93be4e20210ceb Christian Brabandt <cb@256bit.org> parents: 9452 diff changeset	54 " Common with all tests on all systems.
f0201deac2a2 commit https://github.com/vim/vim/commit/89b10421ca1aea55acbafac1df93be4e20210ceb Christian Brabandt <cb@256bit.org> parents: 9452 diff changeset	55 source setup.vim
f0201deac2a2 commit https://github.com/vim/vim/commit/89b10421ca1aea55acbafac1df93be4e20210ceb Christian Brabandt <cb@256bit.org> parents: 9452 diff changeset	56
7391 4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17 Christian Brabandt <cb@256bit.org> parents: 7372 diff changeset	57 " For consistency run all tests with 'nocompatible' set.
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17 Christian Brabandt <cb@256bit.org> parents: 7372 diff changeset	58 " This also enables use of line continuation.
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17 Christian Brabandt <cb@256bit.org> parents: 7372 diff changeset	59 set nocp viminfo+=nviminfo
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17 Christian Brabandt <cb@256bit.org> parents: 7372 diff changeset	60
15607 2dcaa860e3fc patch 8.1.0811: too many #ifdefs Bram Moolenaar <Bram@vim.org> parents: 15377 diff changeset	61 " Use utf-8 by default, instead of whatever the system default happens to be.
15928 857ce36c8412 patch 8.1.0970: text properties test fails when 'encoding' is not utf-8 Bram Moolenaar <Bram@vim.org> parents: 15920 diff changeset	62 " Individual tests can overrule this at the top of the file and use
857ce36c8412 patch 8.1.0970: text properties test fails when 'encoding' is not utf-8 Bram Moolenaar <Bram@vim.org> parents: 15920 diff changeset	63 " g:orig_encoding if needed.
857ce36c8412 patch 8.1.0970: text properties test fails when 'encoding' is not utf-8 Bram Moolenaar <Bram@vim.org> parents: 15920 diff changeset	64 let g:orig_encoding = &encoding
15607 2dcaa860e3fc patch 8.1.0811: too many #ifdefs Bram Moolenaar <Bram@vim.org> parents: 15377 diff changeset	65 set encoding=utf-8
9618 81ba6e4eb72b commit https://github.com/vim/vim/commit/ac105ed3c420660ddbddc501c97875c48220817e Christian Brabandt <cb@256bit.org> parents: 9501 diff changeset	66
14889 ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	67 " REDIR_TEST_TO_NULL has a very permissive SwapExists autocommand which is for
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	68 " the test_name.vim file itself. Replace it here with a more restrictive one,
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	69 " so we still catch mistakes.
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	70 let s:test_script_fname = expand('%')
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	71 au! SwapExists * call HandleSwapExists()
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	72 func HandleSwapExists()
17356 4fe44a3b732f patch 8.1.1677: tests get stuck when running into an existing swapfile Bram Moolenaar <Bram@vim.org> parents: 17123 diff changeset	73 " Ignore finding a swap file for the test script (the user might be
14889 ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	74 " editing it and do ":make test_name") and the output file.
17356 4fe44a3b732f patch 8.1.1677: tests get stuck when running into an existing swapfile Bram Moolenaar <Bram@vim.org> parents: 17123 diff changeset	75 " Report finding another swap file and chose 'q' to avoid getting stuck.
14889 ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	76 if expand('<afile>') == 'messages' \|\| expand('<afile>') =~ s:test_script_fname
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	77 let v:swapchoice = 'e'
17356 4fe44a3b732f patch 8.1.1677: tests get stuck when running into an existing swapfile Bram Moolenaar <Bram@vim.org> parents: 17123 diff changeset	78 else
4fe44a3b732f patch 8.1.1677: tests get stuck when running into an existing swapfile Bram Moolenaar <Bram@vim.org> parents: 17123 diff changeset	79 call assert_report('Unexpected swap file: ' .. v:swapname)
4fe44a3b732f patch 8.1.1677: tests get stuck when running into an existing swapfile Bram Moolenaar <Bram@vim.org> parents: 17123 diff changeset	80 let v:swapchoice = 'q'
14889 ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	81 endif
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	82 endfunc
ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	83
10811 4cb74eab1733 patch 8.0.0295: test_viml hangs Christian Brabandt <cb@256bit.org> parents: 10807 diff changeset	84 " Avoid stopping at the "hit enter" prompt
4cb74eab1733 patch 8.0.0295: test_viml hangs Christian Brabandt <cb@256bit.org> parents: 10807 diff changeset	85 set nomore
4cb74eab1733 patch 8.0.0295: test_viml hangs Christian Brabandt <cb@256bit.org> parents: 10807 diff changeset	86
7391 4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17 Christian Brabandt <cb@256bit.org> parents: 7372 diff changeset	87 " Output all messages in English.
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17 Christian Brabandt <cb@256bit.org> parents: 7372 diff changeset	88 lang mess C
4761fed349ed commit https://github.com/vim/vim/commit/c06624661a3aa6642304c06db9cebe553a4cab17 Christian Brabandt <cb@256bit.org> parents: 7372 diff changeset	89
7623 2720952e9acb commit https://github.com/vim/vim/commit/f60b796fa9870bdfc4cdeb91653bac041916077d Christian Brabandt <cb@256bit.org> parents: 7595 diff changeset	90 " Always use forward slashes.
2720952e9acb commit https://github.com/vim/vim/commit/f60b796fa9870bdfc4cdeb91653bac041916077d Christian Brabandt <cb@256bit.org> parents: 7595 diff changeset	91 set shellslash
2720952e9acb commit https://github.com/vim/vim/commit/f60b796fa9870bdfc4cdeb91653bac041916077d Christian Brabandt <cb@256bit.org> parents: 7595 diff changeset	92
7545 4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	93 let s:srcdir = expand('%:p:h:h')
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	94
9121 7350959e53c3 commit https://github.com/vim/vim/commit/8e8df251bf2505e5decf258397c6069fbe5e2e01 Christian Brabandt <cb@256bit.org> parents: 8881 diff changeset	95 " Prepare for calling test_garbagecollect_now().
8881 ed0b39dd7fd6 commit https://github.com/vim/vim/commit/ebf7dfa6f121c82f97d2adca3d45fbaba9ad8f7e Christian Brabandt <cb@256bit.org> parents: 8796 diff changeset	96 let v:testing = 1
ed0b39dd7fd6 commit https://github.com/vim/vim/commit/ebf7dfa6f121c82f97d2adca3d45fbaba9ad8f7e Christian Brabandt <cb@256bit.org> parents: 8796 diff changeset	97
7545 4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	98 " Support function: get the alloc ID by name.
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	99 function GetAllocId(name)
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	100 exe 'split ' . s:srcdir . '/alloc.h'
7595 99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0 Christian Brabandt <cb@256bit.org> parents: 7545 diff changeset	101 let top = search('typedef enum')
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0 Christian Brabandt <cb@256bit.org> parents: 7545 diff changeset	102 if top == 0
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0 Christian Brabandt <cb@256bit.org> parents: 7545 diff changeset	103 call add(v:errors, 'typedef not found in alloc.h')
99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0 Christian Brabandt <cb@256bit.org> parents: 7545 diff changeset	104 endif
7545 4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	105 let lnum = search('aid_' . a:name . ',')
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	106 if lnum == 0
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	107 call add(v:errors, 'Alloc ID ' . a:name . ' not defined')
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	108 endif
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	109 close
7595 99e93f72ff91 commit https://github.com/vim/vim/commit/065ee9aebf9abe08ae8c0dba7d05cbdcc423c8e0 Christian Brabandt <cb@256bit.org> parents: 7545 diff changeset	110 return lnum - top - 1
7545 4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	111 endfunc
4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	112
11181 13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	113 func RunTheTest(test)
8370 cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	114 echo 'Executing ' . a:test
16967 586d625e21b4 patch 8.1.1484: some tests are slow Bram Moolenaar <Bram@vim.org> parents: 16951 diff changeset	115 if has('reltime')
586d625e21b4 patch 8.1.1484: some tests are slow Bram Moolenaar <Bram@vim.org> parents: 16951 diff changeset	116 let func_start = reltime()
586d625e21b4 patch 8.1.1484: some tests are slow Bram Moolenaar <Bram@vim.org> parents: 16951 diff changeset	117 endif
10807 82c43bca568e patch 8.0.0293: some tests have a one or three second wait Christian Brabandt <cb@256bit.org> parents: 10797 diff changeset	118
82c43bca568e patch 8.0.0293: some tests have a one or three second wait Christian Brabandt <cb@256bit.org> parents: 10797 diff changeset	119 " Avoid stopping at the "hit enter" prompt
82c43bca568e patch 8.0.0293: some tests have a one or three second wait Christian Brabandt <cb@256bit.org> parents: 10797 diff changeset	120 set nomore
82c43bca568e patch 8.0.0293: some tests have a one or three second wait Christian Brabandt <cb@256bit.org> parents: 10797 diff changeset	121
82c43bca568e patch 8.0.0293: some tests have a one or three second wait Christian Brabandt <cb@256bit.org> parents: 10797 diff changeset	122 " Avoid a three second wait when a message is about to be overwritten by the
82c43bca568e patch 8.0.0293: some tests have a one or three second wait Christian Brabandt <cb@256bit.org> parents: 10797 diff changeset	123 " mode message.
82c43bca568e patch 8.0.0293: some tests have a one or three second wait Christian Brabandt <cb@256bit.org> parents: 10797 diff changeset	124 set noshowmode
82c43bca568e patch 8.0.0293: some tests have a one or three second wait Christian Brabandt <cb@256bit.org> parents: 10797 diff changeset	125
11105 7c7e496e625d patch 8.0.0440: not enough test coverage in Insert mode Christian Brabandt <cb@256bit.org> parents: 11056 diff changeset	126 " Clear any overrides.
7c7e496e625d patch 8.0.0440: not enough test coverage in Insert mode Christian Brabandt <cb@256bit.org> parents: 11056 diff changeset	127 call test_override('ALL', 0)
7c7e496e625d patch 8.0.0440: not enough test coverage in Insert mode Christian Brabandt <cb@256bit.org> parents: 11056 diff changeset	128
12692 03a6aeea2096 patch 8.0.1224: still interference between test functions Christian Brabandt <cb@256bit.org> parents: 12688 diff changeset	129 " Some tests wipe out buffers. To be consistent, always wipe out all
03a6aeea2096 patch 8.0.1224: still interference between test functions Christian Brabandt <cb@256bit.org> parents: 12688 diff changeset	130 " buffers.
03a6aeea2096 patch 8.0.1224: still interference between test functions Christian Brabandt <cb@256bit.org> parents: 12688 diff changeset	131 %bwipe!
03a6aeea2096 patch 8.0.1224: still interference between test functions Christian Brabandt <cb@256bit.org> parents: 12688 diff changeset	132
12851 90aaa974594e patch 8.0.1302: still too many old style tests Christian Brabandt <cb@256bit.org> parents: 12769 diff changeset	133 " The test may change the current directory. Save and restore the
90aaa974594e patch 8.0.1302: still too many old style tests Christian Brabandt <cb@256bit.org> parents: 12769 diff changeset	134 " directory after executing the test.
90aaa974594e patch 8.0.1302: still too many old style tests Christian Brabandt <cb@256bit.org> parents: 12769 diff changeset	135 let save_cwd = getcwd()
90aaa974594e patch 8.0.1302: still too many old style tests Christian Brabandt <cb@256bit.org> parents: 12769 diff changeset	136
8370 cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	137 if exists("*SetUp")
10388 6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4 Christian Brabandt <cb@256bit.org> parents: 10229 diff changeset	138 try
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4 Christian Brabandt <cb@256bit.org> parents: 10229 diff changeset	139 call SetUp()
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4 Christian Brabandt <cb@256bit.org> parents: 10229 diff changeset	140 catch
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4 Christian Brabandt <cb@256bit.org> parents: 10229 diff changeset	141 call add(v:errors, 'Caught exception in SetUp() before ' . a:test . ': ' . v:exception . ' @ ' . v:throwpoint)
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4 Christian Brabandt <cb@256bit.org> parents: 10229 diff changeset	142 endtry
8370 cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	143 endif
cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	144
12682 47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	145 if a:test =~ 'Test_nocatch_'
47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	146 " Function handles errors itself. This avoids skipping commands after the
47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	147 " error.
8370 cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	148 exe 'call ' . a:test
12682 47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	149 else
47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	150 try
14053 d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed Christian Brabandt <cb@256bit.org> parents: 13634 diff changeset	151 let s:test = a:test
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed Christian Brabandt <cb@256bit.org> parents: 13634 diff changeset	152 au VimLeavePre * call EarlyExit(s:test)
12682 47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	153 exe 'call ' . a:test
14053 d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed Christian Brabandt <cb@256bit.org> parents: 13634 diff changeset	154 au! VimLeavePre
12682 47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	155 catch /^\cskipped/
47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	156 call add(s:messages, ' Skipped')
47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	157 call add(s:skipped, 'SKIPPED ' . a:test . ': ' . substitute(v:exception, '^\S*\s\+', '', ''))
47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	158 catch
47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	159 call add(v:errors, 'Caught exception in ' . a:test . ': ' . v:exception . ' @ ' . v:throwpoint)
47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	160 endtry
47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	161 endif
8370 cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	162
15377 88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset Bram Moolenaar <Bram@vim.org> parents: 15180 diff changeset	163 " In case 'insertmode' was set and something went wrong, make sure it is
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset Bram Moolenaar <Bram@vim.org> parents: 15180 diff changeset	164 " reset to avoid trouble with anything else.
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset Bram Moolenaar <Bram@vim.org> parents: 15180 diff changeset	165 set noinsertmode
88b0a490816e patch 8.1.0696: when test_edit fails 'insertmode' may not be reset Bram Moolenaar <Bram@vim.org> parents: 15180 diff changeset	166
8370 cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	167 if exists("*TearDown")
10388 6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4 Christian Brabandt <cb@256bit.org> parents: 10229 diff changeset	168 try
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4 Christian Brabandt <cb@256bit.org> parents: 10229 diff changeset	169 call TearDown()
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4 Christian Brabandt <cb@256bit.org> parents: 10229 diff changeset	170 catch
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4 Christian Brabandt <cb@256bit.org> parents: 10229 diff changeset	171 call add(v:errors, 'Caught exception in TearDown() after ' . a:test . ': ' . v:exception . ' @ ' . v:throwpoint)
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4 Christian Brabandt <cb@256bit.org> parents: 10229 diff changeset	172 endtry
8370 cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	173 endif
9748 5440f3442841 commit https://github.com/vim/vim/commit/7cba71d7e3576639679b6a3aedeeb1ac07f7f2f5 Christian Brabandt <cb@256bit.org> parents: 9632 diff changeset	174
17684 e2bfdd281578 patch 8.1.1839: insufficient info when test fails because of screen size Bram Moolenaar <Bram@vim.org> parents: 17356 diff changeset	175 " Clear any autocommands and put back the catch-all for SwapExists.
12692 03a6aeea2096 patch 8.0.1224: still interference between test functions Christian Brabandt <cb@256bit.org> parents: 12688 diff changeset	176 au!
14889 ba1f0c06b4be patch 8.1.0456: running test hangs when the input file is being edited Bram Moolenaar <Bram@vim.org> parents: 14571 diff changeset	177 au SwapExists * call HandleSwapExists()
12692 03a6aeea2096 patch 8.0.1224: still interference between test functions Christian Brabandt <cb@256bit.org> parents: 12688 diff changeset	178
17123 efc6f5e3b543 patch 8.1.1561: popup_setoptions() is not implemented yet Bram Moolenaar <Bram@vim.org> parents: 17091 diff changeset	179 " Close any stray popup windows
18763 49b78d6465e5 patch 8.1.2371: FEAT_TEXT_PROP is a confusing name Bram Moolenaar <Bram@vim.org> parents: 18748 diff changeset	180 if has('popupwin')
17123 efc6f5e3b543 patch 8.1.1561: popup_setoptions() is not implemented yet Bram Moolenaar <Bram@vim.org> parents: 17091 diff changeset	181 call popup_clear()
efc6f5e3b543 patch 8.1.1561: popup_setoptions() is not implemented yet Bram Moolenaar <Bram@vim.org> parents: 17091 diff changeset	182 endif
efc6f5e3b543 patch 8.1.1561: popup_setoptions() is not implemented yet Bram Moolenaar <Bram@vim.org> parents: 17091 diff changeset	183
12688 a6a935b3270e patch 8.0.1222: test functions interfere with each other Christian Brabandt <cb@256bit.org> parents: 12682 diff changeset	184 " Close any extra tab pages and windows and make the current one not modified.
a6a935b3270e patch 8.0.1222: test functions interfere with each other Christian Brabandt <cb@256bit.org> parents: 12682 diff changeset	185 while tabpagenr('$') > 1
12692 03a6aeea2096 patch 8.0.1224: still interference between test functions Christian Brabandt <cb@256bit.org> parents: 12688 diff changeset	186 quit!
12688 a6a935b3270e patch 8.0.1222: test functions interfere with each other Christian Brabandt <cb@256bit.org> parents: 12682 diff changeset	187 endwhile
a6a935b3270e patch 8.0.1222: test functions interfere with each other Christian Brabandt <cb@256bit.org> parents: 12682 diff changeset	188
9949 30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe Christian Brabandt <cb@256bit.org> parents: 9758 diff changeset	189 while 1
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe Christian Brabandt <cb@256bit.org> parents: 9758 diff changeset	190 let wincount = winnr('$')
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe Christian Brabandt <cb@256bit.org> parents: 9758 diff changeset	191 if wincount == 1
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe Christian Brabandt <cb@256bit.org> parents: 9758 diff changeset	192 break
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe Christian Brabandt <cb@256bit.org> parents: 9758 diff changeset	193 endif
9748 5440f3442841 commit https://github.com/vim/vim/commit/7cba71d7e3576639679b6a3aedeeb1ac07f7f2f5 Christian Brabandt <cb@256bit.org> parents: 9632 diff changeset	194 bwipe!
9949 30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe Christian Brabandt <cb@256bit.org> parents: 9758 diff changeset	195 if wincount == winnr('$')
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe Christian Brabandt <cb@256bit.org> parents: 9758 diff changeset	196 " Did not manage to close a window.
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe Christian Brabandt <cb@256bit.org> parents: 9758 diff changeset	197 only!
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe Christian Brabandt <cb@256bit.org> parents: 9758 diff changeset	198 break
30be4b26a37e commit https://github.com/vim/vim/commit/358308dd99abdd56c6540339e505585d8db7bdfe Christian Brabandt <cb@256bit.org> parents: 9758 diff changeset	199 endif
9748 5440f3442841 commit https://github.com/vim/vim/commit/7cba71d7e3576639679b6a3aedeeb1ac07f7f2f5 Christian Brabandt <cb@256bit.org> parents: 9632 diff changeset	200 endwhile
12851 90aaa974594e patch 8.0.1302: still too many old style tests Christian Brabandt <cb@256bit.org> parents: 12769 diff changeset	201
90aaa974594e patch 8.0.1302: still too many old style tests Christian Brabandt <cb@256bit.org> parents: 12769 diff changeset	202 exe 'cd ' . save_cwd
17032 84f308b126de patch 8.1.1516: time reported for a test measured wrong Bram Moolenaar <Bram@vim.org> parents: 16967 diff changeset	203
84f308b126de patch 8.1.1516: time reported for a test measured wrong Bram Moolenaar <Bram@vim.org> parents: 16967 diff changeset	204 let message = 'Executed ' . a:test
84f308b126de patch 8.1.1516: time reported for a test measured wrong Bram Moolenaar <Bram@vim.org> parents: 16967 diff changeset	205 if has('reltime')
84f308b126de patch 8.1.1516: time reported for a test measured wrong Bram Moolenaar <Bram@vim.org> parents: 16967 diff changeset	206 let message ..= ' in ' .. reltimestr(reltime(func_start)) .. ' seconds'
84f308b126de patch 8.1.1516: time reported for a test measured wrong Bram Moolenaar <Bram@vim.org> parents: 16967 diff changeset	207 endif
84f308b126de patch 8.1.1516: time reported for a test measured wrong Bram Moolenaar <Bram@vim.org> parents: 16967 diff changeset	208 call add(s:messages, message)
84f308b126de patch 8.1.1516: time reported for a test measured wrong Bram Moolenaar <Bram@vim.org> parents: 16967 diff changeset	209 let s:done += 1
8370 cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	210 endfunc
7545 4c922651fd78 commit https://github.com/vim/vim/commit/28fb79db6b52d1154e8dc63d227673648c2fce15 Christian Brabandt <cb@256bit.org> parents: 7540 diff changeset	211
11181 13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	212 func AfterTheTest()
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	213 if len(v:errors) > 0
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	214 let s:fail += 1
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	215 call add(s:errors, 'Found errors in ' . s:test . ':')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	216 call extend(s:errors, v:errors)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	217 let v:errors = []
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	218 endif
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	219 endfunc
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	220
14053 d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed Christian Brabandt <cb@256bit.org> parents: 13634 diff changeset	221 func EarlyExit(test)
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed Christian Brabandt <cb@256bit.org> parents: 13634 diff changeset	222 " It's OK for the test we use to test the quit detection.
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed Christian Brabandt <cb@256bit.org> parents: 13634 diff changeset	223 if a:test != 'Test_zz_quit_detected()'
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed Christian Brabandt <cb@256bit.org> parents: 13634 diff changeset	224 call add(v:errors, 'Test caused Vim to exit: ' . a:test)
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed Christian Brabandt <cb@256bit.org> parents: 13634 diff changeset	225 endif
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed Christian Brabandt <cb@256bit.org> parents: 13634 diff changeset	226
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed Christian Brabandt <cb@256bit.org> parents: 13634 diff changeset	227 call FinishTesting()
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed Christian Brabandt <cb@256bit.org> parents: 13634 diff changeset	228 endfunc
d6866c54a311 patch 8.1.0044: if a test function exists Vim this may go unnoticed Christian Brabandt <cb@256bit.org> parents: 13634 diff changeset	229
11181 13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	230 " This function can be called by a test if it wants to abort testing.
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	231 func FinishTesting()
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	232 call AfterTheTest()
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	233
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	234 " Don't write viminfo on exit.
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	235 set viminfo=
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	236
11850 eb339f9af281 patch 8.0.0805: GUI test fails with gnome2 Christian Brabandt <cb@256bit.org> parents: 11205 diff changeset	237 " Clean up files created by setup.vim
eb339f9af281 patch 8.0.0805: GUI test fails with gnome2 Christian Brabandt <cb@256bit.org> parents: 11205 diff changeset	238 call delete('XfakeHOME', 'rf')
eb339f9af281 patch 8.0.0805: GUI test fails with gnome2 Christian Brabandt <cb@256bit.org> parents: 11205 diff changeset	239
11181 13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	240 if s:fail == 0
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	241 " Success, create the .res file so that make knows it's done.
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	242 exe 'split ' . fnamemodify(g:testname, ':r') . '.res'
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	243 write
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	244 endif
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	245
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	246 if len(s:errors) > 0
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	247 " Append errors to test.log
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	248 split test.log
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	249 call append(line('$'), '')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	250 call append(line('$'), 'From ' . g:testname . ':')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	251 call append(line('$'), s:errors)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	252 write
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	253 endif
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	254
13634 2678e38e1de6 patch 8.0.1689: no tests for xxd Christian Brabandt <cb@256bit.org> parents: 13533 diff changeset	255 if s:done == 0
18189 b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active Bram Moolenaar <Bram@vim.org> parents: 18184 diff changeset	256 if s:filtered > 0
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active Bram Moolenaar <Bram@vim.org> parents: 18184 diff changeset	257 let message = "NO tests match $TEST_FILTER: '" .. $TEST_FILTER .. "'"
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active Bram Moolenaar <Bram@vim.org> parents: 18184 diff changeset	258 else
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active Bram Moolenaar <Bram@vim.org> parents: 18184 diff changeset	259 let message = 'NO tests executed'
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active Bram Moolenaar <Bram@vim.org> parents: 18184 diff changeset	260 endif
13634 2678e38e1de6 patch 8.0.1689: no tests for xxd Christian Brabandt <cb@256bit.org> parents: 13533 diff changeset	261 else
18189 b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active Bram Moolenaar <Bram@vim.org> parents: 18184 diff changeset	262 if s:filtered > 0
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active Bram Moolenaar <Bram@vim.org> parents: 18184 diff changeset	263 call add(s:messages, "Filtered " .. s:filtered .. " tests with $TEST_FILTER")
b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active Bram Moolenaar <Bram@vim.org> parents: 18184 diff changeset	264 endif
13634 2678e38e1de6 patch 8.0.1689: no tests for xxd Christian Brabandt <cb@256bit.org> parents: 13533 diff changeset	265 let message = 'Executed ' . s:done . (s:done > 1 ? ' tests' : ' test')
2678e38e1de6 patch 8.0.1689: no tests for xxd Christian Brabandt <cb@256bit.org> parents: 13533 diff changeset	266 endif
18189 b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active Bram Moolenaar <Bram@vim.org> parents: 18184 diff changeset	267 if s:done > 0 && has('reltime')
16967 586d625e21b4 patch 8.1.1484: some tests are slow Bram Moolenaar <Bram@vim.org> parents: 16951 diff changeset	268 let message ..= ' in ' .. reltimestr(reltime(s:start_time)) .. ' seconds'
586d625e21b4 patch 8.1.1484: some tests are slow Bram Moolenaar <Bram@vim.org> parents: 16951 diff changeset	269 endif
11181 13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	270 echo message
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	271 call add(s:messages, message)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	272 if s:fail > 0
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	273 let message = s:fail . ' FAILED:'
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	274 echo message
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	275 call add(s:messages, message)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	276 call extend(s:messages, s:errors)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	277 endif
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	278
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	279 " Add SKIPPED messages
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	280 call extend(s:messages, s:skipped)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	281
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	282 " Append messages to the file "messages"
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	283 split messages
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	284 call append(line('$'), '')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	285 call append(line('$'), 'From ' . g:testname . ':')
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	286 call append(line('$'), s:messages)
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	287 write
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	288
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	289 qall!
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	290 endfunc
13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	291
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	292 " Source the test script. First grab the file name, in case the script
8029 770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	293 " navigates away. g:testname can be used by the tests.
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	294 let g:testname = expand('%')
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	295 let s:done = 0
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	296 let s:fail = 0
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	297 let s:errors = []
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	298 let s:messages = []
10100 26441931dad7 commit https://github.com/vim/vim/commit/dac1947bb366ef43cd6da95acc730554e76d8b84 Christian Brabandt <cb@256bit.org> parents: 9949 diff changeset	299 let s:skipped = []
10942 e05695e59f6d patch 8.0.0360: sometimes VimL is used instead of "Vim script" Christian Brabandt <cb@256bit.org> parents: 10811 diff changeset	300 if expand('%') =~ 'test_vimscript.vim'
8029 770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	301 " this test has intentional s:errors, don't use try/catch.
7352 ddab7ae8796d commit https://github.com/vim/vim/commit/4686b323e4bc0f466500b018959f6c8965f010f9 Christian Brabandt <cb@256bit.org> parents: 7295 diff changeset	302 source %
7454 4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1 Christian Brabandt <cb@256bit.org> parents: 7391 diff changeset	303 else
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1 Christian Brabandt <cb@256bit.org> parents: 7391 diff changeset	304 try
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1 Christian Brabandt <cb@256bit.org> parents: 7391 diff changeset	305 source %
16951 ebdf6cd89910 patch 8.1.1476: no statistics displayed after running tests Bram Moolenaar <Bram@vim.org> parents: 15928 diff changeset	306 catch /^\cskipped/
ebdf6cd89910 patch 8.1.1476: no statistics displayed after running tests Bram Moolenaar <Bram@vim.org> parents: 15928 diff changeset	307 call add(s:messages, ' Skipped')
ebdf6cd89910 patch 8.1.1476: no statistics displayed after running tests Bram Moolenaar <Bram@vim.org> parents: 15928 diff changeset	308 call add(s:skipped, 'SKIPPED ' . expand('%') . ': ' . substitute(v:exception, '^\S*\s\+', '', ''))
7454 4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1 Christian Brabandt <cb@256bit.org> parents: 7391 diff changeset	309 catch
8029 770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	310 let s:fail += 1
770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	311 call add(s:errors, 'Caught exception: ' . v:exception . ' @ ' . v:throwpoint)
7454 4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1 Christian Brabandt <cb@256bit.org> parents: 7391 diff changeset	312 endtry
4ff843da79fd commit https://github.com/vim/vim/commit/a2cce8630756769b2cefdc28c7290ae9262cddb1 Christian Brabandt <cb@256bit.org> parents: 7391 diff changeset	313 endif
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	314
8370 cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	315 " Names of flaky tests.
15040 da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message Bram Moolenaar <Bram@vim.org> parents: 15038 diff changeset	316 let s:flaky_tests = [
18158 ea327d003580 patch 8.1.2074: test for SafeState autocommand is a bit flaky Bram Moolenaar <Bram@vim.org> parents: 18112 diff changeset	317 \ 'Test_autocmd_SafeState()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	318 \ 'Test_call()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	319 \ 'Test_channel_handler()',
11181 13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	320 \ 'Test_client_server()',
10789 214417c527b5 patch 8.0.0284: collapse buffers test failed once Christian Brabandt <cb@256bit.org> parents: 10755 diff changeset	321 \ 'Test_close_and_exit_cb()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	322 \ 'Test_close_callback()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	323 \ 'Test_close_handle()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	324 \ 'Test_close_lambda()',
15180 52e883259d99 patch 8.1.0600: channel test is flaky Bram Moolenaar <Bram@vim.org> parents: 15044 diff changeset	325 \ 'Test_close_output_buffer()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	326 \ 'Test_close_partial()',
10789 214417c527b5 patch 8.0.0284: collapse buffers test failed once Christian Brabandt <cb@256bit.org> parents: 10755 diff changeset	327 \ 'Test_collapse_buffers()',
214417c527b5 patch 8.0.0284: collapse buffers test failed once Christian Brabandt <cb@256bit.org> parents: 10755 diff changeset	328 \ 'Test_communicate()',
13533 9e9e9f10e311 patch 8.0.1640: Test_cwd() is flaky Christian Brabandt <cb@256bit.org> parents: 13266 diff changeset	329 \ 'Test_cwd()',
15018 563ca9655b8f patch 8.1.0520: screen diff test sometimes fails Bram Moolenaar <Bram@vim.org> parents: 14889 diff changeset	330 \ 'Test_diff_screen()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	331 \ 'Test_exit_callback()',
11169 f4971739ea47 patch 8.0.0471: exit callback test sometimes fails Christian Brabandt <cb@256bit.org> parents: 11105 diff changeset	332 \ 'Test_exit_callback_interval()',
17726 586558396b7e patch 8.1.1860: map timeout test is flaky Bram Moolenaar <Bram@vim.org> parents: 17716 diff changeset	333 \ 'Test_map_timeout_with_timer_interrupt()',
10189 2392e065efea commit https://github.com/vim/vim/commit/e1c8c7a6742be6072290f9aa54ae358060d9c42f Christian Brabandt <cb@256bit.org> parents: 10100 diff changeset	334 \ 'Test_nb_basic()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	335 \ 'Test_open_delay()',
12988 da725bc3fd2c patch 8.0.1370: channel test for callback is flaky Christian Brabandt <cb@256bit.org> parents: 12979 diff changeset	336 \ 'Test_out_cb()',
10229 19ced29f773a commit https://github.com/vim/vim/commit/c79d6aa01d86cd98cffddc80825c1c600e833bbf Christian Brabandt <cb@256bit.org> parents: 10223 diff changeset	337 \ 'Test_pipe_through_sort_all()',
10797 e1bddbed43cc patch 8.0.0288: errors reported while running tests Christian Brabandt <cb@256bit.org> parents: 10789 diff changeset	338 \ 'Test_pipe_through_sort_some()',
18675 a0bd0a022bd4 patch 8.1.2329: mouse multiple click test is a bit flaky Bram Moolenaar <Bram@vim.org> parents: 18335 diff changeset	339 \ 'Test_popup_and_window_resize()',
11056 679c12064dbc patch 8.0.0417: test for the clipboard fails sometimes Christian Brabandt <cb@256bit.org> parents: 10990 diff changeset	340 \ 'Test_quoteplus()',
11205 4db196820d3b patch 8.0.0489: clipboard and "* register is not tested Christian Brabandt <cb@256bit.org> parents: 11187 diff changeset	341 \ 'Test_quotestar()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	342 \ 'Test_raw_one_time_callback()',
10789 214417c527b5 patch 8.0.0284: collapse buffers test failed once Christian Brabandt <cb@256bit.org> parents: 10755 diff changeset	343 \ 'Test_reltime()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	344 \ 'Test_server_crash()',
18313 d2fd32a43621 patch 8.1.2151: state test is a bit flaky Bram Moolenaar <Bram@vim.org> parents: 18189 diff changeset	345 \ 'Test_state()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	346 \ 'Test_terminal_ansicolors_default()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	347 \ 'Test_terminal_ansicolors_func()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	348 \ 'Test_terminal_ansicolors_global()',
12682 47f253caa805 patch 8.0.1219: terminal test is flaky Christian Brabandt <cb@256bit.org> parents: 12616 diff changeset	349 \ 'Test_terminal_composing_unicode()',
15920 82e3ffdc1c2b patch 8.1.0966: one terminal test is flaky Bram Moolenaar <Bram@vim.org> parents: 15772 diff changeset	350 \ 'Test_terminal_does_not_truncate_last_newlines()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	351 \ 'Test_terminal_env()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	352 \ 'Test_terminal_hide_buffer()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	353 \ 'Test_terminal_make_change()',
15772 649b81669197 patch 8.1.0893: terminal test is a bit flaky Bram Moolenaar <Bram@vim.org> parents: 15607 diff changeset	354 \ 'Test_terminal_no_cmd()',
12381 d3f8f1c33a20 patch 8.0.1070: terminal test is flaky on Mac Christian Brabandt <cb@256bit.org> parents: 11884 diff changeset	355 \ 'Test_terminal_noblock()',
12769 083ae2cfcc5b patch 8.0.1262: terminal redir test is flaky Christian Brabandt <cb@256bit.org> parents: 12692 diff changeset	356 \ 'Test_terminal_redir_file()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	357 \ 'Test_terminal_response_to_control_sequence()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	358 \ 'Test_terminal_scrollback()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	359 \ 'Test_terminal_split_quit()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	360 \ 'Test_terminal_termwinkey()',
17918 9606c0adc148 patch 8.1.1955: tests contain typos Bram Moolenaar <Bram@vim.org> parents: 17910 diff changeset	361 \ 'Test_terminal_termwinsize_minimum()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	362 \ 'Test_terminal_termwinsize_option_fixed()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	363 \ 'Test_terminal_termwinsize_option_zero()',
12979 6cf6138b6484 patch 8.0.1365: when one channel test fails others fail as well Christian Brabandt <cb@256bit.org> parents: 12905 diff changeset	364 \ 'Test_terminal_tmap()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	365 \ 'Test_terminal_wall()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	366 \ 'Test_terminal_wipe_buffer()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	367 \ 'Test_terminal_wqall()',
18748 c7b16393c248 patch 8.1.2364: termwinscroll test is flaky on FreeBSD Bram Moolenaar <Bram@vim.org> parents: 18691 diff changeset	368 \ 'Test_termwinscroll()',
17716 50b3b9c13ab8 patch 8.1.1855: another failing timer test Bram Moolenaar <Bram@vim.org> parents: 17714 diff changeset	369 \ 'Test_timer_oneshot()',
50b3b9c13ab8 patch 8.1.1855: another failing timer test Bram Moolenaar <Bram@vim.org> parents: 17714 diff changeset	370 \ 'Test_timer_paused()',
50b3b9c13ab8 patch 8.1.1855: another failing timer test Bram Moolenaar <Bram@vim.org> parents: 17714 diff changeset	371 \ 'Test_timer_repeat_many()',
50b3b9c13ab8 patch 8.1.1855: another failing timer test Bram Moolenaar <Bram@vim.org> parents: 17714 diff changeset	372 \ 'Test_timer_repeat_three()',
50b3b9c13ab8 patch 8.1.1855: another failing timer test Bram Moolenaar <Bram@vim.org> parents: 17714 diff changeset	373 \ 'Test_timer_stop_all_in_callback()',
50b3b9c13ab8 patch 8.1.1855: another failing timer test Bram Moolenaar <Bram@vim.org> parents: 17714 diff changeset	374 \ 'Test_timer_stop_in_callback()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	375 \ 'Test_two_channels()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	376 \ 'Test_unlet_handle()',
17716 50b3b9c13ab8 patch 8.1.1855: another failing timer test Bram Moolenaar <Bram@vim.org> parents: 17714 diff changeset	377 \ 'Test_timer_with_partial_callback()',
15038 26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	378 \ 'Test_zero_reply()',
26c266f6fbc6 patch 8.1.0530: channel and terminal tests that start a server can be flaky Bram Moolenaar <Bram@vim.org> parents: 15036 diff changeset	379 \ 'Test_zz1_terminal_in_gui()',
10189 2392e065efea commit https://github.com/vim/vim/commit/e1c8c7a6742be6072290f9aa54ae358060d9c42f Christian Brabandt <cb@256bit.org> parents: 10100 diff changeset	380 \ ]
8370 cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	381
15040 da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message Bram Moolenaar <Bram@vim.org> parents: 15038 diff changeset	382 " Pattern indicating a common flaky test failure.
15044 f47b5dfe5c6d patch 8.1.0533: screendump tests can be flaky Bram Moolenaar <Bram@vim.org> parents: 15040 diff changeset	383 let s:flaky_errors_re = 'StopVimInTerminal\\|VerifyScreenDump'
15040 da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message Bram Moolenaar <Bram@vim.org> parents: 15038 diff changeset	384
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	385 " Locate Test_ functions and execute them.
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	386 redir @q
8116 3c1bdf20c8b9 commit https://github.com/vim/vim/commit/93bf558caef2d507ef6baf56eaf6025b63da1e34 Christian Brabandt <cb@256bit.org> parents: 8029 diff changeset	387 silent function /^Test_
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	388 redir END
8029 770dd91ad9f7 commit https://github.com/vim/vim/commit/00af60bbb6cc7e8ccafddb30a1964f891b800bce Christian Brabandt <cb@256bit.org> parents: 7637 diff changeset	389 let s:tests = split(substitute(@q, 'function $\k*()$', '\1', 'g'))
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	390
8142 71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa Christian Brabandt <cb@256bit.org> parents: 8116 diff changeset	391 " If there is an extra argument filter the function names against it.
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa Christian Brabandt <cb@256bit.org> parents: 8116 diff changeset	392 if argc() > 1
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa Christian Brabandt <cb@256bit.org> parents: 8116 diff changeset	393 let s:tests = filter(s:tests, 'v:val =~ argv(1)')
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa Christian Brabandt <cb@256bit.org> parents: 8116 diff changeset	394 endif
71aabce3142e commit https://github.com/vim/vim/commit/befb366affa6309c6b4a469ec7f729821e3a36fa Christian Brabandt <cb@256bit.org> parents: 8116 diff changeset	395
18184 ef35a3a70c24 patch 8.1.2087: cannot easily select one test function to execute Bram Moolenaar <Bram@vim.org> parents: 18158 diff changeset	396 " If the environment variable $TEST_FILTER is set then filter the function
ef35a3a70c24 patch 8.1.2087: cannot easily select one test function to execute Bram Moolenaar <Bram@vim.org> parents: 18158 diff changeset	397 " names against it.
18189 b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active Bram Moolenaar <Bram@vim.org> parents: 18184 diff changeset	398 let s:filtered = 0
18184 ef35a3a70c24 patch 8.1.2087: cannot easily select one test function to execute Bram Moolenaar <Bram@vim.org> parents: 18158 diff changeset	399 if $TEST_FILTER != ''
18189 b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active Bram Moolenaar <Bram@vim.org> parents: 18184 diff changeset	400 let s:filtered = len(s:tests)
18184 ef35a3a70c24 patch 8.1.2087: cannot easily select one test function to execute Bram Moolenaar <Bram@vim.org> parents: 18158 diff changeset	401 let s:tests = filter(s:tests, 'v:val =~ $TEST_FILTER')
18189 b5bf891fa92c patch 8.1.2089: do not get a hint that $TEST_FILTER was active Bram Moolenaar <Bram@vim.org> parents: 18184 diff changeset	402 let s:filtered -= len(s:tests)
18184 ef35a3a70c24 patch 8.1.2087: cannot easily select one test function to execute Bram Moolenaar <Bram@vim.org> parents: 18158 diff changeset	403 endif
ef35a3a70c24 patch 8.1.2087: cannot easily select one test function to execute Bram Moolenaar <Bram@vim.org> parents: 18158 diff changeset	404
7540 b910bb01832a commit https://github.com/vim/vim/commit/cfc0a350a9fa04f1b0cfa1ba31fbd2847376513f Christian Brabandt <cb@256bit.org> parents: 7454 diff changeset	405 " Execute the tests in alphabetical order.
8116 3c1bdf20c8b9 commit https://github.com/vim/vim/commit/93bf558caef2d507ef6baf56eaf6025b63da1e34 Christian Brabandt <cb@256bit.org> parents: 8029 diff changeset	406 for s:test in sort(s:tests)
12616 4767939d10cc patch 8.0.1186: still quite a few old style tests Christian Brabandt <cb@256bit.org> parents: 12381 diff changeset	407 " Silence, please!
4767939d10cc patch 8.0.1186: still quite a few old style tests Christian Brabandt <cb@256bit.org> parents: 12381 diff changeset	408 set belloff=all
15036 73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	409 let prev_error = ''
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	410 let total_errors = []
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	411 let run_nr = 1
12616 4767939d10cc patch 8.0.1186: still quite a few old style tests Christian Brabandt <cb@256bit.org> parents: 12381 diff changeset	412
8370 cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	413 call RunTheTest(s:test)
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	414
15036 73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	415 " Repeat a flaky test. Give up when:
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	416 " - it fails again with the same message
17918 9606c0adc148 patch 8.1.1955: tests contain typos Bram Moolenaar <Bram@vim.org> parents: 17910 diff changeset	417 " - it fails five times (with a different message)
15040 da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message Bram Moolenaar <Bram@vim.org> parents: 15038 diff changeset	418 if len(v:errors) > 0
da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message Bram Moolenaar <Bram@vim.org> parents: 15038 diff changeset	419 \ && (index(s:flaky_tests, s:test) >= 0
da8a8cdcb8b0 patch 8.1.0531: flaky tests often fail with a common error message Bram Moolenaar <Bram@vim.org> parents: 15038 diff changeset	420 \ \|\| v:errors[0] =~ s:flaky_errors_re)
15036 73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	421 while 1
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	422 call add(s:messages, 'Found errors in ' . s:test . ':')
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	423 call extend(s:messages, v:errors)
11187 515db00c4676 patch 8.0.0480: the remote_peek() test fails on MS-Windows Christian Brabandt <cb@256bit.org> parents: 11185 diff changeset	424
15036 73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	425 call add(total_errors, 'Run ' . run_nr . ':')
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	426 call extend(total_errors, v:errors)
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	427
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	428 if run_nr == 5 \|\| prev_error == v:errors[0]
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	429 call add(total_errors, 'Flaky test failed too often, giving up')
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	430 let v:errors = total_errors
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	431 break
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	432 endif
12905 30b0d784c870 patch 8.0.1329: when a flaky test fails it also often fails the second time Christian Brabandt <cb@256bit.org> parents: 12851 diff changeset	433
15036 73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	434 call add(s:messages, 'Flaky test failed, running it again')
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	435
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	436 " Flakiness is often caused by the system being very busy. Sleep a
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	437 " couple of seconds to have a higher chance of succeeding the second
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	438 " time.
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	439 sleep 2
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	440
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	441 let prev_error = v:errors[0]
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	442 let v:errors = []
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	443 let run_nr += 1
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	444
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	445 call RunTheTest(s:test)
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	446
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	447 if len(v:errors) == 0
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	448 " Test passed on rerun.
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	449 break
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	450 endif
73fef88aa0f9 patch 8.1.0529: flaky test sometimes fails in different ways Bram Moolenaar <Bram@vim.org> parents: 15018 diff changeset	451 endwhile
8370 cd7ea16d1300 commit https://github.com/vim/vim/commit/b5760a1ce5b700fc32b8bd11948ee189a847c59e Christian Brabandt <cb@256bit.org> parents: 8142 diff changeset	452 endif
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	453
11181 13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	454 call AfterTheTest()
7277 6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	455 endfor
6600871bb38c commit https://github.com/vim/vim/commit/43345546ae63710441f066648b8485fb545b3801 Christian Brabandt <cb@256bit.org> parents: diff changeset	456
11181 13544aa85dc0 patch 8.0.0477: the client-server test may hang when failing Christian Brabandt <cb@256bit.org> parents: 11169 diff changeset	457 call FinishTesting()
10388 6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4 Christian Brabandt <cb@256bit.org> parents: 10229 diff changeset	458
6cdb0226d74a commit https://github.com/vim/vim/commit/cc28e2d05d05552d8b72a520be8a193f3d9822d4 Christian Brabandt <cb@256bit.org> parents: 10229 diff changeset	459 " vim: shiftwidth=2 sts=2 expandtab

Mercurial > vim

annotate src/testdir/runtest.vim @ 18825:70f1e352d599 v8.1.2400