Mercurial > vim
annotate src/testdir/README.txt @ 33776:9503dc55b5ed v9.0.2108
patch 9.0.2108: [security]: overflow with count for :s command
Commit: https://github.com/vim/vim/commit/ac63787734fda2e294e477af52b3bd601517fa78
Author: Christian Brabandt <cb@256bit.org>
Date: Tue Nov 14 20:45:48 2023 +0100
patch 9.0.2108: [security]: overflow with count for :s command
Problem: [security]: overflow with count for :s command
Solution: Abort the :s command if the count is too large
If the count after the :s command is larger than what fits into a
(signed) long variable, abort with e_value_too_large.
Adds a test with INT_MAX as count and verify it correctly fails.
It seems the return value on Windows using mingw compiler wraps around,
so the initial test using :s/./b/9999999999999999999999999990 doesn't
fail there, since the count is wrapping around several times and finally
is no longer larger than 2147483647. So let's just use 2147483647 in the
test, which hopefully will always cause a failure
Signed-off-by: Christian Brabandt <cb@256bit.org>
author | Christian Brabandt <cb@256bit.org> |
---|---|
date | Thu, 16 Nov 2023 22:15:10 +0100 |
parents | ba9d53c7c509 |
children | 043b537c852b |
rev | line source |
---|---|
6168 | 1 This directory contains tests for various Vim features. |
15430
d94901eeb762
patch 8.1.0723: cannot easily run specific test when in src/testdir
Bram Moolenaar <Bram@vim.org>
parents:
15034
diff
changeset
|
2 For testing an indent script see runtime/indent/testdir/README.txt. |
6168 | 3 |
7517
9d67399f49c6
commit https://github.com/vim/vim/commit/6602af7fe069246dbcf419c3e904a78b60e7d4dc
Christian Brabandt <cb@256bit.org>
parents:
7478
diff
changeset
|
4 If it makes sense, add a new test method to an already existing file. You may |
9d67399f49c6
commit https://github.com/vim/vim/commit/6602af7fe069246dbcf419c3e904a78b60e7d4dc
Christian Brabandt <cb@256bit.org>
parents:
7478
diff
changeset
|
5 want to separate it from other tests with comment lines. |
6168 | 6 |
7517
9d67399f49c6
commit https://github.com/vim/vim/commit/6602af7fe069246dbcf419c3e904a78b60e7d4dc
Christian Brabandt <cb@256bit.org>
parents:
7478
diff
changeset
|
7 TO ADD A NEW STYLE TEST: |
9d67399f49c6
commit https://github.com/vim/vim/commit/6602af7fe069246dbcf419c3e904a78b60e7d4dc
Christian Brabandt <cb@256bit.org>
parents:
7478
diff
changeset
|
8 |
9d67399f49c6
commit https://github.com/vim/vim/commit/6602af7fe069246dbcf419c3e904a78b60e7d4dc
Christian Brabandt <cb@256bit.org>
parents:
7478
diff
changeset
|
9 1) Create a test_<subject>.vim file. |
15430
d94901eeb762
patch 8.1.0723: cannot easily run specific test when in src/testdir
Bram Moolenaar <Bram@vim.org>
parents:
15034
diff
changeset
|
10 2) Add test_<subject>.res to NEW_TESTS_RES in Make_all.mak in alphabetical |
d94901eeb762
patch 8.1.0723: cannot easily run specific test when in src/testdir
Bram Moolenaar <Bram@vim.org>
parents:
15034
diff
changeset
|
11 order. |
d94901eeb762
patch 8.1.0723: cannot easily run specific test when in src/testdir
Bram Moolenaar <Bram@vim.org>
parents:
15034
diff
changeset
|
12 3) Also add an entry "test_<subject>" to NEW_TESTS in Make_all.mak. |
d94901eeb762
patch 8.1.0723: cannot easily run specific test when in src/testdir
Bram Moolenaar <Bram@vim.org>
parents:
15034
diff
changeset
|
13 4) Use make test_<subject> to run a single test. |
6168 | 14 |
14689
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
15 At 2), instead of running the test separately, it can be included in |
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
16 "test_alot". Do this for quick tests without side effects. The test runs a |
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
17 bit faster, because Vim doesn't have to be started, one Vim instance runs many |
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
18 tests. |
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
19 |
19465
07627d1b4d8c
patch 8.2.0290: running individual test differs from all tests
Bram Moolenaar <Bram@vim.org>
parents:
16087
diff
changeset
|
20 At 4), to run a test in GUI, add "GUI_FLAG=-g" to the make command. |
07627d1b4d8c
patch 8.2.0290: running individual test differs from all tests
Bram Moolenaar <Bram@vim.org>
parents:
16087
diff
changeset
|
21 |
14689
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
22 |
7517
9d67399f49c6
commit https://github.com/vim/vim/commit/6602af7fe069246dbcf419c3e904a78b60e7d4dc
Christian Brabandt <cb@256bit.org>
parents:
7478
diff
changeset
|
23 What you can use (see test_assert.vim for an example): |
14689
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
24 |
10100
26441931dad7
commit https://github.com/vim/vim/commit/dac1947bb366ef43cd6da95acc730554e76d8b84
Christian Brabandt <cb@256bit.org>
parents:
9909
diff
changeset
|
25 - Call assert_equal(), assert_true(), assert_false(), etc. |
14689
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
26 |
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
27 - Use assert_fails() to check for expected errors. |
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
28 |
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
29 - Use try/catch to avoid an exception aborts the test. |
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
30 |
16087
e992f31274b7
patch 8.1.1048: minor issues with tests
Bram Moolenaar <Bram@vim.org>
parents:
15430
diff
changeset
|
31 - Use test_alloc_fail() to have memory allocation fail. This makes it possible |
e992f31274b7
patch 8.1.1048: minor issues with tests
Bram Moolenaar <Bram@vim.org>
parents:
15430
diff
changeset
|
32 to check memory allocation failures are handled gracefully. You need to |
e992f31274b7
patch 8.1.1048: minor issues with tests
Bram Moolenaar <Bram@vim.org>
parents:
15430
diff
changeset
|
33 change the source code to add an ID to the allocation. Add a new one to |
e992f31274b7
patch 8.1.1048: minor issues with tests
Bram Moolenaar <Bram@vim.org>
parents:
15430
diff
changeset
|
34 alloc_id_T, before aid_last. |
14689
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
35 |
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
36 - Use test_override() to make Vim behave differently, e.g. if char_avail() |
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
37 must return FALSE for a while. E.g. to trigger the CursorMovedI autocommand |
16087
e992f31274b7
patch 8.1.1048: minor issues with tests
Bram Moolenaar <Bram@vim.org>
parents:
15430
diff
changeset
|
38 event. See test_cursor_func.vim for an example. |
14689
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
39 |
10100
26441931dad7
commit https://github.com/vim/vim/commit/dac1947bb366ef43cd6da95acc730554e76d8b84
Christian Brabandt <cb@256bit.org>
parents:
9909
diff
changeset
|
40 - If the bug that is being tested isn't fixed yet, you can throw an exception |
14689
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
41 with "Skipped" so that it's clear this still needs work. E.g.: throw |
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
42 "Skipped: Bug with <c-e> and popupmenu not fixed yet" |
796d794cdc88
patch 8.1.0357: instructions for tests are outdated
Christian Brabandt <cb@256bit.org>
parents:
13304
diff
changeset
|
43 |
28542
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
44 - The following environment variables are recognized and can be set to |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
45 influence the behavior of the test suite (see runtest.vim for details) |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
46 |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
47 - $TEST_MAY_FAIL=Test_channel_one - ignore those failing tests |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
48 - $TEST_FILTER=Test_channel - only run test that match this pattern |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
49 - $TEST_SKIP_PAT=Test_channel - skip tests that match this pattern |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
50 - $TEST_NO_RETRY=yes - do not try to re-run failing tests |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
51 You can also set them in Vim: |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
52 :let $TEST_MAY_FAIL = 'Test_channel_one' |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
53 :let $TEST_FILTER = '_set_mode' |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
54 :let $TEST_SKIP_PAT = 'Test_loop_forever' |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
55 :let $TEST_NO_RETRY = 'yes' |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
56 Use an empty string to revert, e.g.: |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
57 :let $TEST_FILTER = '' |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
58 |
9228
ea504064c996
commit https://github.com/vim/vim/commit/fd89d7ea81b18d32363456b16258174dc9e095dc
Christian Brabandt <cb@256bit.org>
parents:
8011
diff
changeset
|
59 - See the start of runtest.vim for more help. |
7517
9d67399f49c6
commit https://github.com/vim/vim/commit/6602af7fe069246dbcf419c3e904a78b60e7d4dc
Christian Brabandt <cb@256bit.org>
parents:
7478
diff
changeset
|
60 |
9d67399f49c6
commit https://github.com/vim/vim/commit/6602af7fe069246dbcf419c3e904a78b60e7d4dc
Christian Brabandt <cb@256bit.org>
parents:
7478
diff
changeset
|
61 |
13304
013c44d9dc09
patch 8.0.1526: no test using a screen dump yet
Christian Brabandt <cb@256bit.org>
parents:
10549
diff
changeset
|
62 TO ADD A SCREEN DUMP TEST: |
013c44d9dc09
patch 8.0.1526: no test using a screen dump yet
Christian Brabandt <cb@256bit.org>
parents:
10549
diff
changeset
|
63 |
15034
6e4e0d43b20b
patch 8.1.0528: various typos in comments
Bram Moolenaar <Bram@vim.org>
parents:
14689
diff
changeset
|
64 Mostly the same as writing a new style test. Additionally, see help on |
13304
013c44d9dc09
patch 8.0.1526: no test using a screen dump yet
Christian Brabandt <cb@256bit.org>
parents:
10549
diff
changeset
|
65 "terminal-dumptest". Put the reference dump in "dumps/Test_func_name.dump". |
013c44d9dc09
patch 8.0.1526: no test using a screen dump yet
Christian Brabandt <cb@256bit.org>
parents:
10549
diff
changeset
|
66 |
21996
808edde1e97d
patch 8.2.1547: various comment problems
Bram Moolenaar <Bram@vim.org>
parents:
21741
diff
changeset
|
67 |
808edde1e97d
patch 8.2.1547: various comment problems
Bram Moolenaar <Bram@vim.org>
parents:
21741
diff
changeset
|
68 OLD STYLE TESTS: |
808edde1e97d
patch 8.2.1547: various comment problems
Bram Moolenaar <Bram@vim.org>
parents:
21741
diff
changeset
|
69 |
808edde1e97d
patch 8.2.1547: various comment problems
Bram Moolenaar <Bram@vim.org>
parents:
21741
diff
changeset
|
70 There are a few tests that are used when Vim was built without the +eval |
808edde1e97d
patch 8.2.1547: various comment problems
Bram Moolenaar <Bram@vim.org>
parents:
21741
diff
changeset
|
71 feature. These cannot use the "assert" functions, therefore they consist of a |
808edde1e97d
patch 8.2.1547: various comment problems
Bram Moolenaar <Bram@vim.org>
parents:
21741
diff
changeset
|
72 .in file that contains Normal mode commands between STARTTEST and ENDTEST. |
27459 | 73 They modify the file and the result gets written in the test.out file. This |
21996
808edde1e97d
patch 8.2.1547: various comment problems
Bram Moolenaar <Bram@vim.org>
parents:
21741
diff
changeset
|
74 is then compared with the .ok file. If they are equal the test passed. If |
808edde1e97d
patch 8.2.1547: various comment problems
Bram Moolenaar <Bram@vim.org>
parents:
21741
diff
changeset
|
75 they differ the test failed. |
26591 | 76 |
77 | |
78 RUNNING THE TESTS: | |
79 | |
80 To run a single test from the src directory: | |
81 | |
82 $ make test_<name> | |
83 | |
84 The below commands should be run from the src/testdir directory. | |
85 | |
86 To run a single test: | |
87 | |
88 $ make test_<name>.res | |
89 | |
90 The file 'messages' contains the messages generated by the test script. If a | |
91 test fails, then the test.log file contains the error messages. If all the | |
92 tests are successful, then this file will be an empty file. | |
93 | |
28542
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
94 - To run a single test function from a test script: |
26591 | 95 |
96 $ ../vim -u NONE -S runtest.vim <test_file>.vim <function_name> | |
97 | |
28542
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
98 - To execute only specific test functions, add a second argument: |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
99 |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
100 $ ../vim -u NONE -S runtest.vim test_channel.vim open_delay |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
101 |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
102 |
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
103 - To run all the tests: |
26591 | 104 |
105 $ make | |
106 | |
28542
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
107 - To run the test on MS-Windows using the MSVC nmake: |
26591 | 108 |
30170
ba9d53c7c509
patch 9.0.0421: MS-Windows makefiles are inconsistently named
Bram Moolenaar <Bram@vim.org>
parents:
28542
diff
changeset
|
109 > nmake -f Make_mvc.mak |
26591 | 110 |
28542
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
111 - To run the tests with GUI Vim: |
26591 | 112 |
113 $ make GUI_FLAG=-g | |
114 | |
115 or | |
116 | |
117 $ make VIMPROG=../gvim | |
118 | |
28542
51b5cab35afa
patch 8.2.4795: 'cursorbind' scrolling depends on whether 'cursorline' is set
Bram Moolenaar <Bram@vim.org>
parents:
27459
diff
changeset
|
119 - To cleanup the temporary files after running the tests: |
26591 | 120 |
121 $ make clean |