Mercurial > vim

comparison README_VIM9.md @ 19181:94eda51ba9ba v8.2.0149

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
patch 8.2.0149: maintaining a Vim9 branch separately is more work Commit: https://github.com/vim/vim/commit/8a7d6542b33e5d2b352262305c3bfdb2d14e1cf8 Author: Bram Moolenaar <Bram@vim.org> Date: Sun Jan 26 15:56:19 2020 +0100 patch 8.2.0149: maintaining a Vim9 branch separately is more work Problem: Maintaining a Vim9 branch separately is more work. Solution: Merge the Vim9 script changes.
author Bram Moolenaar <Bram@vim.org>
date Sun, 26 Jan 2020 16:00:05 +0100
parents
children 1b345fb68ae3
comparison
equal deleted inserted replaced
19180:8edf0aeb71b9 19181:94eda51ba9ba
1 ![Vim Logo](https://github.com/vim/vim/blob/master/runtime/vimlogo.gif)
2
3 # What is Vim9?
4
5 This is an experimental side of [Vim](https://github.com/vim/vim).
6 It explores ways of making Vim script faster and better.
7
8 WARNING: The Vim9 script features are in the early stages of development,
9 anything can break!
10
11 # Why Vim9?
12
13 ## 1. FASTER VIM SCRIPT
14
15 The third item on the poll results of 2018, after popup windows and text
16 properties, is faster Vim script. So how do we do that?
17
18 I have been throwing some ideas around, and soon came to the conclusion
19 that the current way functions are called and executed, with
20 dictionaries for the arguments and local variables, is never going to be
21 very fast. We're lucky if we can make it twice as fast. The overhead
22 of a function call and executing every line is just too high.
23
24 So what then? We can only make something fast by having a new way of
25 defining a function, with similar but different properties of the old
26 way:
27 * Arguments are only available by name, not through the a: dictionary or
28 the a:000 list.
29 * Local variables are not available in an l: dictionary.
30 * A few more things that slow us down, such as exception handling details.
31
32 I Implemented a "proof of concept" and measured the time to run a simple
33 for loop with an addition (Justin used this example in his presentation,
34 full code is below):
35
36 ``` vim
37 let sum = 0
38 for i in range(1, 2999999)
39 let sum += i
40 endfor
41 ```
42
43 | how | time in sec |
44 | --------| -------- |
45 | Vim old | 5.018541 |
46 | Python | 0.369598 |
47 | Lua | 0.078817 |
48 | Vim new | 0.073595 |
49
50 That looks very promising! It's just one example, but it shows how much
51 we can gain, and also that Vim script can be faster than builtin
52 interfaces.
53
54 In practice the script would not do something useless as counting but change
55 the text. For example, re-indent all the lines:
56
57 ``` vim
58 let totallen = 0
59 for i in range(1, 100000)
60 call setline(i, ' ' .. getline(i))
61 let totallen += len(getline(i))
62 endfor
63 ```
64
65 | how | time in sec |
66 | --------| -------- |
67 | Vim old | 0.853752 |
68 | Python | 0.304584 |
69 | Lua | 0.286573 |
70 | Vim new | 0.190276 |
71
72 The differences are smaller, but Vim 9 script is clearly the fastest.
73
74 How does Vim9 script work? The function is first compiled into a sequence of
75 instructions. Each instruction has one or two parameters and a stack is
76 used to store intermediate results. Local variables are also on the
77 stack, space is reserved during compilation. This is a fairly normal
78 way of compilation into an intermediate format, specialized for Vim,
79 e.g. each stack item is a typeval_T. And one of the instructions is
80 "execute Ex command", for commands that are not compiled.
81
82
83 ## 2. PHASING OUT INTERFACES
84
85 Attempts have been made to implement functionality with built-in script
86 languages such as Python, Perl, Lua, Tcl and Ruby. This never gained much
87 foothold, for various reasons.
88
89 Instead of using script language support in Vim:
90 * Encourage implementing external tools in any language and communicate
91 with them. The job and channel support already makes this possible.
92 Really any language can be used, also Java and Go, which are not
93 available built-in.
94 * Phase out the built-in language interfaces, make maintenance a bit easier
95 and executables easier to build. They will be kept for backwards
96 compatibility, no new features.
97 * Improve the Vim script language, it is used to communicate with the external
98 tool and implements the Vim side of the interface. Also, it can be used when
99 an external tool is undesired.
100
101 All together this creates a clear situation: Vim with the +eval feature
102 will be sufficient for most plugins, while some plugins require
103 installing a tool that can be written in any language. No confusion
104 about having Vim but the plugin not working because some specific
105 language is missing. This is a good long term goal.
106
107 Rationale: Why is it better to run a tool separately from Vim than using a
108 built-in interface and interpreter? Take for example something that is
109 written in Python:
110 * The built-in interface uses the embedded python interpreter. This is less
111 well maintained than the python command. Building Vim with it requires
112 installing developer packages. If loaded dynamically there can be a version
113 mismatch.
114 * When running the tool externally the standard python command can be used,
115 which is quite often available by default or can be easily installed.
116 * The built-in interface has an API that is unique for Vim with Python. This is
117 an extra API to learn.
118 * A .py file can be compiled into a .pyc file and execute much faster.
119 * Inside Vim multi-threading can cause problems, since the Vim core is single
120 threaded. In an external tool there are no such problems.
121 * The Vim part is written in .vim files, the Python part is in .py files, this
122 is nicely separated.
123 * Disadvantage: An interface needs to be made between Vim and Python.
124 JSON is available for this, and it's fairly easy to use. But it still
125 requires implementing asynchronous communication.
126
127
128 ## 3. BETTER VIM SCRIPT
129
130 To make Vim faster a new way of defining a function needs to be added.
131 While we are doing that, since the lines in this function won't be fully
132 backwards compatible anyway, we can also make Vim script easier to use.
133 In other words: "less weird". Making it work more like modern
134 programming languages will help. No surprises.
135
136 A good example is how in a function the arguments are prefixed with
137 "a:". No other language I know does that, so let's drop it.
138
139 Taking this one step further is also dropping "s:" for script-local variables;
140 everything at the script level is script-local by default. Since this is not
141 backwards compatible it requires a new script style: Vim9 script!
142
143 It should be possible to convert code from other languages to Vim
144 script. We can add functionality to make this easier. This still needs
145 to be discussed, but we can consider adding type checking and a simple
146 form of classes. If you look at JavaScript for example, it has gone
147 through these stages over time, adding real class support and now
148 TypeScript adds type checking. But we'll have to see how much of that
149 we actually want to include in Vim script. Ideally a conversion tool
150 can take Python, JavaScript or TypeScript code and convert it to Vim
151 script, with only some things that cannot be converted.
152
153 Vim script won't work the same as any specific language, but we can use
154 mechanisms that are commonly known, ideally with the same syntax. One
155 thing I have been thinking of is assignments without ":let". I often
156 make that mistake (after writing JavaScript especially). I think it is
157 possible, if we make local variables shadow commands. That should be OK,
158 if you shadow a command you want to use, just rename the variable.
159 Using "let" and "const" to declare a variable, like in JavaScript and
160 TypeScript, can work:
161
162
163 ``` vim
164 def MyFunction(arg: number): number
165 let local = 1
166 let todo = arg
167 const ADD = 88
168 while todo > 0
169 local += ADD
170 --todo
171 endwhile
172 return local
173 enddef
174 ```
175
176 The similarity with JavaScript/TypeScript can also be used for dependencies
177 between files. Vim currently uses the `:source` command, which has several
178 disadvantages:
179 * In the sourced script, is not clear what it provides. By default all
180 functions are global and can be used elsewhere.
181 * In a script that sources other scripts, it is not clear what function comes
182 from what sourced script. Finding the implementation is a hassle.
183 * Prevention of loading the whole script twice must be manually implemented.
184
185 We can use the `:import` and `:export` commands from the JavaScript standard to
186 make this much better. For example, in script "myfunction.vim" define a
187 function and export it:
188
189 ``` vim
190 vim9script " Vim9 script syntax used here
191
192 let local = 'local variable is not exported, script-local'
193
194 export def MyFunction() " exported function
195 ...
196
197 def LocalFunction() " not exported, script-local
198 ...
199 ```
200
201 And in another script import the function:
202
203 ``` vim
204 vim9script " Vim9 script syntax used here
205
206 import MyFunction from 'myfunction.vim'
207 ```
208
209 This looks like JavaScript/TypeScript, thus many users will understand the
210 syntax.
211
212 These are ideas, this will take time to design, discuss and implement.
213 Eventually this will lead to Vim 9!
214
215
216 ## Code for sum time measurements
217
218 Vim was build with -O2.
219
220 ``` vim
221 func VimOld()
222 let sum = 0
223 for i in range(1, 2999999)
224 let sum += i
225 endfor
226 return sum
227 endfunc
228
229 func Python()
230 py3 << END
231 sum = 0
232 for i in range(1, 3000000):
233 sum += i
234 END
235 return py3eval('sum')
236 endfunc
237
238 func Lua()
239 lua << END
240 sum = 0
241 for i = 1, 2999999 do
242 sum = sum + i
243 end
244 END
245 return luaeval('sum')
246 endfunc
247
248 def VimNew()
249 let sum = 0
250 for i in range(1, 2999999)
251 let sum += i
252 endfor
253 return sum
254 enddef
255
256 let start = reltime()
257 echo VimOld()
258 echo 'Vim old: ' .. reltimestr(reltime(start))
259
260 let start = reltime()
261 echo Python()
262 echo 'Python: ' .. reltimestr(reltime(start))
263
264 let start = reltime()
265 echo Lua()
266 echo 'Lua: ' .. reltimestr(reltime(start))
267
268 let start = reltime()
269 echo VimNew()
270 echo 'Vim new: ' .. reltimestr(reltime(start))
271 ```
272
273 ## Code for indent time measurements
274
275 ``` vim
276 def VimNew(): number
277 let totallen = 0
278 for i in range(1, 100000)
279 setline(i, ' ' .. getline(i))
280 totallen += len(getline(i))
281 endfor
282 return totallen
283 enddef
284
285 func VimOld()
286 let totallen = 0
287 for i in range(1, 100000)
288 call setline(i, ' ' .. getline(i))
289 let totallen += len(getline(i))
290 endfor
291 return totallen
292 endfunc
293
294 func Lua()
295 lua << END
296 b = vim.buffer()
297 totallen = 0
298 for i = 1, 100000 do
299 b[i] = " " .. b[i]
300 totallen = totallen + string.len(b[i])
301 end
302 END
303 return luaeval('totallen')
304 endfunc
305
306 func Python()
307 py3 << END
308 cb = vim.current.buffer
309 totallen = 0
310 for i in range(0, 100000):
311 cb[i] = ' ' + cb[i]
312 totallen += len(cb[i])
313 END
314 return py3eval('totallen')
315 endfunc
316
317 new
318 call setline(1, range(100000))
319 let start = reltime()
320 echo VimOld()
321 echo 'Vim old: ' .. reltimestr(reltime(start))
322 bwipe!
323
324 new
325 call setline(1, range(100000))
326 let start = reltime()
327 echo Python()
328 echo 'Python: ' .. reltimestr(reltime(start))
329 bwipe!
330
331 new
332 call setline(1, range(100000))
333 let start = reltime()
334 echo Lua()
335 echo 'Lua: ' .. reltimestr(reltime(start))
336 bwipe!
337
338 new
339 call setline(1, range(100000))
340 let start = reltime()
341 echo VimNew()
342 echo 'Vim new: ' .. reltimestr(reltime(start))
343 bwipe!
344 ```