Mercurial > vim
view runtime/doc/usr_29.txt @ 32936:c517845bd10e v9.0.1776
patch 9.0.1776: No support for stable Python 3 ABI
Commit: https://github.com/vim/vim/commit/c13b3d1350b60b94fe87f0761ea31c0e7fb6ebf3
Author: Yee Cheng Chin <ychin.git@gmail.com>
Date: Sun Aug 20 21:18:38 2023 +0200
patch 9.0.1776: No support for stable Python 3 ABI
Problem: No support for stable Python 3 ABI
Solution: Support Python 3 stable ABI
Commits:
1) Support Python 3 stable ABI to allow mixed version interoperatbility
Vim currently supports embedding Python for use with plugins, and the
"dynamic" linking option allows the user to specify a locally installed
version of Python by setting `pythonthreedll`. However, one caveat is
that the Python 3 libs are not binary compatible across minor versions,
and mixing versions can potentially be dangerous (e.g. let's say Vim was
linked against the Python 3.10 SDK, but the user sets `pythonthreedll`
to a 3.11 lib). Usually, nothing bad happens, but in theory this could
lead to crashes, memory corruption, and other unpredictable behaviors.
It's also difficult for the user to tell something is wrong because Vim
has no way of reporting what Python 3 version Vim was linked with.
For Vim installed via a package manager, this usually isn't an issue
because all the dependencies would already be figured out. For prebuilt
Vim binaries like MacVim (my motivation for working on this), AppImage,
and Win32 installer this could potentially be an issue as usually a
single binary is distributed. This is more tricky when a new Python
version is released, as there's a chicken-and-egg issue with deciding
what Python version to build against and hard to keep in sync when a new
Python version just drops and we have a mix of users of different Python
versions, and a user just blindly upgrading to a new Python could lead to
bad interactions with Vim.
Python 3 does have a solution for this problem: stable ABI / limited API
(see https://docs.python.org/3/c-api/stable.html). The C SDK limits the
API to a set of functions that are promised to be stable across
versions. This pull request adds an ifdef config that allows us to turn
it on when building Vim. Vim binaries built with this option should be
safe to freely link with any Python 3 libraies without having the
constraint of having to use the same minor version.
Note: Python 2 has no such concept and this doesn't change how Python 2
integration works (not that there is going to be a new version of Python
2 that would cause compatibility issues in the future anyway).
---
Technical details:
======
The stable ABI can be accessed when we compile with the Python 3 limited
API (by defining `Py_LIMITED_API`). The Python 3 code (in `if_python3.c`
and `if_py_both.h`) would now handle this and switch to limited API
mode. Without it set, Vim will still use the full API as before so this
is an opt-in change.
The main difference is that `PyType_Object` is now an opaque struct that
we can't directly create "static types" out of, and we have to create
type objects as "heap types" instead. This is because the struct is not
stable and changes from version to version (e.g. 3.8 added a
`tp_vectorcall` field to it). I had to change all the types to be
allocated on the heap instead with just a pointer to them.
Other functions are also simply missing in limited API, or they are
introduced too late (e.g. `PyUnicode_AsUTF8AndSize` in 3.10) to it that
we need some other ways to do the same thing, so I had to abstract a few
things into macros, and sometimes re-implement functions like
`PyObject_NEW`.
One caveat is that in limited API, `OutputType` (used for replacing
`sys.stdout`) no longer inherits from `PyStdPrinter_Type` which I don't
think has any real issue other than minor differences in how they
convert to a string and missing a couple functions like `mode()` and
`fileno()`.
Also fixed an existing bug where `tp_basicsize` was set incorrectly for
`BufferObject`, `TabListObject, `WinListObject`.
Technically, there could be a small performance drop, there is a little
more indirection with accessing type objects, and some APIs like
`PyUnicode_AsUTF8AndSize` are missing, but in practice I didn't see any
difference, and any well-written Python plugin should try to avoid
excessing callbacks to the `vim` module in Python anyway.
I only tested limited API mode down to Python 3.7, which seemes to
compile and work fine. I haven't tried earlier Python versions.
2) Fix PyIter_Check on older Python vers / type##Ptr unused warning
For PyIter_Check, older versions exposed them as either macros (used in
full API), or a function (for use in limited API). A previous change
exposed PyIter_Check to the dynamic build because Python just moved it
to function-only in 3.10 anyway. Because of that, just make sure we
always grab the function in dynamic builds in earlier versions since
that's what Python eventually did anyway.
3) Move Py_LIMITED_API define to configure script
Can now use --with-python-stable-abi flag to customize what stable ABI
version to target. Can also use an env var to do so as well.
4) Show +python/dyn-stable in :version, and allow has() feature query
Not sure if the "/dyn-stable" suffix would break things, or whether we
should do it another way. Or just don't show it in version and rely on
has() feature checking.
5) Documentation first draft. Still need to implement v:python3_version
6) Fix PyIter_Check build breaks when compiling against Python 3.8
7) Add CI coverage stable ABI on Linux/Windows / make configurable on Windows
This adds configurable options for Windows make files (both MinGW and
MSVC). CI will also now exercise both traditional full API and stable
ABI for Linux and Windows in the matrix for coverage.
Also added a "dynamic" option to Linux matrix as a drive-by change to
make other scripting languages like Ruby / Perl testable under both
static and dynamic builds.
8) Fix inaccuracy in Windows docs
Python's own docs are confusing but you don't actually want to use
`python3.dll` for the dynamic linkage.
9) Add generated autoconf file
10) Add v:python3_version support
This variable indicates the version of Python3 that Vim was built
against (PY_VERSION_HEX), and will be useful to check whether the Python
library you are loading in dynamically actually fits it. When built with
stable ABI, it will be the limited ABI version instead
(`Py_LIMITED_API`), which indicates the minimum version of Python 3 the
user should have, rather than the exact match. When stable ABI is used,
we won't be exposing PY_VERSION_HEX in this var because it just doesn't
seem necessary to do so (the whole point of stable ABI is the promise
that it will work across versions), and I don't want to confuse the user
with too many variables.
Also, cleaned up some documentation, and added help tags.
11) Fix Python 3.7 compat issues
Fix a couple issues when using limited API < 3.8
- Crash on exit: In Python 3.7, if a heap-allocated type is destroyed
before all instances are, it would cause a crash later. This happens
when we destroyed `OptionsType` before calling `Py_Finalize` when
using the limited API. To make it worse, later versions changed the
semantics and now each instance has a strong reference to its own type
and the recommendation has changed to have each instance de-ref its
own type and have its type in GC traversal. To avoid dealing with
these cross-version variations, we just don't free the heap type. They
are static types in non-limited-API anyway and are designed to last
through the entirety of the app, and we also don't restart the Python
runtime and therefore do not need it to have absolutely 0 leaks.
See:
- https://docs.python.org/3/whatsnew/3.8.html#changes-in-the-c-api
- https://docs.python.org/3/whatsnew/3.9.html#changes-in-the-c-api
- PyIter_Check: This function is not provided in limited APIs older than
3.8. Previously I was trying to mock it out using manual
PyType_GetSlot() but it was brittle and also does not actually work
properly for static types (it will generate a Python error). Just
return false. It does mean using limited API < 3.8 is not recommended
as you lose the functionality to handle iterators, but from playing
with plugins I couldn't find it to be an issue.
- Fix loading of PyIter_Check so it will be done when limited API < 3.8.
Otherwise loading a 3.7 Python lib will fail even if limited API was
specified to use it.
12) Make sure to only load `PyUnicode_AsUTF8AndSize` in needed in limited API
We don't use this function unless limited API >= 3.10, but we were
loading it regardless. Usually it's ok in Unix-like systems where Python
just has a single lib that we load from, but in Windows where there is a
separate python3.dll this would not work as the symbol would not have
been exposed in this more limited DLL file. This makes it much clearer
under what condition is this function needed.
closes: #12032
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Yee Cheng Chin <ychin.git@gmail.com>
author | Christian Brabandt <cb@256bit.org> |
---|---|
date | Sun, 20 Aug 2023 21:30:04 +0200 |
parents | f8116058ca76 |
children | 4635e43f2c6f |
line wrap: on
line source
*usr_29.txt* For Vim version 9.0. Last change: 2022 Mar 13 VIM USER MANUAL - by Bram Moolenaar Moving through programs The creator of Vim is a computer programmer. It's no surprise that Vim contains many features to aid in writing programs. Jump around to find where identifiers are defined and used. Preview declarations in a separate window. There is more in the next chapter. |29.1| Using tags |29.2| The preview window |29.3| Moving through a program |29.4| Finding global identifiers |29.5| Finding local identifiers Next chapter: |usr_30.txt| Editing programs Previous chapter: |usr_28.txt| Folding Table of contents: |usr_toc.txt| ============================================================================== *29.1* Using tags What is a tag? It is a location where an identifier is defined. An example is a function definition in a C or C++ program. A list of tags is kept in a tags file. This can be used by Vim to directly jump from any place to the tag, the place where an identifier is defined. To generate the tags file for all C files in the current directory, use the following command: > ctags *.c "ctags" is a separate program. Most Unix systems already have it installed. If you do not have it yet, you can find Universal/Exuberant ctags at: http://ctags.io ~ http://ctags.sf.net ~ Universal ctags is preferred, Exuberant ctags is no longer being developed. Now when you are in Vim and you want to go to a function definition, you can jump to it by using the following command: > :tag startlist This command will find the function "startlist" even if it is in another file. The CTRL-] command jumps to the tag of the word that is under the cursor. This makes it easy to explore a tangle of C code. Suppose, for example, that you are in the function "write_block". You can see that it calls "write_line". But what does "write_line" do? By placing the cursor on the call to "write_line" and pressing CTRL-], you jump to the definition of this function. The "write_line" function calls "write_char". You need to figure out what it does. So you position the cursor over the call to "write_char" and press CTRL-]. Now you are at the definition of "write_char". +-------------------------------------+ |void write_block(char **s; int cnt) | |{ | | int i; | | for (i = 0; i < cnt; ++i) | | write_line(s[i]); | |} | | +-----------|-------------------------+ | CTRL-] | | +----------------------------+ +--> |void write_line(char *s) | |{ | | while (*s != 0) | | write_char(*s++); | |} | | +--------|-------------------+ | CTRL-] | | +------------------------------------+ +--> |void write_char(char c) | |{ | | putchar((int)(unsigned char)c); | |} | +------------------------------------+ The ":tags" command shows the list of tags that you traversed through: :tags # TO tag FROM line in file/text ~ 1 1 write_line 8 write_block.c ~ 2 1 write_char 7 write_line.c ~ > ~ > Now to go back. The CTRL-T command goes to the preceding tag. In the example above you get back to the "write_line" function, in the call to "write_char". This command takes a count argument that indicates how many tags to jump back. You have gone forward, and now back. Let's go forward again. The following command goes to the tag on top of the list: > :tag You can prefix it with a count and jump forward that many tags. For example: ":3tag". CTRL-T also can be preceded with a count. These commands thus allow you to go down a call tree with CTRL-] and back up again with CTRL-T. Use ":tags" to find out where you are. SPLIT WINDOWS The ":tag" command replaces the file in the current window with the one containing the new function. But suppose you want to see not only the old function but also the new one? You can split the window using the ":split" command followed by the ":tag" command. Vim has a shorthand command that does both: > :stag tagname To split the current window and jump to the tag under the cursor use this command: > CTRL-W ] If a count is specified, the new window will be that many lines high. MORE TAGS FILES When you have files in many directories, you can create a tags file in each of them. Vim will then only be able to jump to tags within that directory. To find more tags files, set the 'tags' option to include all the relevant tags files. Example: > :set tags=./tags,./../tags,./*/tags This finds a tags file in the same directory as the current file, one directory level higher and in all subdirectories. This is quite a number of tags files, but it may still not be enough. For example, when editing a file in "~/proj/src", you will not find the tags file "~/proj/sub/tags". For this situation Vim offers to search a whole directory tree for tags files. Example: > :set tags=~/proj/**/tags ONE TAGS FILE When Vim has to search many places for tags files, you can hear the disk rattling. It may get a bit slow. In that case it's better to spend this time while generating one big tags file. You might do this overnight. This requires the Universal or Exuberant ctags program, mentioned above. It offers an argument to search a whole directory tree: > cd ~/proj ctags -R . The nice thing about this is that Universal/Exuberant ctags recognizes various file types. Thus this doesn't work just for C and C++ programs, also for Eiffel and even Vim scripts. See the ctags documentation to tune this. Now you only need to tell Vim where your big tags file is: > :set tags=~/proj/tags MULTIPLE MATCHES When a function is defined multiple times (or a method in several classes), the ":tag" command will jump to the first one. If there is a match in the current file, that one is used first. You can now jump to other matches for the same tag with: > :tnext Repeat this to find further matches. If there are many, you can select which one to jump to: > :tselect tagname Vim will present you with a list of choices: # pri kind tag file ~ 1 F f mch_init os_amiga.c ~ mch_init() ~ 2 F f mch_init os_mac.c ~ mch_init() ~ 3 F f mch_init os_msdos.c ~ mch_init(void) ~ 4 F f mch_init os_riscos.c ~ mch_init() ~ Enter nr of choice (<CR> to abort): ~ You can now enter the number (in the first column) of the match that you would like to jump to. The information in the other columns give you a good idea of where the match is defined. To move between the matching tags, these commands can be used: :tfirst go to first match :[count]tprevious go to [count] previous match :[count]tnext go to [count] next match :tlast go to last match If [count] is omitted then one is used. GUESSING TAG NAMES Command line completion is a good way to avoid typing a long tag name. Just type the first bit and press <Tab>: > :tag write_<Tab> You will get the first match. If it's not the one you want, press <Tab> until you find the right one. Sometimes you only know part of the name of a function. Or you have many tags that start with the same string, but end differently. Then you can tell Vim to use a pattern to find the tag. Suppose you want to jump to a tag that contains "block". First type this: > :tag /block Now use command line completion: press <Tab>. Vim will find all tags that contain "block" and use the first match. The "/" before a tag name tells Vim that what follows is not a literal tag name, but a pattern. You can use all the items for search patterns here. For example, suppose you want to select a tag that starts with "write_": > :tselect /^write_ The "^" specifies that the tag starts with "write_". Otherwise it would also be found halfway a tag name. Similarly "$" at the end makes sure the pattern matches until the end of a tag. A TAGS BROWSER Since CTRL-] takes you to the definition of the identifier under the cursor, you can use a list of identifier names as a table of contents. Here is an example. First create a list of identifiers (this requires Universal or Exuberant ctags): > ctags --c-types=f -f functions *.c Now start Vim without a file, and edit this file in Vim, in a vertically split window: > vim :vsplit functions The window contains a list of all the functions. There is some more stuff, but you can ignore that. Do ":setlocal ts=99" to clean it up a bit. In this window, define a mapping: > :nnoremap <buffer> <CR> 0ye<C-W>w:tag <C-R>"<CR> Move the cursor to the line that contains the function you want to go to. Now press <Enter>. Vim will go to the other window and jump to the selected function. RELATED ITEMS To make case in tag names be ignored, you can set 'ignorecase' while leaving 'tagcase' as "followic", or set 'tagcase' to "ignore". The 'tagbsearch' option tells if the tags file is sorted or not. The default is to assume a sorted tags file, which makes a tags search a lot faster, but doesn't work if the tags file isn't sorted. The 'taglength' option can be used to tell Vim the number of significant characters in a tag. Cscope is a free program. It does not only find places where an identifier is declared, but also where it is used. See |cscope|. ============================================================================== *29.2* The preview window When you edit code that contains a function call, you need to use the correct arguments. To know what values to pass you can look at how the function is defined. The tags mechanism works very well for this. Preferably the definition is displayed in another window. For this the preview window can be used. To open a preview window to display the function "write_char": > :ptag write_char Vim will open a window, and jumps to the tag "write_char". Then it takes you back to the original position. Thus you can continue typing without the need to use a CTRL-W command. If the name of a function appears in the text, you can get its definition in the preview window with: > CTRL-W } There is a script that automatically displays the text where the word under the cursor was defined. See |CursorHold-example|. To close the preview window use this command: > :pclose To edit a specific file in the preview window, use ":pedit". This can be useful to edit a header file, for example: > :pedit defs.h Finally, ":psearch" can be used to find a word in the current file and any included files and display the match in the preview window. This is especially useful when using library functions, for which you do not have a tags file. Example: > :psearch popen This will show the "stdio.h" file in the preview window, with the function prototype for popen(): FILE *popen __P((const char *, const char *)); ~ You can specify the height of the preview window, when it is opened, with the 'previewheight' option. ============================================================================== *29.3* Moving through a program Since a program is structured, Vim can recognize items in it. Specific commands can be used to move around. C programs often contain constructs like this: #ifdef USE_POPEN ~ fd = popen("ls", "r") ~ #else ~ fd = fopen("tmp", "w") ~ #endif ~ But then much longer, and possibly nested. Position the cursor on the "#ifdef" and press %. Vim will jump to the "#else". Pressing % again takes you to the "#endif". Another % takes you to the "#ifdef" again. When the construct is nested, Vim will find the matching items. This is a good way to check if you didn't forget an "#endif". When you are somewhere inside a "#if" - "#endif", you can jump to the start of it with: > [# If you are not after a "#if" or "#ifdef" Vim will beep. To jump forward to the next "#else" or "#endif" use: > ]# These two commands skip any "#if" - "#endif" blocks that they encounter. Example: #if defined(HAS_INC_H) ~ a = a + inc(); ~ # ifdef USE_THEME ~ a += 3; ~ # endif ~ set_width(a); ~ With the cursor in the last line, "[#" moves to the first line. The "#ifdef" - "#endif" block in the middle is skipped. MOVING IN CODE BLOCKS In C code blocks are enclosed in {}. These can get pretty long. To move to the start of the outer block use the "[[" command. Use "][" to find the end. This assumes that the "{" and "}" are in the first column. The "[{" command moves to the start of the current block. It skips over pairs of {} at the same level. "]}" jumps to the end. An overview: function(int a) +-> { | if (a) | +-> { [[ | | for (;;) --+ | | +-> { | | [{ | | foo(32); | --+ | | [{ | if (bar(a)) --+ | ]} | +-- | +-- break; | ]} | | | } <-+ | | ][ +-- foobar(a) | | } <-+ | } <-+ When writing C++ or Java, the outer {} block is for the class. The next level of {} is for a method. When somewhere inside a class use "[m" to find the previous start of a method. "]m" finds the next start of a method. Additionally, "[]" moves backward to the end of a function and "]]" moves forward to the start of the next function. The end of a function is defined by a "}" in the first column. int func1(void) { return 1; +----------> } | [] | int func2(void) | +-> { | [[ | if (flag) start +-- +-- return flag; | ][ | return 2; | +-> } ]] | | int func3(void) +----------> { return 3; } Don't forget you can also use "%" to move between matching (), {} and []. That also works when they are many lines apart. MOVING IN BRACES The "[(" and "])" commands work similar to "[{" and "]}", except that they work on () pairs instead of {} pairs. > [( < <-------------------------------- <------- if (a == b && (c == d || (e > f)) && x > y) ~ --------------> --------------------------------> > ]) MOVING IN COMMENTS To move back to the start of a comment use "[/". Move forward to the end of a comment with "]/". This only works for /* - */ comments. +-> +-> /* | [/ | * A comment about --+ [/ | +-- * wonderful life. | ]/ | */ <-+ | +-- foo = bar * 3; --+ | ]/ /* a short comment */ <-+ ============================================================================== *29.4* Finding global identifiers You are editing a C program and wonder if a variable is declared as "int" or "unsigned". A quick way to find this is with the "[I" command. Suppose the cursor is on the word "column". Type: > [I Vim will list the matching lines it can find. Not only in the current file, but also in all included files (and files included in them, etc.). The result looks like this: structs.h ~ 1: 29 unsigned column; /* column number */ ~ The advantage over using tags or the preview window is that included files are searched. In most cases this results in the right declaration to be found. Also when the tags file is out of date. Also when you don't have tags for the included files. However, a few things must be right for "[I" to do its work. First of all, the 'include' option must specify how a file is included. The default value works for C and C++. For other languages you will have to change it. LOCATING INCLUDED FILES Vim will find included files in the places specified with the 'path' option. If a directory is missing, some include files will not be found. You can discover this with this command: > :checkpath It will list the include files that could not be found. Also files included by the files that could be found. An example of the output: --- Included files not found in path --- ~ <io.h> ~ vim.h --> ~ <functions.h> ~ <clib/exec_protos.h> ~ The "io.h" file is included by the current file and can't be found. "vim.h" can be found, thus ":checkpath" goes into this file and checks what it includes. The "functions.h" and "clib/exec_protos.h" files, included by "vim.h" are not found. Note: Vim is not a compiler. It does not recognize "#ifdef" statements. This means every "#include" statement is used, also when it comes after "#if NEVER". To fix the files that could not be found, add a directory to the 'path' option. A good place to find out about this is the Makefile. Look out for lines that contain "-I" items, like "-I/usr/local/X11". To add this directory use: > :set path+=/usr/local/X11 When there are many subdirectories, you can use the "*" wildcard. Example: > :set path+=/usr/*/include This would find files in "/usr/local/include" as well as "/usr/X11/include". When working on a project with a whole nested tree of included files, the "**" items is useful. This will search down in all subdirectories. Example: > :set path+=/projects/invent/**/include This will find files in the directories: /projects/invent/include ~ /projects/invent/main/include ~ /projects/invent/main/os/include ~ etc. There are even more possibilities. Check out the 'path' option for info. If you want to see which included files are actually found, use this command: > :checkpath! You will get a (very long) list of included files, the files they include, and so on. To shorten the list a bit, Vim shows "(Already listed)" for files that were found before and doesn't list the included files in there again. JUMPING TO A MATCH "[I" produces a list with only one line of text. When you want to have a closer look at the first item, you can jump to that line with the command: > [<Tab> You can also use "[ CTRL-I", since CTRL-I is the same as pressing <Tab>. The list that "[I" produces has a number at the start of each line. When you want to jump to another item than the first one, type the number first: > 3[<Tab> Will jump to the third item in the list. Remember that you can use CTRL-O to jump back to where you started from. RELATED COMMANDS [i only lists the first match ]I only lists items below the cursor ]i only lists the first item below the cursor FINDING DEFINED IDENTIFIERS The "[I" command finds any identifier. To find only macros, defined with "#define" use: > [D Again, this searches in included files. The 'define' option specifies what a line looks like that defines the items for "[D". You could change it to make it work with other languages than C or C++. The commands related to "[D" are: [d only lists the first match ]D only lists items below the cursor ]d only lists the first item below the cursor ============================================================================== *29.5* Finding local identifiers The "[I" command searches included files. To search in the current file only, and jump to the first place where the word under the cursor is used: > gD Hint: Goto Definition. This command is very useful to find a variable or function that was declared locally ("static", in C terms). Example (cursor on "counter"): +-> static int counter = 0; | | int get_counter(void) gD | { | ++counter; +-- return counter; } To restrict the search even further, and look only in the current function, use this command: > gd This will go back to the start of the current function and find the first occurrence of the word under the cursor. Actually, it searches backwards to an empty line above a "{" in the first column. From there it searches forward for the identifier. Example (cursor on "idx"): int find_entry(char *name) { +-> int idx; | gd | for (idx = 0; idx < table_len; ++idx) | if (strcmp(table[idx].name, name) == 0) +-- return idx; } ============================================================================== Next chapter: |usr_30.txt| Editing programs Copyright: see |manual-copyright| vim:tw=78:ts=8:noet:ft=help:norl: