Mercurial > vim
view runtime/doc/helphelp.txt @ 32936:c517845bd10e v9.0.1776
patch 9.0.1776: No support for stable Python 3 ABI
Commit: https://github.com/vim/vim/commit/c13b3d1350b60b94fe87f0761ea31c0e7fb6ebf3
Author: Yee Cheng Chin <ychin.git@gmail.com>
Date: Sun Aug 20 21:18:38 2023 +0200
patch 9.0.1776: No support for stable Python 3 ABI
Problem: No support for stable Python 3 ABI
Solution: Support Python 3 stable ABI
Commits:
1) Support Python 3 stable ABI to allow mixed version interoperatbility
Vim currently supports embedding Python for use with plugins, and the
"dynamic" linking option allows the user to specify a locally installed
version of Python by setting `pythonthreedll`. However, one caveat is
that the Python 3 libs are not binary compatible across minor versions,
and mixing versions can potentially be dangerous (e.g. let's say Vim was
linked against the Python 3.10 SDK, but the user sets `pythonthreedll`
to a 3.11 lib). Usually, nothing bad happens, but in theory this could
lead to crashes, memory corruption, and other unpredictable behaviors.
It's also difficult for the user to tell something is wrong because Vim
has no way of reporting what Python 3 version Vim was linked with.
For Vim installed via a package manager, this usually isn't an issue
because all the dependencies would already be figured out. For prebuilt
Vim binaries like MacVim (my motivation for working on this), AppImage,
and Win32 installer this could potentially be an issue as usually a
single binary is distributed. This is more tricky when a new Python
version is released, as there's a chicken-and-egg issue with deciding
what Python version to build against and hard to keep in sync when a new
Python version just drops and we have a mix of users of different Python
versions, and a user just blindly upgrading to a new Python could lead to
bad interactions with Vim.
Python 3 does have a solution for this problem: stable ABI / limited API
(see https://docs.python.org/3/c-api/stable.html). The C SDK limits the
API to a set of functions that are promised to be stable across
versions. This pull request adds an ifdef config that allows us to turn
it on when building Vim. Vim binaries built with this option should be
safe to freely link with any Python 3 libraies without having the
constraint of having to use the same minor version.
Note: Python 2 has no such concept and this doesn't change how Python 2
integration works (not that there is going to be a new version of Python
2 that would cause compatibility issues in the future anyway).
---
Technical details:
======
The stable ABI can be accessed when we compile with the Python 3 limited
API (by defining `Py_LIMITED_API`). The Python 3 code (in `if_python3.c`
and `if_py_both.h`) would now handle this and switch to limited API
mode. Without it set, Vim will still use the full API as before so this
is an opt-in change.
The main difference is that `PyType_Object` is now an opaque struct that
we can't directly create "static types" out of, and we have to create
type objects as "heap types" instead. This is because the struct is not
stable and changes from version to version (e.g. 3.8 added a
`tp_vectorcall` field to it). I had to change all the types to be
allocated on the heap instead with just a pointer to them.
Other functions are also simply missing in limited API, or they are
introduced too late (e.g. `PyUnicode_AsUTF8AndSize` in 3.10) to it that
we need some other ways to do the same thing, so I had to abstract a few
things into macros, and sometimes re-implement functions like
`PyObject_NEW`.
One caveat is that in limited API, `OutputType` (used for replacing
`sys.stdout`) no longer inherits from `PyStdPrinter_Type` which I don't
think has any real issue other than minor differences in how they
convert to a string and missing a couple functions like `mode()` and
`fileno()`.
Also fixed an existing bug where `tp_basicsize` was set incorrectly for
`BufferObject`, `TabListObject, `WinListObject`.
Technically, there could be a small performance drop, there is a little
more indirection with accessing type objects, and some APIs like
`PyUnicode_AsUTF8AndSize` are missing, but in practice I didn't see any
difference, and any well-written Python plugin should try to avoid
excessing callbacks to the `vim` module in Python anyway.
I only tested limited API mode down to Python 3.7, which seemes to
compile and work fine. I haven't tried earlier Python versions.
2) Fix PyIter_Check on older Python vers / type##Ptr unused warning
For PyIter_Check, older versions exposed them as either macros (used in
full API), or a function (for use in limited API). A previous change
exposed PyIter_Check to the dynamic build because Python just moved it
to function-only in 3.10 anyway. Because of that, just make sure we
always grab the function in dynamic builds in earlier versions since
that's what Python eventually did anyway.
3) Move Py_LIMITED_API define to configure script
Can now use --with-python-stable-abi flag to customize what stable ABI
version to target. Can also use an env var to do so as well.
4) Show +python/dyn-stable in :version, and allow has() feature query
Not sure if the "/dyn-stable" suffix would break things, or whether we
should do it another way. Or just don't show it in version and rely on
has() feature checking.
5) Documentation first draft. Still need to implement v:python3_version
6) Fix PyIter_Check build breaks when compiling against Python 3.8
7) Add CI coverage stable ABI on Linux/Windows / make configurable on Windows
This adds configurable options for Windows make files (both MinGW and
MSVC). CI will also now exercise both traditional full API and stable
ABI for Linux and Windows in the matrix for coverage.
Also added a "dynamic" option to Linux matrix as a drive-by change to
make other scripting languages like Ruby / Perl testable under both
static and dynamic builds.
8) Fix inaccuracy in Windows docs
Python's own docs are confusing but you don't actually want to use
`python3.dll` for the dynamic linkage.
9) Add generated autoconf file
10) Add v:python3_version support
This variable indicates the version of Python3 that Vim was built
against (PY_VERSION_HEX), and will be useful to check whether the Python
library you are loading in dynamically actually fits it. When built with
stable ABI, it will be the limited ABI version instead
(`Py_LIMITED_API`), which indicates the minimum version of Python 3 the
user should have, rather than the exact match. When stable ABI is used,
we won't be exposing PY_VERSION_HEX in this var because it just doesn't
seem necessary to do so (the whole point of stable ABI is the promise
that it will work across versions), and I don't want to confuse the user
with too many variables.
Also, cleaned up some documentation, and added help tags.
11) Fix Python 3.7 compat issues
Fix a couple issues when using limited API < 3.8
- Crash on exit: In Python 3.7, if a heap-allocated type is destroyed
before all instances are, it would cause a crash later. This happens
when we destroyed `OptionsType` before calling `Py_Finalize` when
using the limited API. To make it worse, later versions changed the
semantics and now each instance has a strong reference to its own type
and the recommendation has changed to have each instance de-ref its
own type and have its type in GC traversal. To avoid dealing with
these cross-version variations, we just don't free the heap type. They
are static types in non-limited-API anyway and are designed to last
through the entirety of the app, and we also don't restart the Python
runtime and therefore do not need it to have absolutely 0 leaks.
See:
- https://docs.python.org/3/whatsnew/3.8.html#changes-in-the-c-api
- https://docs.python.org/3/whatsnew/3.9.html#changes-in-the-c-api
- PyIter_Check: This function is not provided in limited APIs older than
3.8. Previously I was trying to mock it out using manual
PyType_GetSlot() but it was brittle and also does not actually work
properly for static types (it will generate a Python error). Just
return false. It does mean using limited API < 3.8 is not recommended
as you lose the functionality to handle iterators, but from playing
with plugins I couldn't find it to be an issue.
- Fix loading of PyIter_Check so it will be done when limited API < 3.8.
Otherwise loading a 3.7 Python lib will fail even if limited API was
specified to use it.
12) Make sure to only load `PyUnicode_AsUTF8AndSize` in needed in limited API
We don't use this function unless limited API >= 3.10, but we were
loading it regardless. Usually it's ok in Unix-like systems where Python
just has a single lib that we load from, but in Windows where there is a
separate python3.dll this would not work as the symbol would not have
been exposed in this more limited DLL file. This makes it much clearer
under what condition is this function needed.
closes: #12032
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Yee Cheng Chin <ychin.git@gmail.com>
author | Christian Brabandt <cb@256bit.org> |
---|---|
date | Sun, 20 Aug 2023 21:30:04 +0200 |
parents | a9b5ffbc0428 |
children | 35aaf01f7d61 |
line wrap: on
line source
*helphelp.txt* For Vim version 9.0. Last change: 2022 Jan 08 VIM REFERENCE MANUAL by Bram Moolenaar Help on help files *helphelp* 1. Help commands |online-help| 2. Translated help files |help-translated| 3. Writing help files |help-writing| ============================================================================== 1. Help commands *online-help* *help* *<Help>* *:h* *:help* *<F1>* *i_<F1>* *i_<Help>* <Help> or :h[elp] Open a window and display the help file in read-only mode. If there is a help window open already, use that one. Otherwise, if the current window uses the full width of the screen or is at least 80 characters wide, the help window will appear just above the current window. Otherwise the new window is put at the very top. The 'helplang' option is used to select a language, if the main help file is available in several languages. *{subject}* *E149* *E661* :h[elp] {subject} Like ":help", additionally jump to the tag {subject}. For example: > :help options < {subject} can include wildcards such as "*", "?" and "[a-z]": :help z? jump to help for any "z" command :help z. jump to the help for "z." But when a tag exists it is taken literally: :help :? jump to help for ":?" If there is no full match for the pattern, or there are several matches, the "best" match will be used. A sophisticated algorithm is used to decide which match is better than another one. These items are considered in the computation: - A match with same case is much better than a match with different case. - A match that starts after a non-alphanumeric character is better than a match in the middle of a word. - A match at or near the beginning of the tag is better than a match further on. - The more alphanumeric characters match, the better. - The shorter the length of the match, the better. The 'helplang' option is used to select a language, if the {subject} is available in several languages. To find a tag in a specific language, append "@ab", where "ab" is the two-letter language code. See |help-translated|. Note that the longer the {subject} you give, the less matches will be found. You can get an idea how this all works by using commandline completion (type CTRL-D after ":help subject" |c_CTRL-D|). If there are several matches, you can have them listed by hitting CTRL-D. Example: > :help cont<Ctrl-D> < Instead of typing ":help CTRL-V" to search for help for CTRL-V you can type: > :help ^V < This also works together with other characters, for example to find help for CTRL-V in Insert mode: > :help i^V < It is also possible to first do ":help" and then use ":tag {pattern}" in the help window. The ":tnext" command can then be used to jump to other matches, "tselect" to list matches and choose one. > :help index :tselect /.*mode < When there is no argument you will see matches for "help", to avoid listing all possible matches (that would be very slow). The number of matches displayed is limited to 300. The `:help` command can be followed by '|' and another command, but you don't need to escape the '|' inside a help command. So these both work: > :help | :help k| only < Note that a space before the '|' is seen as part of the ":help" argument. You can also use <NL> or <CR> to separate the help command from a following command. You need to type CTRL-V first to insert the <NL> or <CR>. Example: > :help so<C-V><CR>only :h[elp]! [subject] Like ":help", but in non-English help files prefer to find a tag in a file with the same language as the current file. See |help-translated|. *:helpc* *:helpclose* :helpc[lose] Close one help window, if there is one. Vim will try to restore the window layout (including cursor position) to the same layout it was before opening the help window initially. This might cause triggering several autocommands. *:helpg* *:helpgrep* :helpg[rep] {pattern}[@xx] Search all help text files and make a list of lines in which {pattern} matches. Jumps to the first match. The optional [@xx] specifies that only matches in the "xx" language are to be found. You can navigate through the matches with the |quickfix| commands, e.g., |:cnext| to jump to the next one. Or use |:cwindow| to get the list of matches in the quickfix window. {pattern} is used as a Vim regexp |pattern|. 'ignorecase' is not used, add "\c" to ignore case. Example for case sensitive search: > :helpgrep Uganda < Example for case ignoring search: > :helpgrep uganda\c < Example for searching in French help: > :helpgrep backspace@fr < The pattern does not support line breaks, it must match within one line. You can use |:grep| instead, but then you need to get the list of help files in a complicated way. Cannot be followed by another command, everything is used as part of the pattern. But you can use |:execute| when needed. Compressed help files will not be searched (Fedora compresses the help files). *:lh* *:lhelpgrep* :lh[elpgrep] {pattern}[@xx] Same as ":helpgrep", except the location list is used instead of the quickfix list. If the help window is already opened, then the location list for that window is used. Otherwise, a new help window is opened and the location list for that window is set. The location list for the current window is not changed then. *:exu* *:exusage* :exu[sage] Show help on Ex commands. Added to simulate the Nvi command. *:viu* *:viusage* :viu[sage] Show help on Normal mode commands. Added to simulate the Nvi command. When no argument is given to |:help| the file given with the 'helpfile' option will be opened. Otherwise the specified tag is searched for in all "doc/tags" files in the directories specified in the 'runtimepath' option. If you would like to open the help in the current window, see this tip: |help-curwin|. The initial height of the help window can be set with the 'helpheight' option (default 20). *help-buffer-options* When the help buffer is created, several local options are set to make sure the help text is displayed as it was intended: 'iskeyword' nearly all ASCII chars except ' ', '*', '"' and '|' 'foldmethod' "manual" 'tabstop' 8 'arabic' off 'binary' off 'buflisted' off 'cursorbind' off 'diff' off 'foldenable' off 'list' off 'modifiable' off 'number' off 'relativenumber' off 'rightleft' off 'scrollbind' off 'spell' off Jump to specific subjects by using tags. This can be done in two ways: - Use the "CTRL-]" command while standing on the name of a command or option. This only works when the tag is a keyword. "<C-Leftmouse>" and "g<LeftMouse>" work just like "CTRL-]". - use the ":ta {subject}" command. This also works with non-keyword characters. Use CTRL-T or CTRL-O to jump back. Use ":q" to close the help window. If there are several matches for an item you are looking for, this is how you can jump to each one of them: 1. Open a help window 2. Use the ":tag" command with a slash prepended to the tag. E.g.: > :tag /min 3. Use ":tnext" to jump to the next matching tag. It is possible to add help files for plugins and other items. You don't need to change the distributed help files for that. See |add-local-help|. To write a local help file, see |write-local-help|. Note that the title lines from the local help files are automagically added to the "LOCAL ADDITIONS" section in the "help.txt" help file |local-additions|. This is done when viewing the file in Vim, the file itself is not changed. It is done by going through all help files and obtaining the first line of each file. The files in $VIMRUNTIME/doc are skipped. *help-xterm-window* If you want to have the help in another xterm window, you could use this command: > :!xterm -e vim +help & < *:helpfind* *:helpf* :helpf[ind] Like |:help|, but use a dialog to enter the argument. Only for backwards compatibility. It now executes the ToolBar.FindHelp menu entry instead of using a builtin dialog. {only when compiled with |+GUI_GTK|} *:helpt* *:helptags* *E150* *E151* *E152* *E153* *E154* *E670* :helpt[ags] [++t] {dir} Generate the help tags file(s) for directory {dir}. When {dir} is ALL then all "doc" directories in 'runtimepath' will be used. All "*.txt" and "*.??x" files in the directory and sub-directories are scanned for a help tag definition in between stars. The "*.??x" files are for translated docs, they generate the "tags-??" file, see |help-translated|. The generated tags files are sorted. When there are duplicates an error message is given. An existing tags file is silently overwritten. The optional "++t" argument forces adding the "help-tags" tag. This is also done when the {dir} is equal to $VIMRUNTIME/doc. To rebuild the help tags in the runtime directory (requires write permission there): > :helptags $VIMRUNTIME/doc ============================================================================== 2. Translated help files *help-translated* It is possible to add translated help files, next to the original English help files. Vim will search for all help in "doc" directories in 'runtimepath'. This is only available when compiled with the |+multi_lang| feature. At this moment translations are available for: Chinese - multiple authors French - translated by David Blanchet Italian - translated by Antonio Colombo Japanese - multiple authors Polish - translated by Mikolaj Machowski Russian - translated by Vassily Ragosin See the Vim website to find them: http://www.vim.org/translations.php A set of translated help files consists of these files: help.abx howto.abx ... tags-ab "ab" is the two-letter language code. Thus for Italian the names are: help.itx howto.itx ... tags-it The 'helplang' option can be set to the preferred language(s). The default is set according to the environment. Vim will first try to find a matching tag in the preferred language(s). English is used when it cannot be found. To find a tag in a specific language, append "@ab" to a tag, where "ab" is the two-letter language code. Example: > :he user-manual@it :he user-manual@en The first one finds the Italian user manual, even when 'helplang' is empty. The second one finds the English user manual, even when 'helplang' is set to "it". When using command-line completion for the ":help" command, the "@en" extension is only shown when a tag exists for multiple languages. When the tag only exists for English "@en" is omitted. When the first candidate has an "@ab" extension and it matches the first language in 'helplang' "@ab" is also omitted. When using |CTRL-]| or ":help!" in a non-English help file Vim will try to find the tag in the same language. If not found then 'helplang' will be used to select a language. Help files must use latin1 or utf-8 encoding. Vim assumes the encoding is utf-8 when finding non-ASCII characters in the first line. Thus you must translate the header with "For Vim version". The same encoding must be used for the help files of one language in one directory. You can use a different encoding for different languages and use a different encoding for help files of the same language but in a different directory. Hints for translators: - Do not translate the tags. This makes it possible to use 'helplang' to specify the preferred language. You may add new tags in your language. - When you do not translate a part of a file, add tags to the English version, using the "tag@en" notation. - Make a package with all the files and the tags file available for download. Users can drop it in one of the "doc" directories and start use it. Report this to Bram, so that he can add a link on www.vim.org. - Use the |:helptags| command to generate the tags files. It will find all languages in the specified directory. ============================================================================== 3. Writing help files *help-writing* For ease of use, a Vim help file for a plugin should follow the format of the standard Vim help files, except for the first line. If you are writing a new help file it's best to copy one of the existing files and use it as a template. The first line in a help file should have the following format: *plugin_name.txt* {short description of the plugin} The first field is a help tag where ":help plugin_name" will jump to. The remainder of the line, after a Tab, describes the plugin purpose in a short way. This will show up in the "LOCAL ADDITIONS" section of the main help file. Check there that it shows up properly: |local-additions|. If you want to add a version number or last modification date, put it in the second line, right aligned. At the bottom of the help file, place a Vim modeline to set the 'textwidth' and 'tabstop' options and the 'filetype' to "help". Never set a global option in such a modeline, that can have undesired consequences. TAGS To define a help tag, place the name between asterisks (*tag-name*). The tag-name should be different from all the Vim help tag names and ideally should begin with the name of the Vim plugin. The tag name is usually right aligned on a line. When referring to an existing help tag and to create a hot-link, place the name between two bars (|) eg. |help-writing|. When referring to a Vim command and to create a hot-link, place the name between two backticks, eg. inside `:filetype`. You will see this is highlighted as a command, like a code block (see below). When referring to a Vim option in the help file, place the option name between two single quotes, eg. 'statusline' HIGHLIGHTING To define a column heading, use a tilde character at the end of the line. This will highlight the column heading in a different color. E.g. Column heading~ To separate sections in a help file, place a series of '=' characters in a line starting from the first column. The section separator line is highlighted differently. To quote a block of ex-commands verbatim, place a greater than (>) character at the end of the line before the block and a less than (<) character as the first non-blank on a line following the block. Any line starting in column 1 also implicitly stops the block of ex-commands before it. E.g. > function Example_Func() echo "Example" endfunction < The following are highlighted differently in a Vim help file: - a special key name expressed either in <> notation as in <PageDown>, or as a Ctrl character as in CTRL-X - anything between {braces}, e.g. {lhs} and {rhs} The word "Note", "Notes" and similar automagically receive distinctive highlighting. So do these: *Todo something to do *Error something wrong You can find the details in $VIMRUNTIME/syntax/help.vim *inclusion* Vim is for everybody, no matter race, gender or anything. Some people make a big deal about using "he" or "his" when referring to the user, thinking it means we assume the user is male. That is not the case, it's just a habit of writing help text, which quite often is many years old. Also, a lot of the text is written by contributors for whom English is not their first language. We do not make any assumptions about the gender of the user, no matter how the text is phrased. Some people have suggested using "they", but that is not regular English. We do not want to spend much time on this discussion. The goal is that the reader understands how Vim works, the exact wording is secondary. vim:tw=78:ts=8:noet:ft=help:norl: