Mercurial > vim
view runtime/doc/usr_28.txt @ 32936:c517845bd10e v9.0.1776
patch 9.0.1776: No support for stable Python 3 ABI
Commit: https://github.com/vim/vim/commit/c13b3d1350b60b94fe87f0761ea31c0e7fb6ebf3
Author: Yee Cheng Chin <ychin.git@gmail.com>
Date: Sun Aug 20 21:18:38 2023 +0200
patch 9.0.1776: No support for stable Python 3 ABI
Problem: No support for stable Python 3 ABI
Solution: Support Python 3 stable ABI
Commits:
1) Support Python 3 stable ABI to allow mixed version interoperatbility
Vim currently supports embedding Python for use with plugins, and the
"dynamic" linking option allows the user to specify a locally installed
version of Python by setting `pythonthreedll`. However, one caveat is
that the Python 3 libs are not binary compatible across minor versions,
and mixing versions can potentially be dangerous (e.g. let's say Vim was
linked against the Python 3.10 SDK, but the user sets `pythonthreedll`
to a 3.11 lib). Usually, nothing bad happens, but in theory this could
lead to crashes, memory corruption, and other unpredictable behaviors.
It's also difficult for the user to tell something is wrong because Vim
has no way of reporting what Python 3 version Vim was linked with.
For Vim installed via a package manager, this usually isn't an issue
because all the dependencies would already be figured out. For prebuilt
Vim binaries like MacVim (my motivation for working on this), AppImage,
and Win32 installer this could potentially be an issue as usually a
single binary is distributed. This is more tricky when a new Python
version is released, as there's a chicken-and-egg issue with deciding
what Python version to build against and hard to keep in sync when a new
Python version just drops and we have a mix of users of different Python
versions, and a user just blindly upgrading to a new Python could lead to
bad interactions with Vim.
Python 3 does have a solution for this problem: stable ABI / limited API
(see https://docs.python.org/3/c-api/stable.html). The C SDK limits the
API to a set of functions that are promised to be stable across
versions. This pull request adds an ifdef config that allows us to turn
it on when building Vim. Vim binaries built with this option should be
safe to freely link with any Python 3 libraies without having the
constraint of having to use the same minor version.
Note: Python 2 has no such concept and this doesn't change how Python 2
integration works (not that there is going to be a new version of Python
2 that would cause compatibility issues in the future anyway).
---
Technical details:
======
The stable ABI can be accessed when we compile with the Python 3 limited
API (by defining `Py_LIMITED_API`). The Python 3 code (in `if_python3.c`
and `if_py_both.h`) would now handle this and switch to limited API
mode. Without it set, Vim will still use the full API as before so this
is an opt-in change.
The main difference is that `PyType_Object` is now an opaque struct that
we can't directly create "static types" out of, and we have to create
type objects as "heap types" instead. This is because the struct is not
stable and changes from version to version (e.g. 3.8 added a
`tp_vectorcall` field to it). I had to change all the types to be
allocated on the heap instead with just a pointer to them.
Other functions are also simply missing in limited API, or they are
introduced too late (e.g. `PyUnicode_AsUTF8AndSize` in 3.10) to it that
we need some other ways to do the same thing, so I had to abstract a few
things into macros, and sometimes re-implement functions like
`PyObject_NEW`.
One caveat is that in limited API, `OutputType` (used for replacing
`sys.stdout`) no longer inherits from `PyStdPrinter_Type` which I don't
think has any real issue other than minor differences in how they
convert to a string and missing a couple functions like `mode()` and
`fileno()`.
Also fixed an existing bug where `tp_basicsize` was set incorrectly for
`BufferObject`, `TabListObject, `WinListObject`.
Technically, there could be a small performance drop, there is a little
more indirection with accessing type objects, and some APIs like
`PyUnicode_AsUTF8AndSize` are missing, but in practice I didn't see any
difference, and any well-written Python plugin should try to avoid
excessing callbacks to the `vim` module in Python anyway.
I only tested limited API mode down to Python 3.7, which seemes to
compile and work fine. I haven't tried earlier Python versions.
2) Fix PyIter_Check on older Python vers / type##Ptr unused warning
For PyIter_Check, older versions exposed them as either macros (used in
full API), or a function (for use in limited API). A previous change
exposed PyIter_Check to the dynamic build because Python just moved it
to function-only in 3.10 anyway. Because of that, just make sure we
always grab the function in dynamic builds in earlier versions since
that's what Python eventually did anyway.
3) Move Py_LIMITED_API define to configure script
Can now use --with-python-stable-abi flag to customize what stable ABI
version to target. Can also use an env var to do so as well.
4) Show +python/dyn-stable in :version, and allow has() feature query
Not sure if the "/dyn-stable" suffix would break things, or whether we
should do it another way. Or just don't show it in version and rely on
has() feature checking.
5) Documentation first draft. Still need to implement v:python3_version
6) Fix PyIter_Check build breaks when compiling against Python 3.8
7) Add CI coverage stable ABI on Linux/Windows / make configurable on Windows
This adds configurable options for Windows make files (both MinGW and
MSVC). CI will also now exercise both traditional full API and stable
ABI for Linux and Windows in the matrix for coverage.
Also added a "dynamic" option to Linux matrix as a drive-by change to
make other scripting languages like Ruby / Perl testable under both
static and dynamic builds.
8) Fix inaccuracy in Windows docs
Python's own docs are confusing but you don't actually want to use
`python3.dll` for the dynamic linkage.
9) Add generated autoconf file
10) Add v:python3_version support
This variable indicates the version of Python3 that Vim was built
against (PY_VERSION_HEX), and will be useful to check whether the Python
library you are loading in dynamically actually fits it. When built with
stable ABI, it will be the limited ABI version instead
(`Py_LIMITED_API`), which indicates the minimum version of Python 3 the
user should have, rather than the exact match. When stable ABI is used,
we won't be exposing PY_VERSION_HEX in this var because it just doesn't
seem necessary to do so (the whole point of stable ABI is the promise
that it will work across versions), and I don't want to confuse the user
with too many variables.
Also, cleaned up some documentation, and added help tags.
11) Fix Python 3.7 compat issues
Fix a couple issues when using limited API < 3.8
- Crash on exit: In Python 3.7, if a heap-allocated type is destroyed
before all instances are, it would cause a crash later. This happens
when we destroyed `OptionsType` before calling `Py_Finalize` when
using the limited API. To make it worse, later versions changed the
semantics and now each instance has a strong reference to its own type
and the recommendation has changed to have each instance de-ref its
own type and have its type in GC traversal. To avoid dealing with
these cross-version variations, we just don't free the heap type. They
are static types in non-limited-API anyway and are designed to last
through the entirety of the app, and we also don't restart the Python
runtime and therefore do not need it to have absolutely 0 leaks.
See:
- https://docs.python.org/3/whatsnew/3.8.html#changes-in-the-c-api
- https://docs.python.org/3/whatsnew/3.9.html#changes-in-the-c-api
- PyIter_Check: This function is not provided in limited APIs older than
3.8. Previously I was trying to mock it out using manual
PyType_GetSlot() but it was brittle and also does not actually work
properly for static types (it will generate a Python error). Just
return false. It does mean using limited API < 3.8 is not recommended
as you lose the functionality to handle iterators, but from playing
with plugins I couldn't find it to be an issue.
- Fix loading of PyIter_Check so it will be done when limited API < 3.8.
Otherwise loading a 3.7 Python lib will fail even if limited API was
specified to use it.
12) Make sure to only load `PyUnicode_AsUTF8AndSize` in needed in limited API
We don't use this function unless limited API >= 3.10, but we were
loading it regardless. Usually it's ok in Unix-like systems where Python
just has a single lib that we load from, but in Windows where there is a
separate python3.dll this would not work as the symbol would not have
been exposed in this more limited DLL file. This makes it much clearer
under what condition is this function needed.
closes: #12032
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Yee Cheng Chin <ychin.git@gmail.com>
author | Christian Brabandt <cb@256bit.org> |
---|---|
date | Sun, 20 Aug 2023 21:30:04 +0200 |
parents | f8116058ca76 |
children | 4635e43f2c6f |
line wrap: on
line source
*usr_28.txt* For Vim version 9.0. Last change: 2008 Jun 14 VIM USER MANUAL - by Bram Moolenaar Folding Structured text can be separated in sections. And sections in sub-sections. Folding allows you to display a section as one line, providing an overview. This chapter explains the different ways this can be done. |28.1| What is folding? |28.2| Manual folding |28.3| Working with folds |28.4| Saving and restoring folds |28.5| Folding by indent |28.6| Folding with markers |28.7| Folding by syntax |28.8| Folding by expression |28.9| Folding unchanged lines |28.10| Which fold method to use? Next chapter: |usr_29.txt| Moving through programs Previous chapter: |usr_27.txt| Search commands and patterns Table of contents: |usr_toc.txt| ============================================================================== *28.1* What is folding? Folding is used to show a range of lines in the buffer as a single line on the screen. Like a piece of paper which is folded to make it shorter: +------------------------+ | line 1 | | line 2 | | line 3 | |_______________________ | \ \ \________________________\ / folded lines / /________________________/ | line 12 | | line 13 | | line 14 | +------------------------+ The text is still in the buffer, unchanged. Only the way lines are displayed is affected by folding. The advantage of folding is that you can get a better overview of the structure of text, by folding lines of a section and replacing it with a line that indicates that there is a section. ============================================================================== *28.2* Manual folding Try it out: Position the cursor in a paragraph and type: > zfap You will see that the paragraph is replaced by a highlighted line. You have created a fold. |zf| is an operator and |ap| a text object selection. You can use the |zf| operator with any movement command to create a fold for the text that it moved over. |zf| also works in Visual mode. To view the text again, open the fold by typing: > zo And you can close the fold again with: > zc All the folding commands start with "z". With some fantasy, this looks like a folded piece of paper, seen from the side. The letter after the "z" has a mnemonic meaning to make it easier to remember the commands: zf F-old creation zo O-pen a fold zc C-lose a fold Folds can be nested: A region of text that contains folds can be folded again. For example, you can fold each paragraph in this section, and then fold all the sections in this chapter. Try it out. You will notice that opening the fold for the whole chapter will restore the nested folds as they were, some may be open and some may be closed. Suppose you have created several folds, and now want to view all the text. You could go to each fold and type "zo". To do this faster, use this command: > zr This will R-educe the folding. The opposite is: > zm This folds M-ore. You can repeat "zr" and "zm" to open and close nested folds of several levels. If you have nested several levels deep, you can open all of them with: > zR This R-educes folds until there are none left. And you can close all folds with: > zM This folds M-ore and M-ore. You can quickly disable the folding with the |zn| command. Then |zN| brings back the folding as it was. |zi| toggles between the two. This is a useful way of working: - create folds to get overview on your file - move around to where you want to do your work - do |zi| to look at the text and edit it - do |zi| again to go back to moving around More about manual folding in the reference manual: |fold-manual| ============================================================================== *28.3* Working with folds When some folds are closed, movement commands like "j" and "k" move over a fold like it was a single, empty line. This allows you to quickly move around over folded text. You can yank, delete and put folds as if it was a single line. This is very useful if you want to reorder functions in a program. First make sure that each fold contains a whole function (or a bit less) by selecting the right 'foldmethod'. Then delete the function with "dd", move the cursor and put it with "p". If some lines of the function are above or below the fold, you can use Visual selection: - put the cursor on the first line to be moved - hit "V" to start Visual mode - put the cursor on the last line to be moved - hit "d" to delete the selected lines. - move the cursor to the new position and "p"ut the lines there. It is sometimes difficult to see or remember where a fold is located, thus where a |zo| command would actually work. To see the defined folds: > :set foldcolumn=4 This will show a small column on the left of the window to indicate folds. A "+" is shown for a closed fold. A "-" is shown at the start of each open fold and "|" at following lines of the fold. You can use the mouse to open a fold by clicking on the "+" in the foldcolumn. Clicking on the "-" or a "|" below it will close an open fold. To open all folds at the cursor line use |zO|. To close all folds at the cursor line use |zC|. To delete a fold at the cursor line use |zd|. To delete all folds at the cursor line use |zD|. When in Insert mode, the fold at the cursor line is never closed. That allows you to see what you type! Folds are opened automatically when jumping around or moving the cursor left or right. For example, the "0" command opens the fold under the cursor (if 'foldopen' contains "hor", which is the default). The 'foldopen' option can be changed to open folds for specific commands. If you want the line under the cursor always to be open, do this: > :set foldopen=all Warning: You won't be able to move onto a closed fold then. You might want to use this only temporarily and then set it back to the default: > :set foldopen& You can make folds close automatically when you move out of it: > :set foldclose=all This will re-apply 'foldlevel' to all folds that don't contain the cursor. You have to try it out if you like how this feels. Use |zm| to fold more and |zr| to fold less (reduce folds). The folding is local to the window. This allows you to open two windows on the same buffer, one with folds and one without folds. Or one with all folds closed and one with all folds open. ============================================================================== *28.4* Saving and restoring folds When you abandon a file (starting to edit another one), the state of the folds is lost. If you come back to the same file later, all manually opened and closed folds are back to their default. When folds have been created manually, all folds are gone! To save the folds use the |:mkview| command: > :mkview This will store the settings and other things that influence the view on the file. You can change what is stored with the 'viewoptions' option. When you come back to the same file later, you can load the view again: > :loadview You can store up to ten views on one file. For example, to save the current setup as the third view and load the second view: > :mkview 3 :loadview 2 Note that when you insert or delete lines the views might become invalid. Also check out the 'viewdir' option, which specifies where the views are stored. You might want to delete old views now and then. ============================================================================== *28.5* Folding by indent Defining folds with |zf| is a lot of work. If your text is structured by giving lower level items a larger indent, you can use the indent folding method. This will create folds for every sequence of lines with the same indent. Lines with a larger indent will become nested folds. This works well with many programming languages. Try this by setting the 'foldmethod' option: > :set foldmethod=indent Then you can use the |zm| and |zr| commands to fold more and reduce folding. It's easy to see on this example text: This line is not indented This line is indented once This line is indented twice This line is indented twice This line is indented once This line is not indented This line is indented once This line is indented once Note that the relation between the amount of indent and the fold depth depends on the 'shiftwidth' option. Each 'shiftwidth' worth of indent adds one to the depth of the fold. This is called a fold level. When you use the |zr| and |zm| commands you actually increase or decrease the 'foldlevel' option. You could also set it directly: > :set foldlevel=3 This means that all folds with three times a 'shiftwidth' indent or more will be closed. The lower the foldlevel, the more folds will be closed. When 'foldlevel' is zero, all folds are closed. |zM| does set 'foldlevel' to zero. The opposite command |zR| sets 'foldlevel' to the deepest fold level that is present in the file. Thus there are two ways to open and close the folds: (A) By setting the fold level. This gives a very quick way of "zooming out" to view the structure of the text, move the cursor, and "zoom in" on the text again. (B) By using |zo| and |zc| commands to open or close specific folds. This allows opening only those folds that you want to be open, while other folds remain closed. This can be combined: You can first close most folds by using |zm| a few times and then open a specific fold with |zo|. Or open all folds with |zR| and then close specific folds with |zc|. But you cannot manually define folds when 'foldmethod' is "indent", as that would conflict with the relation between the indent and the fold level. More about folding by indent in the reference manual: |fold-indent| ============================================================================== *28.6* Folding with markers Markers in the text are used to specify the start and end of a fold region. This gives precise control over which lines are included in a fold. The disadvantage is that the text needs to be modified. Try it: > :set foldmethod=marker Example text, as it could appear in a C program: /* foobar () {{{ */ int foobar() { /* return a value {{{ */ return 42; /* }}} */ } /* }}} */ Notice that the folded line will display the text before the marker. This is very useful to tell what the fold contains. It's quite annoying when the markers don't pair up correctly after moving some lines around. This can be avoided by using numbered markers. Example: /* global variables {{{1 */ int varA, varB; /* functions {{{1 */ /* funcA() {{{2 */ void funcA() {} /* funcB() {{{2 */ void funcB() {} /* }}}1 */ At every numbered marker a fold at the specified level begins. This will make any fold at a higher level stop here. You can just use numbered start markers to define all folds. Only when you want to explicitly stop a fold before another starts you need to add an end marker. More about folding with markers in the reference manual: |fold-marker| ============================================================================== *28.7* Folding by syntax For each language Vim uses a different syntax file. This defines the colors for various items in the file. If you are reading this in Vim, in a terminal that supports colors, the colors you see are made with the "help" syntax file. In the syntax files it is possible to add syntax items that have the "fold" argument. These define a fold region. This requires writing a syntax file and adding these items in it. That's not so easy to do. But once it's done, all folding happens automatically. Here we'll assume you are using an existing syntax file. Then there is nothing more to explain. You can open and close folds as explained above. The folds will be created and deleted automatically when you edit the file. More about folding by syntax in the reference manual: |fold-syntax| ============================================================================== *28.8* Folding by expression This is similar to folding by indent, but instead of using the indent of a line a user function is called to compute the fold level of a line. You can use this for text where something in the text indicates which lines belong together. An example is an e-mail message where the quoted text is indicated by a ">" before the line. To fold these quotes use this: > :set foldmethod=expr :set foldexpr=strlen(substitute(substitute(getline(v:lnum),'\\s','',\"g\"),'[^>].*','','')) You can try it out on this text: > quoted text he wrote > quoted text he wrote > > double quoted text I wrote > > double quoted text I wrote Explanation for the 'foldexpr' used in the example (inside out): getline(v:lnum) gets the current line substitute(...,'\\s','','g') removes all white space from the line substitute(...,'[^>].*','','') removes everything after leading '>'s strlen(...) counts the length of the string, which is the number of '>'s found Note that a backslash must be inserted before every space, double quote and backslash for the ":set" command. If this confuses you, do > :set foldexpr to check the actual resulting value. To correct a complicated expression, use the command-line completion: > :set foldexpr=<Tab> Where <Tab> is a real Tab. Vim will fill in the previous value, which you can then edit. When the expression gets more complicated you should put it in a function and set 'foldexpr' to call that function. More about folding by expression in the reference manual: |fold-expr| ============================================================================== *28.9* Folding unchanged lines This is useful when you set the 'diff' option in the same window. The |vimdiff| command does this for you. Example: > :setlocal diff foldmethod=diff scrollbind nowrap foldlevel=1 Do this in every window that shows a different version of the same file. You will clearly see the differences between the files, while the text that didn't change is folded. For more details see |fold-diff|. ============================================================================== *28.10* Which fold method to use? All these possibilities make you wonder which method you should choose. Unfortunately, there is no golden rule. Here are some hints. If there is a syntax file with folding for the language you are editing, that is probably the best choice. If there isn't one, you might try to write it. This requires a good knowledge of search patterns. It's not easy, but when it's working you will not have to define folds manually. Typing commands to manually fold regions can be used for unstructured text. Then use the |:mkview| command to save and restore your folds. The marker method requires you to change the file. If you are sharing the files with other people or you have to meet company standards, you might not be allowed to add them. The main advantage of markers is that you can put them exactly where you want them. That avoids that a few lines are missed when you cut and paste folds. And you can add a comment about what is contained in the fold. Folding by indent is something that works in many files, but not always very well. Use it when you can't use one of the other methods. However, it is very useful for outlining. Then you specifically use one 'shiftwidth' for each nesting level. Folding with expressions can make folds in almost any structured text. It is quite simple to specify, especially if the start and end of a fold can easily be recognized. If you use the "expr" method to define folds, but they are not exactly how you want them, you could switch to the "manual" method. This will not remove the defined folds. Then you can delete or add folds manually. ============================================================================== Next chapter: |usr_29.txt| Moving through programs Copyright: see |manual-copyright| vim:tw=78:ts=8:noet:ft=help:norl: