view runtime/doc/recover.txt @ 32936:c517845bd10e v9.0.1776

patch 9.0.1776: No support for stable Python 3 ABI Commit: https://github.com/vim/vim/commit/c13b3d1350b60b94fe87f0761ea31c0e7fb6ebf3 Author: Yee Cheng Chin <ychin.git@gmail.com> Date: Sun Aug 20 21:18:38 2023 +0200 patch 9.0.1776: No support for stable Python 3 ABI Problem: No support for stable Python 3 ABI Solution: Support Python 3 stable ABI Commits: 1) Support Python 3 stable ABI to allow mixed version interoperatbility Vim currently supports embedding Python for use with plugins, and the "dynamic" linking option allows the user to specify a locally installed version of Python by setting `pythonthreedll`. However, one caveat is that the Python 3 libs are not binary compatible across minor versions, and mixing versions can potentially be dangerous (e.g. let's say Vim was linked against the Python 3.10 SDK, but the user sets `pythonthreedll` to a 3.11 lib). Usually, nothing bad happens, but in theory this could lead to crashes, memory corruption, and other unpredictable behaviors. It's also difficult for the user to tell something is wrong because Vim has no way of reporting what Python 3 version Vim was linked with. For Vim installed via a package manager, this usually isn't an issue because all the dependencies would already be figured out. For prebuilt Vim binaries like MacVim (my motivation for working on this), AppImage, and Win32 installer this could potentially be an issue as usually a single binary is distributed. This is more tricky when a new Python version is released, as there's a chicken-and-egg issue with deciding what Python version to build against and hard to keep in sync when a new Python version just drops and we have a mix of users of different Python versions, and a user just blindly upgrading to a new Python could lead to bad interactions with Vim. Python 3 does have a solution for this problem: stable ABI / limited API (see https://docs.python.org/3/c-api/stable.html). The C SDK limits the API to a set of functions that are promised to be stable across versions. This pull request adds an ifdef config that allows us to turn it on when building Vim. Vim binaries built with this option should be safe to freely link with any Python 3 libraies without having the constraint of having to use the same minor version. Note: Python 2 has no such concept and this doesn't change how Python 2 integration works (not that there is going to be a new version of Python 2 that would cause compatibility issues in the future anyway). --- Technical details: ====== The stable ABI can be accessed when we compile with the Python 3 limited API (by defining `Py_LIMITED_API`). The Python 3 code (in `if_python3.c` and `if_py_both.h`) would now handle this and switch to limited API mode. Without it set, Vim will still use the full API as before so this is an opt-in change. The main difference is that `PyType_Object` is now an opaque struct that we can't directly create "static types" out of, and we have to create type objects as "heap types" instead. This is because the struct is not stable and changes from version to version (e.g. 3.8 added a `tp_vectorcall` field to it). I had to change all the types to be allocated on the heap instead with just a pointer to them. Other functions are also simply missing in limited API, or they are introduced too late (e.g. `PyUnicode_AsUTF8AndSize` in 3.10) to it that we need some other ways to do the same thing, so I had to abstract a few things into macros, and sometimes re-implement functions like `PyObject_NEW`. One caveat is that in limited API, `OutputType` (used for replacing `sys.stdout`) no longer inherits from `PyStdPrinter_Type` which I don't think has any real issue other than minor differences in how they convert to a string and missing a couple functions like `mode()` and `fileno()`. Also fixed an existing bug where `tp_basicsize` was set incorrectly for `BufferObject`, `TabListObject, `WinListObject`. Technically, there could be a small performance drop, there is a little more indirection with accessing type objects, and some APIs like `PyUnicode_AsUTF8AndSize` are missing, but in practice I didn't see any difference, and any well-written Python plugin should try to avoid excessing callbacks to the `vim` module in Python anyway. I only tested limited API mode down to Python 3.7, which seemes to compile and work fine. I haven't tried earlier Python versions. 2) Fix PyIter_Check on older Python vers / type##Ptr unused warning For PyIter_Check, older versions exposed them as either macros (used in full API), or a function (for use in limited API). A previous change exposed PyIter_Check to the dynamic build because Python just moved it to function-only in 3.10 anyway. Because of that, just make sure we always grab the function in dynamic builds in earlier versions since that's what Python eventually did anyway. 3) Move Py_LIMITED_API define to configure script Can now use --with-python-stable-abi flag to customize what stable ABI version to target. Can also use an env var to do so as well. 4) Show +python/dyn-stable in :version, and allow has() feature query Not sure if the "/dyn-stable" suffix would break things, or whether we should do it another way. Or just don't show it in version and rely on has() feature checking. 5) Documentation first draft. Still need to implement v:python3_version 6) Fix PyIter_Check build breaks when compiling against Python 3.8 7) Add CI coverage stable ABI on Linux/Windows / make configurable on Windows This adds configurable options for Windows make files (both MinGW and MSVC). CI will also now exercise both traditional full API and stable ABI for Linux and Windows in the matrix for coverage. Also added a "dynamic" option to Linux matrix as a drive-by change to make other scripting languages like Ruby / Perl testable under both static and dynamic builds. 8) Fix inaccuracy in Windows docs Python's own docs are confusing but you don't actually want to use `python3.dll` for the dynamic linkage. 9) Add generated autoconf file 10) Add v:python3_version support This variable indicates the version of Python3 that Vim was built against (PY_VERSION_HEX), and will be useful to check whether the Python library you are loading in dynamically actually fits it. When built with stable ABI, it will be the limited ABI version instead (`Py_LIMITED_API`), which indicates the minimum version of Python 3 the user should have, rather than the exact match. When stable ABI is used, we won't be exposing PY_VERSION_HEX in this var because it just doesn't seem necessary to do so (the whole point of stable ABI is the promise that it will work across versions), and I don't want to confuse the user with too many variables. Also, cleaned up some documentation, and added help tags. 11) Fix Python 3.7 compat issues Fix a couple issues when using limited API < 3.8 - Crash on exit: In Python 3.7, if a heap-allocated type is destroyed before all instances are, it would cause a crash later. This happens when we destroyed `OptionsType` before calling `Py_Finalize` when using the limited API. To make it worse, later versions changed the semantics and now each instance has a strong reference to its own type and the recommendation has changed to have each instance de-ref its own type and have its type in GC traversal. To avoid dealing with these cross-version variations, we just don't free the heap type. They are static types in non-limited-API anyway and are designed to last through the entirety of the app, and we also don't restart the Python runtime and therefore do not need it to have absolutely 0 leaks. See: - https://docs.python.org/3/whatsnew/3.8.html#changes-in-the-c-api - https://docs.python.org/3/whatsnew/3.9.html#changes-in-the-c-api - PyIter_Check: This function is not provided in limited APIs older than 3.8. Previously I was trying to mock it out using manual PyType_GetSlot() but it was brittle and also does not actually work properly for static types (it will generate a Python error). Just return false. It does mean using limited API < 3.8 is not recommended as you lose the functionality to handle iterators, but from playing with plugins I couldn't find it to be an issue. - Fix loading of PyIter_Check so it will be done when limited API < 3.8. Otherwise loading a 3.7 Python lib will fail even if limited API was specified to use it. 12) Make sure to only load `PyUnicode_AsUTF8AndSize` in needed in limited API We don't use this function unless limited API >= 3.10, but we were loading it regardless. Usually it's ok in Unix-like systems where Python just has a single lib that we load from, but in Windows where there is a separate python3.dll this would not work as the symbol would not have been exposed in this more limited DLL file. This makes it much clearer under what condition is this function needed. closes: #12032 Signed-off-by: Christian Brabandt <cb@256bit.org> Co-authored-by: Yee Cheng Chin <ychin.git@gmail.com>
author Christian Brabandt <cb@256bit.org>
date Sun, 20 Aug 2023 21:30:04 +0200
parents b2e8663e6dcc
children 4635e43f2c6f
line wrap: on
line source

*recover.txt*   For Vim version 9.0.  Last change: 2023 Apr 22


		  VIM REFERENCE MANUAL    by Bram Moolenaar


Recovery after a crash					*crash-recovery*

You have spent several hours typing in that text that has to be finished
next morning, and then disaster strikes: Your computer crashes.

			DON'T PANIC!

You can recover most of your changes from the files that Vim uses to store
the contents of the file.  Mostly you can recover your work with one command:
	vim -r filename

1. The swap file	|swap-file|
2. Recovery		|recovery|

==============================================================================
1. The swap file					*swap-file*

Vim stores the things you changed in a swap file.  Using the original file
you started from plus the swap file you can mostly recover your work.

You can see the name of the current swap file being used with the command:

	:sw[apname]					*:sw* *:swapname*

Or you can use the |swapname()| function, which also allows for seeing the
swap file name of other buffers.

The name of the swap file is normally the same as the file you are editing,
with the extension ".swp".
- On Unix, a '.' is prepended to swap file names in the same directory as the
  edited file.  This avoids that the swap file shows up in a directory
  listing.
- On MS-Windows machines and when the 'shortname' option is on, any '.' in the
  original file name is replaced with '_'.
- If this file already exists (e.g., when you are recovering from a crash) a
  warning is given and another extension is used, ".swo", ".swn", etc.
- An existing file will never be overwritten.
- The swap file is deleted as soon as Vim stops editing the file.

Technical: The replacement of '.' with '_' is done to avoid problems with
	   MS-DOS compatible filesystems (e.g., crossdos, multidos).  If Vim
	   is able to detect that the file is on an MS-DOS-like filesystem, a
	   flag is set that has the same effect as the 'shortname' option.
	   This flag is reset when you start editing another file.
							*E326*
	   If the ".swp" file name already exists, the last character is
	   decremented until there is no file with that name or ".saa" is
	   reached.  In the last case, no swap file is created.

By setting the 'directory' option you can place the swap file in another place
than where the edited file is.
Advantages:
- You will not pollute the directories with ".swp" files.
- When the 'directory' is on another partition, reduce the risk of damaging
  the file system where the file is (in a crash).
Disadvantages:
- You can get name collisions from files with the same name but in different
  directories (although Vim tries to avoid that by comparing the path name).
  This will result in bogus ATTENTION warning messages.
- When you use your home directory, and somebody else tries to edit the same
  file, that user will not see your swap file and will not get the ATTENTION
  warning message.
On the Amiga you can also use a recoverable ram disk, but there is no 100%
guarantee that this works.  Putting swap files in a normal ram disk (like RAM:
on the Amiga) or in a place that is cleared when rebooting (like /tmp on Unix)
makes no sense, you will lose the swap file in a crash.

If you want to put swap files in a fixed place, put a command resembling the
following ones in your .vimrc:
	:set dir=dh2:tmp	(for Amiga)
	:set dir=~/tmp		(for Unix)
	:set dir=c:\\tmp	(for Win32)
This is also very handy when editing files on floppy.  Of course you will have
to create that "tmp" directory for this to work!

For read-only files, a swap file is not used.  Unless the file is big, causing
the amount of memory used to be higher than given with 'maxmem' or
'maxmemtot'.  And when making a change to a read-only file, the swap file is
created anyway.

The 'swapfile' option can be reset to avoid creating a swapfile.  And the
|:noswapfile| modifier can be used to not create a swapfile for a new buffer.

:nos[wapfile]   {command}			*:nos* *:noswapfile*
		Execute {command}. If it contains a command that loads a new
		buffer, it will be loaded without creating a swapfile and the
		'swapfile' option will be reset.  If a buffer already had a
		swapfile it is not removed and 'swapfile' is not reset.


Detecting an existing swap file ~

You can find this in the user manual, section |11.3|.


Updating the swapfile ~

The swap file is updated after typing 200 characters or when you have not
typed anything for four seconds.  This only happens if the buffer was
changed, not when you only moved around.  The reason why it is not kept up to
date all the time is that this would slow down normal work too much.  You can
change the 200 character count with the 'updatecount' option.  You can set
the time with the 'updatetime' option.  The time is given in milliseconds.
After writing to the swap file Vim syncs the file to disk.  This takes some
time, especially on busy Unix systems.  If you don't want this you can set the
'swapsync' option to an empty string.  The risk of losing work becomes bigger
though.  On some non-Unix systems (MS-Windows, Amiga) the swap file won't be
written at all.

If the writing to the swap file is not wanted, it can be switched off by
setting the 'updatecount' option to 0.  The same is done when starting Vim
with the "-n" option.  Writing can be switched back on by setting the
'updatecount' option to non-zero.  Swap files will be created for all buffers
when doing this.  But when setting 'updatecount' to zero, the existing swap
files will not be removed, it will only affect files that will be opened
after this.

If you want to make sure that your changes are in the swap file use this
command:

					*:pre* *:preserve* *E313* *E314*
:pre[serve]		Write all text for the current buffer into its swap
			file.  The original file is no longer needed for
			recovery.  This sets a flag in the current buffer.
			When the '&' flag is present in 'cpoptions' the swap
			file will not be deleted for this buffer when Vim
			exits and the buffer is still loaded |cpo-&|.

A Vim swap file can be recognized by the first six characters: "b0VIM ".
After that comes the version number, e.g., "3.0".


Links and symbolic links ~

On Unix it is possible to have two names for the same file.  This can be done
with hard links and with symbolic links (symlinks).

For hard links Vim does not know the other name of the file.  Therefore, the
name of the swapfile will be based on the name you used to edit the file.
There is no check for editing the same file by the other name too, because Vim
cannot find the other swapfile (except for searching all of your harddisk,
which would be very slow).

For symbolic links Vim resolves the links to find the name of the actual file.
The swap file name is based on that name.  Thus it doesn't matter by what name
you edit the file, the swap file name will normally be the same.  However,
there are exceptions:
- When the directory of the actual file is not writable the swapfile is put
  elsewhere.
- When the symbolic links somehow create a loop you get an *E773* error
  message and the unmodified file name will be used.  You won't be able to
  save your file normally.

==============================================================================
2. Recovery					*recovery* *E308* *E311*

Basic file recovery is explained in the user manual: |usr_11.txt|.

Another way to do recovery is to start Vim and use the ":recover" command.
This is easy when you start Vim to edit a file and you get the "ATTENTION:
Found a swap file ..." message.  In this case the single command ":recover"
will do the work.  You can also give the name of the file or the swap file to
the recover command:
					*:rec* *:recover* *E305* *E306* *E307*
:rec[over] [file]	Try to recover [file] from the swap file.  If [file]
			is not given use the file name for the current
			buffer.  The current contents of the buffer are lost.
			This command fails if the buffer was modified.

:rec[over]! [file]	Like ":recover", but any changes in the current
			buffer are lost.

						*E312* *E309* *E310* *E1364*
Vim has some intelligence about what to do if the swap file is corrupt in
some way.  If Vim has doubt about what it found, it will give an error
message and insert lines with "???" in the text.  If you see an error message
while recovering, search in the file for "???" to see what is wrong.  You may
want to cut and paste to get the text you need.

The most common remark is "???LINES MISSING".  This means that Vim cannot read
the text from the original file.  This can happen if the system crashed and
parts of the original file were not written to disk.

Be sure that the recovery was successful before overwriting the original
file or deleting the swap file.  It is good practice to write the recovered
file elsewhere and run 'diff' to find out if the changes you want are in the
recovered file.  Or use |:DiffOrig|.

Once you are sure the recovery is ok delete the swap file.  Otherwise, you
will continue to get warning messages that the ".swp" file already exists.



ENCRYPTION AND THE SWAP FILE				*:recover-crypt*

When the text file is encrypted the swap file is encrypted as well.  This
makes recovery a bit more complicated.  When recovering from a swap file and
encryption has been used, you will be asked to enter one or two crypt keys.

If the text file does not exist you will only be asked to enter the crypt key
for the swap file.

If the text file does exist, it may be encrypted in a different way than the
swap file.  You will be asked for the crypt key twice:

	Need encryption key for "/tmp/tt" ~
	Enter encryption key: ****** ~
	"/tmp/tt" [crypted] 23200L, 522129C ~
	Using swap file "/tmp/.tt.swp" ~
	Original file "/tmp/tt" ~
	Swap file is encrypted: "/tmp/.tt.swp" ~
	If you entered a new crypt key but did not write the text file, ~
	enter the new crypt key. ~
	If you wrote the text file after changing the crypt key press enter ~
	to use the same key for text file and swap file ~
	Enter encryption key:  ~

You can be in one of these two situations:

1. The encryption key was not changed, or after changing the key the text file
   was written.  You will be prompted for the crypt key twice.  The second
   time you can simply press Enter.  That means the same key is used for the
   text file and the swap file.
2. You entered a new encryption key, but did not save the text file.  Vim will
   then use the new key for the swap file, and the text file will still be
   encrypted with the old key.  At the second prompt enter the new key.

Note that after recovery the key of the swap file will be used for the text
file.  Thus if you write the text file, you need to use that new key.


 vim:tw=78:ts=8:noet:ft=help:norl: