Mercurial > vim
view src/vim9.h @ 32936:c517845bd10e v9.0.1776
patch 9.0.1776: No support for stable Python 3 ABI
Commit: https://github.com/vim/vim/commit/c13b3d1350b60b94fe87f0761ea31c0e7fb6ebf3
Author: Yee Cheng Chin <ychin.git@gmail.com>
Date: Sun Aug 20 21:18:38 2023 +0200
patch 9.0.1776: No support for stable Python 3 ABI
Problem: No support for stable Python 3 ABI
Solution: Support Python 3 stable ABI
Commits:
1) Support Python 3 stable ABI to allow mixed version interoperatbility
Vim currently supports embedding Python for use with plugins, and the
"dynamic" linking option allows the user to specify a locally installed
version of Python by setting `pythonthreedll`. However, one caveat is
that the Python 3 libs are not binary compatible across minor versions,
and mixing versions can potentially be dangerous (e.g. let's say Vim was
linked against the Python 3.10 SDK, but the user sets `pythonthreedll`
to a 3.11 lib). Usually, nothing bad happens, but in theory this could
lead to crashes, memory corruption, and other unpredictable behaviors.
It's also difficult for the user to tell something is wrong because Vim
has no way of reporting what Python 3 version Vim was linked with.
For Vim installed via a package manager, this usually isn't an issue
because all the dependencies would already be figured out. For prebuilt
Vim binaries like MacVim (my motivation for working on this), AppImage,
and Win32 installer this could potentially be an issue as usually a
single binary is distributed. This is more tricky when a new Python
version is released, as there's a chicken-and-egg issue with deciding
what Python version to build against and hard to keep in sync when a new
Python version just drops and we have a mix of users of different Python
versions, and a user just blindly upgrading to a new Python could lead to
bad interactions with Vim.
Python 3 does have a solution for this problem: stable ABI / limited API
(see https://docs.python.org/3/c-api/stable.html). The C SDK limits the
API to a set of functions that are promised to be stable across
versions. This pull request adds an ifdef config that allows us to turn
it on when building Vim. Vim binaries built with this option should be
safe to freely link with any Python 3 libraies without having the
constraint of having to use the same minor version.
Note: Python 2 has no such concept and this doesn't change how Python 2
integration works (not that there is going to be a new version of Python
2 that would cause compatibility issues in the future anyway).
---
Technical details:
======
The stable ABI can be accessed when we compile with the Python 3 limited
API (by defining `Py_LIMITED_API`). The Python 3 code (in `if_python3.c`
and `if_py_both.h`) would now handle this and switch to limited API
mode. Without it set, Vim will still use the full API as before so this
is an opt-in change.
The main difference is that `PyType_Object` is now an opaque struct that
we can't directly create "static types" out of, and we have to create
type objects as "heap types" instead. This is because the struct is not
stable and changes from version to version (e.g. 3.8 added a
`tp_vectorcall` field to it). I had to change all the types to be
allocated on the heap instead with just a pointer to them.
Other functions are also simply missing in limited API, or they are
introduced too late (e.g. `PyUnicode_AsUTF8AndSize` in 3.10) to it that
we need some other ways to do the same thing, so I had to abstract a few
things into macros, and sometimes re-implement functions like
`PyObject_NEW`.
One caveat is that in limited API, `OutputType` (used for replacing
`sys.stdout`) no longer inherits from `PyStdPrinter_Type` which I don't
think has any real issue other than minor differences in how they
convert to a string and missing a couple functions like `mode()` and
`fileno()`.
Also fixed an existing bug where `tp_basicsize` was set incorrectly for
`BufferObject`, `TabListObject, `WinListObject`.
Technically, there could be a small performance drop, there is a little
more indirection with accessing type objects, and some APIs like
`PyUnicode_AsUTF8AndSize` are missing, but in practice I didn't see any
difference, and any well-written Python plugin should try to avoid
excessing callbacks to the `vim` module in Python anyway.
I only tested limited API mode down to Python 3.7, which seemes to
compile and work fine. I haven't tried earlier Python versions.
2) Fix PyIter_Check on older Python vers / type##Ptr unused warning
For PyIter_Check, older versions exposed them as either macros (used in
full API), or a function (for use in limited API). A previous change
exposed PyIter_Check to the dynamic build because Python just moved it
to function-only in 3.10 anyway. Because of that, just make sure we
always grab the function in dynamic builds in earlier versions since
that's what Python eventually did anyway.
3) Move Py_LIMITED_API define to configure script
Can now use --with-python-stable-abi flag to customize what stable ABI
version to target. Can also use an env var to do so as well.
4) Show +python/dyn-stable in :version, and allow has() feature query
Not sure if the "/dyn-stable" suffix would break things, or whether we
should do it another way. Or just don't show it in version and rely on
has() feature checking.
5) Documentation first draft. Still need to implement v:python3_version
6) Fix PyIter_Check build breaks when compiling against Python 3.8
7) Add CI coverage stable ABI on Linux/Windows / make configurable on Windows
This adds configurable options for Windows make files (both MinGW and
MSVC). CI will also now exercise both traditional full API and stable
ABI for Linux and Windows in the matrix for coverage.
Also added a "dynamic" option to Linux matrix as a drive-by change to
make other scripting languages like Ruby / Perl testable under both
static and dynamic builds.
8) Fix inaccuracy in Windows docs
Python's own docs are confusing but you don't actually want to use
`python3.dll` for the dynamic linkage.
9) Add generated autoconf file
10) Add v:python3_version support
This variable indicates the version of Python3 that Vim was built
against (PY_VERSION_HEX), and will be useful to check whether the Python
library you are loading in dynamically actually fits it. When built with
stable ABI, it will be the limited ABI version instead
(`Py_LIMITED_API`), which indicates the minimum version of Python 3 the
user should have, rather than the exact match. When stable ABI is used,
we won't be exposing PY_VERSION_HEX in this var because it just doesn't
seem necessary to do so (the whole point of stable ABI is the promise
that it will work across versions), and I don't want to confuse the user
with too many variables.
Also, cleaned up some documentation, and added help tags.
11) Fix Python 3.7 compat issues
Fix a couple issues when using limited API < 3.8
- Crash on exit: In Python 3.7, if a heap-allocated type is destroyed
before all instances are, it would cause a crash later. This happens
when we destroyed `OptionsType` before calling `Py_Finalize` when
using the limited API. To make it worse, later versions changed the
semantics and now each instance has a strong reference to its own type
and the recommendation has changed to have each instance de-ref its
own type and have its type in GC traversal. To avoid dealing with
these cross-version variations, we just don't free the heap type. They
are static types in non-limited-API anyway and are designed to last
through the entirety of the app, and we also don't restart the Python
runtime and therefore do not need it to have absolutely 0 leaks.
See:
- https://docs.python.org/3/whatsnew/3.8.html#changes-in-the-c-api
- https://docs.python.org/3/whatsnew/3.9.html#changes-in-the-c-api
- PyIter_Check: This function is not provided in limited APIs older than
3.8. Previously I was trying to mock it out using manual
PyType_GetSlot() but it was brittle and also does not actually work
properly for static types (it will generate a Python error). Just
return false. It does mean using limited API < 3.8 is not recommended
as you lose the functionality to handle iterators, but from playing
with plugins I couldn't find it to be an issue.
- Fix loading of PyIter_Check so it will be done when limited API < 3.8.
Otherwise loading a 3.7 Python lib will fail even if limited API was
specified to use it.
12) Make sure to only load `PyUnicode_AsUTF8AndSize` in needed in limited API
We don't use this function unless limited API >= 3.10, but we were
loading it regardless. Usually it's ok in Unix-like systems where Python
just has a single lib that we load from, but in Windows where there is a
separate python3.dll this would not work as the symbol would not have
been exposed in this more limited DLL file. This makes it much clearer
under what condition is this function needed.
closes: #12032
Signed-off-by: Christian Brabandt <cb@256bit.org>
Co-authored-by: Yee Cheng Chin <ychin.git@gmail.com>
author | Christian Brabandt <cb@256bit.org> |
---|---|
date | Sun, 20 Aug 2023 21:30:04 +0200 |
parents | 695b50472e85 |
children | 9efd99a717c1 |
line wrap: on
line source
/* vi:set ts=8 sts=4 sw=4 noet: * * VIM - Vi IMproved by Bram Moolenaar * * Do ":help uganda" in Vim to read copying and usage conditions. * Do ":help credits" in Vim to see a list of people who contributed. * See README.txt for an overview of the Vim source code. */ /* * vim9.h: types and globals used for Vim9 script. */ #ifdef VMS # include <float.h> #endif typedef enum { ISN_EXEC, // execute Ex command line isn_arg.string ISN_EXECCONCAT, // execute Ex command from isn_arg.number items on stack ISN_EXEC_SPLIT, // execute Ex command from isn_arg.string split at NL ISN_EXECRANGE, // execute EX command that is only a range ISN_LEGACY_EVAL, // evaluate expression isn_arg.string with legacy syntax. ISN_ECHO, // :echo with isn_arg.echo.echo_count items on top of stack ISN_EXECUTE, // :execute with isn_arg.number items on top of stack ISN_ECHOMSG, // :echomsg with isn_arg.number items on top of stack ISN_ECHOCONSOLE, // :echoconsole with isn_arg.number items on top of stack ISN_ECHOWINDOW, // :echowindow with isn_arg.number items on top of stack ISN_ECHOERR, // :echoerr with isn_arg.number items on top of stack ISN_RANGE, // compute range from isn_arg.string, push to stack ISN_SUBSTITUTE, // :s command with expression ISN_SOURCE, // source autoload script, isn_arg.number is the script ID ISN_INSTR, // instructions compiled from expression ISN_CONSTRUCT, // construct an object, using contstruct_T ISN_GET_OBJ_MEMBER, // object member, index is isn_arg.number ISN_GET_ITF_MEMBER, // interface member, index is isn_arg.classmember ISN_STORE_THIS, // store value in "this" object member, index is // isn_arg.number ISN_LOAD_CLASSMEMBER, // load class member, using isn_arg.classmember ISN_STORE_CLASSMEMBER, // store in class member, using isn_arg.classmember // get and set variables ISN_LOAD, // push local variable isn_arg.number ISN_LOADV, // push v: variable isn_arg.number ISN_LOADG, // push g: variable isn_arg.string ISN_LOADAUTO, // push g: autoload variable isn_arg.string ISN_LOADB, // push b: variable isn_arg.string ISN_LOADW, // push w: variable isn_arg.string ISN_LOADT, // push t: variable isn_arg.string ISN_LOADGDICT, // push g: dict ISN_LOADBDICT, // push b: dict ISN_LOADWDICT, // push w: dict ISN_LOADTDICT, // push t: dict ISN_LOADS, // push s: variable isn_arg.loadstore ISN_LOADEXPORT, // push exported variable isn_arg.loadstore ISN_LOADOUTER, // push variable from outer scope isn_arg.outer ISN_LOADSCRIPT, // push script-local variable isn_arg.script. ISN_LOADOPT, // push option isn_arg.string ISN_LOADENV, // push environment variable isn_arg.string ISN_LOADREG, // push register isn_arg.number ISN_STORE, // pop into local variable isn_arg.number ISN_STOREV, // pop into v: variable isn_arg.number ISN_STOREG, // pop into global variable isn_arg.string ISN_STOREAUTO, // pop into global autoload variable isn_arg.string ISN_STOREB, // pop into buffer-local variable isn_arg.string ISN_STOREW, // pop into window-local variable isn_arg.string ISN_STORET, // pop into tab-local variable isn_arg.string ISN_STORES, // pop into script variable isn_arg.loadstore ISN_STOREEXPORT, // pop into exported script variable isn_arg.loadstore ISN_STOREOUTER, // pop variable into outer scope isn_arg.outer ISN_STORESCRIPT, // pop into script variable isn_arg.script ISN_STOREOPT, // pop into option isn_arg.storeopt ISN_STOREFUNCOPT, // pop into option isn_arg.storeopt ISN_STOREENV, // pop into environment variable isn_arg.string ISN_STOREREG, // pop into register isn_arg.number // ISN_STOREOTHER, // pop into other script variable isn_arg.other. ISN_STORENR, // store number into local variable isn_arg.storenr.stnr_idx ISN_STOREINDEX, // store into list or dictionary, using // isn_arg.storeindex; value/index/variable on stack ISN_STORERANGE, // store into blob, // value/index 1/index 2/variable on stack ISN_UNLET, // unlet variable isn_arg.unlet.ul_name ISN_UNLETENV, // unlet environment variable isn_arg.unlet.ul_name ISN_UNLETINDEX, // unlet item of list or dict ISN_UNLETRANGE, // unlet items of list ISN_LOCKUNLOCK, // :lock and :unlock for local variable member ISN_LOCKCONST, // lock constant value // constants ISN_PUSHNR, // push number isn_arg.number ISN_PUSHBOOL, // push bool value isn_arg.number ISN_PUSHSPEC, // push special value isn_arg.number ISN_PUSHF, // push float isn_arg.fnumber ISN_PUSHS, // push string isn_arg.string ISN_PUSHBLOB, // push blob isn_arg.blob ISN_PUSHFUNC, // push func isn_arg.string ISN_PUSHCHANNEL, // push NULL channel ISN_PUSHJOB, // push NULL job ISN_PUSHOBJ, // push NULL object ISN_PUSHCLASS, // push class, uses isn_arg.classarg ISN_NEWLIST, // push list from stack items, size is isn_arg.number // -1 for null_list ISN_NEWDICT, // push dict from stack items, size is isn_arg.number // -1 for null_dict ISN_NEWPARTIAL, // push NULL partial ISN_AUTOLOAD, // get item from autoload import, function or variable // function call ISN_BCALL, // call builtin function isn_arg.bfunc ISN_DCALL, // call def function isn_arg.dfunc ISN_METHODCALL, // call method on interface, uses isn_arg.mfunc ISN_UCALL, // call user function or funcref/partial isn_arg.ufunc ISN_PCALL, // call partial, use isn_arg.pfunc ISN_PCALL_END, // cleanup after ISN_PCALL with cpf_top set ISN_RETURN, // return, result is on top of stack ISN_RETURN_VOID, // Push void, then return ISN_RETURN_OBJECT, // Push constructed object, then return ISN_FUNCREF, // push a function ref to dfunc isn_arg.funcref ISN_NEWFUNC, // create a global function from a lambda function ISN_DEF, // list functions ISN_DEFER, // :defer argument count is isn_arg.number ISN_DEFEROBJ, // idem, function is an object method // expression operations ISN_JUMP, // jump if condition is matched isn_arg.jump ISN_JUMP_IF_ARG_SET, // jump if argument is already set, uses // isn_arg.jumparg ISN_JUMP_IF_ARG_NOT_SET, // jump if argument is not set, uses // isn_arg.jumparg // loop ISN_FOR, // get next item from a list, uses isn_arg.forloop ISN_WHILE, // jump if condition false, store funcref count, uses // isn_arg.whileloop ISN_ENDLOOP, // handle variables for closures, uses isn_arg.endloop ISN_TRY, // add entry to ec_trystack, uses isn_arg.tryref ISN_THROW, // pop value of stack, store in v:exception ISN_PUSHEXC, // push v:exception ISN_CATCH, // drop v:exception ISN_FINALLY, // start of :finally block ISN_ENDTRY, // take entry off from ec_trystack ISN_TRYCONT, // handle :continue or :break inside a :try statement // more expression operations ISN_ADDLIST, // add two lists ISN_ADDBLOB, // add two blobs // operation with two arguments; isn_arg.op.op_type is exprtype_T ISN_OPNR, ISN_OPFLOAT, ISN_OPANY, // comparative operations; isn_arg.op.op_type is exprtype_T, op_ic used ISN_COMPAREBOOL, ISN_COMPARESPECIAL, ISN_COMPARENULL, ISN_COMPARENR, ISN_COMPAREFLOAT, ISN_COMPARESTRING, ISN_COMPAREBLOB, ISN_COMPARELIST, ISN_COMPAREDICT, ISN_COMPAREFUNC, ISN_COMPAREANY, ISN_COMPARECLASS, ISN_COMPAREOBJECT, // expression operations ISN_CONCAT, // concatenate isn_arg.number strings ISN_STRINDEX, // [expr] string index ISN_STRSLICE, // [expr:expr] string slice ISN_LISTAPPEND, // append to a list, like add() ISN_LISTINDEX, // [expr] list index ISN_LISTSLICE, // [expr:expr] list slice ISN_BLOBINDEX, // [expr] blob index ISN_BLOBSLICE, // [expr:expr] blob slice ISN_ANYINDEX, // [expr] runtime index ISN_ANYSLICE, // [expr:expr] runtime slice ISN_SLICE, // drop isn_arg.number items from start of list ISN_BLOBAPPEND, // append to a blob, like add() ISN_GETITEM, // push list item, isn_arg.number is the index ISN_MEMBER, // dict[member] ISN_STRINGMEMBER, // dict.member using isn_arg.string ISN_2BOOL, // falsy/truthy to bool, uses isn_arg.tobool ISN_COND2BOOL, // convert value to bool ISN_2STRING, // convert value to string at isn_arg.tostring on stack ISN_2STRING_ANY, // like ISN_2STRING but check type ISN_NEGATENR, // apply "-" to number ISN_CHECKTYPE, // check value type is isn_arg.type.ct_type ISN_CHECKLEN, // check list length is isn_arg.checklen.cl_min_len ISN_SETTYPE, // set dict type to isn_arg.type.ct_type ISN_CLEARDICT, // clear dict saved by ISN_MEMBER/ISN_STRINGMEMBER ISN_USEDICT, // use or clear dict saved by ISN_MEMBER/ISN_STRINGMEMBER ISN_PUT, // ":put", uses isn_arg.put ISN_CMDMOD, // set cmdmod ISN_CMDMOD_REV, // undo ISN_CMDMOD ISN_PROF_START, // start a line for profiling ISN_PROF_END, // end a line for profiling ISN_DEBUG, // check for debug breakpoint, uses isn_arg.debug ISN_UNPACK, // unpack list into items, uses isn_arg.unpack ISN_SHUFFLE, // move item on stack up or down ISN_DROP, // pop stack and discard value ISN_REDIRSTART, // :redir => ISN_REDIREND, // :redir END, isn_arg.number == 1 for append ISN_CEXPR_AUCMD, // first part of :cexpr isn_arg.number is cmdidx ISN_CEXPR_CORE, // second part of :cexpr, uses isn_arg.cexpr ISN_FINISH // end marker in list of instructions } isntype_T; // arguments to ISN_BCALL typedef struct { int cbf_idx; // index in "global_functions" int cbf_argcount; // number of arguments on top of stack } cbfunc_T; // arguments to ISN_DCALL typedef struct { int cdf_idx; // index in "def_functions" for ISN_DCALL int cdf_argcount; // number of arguments on top of stack } cdfunc_T; // arguments to ISN_METHODCALL typedef struct { class_T *cmf_itf; // interface used int cmf_idx; // index in "def_functions" for ISN_DCALL int cmf_argcount; // number of arguments on top of stack } cmfunc_T; // arguments to ISN_PCALL typedef struct { int cpf_top; // when TRUE partial is above the arguments int cpf_argcount; // number of arguments on top of stack } cpfunc_T; // arguments to ISN_UCALL and ISN_XCALL typedef struct { char_u *cuf_name; int cuf_argcount; // number of arguments on top of stack } cufunc_T; // arguments to ISN_GETITEM typedef struct { varnumber_T gi_index; int gi_with_op; } getitem_T; typedef enum { JUMP_ALWAYS, JUMP_NEVER, JUMP_IF_FALSE, // pop and jump if false JUMP_WHILE_FALSE, // pop and jump if false for :while JUMP_AND_KEEP_IF_TRUE, // jump if top of stack is truthy, drop if not JUMP_IF_COND_TRUE, // jump if top of stack is true, drop if not JUMP_IF_COND_FALSE, // jump if top of stack is false, drop if not } jumpwhen_T; // arguments to ISN_JUMP typedef struct { jumpwhen_T jump_when; int jump_where; // position to jump to } jump_T; // arguments to ISN_JUMP_IF_ARG_SET and ISN_JUMP_IF_ARG_NOT_SET typedef struct { int jump_arg_off; // argument index, negative int jump_where; // position to jump to } jumparg_T; // arguments to ISN_FOR typedef struct { short for_loop_idx; // loop variable index int for_end; // position to jump to after done } forloop_T; // arguments to ISN_WHILE typedef struct { short while_funcref_idx; // variable index for funcref count int while_end; // position to jump to after done } whileloop_T; // arguments to ISN_ENDLOOP typedef struct { short end_funcref_idx; // variable index of funcrefs.ga_len short end_depth; // nested loop depth short end_var_idx; // first variable declared in the loop short end_var_count; // number of variables declared in the loop } endloop_T; // indirect arguments to ISN_TRY typedef struct { int try_catch; // position to jump to on throw int try_finally; // :finally or :endtry position to jump to int try_endtry; // :endtry position to jump to } tryref_T; // arguments to ISN_TRY typedef struct { tryref_T *try_ref; } try_T; // arguments to ISN_TRYCONT typedef struct { int tct_levels; // number of nested try statements int tct_where; // position to jump to, WHILE or FOR } trycont_T; // arguments to ISN_ECHO typedef struct { int echo_with_white; // :echo instead of :echon int echo_count; // number of expressions } echo_T; // arguments to ISN_OPNR, ISN_OPFLOAT, etc. typedef struct { exprtype_T op_type; int op_ic; // TRUE with '#', FALSE with '?', else MAYBE } opexpr_T; // arguments to ISN_CHECKTYPE typedef struct { type_T *ct_type; int8_T ct_off; // offset in stack, -1 is bottom int8_T ct_arg_idx; // argument index or zero int8_T ct_is_var; // when TRUE checking variable instead of arg } checktype_T; // arguments to ISN_STORENR typedef struct { int stnr_idx; varnumber_T stnr_val; } storenr_T; // arguments to ISN_STOREOPT and ISN_STOREFUNCOPT typedef struct { char_u *so_name; int so_flags; } storeopt_T; // arguments to ISN_LOADS and ISN_STORES typedef struct { char_u *ls_name; // variable name (with s: for ISN_STORES) int ls_sid; // script ID } loadstore_T; // arguments to ISN_LOADSCRIPT and ISN_STORESCRIPT typedef struct { int sref_sid; // script ID int sref_idx; // index in sn_var_vals int sref_seq; // sn_script_seq when compiled type_T *sref_type; // type of the variable when compiled } scriptref_T; typedef struct { scriptref_T *scriptref; } script_T; // arguments to ISN_UNLET typedef struct { char_u *ul_name; // variable name with g:, w:, etc. int ul_forceit; // forceit flag } unlet_T; // extra arguments for funcref_T typedef struct { char_u *fre_func_name; // function name for legacy function loopvarinfo_T fre_loopvar_info; // info about variables inside loops class_T *fre_class; // class for a method int fre_method_idx; // method index on "fre_class" } funcref_extra_T; // arguments to ISN_FUNCREF typedef struct { int fr_dfunc_idx; // function index for :def function funcref_extra_T *fr_extra; // optional extra information } funcref_T; // arguments to ISN_NEWFUNC typedef struct { char_u *nfa_lambda; // name of the lambda already defined char_u *nfa_global; // name of the global function to be created loopvarinfo_T nfa_loopvar_info; // ifno about variables inside loops } newfuncarg_T; typedef struct { newfuncarg_T *nf_arg; } newfunc_T; // arguments to ISN_CHECKLEN typedef struct { int cl_min_len; // minimum length int cl_more_OK; // longer is allowed } checklen_T; // arguments to ISN_SHUFFLE typedef struct { int shfl_item; // item to move (relative to top of stack) int shfl_up; // places to move upwards } shuffle_T; // arguments to ISN_PUT typedef struct { int put_regname; // register, can be NUL linenr_T put_lnum; // line number to put below } put_T; // arguments to ISN_CMDMOD typedef struct { cmdmod_T *cf_cmdmod; // allocated } cmod_T; // arguments to ISN_UNPACK typedef struct { int unp_count; // number of items to produce int unp_semicolon; // last item gets list of remainder } unpack_T; // arguments to ISN_LOADOUTER and ISN_STOREOUTER typedef struct { int outer_idx; // index int outer_depth; // nesting level, stack frames to go up } isn_outer_T; #define OUTER_LOOP_DEPTH -9 // used for outer_depth for loop variables // arguments to ISN_SUBSTITUTE typedef struct { char_u *subs_cmd; // :s command isn_T *subs_instr; // sequence of instructions } subs_T; // indirect arguments to ISN_TRY typedef struct { int cer_cmdidx; char_u *cer_cmdline; int cer_forceit; } cexprref_T; // arguments to ISN_CEXPR_CORE typedef struct { cexprref_T *cexpr_ref; } cexpr_T; // arguments to ISN_2STRING and ISN_2STRING_ANY typedef struct { int offset; int tolerant; } tostring_T; // arguments to ISN_2BOOL typedef struct { int offset; int invert; } tobool_T; // arguments to ISN_DEBUG typedef struct { varnumber_T dbg_var_names_len; // current number of local variables int dbg_break_lnum; // first line to break after } debug_T; // arguments to ISN_DEFER typedef struct { int defer_var_idx; // local variable index for defer list int defer_argcount; // number of arguments } deferins_T; // arguments to ISN_ECHOWINDOW typedef struct { int ewin_count; // number of arguments long ewin_time; // time argument (msec) } echowin_T; // arguments to ISN_CONSTRUCT typedef struct { int construct_size; // size of object in bytes class_T *construct_class; // class the object is created from } construct_T; // arguments to ISN_STORE_CLASSMEMBER, ISN_LOAD_CLASSMEMBER, ISN_GET_ITF_MEMBER typedef struct { class_T *cm_class; int cm_idx; } classmember_T; // arguments to ISN_STOREINDEX typedef struct { vartype_T si_vartype; class_T *si_class; } storeindex_T; /* * Instruction */ struct isn_S { isntype_T isn_type; int isn_lnum; union { char_u *string; varnumber_T number; blob_T *blob; vartype_T vartype; float_T fnumber; channel_T *channel; job_T *job; partial_T *partial; class_T *classarg; jump_T jump; jumparg_T jumparg; forloop_T forloop; whileloop_T whileloop; endloop_T endloop; try_T tryref; trycont_T trycont; cbfunc_T bfunc; cdfunc_T dfunc; cmfunc_T *mfunc; cpfunc_T pfunc; cufunc_T ufunc; echo_T echo; opexpr_T op; checktype_T type; storenr_T storenr; storeopt_T storeopt; loadstore_T loadstore; script_T script; unlet_T unlet; funcref_T funcref; newfunc_T newfunc; checklen_T checklen; shuffle_T shuffle; put_T put; cmod_T cmdmod; unpack_T unpack; isn_outer_T outer; subs_T subs; cexpr_T cexpr; isn_T *instr; tostring_T tostring; tobool_T tobool; getitem_T getitem; debug_T debug; deferins_T defer; echowin_T echowin; construct_T construct; classmember_T classmember; storeindex_T storeindex; } isn_arg; }; /* * Info about a function defined with :def. Used in "def_functions". */ struct dfunc_S { ufunc_T *df_ufunc; // struct containing most stuff int df_refcount; // how many ufunc_T point to this dfunc_T int df_idx; // index in def_functions char df_deleted; // if TRUE function was deleted char df_delete_busy; // TRUE when in // delete_def_function_contents() int df_script_seq; // Value of sctx_T sc_seq when the function // was compiled. char_u *df_name; // name used for error messages garray_T df_def_args_isn; // default argument instructions garray_T df_var_names; // names of local vars // After compiling "df_instr" and/or "df_instr_prof" is not NULL. isn_T *df_instr; // function body to be executed int df_instr_count; // size of "df_instr" int df_instr_debug_count; // size of "df_instr_debug" isn_T *df_instr_debug; // like "df_instr" with debugging #ifdef FEAT_PROFILE isn_T *df_instr_prof; // like "df_instr" with profiling int df_instr_prof_count; // size of "df_instr_prof" #endif int df_varcount; // number of local variables int df_has_closure; // one if a closure was created int df_defer_var_idx; // index of local variable that has a list // of deferred function calls; zero if not // set }; // Number of entries used by stack frame for a function call. // - ec_dfunc_idx: function index // - ec_iidx: instruction index // - ec_instr: instruction list pointer // - ec_outer: stack used for closures // - funclocal: function-local data // - ec_frame_idx: previous frame index #define STACK_FRAME_FUNC_OFF 0 #define STACK_FRAME_IIDX_OFF 1 #define STACK_FRAME_INSTR_OFF 2 #define STACK_FRAME_OUTER_OFF 3 #define STACK_FRAME_FUNCLOCAL_OFF 4 #define STACK_FRAME_IDX_OFF 5 #define STACK_FRAME_SIZE 6 extern garray_T def_functions; // Used for "lnum" when a range is to be taken from the stack. #define LNUM_VARIABLE_RANGE (-999) // Used for "lnum" when a range is to be taken from the stack and "!" is used. #define LNUM_VARIABLE_RANGE_ABOVE (-888) // Keep in sync with get_compile_type() #ifdef FEAT_PROFILE # define INSTRUCTIONS(dfunc) \ (debug_break_level > 0 || may_break_in_function(dfunc->df_ufunc) \ ? (dfunc)->df_instr_debug \ : ((do_profiling == PROF_YES && (dfunc->df_ufunc)->uf_profiling) \ ? (dfunc)->df_instr_prof \ : (dfunc)->df_instr)) #else # define INSTRUCTIONS(dfunc) \ (debug_break_level > 0 || may_break_in_function((dfunc)->df_ufunc) \ ? (dfunc)->df_instr_debug \ : (dfunc)->df_instr) #endif // Structure passed between the compile_expr* functions to keep track of // constants that have been parsed but for which no code was produced yet. If // possible expressions on these constants are applied at compile time. If // that is not possible, the code to push the constants needs to be generated // before other instructions. // Using 50 should be more than enough of 5 levels of (). #define PPSIZE 50 typedef struct { typval_T pp_tv[PPSIZE]; // stack of ppconst constants int pp_used; // active entries in pp_tv[] int pp_is_const; // all generated code was constants, used for a // list or dict with constant members } ppconst_T; // values for ctx_skip typedef enum { SKIP_NOT, // condition is a constant, produce code SKIP_YES, // condition is a constant, do NOT produce code SKIP_UNKNOWN // condition is not a constant, produce code } skip_T; /* * Chain of jump instructions where the end label needs to be set. */ typedef struct endlabel_S endlabel_T; struct endlabel_S { endlabel_T *el_next; // chain end_label locations int el_end_label; // instruction idx where to set end }; /* * info specific for the scope of :if / elseif / else */ typedef struct { int is_seen_else; int is_seen_skip_not; // a block was unconditionally executed int is_had_return; // every block ends in :return int is_if_label; // instruction idx at IF or ELSEIF endlabel_T *is_end_label; // instructions to set end label } ifscope_T; // info used by :for and :while needed for ENDLOOP typedef struct { int li_local_count; // ctx_locals.ga_len at loop start int li_closure_count; // ctx_closure_count at loop start int li_funcref_idx; // index of var that holds funcref count int li_depth; // nested loop depth } loop_info_T; /* * info specific for the scope of :while */ typedef struct { int ws_top_label; // instruction idx at WHILE endlabel_T *ws_end_label; // instructions to set end loop_info_T ws_loop_info; // info for LOOPEND } whilescope_T; /* * info specific for the scope of :for */ typedef struct { int fs_top_label; // instruction idx at FOR endlabel_T *fs_end_label; // break instructions loop_info_T fs_loop_info; // info for LOOPEND } forscope_T; /* * info specific for the scope of :try */ typedef struct { int ts_try_label; // instruction idx at TRY endlabel_T *ts_end_label; // jump to :finally or :endtry int ts_catch_label; // instruction idx of last CATCH int ts_caught_all; // "catch" without argument encountered int ts_has_finally; // "finally" encountered int ts_no_return; // one of the blocks did not end in return } tryscope_T; typedef enum { NO_SCOPE, IF_SCOPE, WHILE_SCOPE, FOR_SCOPE, TRY_SCOPE, BLOCK_SCOPE } scopetype_T; /* * Info for one scope, pointed to by "ctx_scope". */ typedef struct scope_S scope_T; struct scope_S { scope_T *se_outer; // scope containing this one scopetype_T se_type; int se_local_count; // ctx_locals.ga_len before scope skip_T se_skip_save; // ctx_skip before the block int se_loop_depth; // number of loop scopes, including this union { ifscope_T se_if; whilescope_T se_while; forscope_T se_for; tryscope_T se_try; } se_u; }; /* * Entry for "ctx_locals". Used for arguments and local variables. */ typedef struct { char_u *lv_name; type_T *lv_type; int lv_idx; // index of the variable on the stack int lv_loop_depth; // depth for variable inside a loop or -1 int lv_loop_idx; // index of first variable inside a loop or -1 int lv_from_outer; // nesting level, using ctx_outer scope int lv_const; // ASSIGN_VAR (can be assigned to), // ASSIGN_FINAL (no assignment) or ASSIGN_CONST // (value cannot be changed) int lv_arg; // when TRUE this is an argument } lvar_T; // Destination for an assignment or ":unlet" with an index. typedef enum { dest_local, dest_option, dest_func_option, dest_env, dest_global, dest_buffer, dest_window, dest_tab, dest_vimvar, dest_class_member, dest_script, dest_reg, dest_expr, } assign_dest_T; // Used by compile_lhs() to store information about the LHS of an assignment // and one argument of ":unlet" with an index. typedef struct { assign_dest_T lhs_dest; // type of destination char_u *lhs_name; // allocated name excluding the last // "[expr]" or ".name". size_t lhs_varlen; // length of the variable without // "[expr]" or ".name" char_u *lhs_whole; // allocated name including the last // "[expr]" or ".name" for :redir size_t lhs_varlen_total; // length of the variable including // any "[expr]" or ".name" char_u *lhs_dest_end; // end of the destination, including // "[expr]" or ".name". char_u *lhs_end; // end including any type int lhs_has_index; // has "[expr]" or ".name" int lhs_new_local; // create new local variable int lhs_opt_flags; // for when destination is an option int lhs_vimvaridx; // for when destination is a v:var lvar_T lhs_local_lvar; // used for existing local destination lvar_T lhs_arg_lvar; // used for argument destination lvar_T *lhs_lvar; // points to destination lvar class_T *lhs_class; // for dest_class_member int lhs_classmember_idx; // for dest_class_member int lhs_scriptvar_sid; int lhs_scriptvar_idx; int lhs_has_type; // type was specified type_T *lhs_type; int lhs_member_idx; // object member index type_T *lhs_member_type; // list/dict/object member type int lhs_append; // used by ISN_REDIREND } lhs_T; /* * Context for compiling lines of a :def function. * Stores info about the local variables and condition stack. */ struct cctx_S { ufunc_T *ctx_ufunc; // current function int ctx_lnum; // line number in current function char_u *ctx_line_start; // start of current line or NULL garray_T ctx_instr; // generated instructions int ctx_prev_lnum; // line number below previous command, for // debugging compiletype_T ctx_compile_type; garray_T ctx_locals; // currently visible local variables int ctx_has_closure; // set to one if a FUNCREF was used in the // function int ctx_closure_count; // incremented for each closure created in // the function. skip_T ctx_skip; scope_T *ctx_scope; // current scope, NULL at toplevel int ctx_had_return; // last seen statement was "return" int ctx_had_throw; // last seen statement was "throw" cctx_T *ctx_outer; // outer scope for lambda or nested // function int ctx_outer_used; // var in ctx_outer was used garray_T ctx_type_stack; // type of each item on the stack garray_T *ctx_type_list; // list of pointers to allocated types int ctx_has_cmdmod; // ISN_CMDMOD was generated lhs_T ctx_redir_lhs; // LHS for ":redir => var", valid when // lhs_name is not NULL }; /* * List of special functions for "compile_arguments()". */ typedef enum { CA_NOT_SPECIAL, CA_SEARCHPAIR, // {skip} in searchpair() and searchpairpos() CA_SUBSTITUTE, // {sub} in substitute(), when prefixed with \= } ca_special_T; // flags for typval2type() #define TVTT_DO_MEMBER 1 #define TVTT_MORE_SPECIFIC 2 // get most specific type for member // flags for call_def_function() #define DEF_USE_PT_ARGV 1 // use the partial arguments