# HG changeset patch # User Bram Moolenaar # Date 1547330406 -3600 # Node ID 1d2b5c016f17a4d597816efa0992550c2c19beff # Parent cdee6e827112e6d1bd4b95a391e6afd83d3b5861 patch 8.1.0735: cannot handle binary data commit https://github.com/vim/vim/commit/6e5ea8d2a995b32bbc5972edc4f827b959f2702f Author: Bram Moolenaar Date: Sat Jan 12 22:47:31 2019 +0100 patch 8.1.0735: cannot handle binary data Problem: Cannot handle binary data. Solution: Add the Blob type. (Yasuhiro Matsumoto, closes https://github.com/vim/vim/issues/3638) diff --git a/runtime/doc/eval.txt b/runtime/doc/eval.txt --- a/runtime/doc/eval.txt +++ b/runtime/doc/eval.txt @@ -72,6 +72,10 @@ Job Used for a job, see |job_start()|. Channel Used for a channel, see |ch_open()|. *Channel* *Channels* +Blob Binary Large Object. Stores any sequence of bytes. *Blob* + Example: 0zFF00ED015DAF + 0z is an empty Blob. + The Number and String types are converted automatically, depending on how they are used. @@ -124,7 +128,8 @@ Note that " " and "0" are also non-empty A List, Dictionary or Float is not a Number or String, thus evaluate to FALSE. *E745* *E728* *E703* *E729* *E730* *E731* *E908* *E910* *E913* -List, Dictionary, Funcref, Job and Channel types are not automatically + *E974* *E975* *E976* +List, Dictionary, Funcref, Job, Channel and Blob types are not automatically converted. *E805* *E806* *E808* @@ -1017,6 +1022,12 @@ just above. Also see |sublist| below. E :let l = mylist[4:4] " List with one item :let l = mylist[:] " shallow copy of a List +If expr8 is a |Blob| this results in a new |Blob| with the bytes in the +indexes expr1a and expr1b, inclusive. Examples: > + :let b = 0zDEADBEEF + :let bs = b[1:2] " 0zADBE + :let bs = b[] " copy ov 0zDEADBEEF + Using expr8[expr1] or expr8[expr1a : expr1b] on a |Funcref| results in an error. @@ -1156,6 +1167,14 @@ of 'encoding'. Note that "\000" and "\x00" force the end of the string. +blob-literal *blob-literal* *E973* *E977* *E978* +------------ + +Hexadecimal starting with 0z or 0Z, with an arbitrary number of bytes. +The sequence must be an even number of hex characters. Example: > + :let b = 0zFF00ED015DAF + + literal-string *literal-string* *E115* --------------- 'string' string constant *expr-'* @@ -1911,6 +1930,8 @@ v:t_none Value of None type. Read-only. v:t_number Value of Number type. Read-only. See: |type()| *v:t_string* *t_string-variable* v:t_string Value of String type. Read-only. See: |type()| + *v:t_blob* *t_blob-variable* +v:t_blob Value of Blob type. Read-only. See: |type()| *v:termresponse* *termresponse-variable* v:termresponse The escape sequence returned by the terminal for the |t_RV| @@ -2094,12 +2115,14 @@ ch_logfile({fname} [, {mode}]) none star ch_open({address} [, {options}]) Channel open a channel to {address} ch_read({handle} [, {options}]) String read from {handle} +ch_readblob({handle} [, {options}]) + Blob read Blob from {handle} ch_readraw({handle} [, {options}]) String read raw from {handle} ch_sendexpr({handle}, {expr} [, {options}]) any send {expr} over JSON {handle} -ch_sendraw({handle}, {string} [, {options}]) - any send {string} over raw {handle} +ch_sendraw({handle}, {expr} [, {options}]) + any send {expr} over raw {handle} ch_setoptions({handle}, {options}) none set options for {handle} ch_status({handle} [, {options}]) @@ -2239,8 +2262,8 @@ hlID({name}) Number syntax ID of highl hostname() String name of the machine Vim is running on iconv({expr}, {from}, {to}) String convert encoding of {expr} indent({lnum}) Number indent of line {lnum} -index({list}, {expr} [, {start} [, {ic}]]) - Number index in {list} where {expr} appears +index({object}, {expr} [, {start} [, {ic}]]) + Number index in {object} where {expr} appears input({prompt} [, {text} [, {completion}]]) String get input from the user inputdialog({prompt} [, {text} [, {completion}]]) @@ -2249,7 +2272,7 @@ inputlist({textlist}) Number let the us inputrestore() Number restore typeahead inputsave() Number save and clear typeahead inputsecret({prompt} [, {text}]) String like input() but hiding the text -insert({list}, {item} [, {idx}]) List insert {item} in {list} [before {idx}] +insert({object}, {item} [, {idx}]) List insert {item} in {object} [before {idx}] invert({expr}) Number bitwise invert isdirectory({directory}) Number |TRUE| if {directory} is a directory islocked({expr}) Number |TRUE| if {expr} is locked @@ -2339,7 +2362,7 @@ py3eval({expr}) any evaluate |python3| pyxeval({expr}) any evaluate |python_x| expression range({expr} [, {max} [, {stride}]]) List items from {expr} to {max} -readfile({fname} [, {binary} [, {max}]]) +readfile({fname} [, {type} [, {max}]]) List get list of lines from file {fname} reg_executing() String get the executing register name reg_recording() String get the recording register name @@ -2554,8 +2577,8 @@ winrestview({dict}) none restore view o winsaveview() Dict save view of current window winwidth({nr}) Number width of window {nr} wordcount() Dict get byte/char/word statistics -writefile({list}, {fname} [, {flags}]) - Number write list of lines to file {fname} +writefile({object}, {fname} [, {flags}]) + Number write |Blob| or |List| of lines to file xor({expr}, {expr}) Number bitwise XOR @@ -3199,6 +3222,11 @@ ch_read({handle} [, {options}]) *ch_ See |channel-more|. {only available when compiled with the |+channel| feature} +ch_readblob({handle} [, {options}]) *ch_readblob()* + Like ch_read() but reads binary data and returns a Blob. + See |channel-more|. + {only available when compiled with the |+channel| feature} + ch_readraw({handle} [, {options}]) *ch_readraw()* Like ch_read() but for a JS and JSON channel does not decode the message. For a NL channel it does not block waiting for @@ -3215,8 +3243,8 @@ ch_sendexpr({handle}, {expr} [, {options {only available when compiled with the |+channel| feature} -ch_sendraw({handle}, {string} [, {options}]) *ch_sendraw()* - Send {string} over {handle}. +ch_sendraw({handle}, {expr} [, {options}]) *ch_sendraw()* + Send string or Blob {expr} over {handle}. Works like |ch_sendexpr()|, but does not encode the request or decode the response. The caller is responsible for the correct contents. Also does not add a newline for a channel @@ -5375,17 +5403,21 @@ indent({lnum}) The result is a Number, w When {lnum} is invalid -1 is returned. -index({list}, {expr} [, {start} [, {ic}]]) *index()* - Return the lowest index in |List| {list} where the item has a - value equal to {expr}. There is no automatic conversion, so - the String "4" is different from the Number 4. And the number - 4 is different from the Float 4.0. The value of 'ignorecase' - is not used here, case always matters. +index({object}, {expr} [, {start} [, {ic}]]) *index()* + If {object} is a |List| return the lowest index where the item + has a value equal to {expr}. There is no automatic + conversion, so the String "4" is different from the Number 4. + And the number 4 is different from the Float 4.0. The value + of 'ignorecase' is not used here, case always matters. + + If {object} is |Blob| return the lowest index where the byte + value is equal to {expr}. + If {start} is given then start looking at the item with index {start} (may be negative for an item relative to the end). When {ic} is given and it is |TRUE|, ignore case. Otherwise case must match. - -1 is returned when {expr} is not found in {list}. + -1 is returned when {expr} is not found in {object}. Example: > :let idx = index(words, "the") :if index(numbers, 123) >= 0 @@ -5491,13 +5523,16 @@ inputsecret({prompt} [, {text}]) *inpu typed on the command-line in response to the issued prompt. NOTE: Command-line completion is not supported. -insert({list}, {item} [, {idx}]) *insert()* - Insert {item} at the start of |List| {list}. +insert({object}, {item} [, {idx}]) *insert()* + When {object} is a |List| or a |Blob| insert {item} at the start + of it. + If {idx} is specified insert {item} before the item with index {idx}. If {idx} is zero it goes before the first item, just like omitting {idx}. A negative {idx} is also possible, see |list-index|. -1 inserts just before the last item. - Returns the resulting |List|. Examples: > + + Returns the resulting |List| or |Blob|. Examples: > :let mylist = insert([2, 3, 5], 1) :call insert(mylist, 4, -1) :call insert(mylist, 6, len(mylist)) @@ -5763,6 +5798,7 @@ json_encode({expr}) *json_encode()* used recursively: [] Dict as an object (possibly null); when used recursively: {} + Blob as an array of the individual bytes v:false "false" v:true "true" v:none "null" @@ -6947,16 +6983,18 @@ range({expr} [, {max} [, {stride}]]) range(2, 0) " error! < *readfile()* -readfile({fname} [, {binary} [, {max}]]) +readfile({fname} [, {type} [, {max}]]) Read file {fname} and return a |List|, each line of the file as an item. Lines are broken at NL characters. Macintosh files separated with CR will result in a single long line (unless a NL appears somewhere). All NUL characters are replaced with a NL character. - When {binary} contains "b" binary mode is used: + When {type} contains "b" binary mode is used: - When the last line ends in a NL an extra empty list item is added. - No CR characters are removed. + When {type} contains "B" a |Blob| is returned with the binary + data of the file unmodified. Otherwise: - CR characters that appear before a NL are removed. - Whether the last line ends in a NL or not does not matter. @@ -7132,6 +7170,16 @@ remove({list}, {idx} [, {end}]) *remo Example: > :echo "last item: " . remove(mylist, -1) :call remove(mylist, 0, 9) +remove({blob}, {idx} [, {end}]) + Without {end}: Remove the byte at {idx} from |Blob| {blob} and + return the byte. + With {end}: Remove bytes from {idx} to {end} (inclusive) and + return a |Blob| with these bytes. When {idx} points to the same + byte as {end} a |Blob| with one byte is returned. When {end} + points to a byte before {idx} this is an error. + Example: > + :echo "last byte: " . remove(myblob, -1) + :call remove(mylist, 0, 9) remove({dict}, {key}) Remove the entry from {dict} with key {key}. Example: > :echo "removed " . remove(dict, "one") @@ -7172,9 +7220,11 @@ resolve({filename}) *resolve()* *E65 path name) and also keeps a trailing path separator. *reverse()* -reverse({list}) Reverse the order of items in {list} in-place. Returns - {list}. - If you want a list to remain unmodified make a copy first: > +reverse({object}) + Reverse the order of items in {object} in-place. + {object} can be a |List| or a |Blob|. + Returns {object}. + If you want an object to remain unmodified make a copy first: > :let revlist = reverse(copy(mylist)) round({expr}) *round()* @@ -9518,6 +9568,7 @@ type({expr}) The result is a Number repr None 7 |v:t_none| (v:null and v:none) Job 8 |v:t_job| Channel 9 |v:t_channel| + Blob 10 |v:t_blob| For backward compatibility, this method can be used: > :if type(myvar) == type(0) :if type(myvar) == type("") @@ -9865,14 +9916,17 @@ wordcount() *wordcount()* *writefile()* -writefile({list}, {fname} [, {flags}]) - Write |List| {list} to file {fname}. Each list item is - separated with a NL. Each list item must be a String or - Number. +writefile({object}, {fname} [, {flags}]) + When {object} is a |List| write it to file {fname}. Each list + item is separated with a NL. Each list item must be a String + or Number. When {flags} contains "b" then binary mode is used: There will not be a NL after the last list item. An empty item at the end does cause the last line in the file to end in a NL. + When {object} is a |Blob| write the bytes to file {fname} + unmodified. + When {flags} contains "a" then append mode is used, lines are appended to the file: > :call writefile(["foo"], "event.log", "a") @@ -10575,7 +10629,10 @@ 7. Commands *expression-commands* This cannot be used to set a byte in a String. You can do that like this: > :let var = var[0:2] . 'X' . var[4:] -< +< When {var-name} is a |Blob| then {idx} can be the + length of the blob, in which case one byte is + appended. + *E711* *E719* :let {var-name}[{idx1}:{idx2}] = {expr1} *E708* *E709* *E710* Set a sequence of items in a |List| to the result of diff --git a/runtime/doc/if_perl.txt b/runtime/doc/if_perl.txt --- a/runtime/doc/if_perl.txt +++ b/runtime/doc/if_perl.txt @@ -191,6 +191,9 @@ VIM::Eval({expr}) Evaluates {expr} and r A |List| is turned into a string by joining the items and inserting line breaks. + *perl-Blob* +VIM::Blob({expr}) Return Blob literal string 0zXXXX from scalar value. + *perl-SetHeight* Window->SetHeight({height}) Sets the Window height to {height}, within screen diff --git a/runtime/doc/if_ruby.txt b/runtime/doc/if_ruby.txt --- a/runtime/doc/if_ruby.txt +++ b/runtime/doc/if_ruby.txt @@ -110,6 +110,10 @@ Module Functions: Vim::message({msg}) Displays the message {msg}. + *ruby-blob* +Vim::blob({arg}) + Return Blob literal string from {arg}. + *ruby-set_option* Vim::set_option({arg}) Sets a vim option. {arg} can be any argument that the ":set" command diff --git a/src/Make_cyg_ming.mak b/src/Make_cyg_ming.mak --- a/src/Make_cyg_ming.mak +++ b/src/Make_cyg_ming.mak @@ -696,6 +696,7 @@ CUIOBJ = $(OUTDIR)/iscygpty.o OBJ = \ $(OUTDIR)/arabic.o \ $(OUTDIR)/beval.o \ + $(OUTDIR)/blob.o \ $(OUTDIR)/blowfish.o \ $(OUTDIR)/buffer.o \ $(OUTDIR)/charset.o \ diff --git a/src/Make_mvc.mak b/src/Make_mvc.mak --- a/src/Make_mvc.mak +++ b/src/Make_mvc.mak @@ -701,6 +701,7 @@ INCL = vim.h alloc.h arabic.h ascii.h ex OBJ = \ $(OUTDIR)\arabic.obj \ $(OUTDIR)\beval.obj \ + $(OUTDIR)\blob.obj \ $(OUTDIR)\blowfish.obj \ $(OUTDIR)\buffer.obj \ $(OUTDIR)\charset.obj \ @@ -1346,6 +1347,8 @@ testclean: $(OUTDIR)/beval.obj: $(OUTDIR) beval.c $(INCL) +$(OUTDIR)/blob.obj: $(OUTDIR) blob.c $(INCL) + $(OUTDIR)/blowfish.obj: $(OUTDIR) blowfish.c $(INCL) $(OUTDIR)/buffer.obj: $(OUTDIR) buffer.c $(INCL) @@ -1616,6 +1619,7 @@ auto: # End Custom Build proto.h: \ proto/arabic.pro \ + proto/blob.pro \ proto/blowfish.pro \ proto/buffer.pro \ proto/charset.pro \ diff --git a/src/Makefile b/src/Makefile --- a/src/Makefile +++ b/src/Makefile @@ -1577,6 +1577,7 @@ include testdir/Make_all.mak BASIC_SRC = \ arabic.c \ beval.c \ + blob.c \ blowfish.c \ buffer.c \ charset.c \ @@ -1693,6 +1694,7 @@ OBJ_COMMON = \ objects/arabic.o \ objects/beval.o \ objects/buffer.o \ + objects/blob.o \ objects/blowfish.o \ objects/crypt.o \ objects/crypt_zip.o \ @@ -2943,6 +2945,9 @@ objects/.dirstamp: objects/arabic.o: arabic.c $(CCC) -o $@ arabic.c +objects/blob.o: blob.c + $(CCC) -o $@ blob.c + objects/blowfish.o: blowfish.c $(CCC) -o $@ blowfish.c @@ -3395,6 +3400,10 @@ objects/beval.o: beval.c vim.h protodef. auto/osdef.h ascii.h keymap.h term.h macros.h option.h beval.h \ proto/gui_beval.pro structs.h regexp.h gui.h alloc.h ex_cmds.h spell.h \ proto.h globals.h farsi.h arabic.h +objects/blob.o: blob.c vim.h protodef.h auto/config.h feature.h os_unix.h \ + auto/osdef.h ascii.h keymap.h term.h macros.h option.h beval.h \ + proto/gui_beval.pro structs.h regexp.h gui.h alloc.h ex_cmds.h spell.h \ + proto.h globals.h farsi.h arabic.h objects/blowfish.o: blowfish.c vim.h protodef.h auto/config.h feature.h os_unix.h \ auto/osdef.h ascii.h keymap.h term.h macros.h option.h beval.h \ proto/gui_beval.pro structs.h regexp.h gui.h alloc.h ex_cmds.h spell.h \ diff --git a/src/blob.c b/src/blob.c new file mode 100644 --- /dev/null +++ b/src/blob.c @@ -0,0 +1,167 @@ +/* vi:set ts=8 sts=4 sw=4 noet: + * + * VIM - Vi IMproved by Bram Moolenaar + * + * Do ":help uganda" in Vim to read copying and usage conditions. + * Do ":help credits" in Vim to see a list of people who contributed. + * See README.txt for an overview of the Vim source code. + */ + +/* + * blob.c: Blob support by Yasuhiro Matsumoto + */ + +#include "vim.h" + +#if defined(FEAT_EVAL) || defined(PROTO) + +/* + * Allocate an empty blob. + * Caller should take care of the reference count. + */ + blob_T * +blob_alloc(void) +{ + blob_T *blob = (blob_T *)alloc_clear(sizeof(blob_T)); + + if (blob != NULL) + ga_init2(&blob->bv_ga, 1, 100); + return blob; +} + +/* + * Allocate an empty blob for a return value, with reference count set. + * Returns OK or FAIL. + */ + int +rettv_blob_alloc(typval_T *rettv) +{ + blob_T *b = blob_alloc(); + + if (b == NULL) + return FAIL; + + rettv_blob_set(rettv, b); + return OK; +} + +/* + * Set a blob as the return value. + */ + void +rettv_blob_set(typval_T *rettv, blob_T *b) +{ + rettv->v_type = VAR_BLOB; + rettv->vval.v_blob = b; + if (b != NULL) + ++b->bv_refcount; +} + + void +blob_free(blob_T *b) +{ + ga_clear(&b->bv_ga); + vim_free(b); +} + +/* + * Unreference a blob: decrement the reference count and free it when it + * becomes zero. + */ + void +blob_unref(blob_T *b) +{ + if (b != NULL && --b->bv_refcount <= 0) + blob_free(b); +} + +/* + * Get the length of data. + */ + long +blob_len(blob_T *b) +{ + if (b == NULL) + return 0L; + return b->bv_ga.ga_len; +} + +/* + * Get byte "idx" in blob "b". + * Caller must check that "idx" is valid. + */ + char_u +blob_get(blob_T *b, int idx) +{ + return ((char_u*)b->bv_ga.ga_data)[idx]; +} + +/* + * Store one byte "c" in blob "b" at "idx". + * Caller must make sure that "idx" is valid. + */ + void +blob_set(blob_T *b, int idx, char_u c) +{ + ((char_u*)b->bv_ga.ga_data)[idx] = c; +} + +/* + * Return TRUE when two blobs have exactly the same values. + */ + int +blob_equal( + blob_T *b1, + blob_T *b2) +{ + int i; + + if (b1 == NULL || b2 == NULL) + return FALSE; + if (b1 == b2) + return TRUE; + if (blob_len(b1) != blob_len(b2)) + return FALSE; + + for (i = 0; i < b1->bv_ga.ga_len; i++) + if (blob_get(b1, i) != blob_get(b2, i)) return FALSE; + return TRUE; +} + +/* + * Read "blob" from file "fd". + * Return OK or FAIL. + */ + int +read_blob(FILE *fd, blob_T *blob) +{ + struct stat st; + + if (fstat(fileno(fd), &st) < 0) + return FAIL; + if (ga_grow(&blob->bv_ga, st.st_size) == FAIL) + return FAIL; + blob->bv_ga.ga_len = st.st_size; + if (fread(blob->bv_ga.ga_data, 1, blob->bv_ga.ga_len, fd) + < (size_t)blob->bv_ga.ga_len) + return FAIL; + return OK; +} + +/* + * Write "blob" to file "fd". + * Return OK or FAIL. + */ + int +write_blob(FILE *fd, blob_T *blob) +{ + if (fwrite(blob->bv_ga.ga_data, 1, blob->bv_ga.ga_len, fd) + < (size_t)blob->bv_ga.ga_len) + { + EMSG(_(e_write)); + return FAIL; + } + return OK; +} + +#endif /* defined(FEAT_EVAL) */ diff --git a/src/channel.c b/src/channel.c --- a/src/channel.c +++ b/src/channel.c @@ -1665,7 +1665,7 @@ channel_first_nl(readq_T *node) * Returns NULL if there is nothing. */ char_u * -channel_get(channel_T *channel, ch_part_T part) +channel_get(channel_T *channel, ch_part_T part, int *outlen) { readq_T *head = &channel->ch_part[part].ch_head; readq_T *node = head->rq_next; @@ -1673,6 +1673,8 @@ channel_get(channel_T *channel, ch_part_ if (node == NULL) return NULL; + if (outlen != NULL) + *outlen += node->rq_buflen; /* dispose of the node but keep the buffer */ p = node->rq_buffer; head->rq_next = node->rq_next; @@ -1689,7 +1691,7 @@ channel_get(channel_T *channel, ch_part_ * Replaces NUL bytes with NL. */ static char_u * -channel_get_all(channel_T *channel, ch_part_T part) +channel_get_all(channel_T *channel, ch_part_T part, int *outlen) { readq_T *head = &channel->ch_part[part].ch_head; readq_T *node = head->rq_next; @@ -1699,7 +1701,7 @@ channel_get_all(channel_T *channel, ch_p /* If there is only one buffer just get that one. */ if (head->rq_next == NULL || head->rq_next->rq_next == NULL) - return channel_get(channel, part); + return channel_get(channel, part, outlen); /* Concatenate everything into one buffer. */ for (node = head->rq_next; node != NULL; node = node->rq_next) @@ -1718,10 +1720,16 @@ channel_get_all(channel_T *channel, ch_p /* Free all buffers */ do { - p = channel_get(channel, part); + p = channel_get(channel, part, NULL); vim_free(p); } while (p != NULL); + if (outlen != NULL) + { + *outlen += len; + return res; + } + /* turn all NUL into NL */ while (len > 0) { @@ -1893,7 +1901,7 @@ channel_fill(js_read_T *reader) { channel_T *channel = (channel_T *)reader->js_cookie; ch_part_T part = reader->js_cookie_arg; - char_u *next = channel_get(channel, part); + char_u *next = channel_get(channel, part, NULL); int keeplen; int addlen; char_u *p; @@ -1942,7 +1950,7 @@ channel_parse_json(channel_T *channel, c if (channel_peek(channel, part) == NULL) return FALSE; - reader.js_buf = channel_get(channel, part); + reader.js_buf = channel_get(channel, part, NULL); reader.js_used = 0; reader.js_fill = channel_fill; reader.js_cookie = channel; @@ -2475,7 +2483,7 @@ drop_messages(channel_T *channel, ch_par { char_u *msg; - while ((msg = channel_get(channel, part)) != NULL) + while ((msg = channel_get(channel, part, NULL)) != NULL) { ch_log(channel, "Dropping message '%s'", (char *)msg); vim_free(msg); @@ -2639,7 +2647,7 @@ may_invoke_callback(channel_T *channel, if (nl + 1 == buf + node->rq_buflen) { /* get the whole buffer, drop the NL */ - msg = channel_get(channel, part); + msg = channel_get(channel, part, NULL); *nl = NUL; } else @@ -2655,7 +2663,7 @@ may_invoke_callback(channel_T *channel, /* For a raw channel we don't know where the message ends, just * get everything we have. * Convert NUL to NL, the internal representation. */ - msg = channel_get_all(channel, part); + msg = channel_get_all(channel, part, NULL); } if (msg == NULL) @@ -3007,7 +3015,7 @@ channel_clear_one(channel_T *channel, ch cbq_T *cb_head = &ch_part->ch_cb_head; while (channel_peek(channel, part) != NULL) - vim_free(channel_get(channel, part)); + vim_free(channel_get(channel, part, NULL)); while (cb_head->cq_next != NULL) { @@ -3381,7 +3389,8 @@ channel_read(channel_T *channel, ch_part * Returns NULL in case of error or timeout. */ static char_u * -channel_read_block(channel_T *channel, ch_part_T part, int timeout, int raw) +channel_read_block( + channel_T *channel, ch_part_T part, int timeout, int raw, int *outlen) { char_u *buf; char_u *msg; @@ -3422,9 +3431,9 @@ channel_read_block(channel_T *channel, c } /* We have a complete message now. */ - if (mode == MODE_RAW) + if (mode == MODE_RAW || outlen != NULL) { - msg = channel_get_all(channel, part); + msg = channel_get_all(channel, part, outlen); } else { @@ -3441,12 +3450,12 @@ channel_read_block(channel_T *channel, c if (nl == NULL) { /* must be a closed channel with missing NL */ - msg = channel_get(channel, part); + msg = channel_get(channel, part, NULL); } else if (nl + 1 == buf + node->rq_buflen) { /* get the whole buffer */ - msg = channel_get(channel, part); + msg = channel_get(channel, part, NULL); *nl = NUL; } else @@ -3554,7 +3563,7 @@ channel_read_json_block( * Common for ch_read() and ch_readraw(). */ void -common_channel_read(typval_T *argvars, typval_T *rettv, int raw) +common_channel_read(typval_T *argvars, typval_T *rettv, int raw, int blob) { channel_T *channel; ch_part_T part = PART_COUNT; @@ -3585,9 +3594,32 @@ common_channel_read(typval_T *argvars, t if (opt.jo_set & JO_TIMEOUT) timeout = opt.jo_timeout; - if (raw || mode == MODE_RAW || mode == MODE_NL) + if (blob) + { + int outlen = 0; + char_u *p = channel_read_block(channel, part, + timeout, TRUE, &outlen); + if (p != NULL) + { + blob_T *b = blob_alloc(); + + if (b != NULL) + { + b->bv_ga.ga_len = outlen; + if (ga_grow(&b->bv_ga, outlen) == FAIL) + blob_free(b); + else + { + memcpy(b->bv_ga.ga_data, p, outlen); + rettv_blob_set(rettv, b); + } + } + vim_free(p); + } + } + else if (raw || mode == MODE_RAW || mode == MODE_NL) rettv->vval.v_string = channel_read_block(channel, part, - timeout, raw); + timeout, raw, NULL); else { if (opt.jo_set & JO_ID) @@ -3905,6 +3937,7 @@ channel_send( send_common( typval_T *argvars, char_u *text, + int len, int id, int eval, jobopt_T *opt, @@ -3938,7 +3971,7 @@ send_common( opt->jo_callback, opt->jo_partial, id); } - if (channel_send(channel, part_send, text, (int)STRLEN(text), fun) == OK + if (channel_send(channel, part_send, text, len, fun) == OK && opt->jo_callback == NULL) return channel; return NULL; @@ -3982,7 +4015,7 @@ ch_expr_common(typval_T *argvars, typval if (text == NULL) return; - channel = send_common(argvars, text, id, eval, &opt, + channel = send_common(argvars, text, (int)STRLEN(text), id, eval, &opt, eval ? "ch_evalexpr" : "ch_sendexpr", &part_read); vim_free(text); if (channel != NULL && eval) @@ -4014,6 +4047,7 @@ ch_raw_common(typval_T *argvars, typval_ { char_u buf[NUMBUFLEN]; char_u *text; + int len; channel_T *channel; ch_part_T part_read; jobopt_T opt; @@ -4023,8 +4057,17 @@ ch_raw_common(typval_T *argvars, typval_ rettv->v_type = VAR_STRING; rettv->vval.v_string = NULL; - text = tv_get_string_buf(&argvars[1], buf); - channel = send_common(argvars, text, 0, eval, &opt, + if (argvars[1].v_type == VAR_BLOB) + { + text = argvars[1].vval.v_blob->bv_ga.ga_data; + len = argvars[1].vval.v_blob->bv_ga.ga_len; + } + else + { + text = tv_get_string_buf(&argvars[1], buf); + len = STRLEN(text); + } + channel = send_common(argvars, text, len, 0, eval, &opt, eval ? "ch_evalraw" : "ch_sendraw", &part_read); if (channel != NULL && eval) { @@ -4033,7 +4076,7 @@ ch_raw_common(typval_T *argvars, typval_ else timeout = channel_get_timeout(channel, part_read); rettv->vval.v_string = channel_read_block(channel, part_read, - timeout, TRUE); + timeout, TRUE, NULL); } free_job_options(&opt); } diff --git a/src/eval.c b/src/eval.c --- a/src/eval.c +++ b/src/eval.c @@ -78,6 +78,8 @@ typedef struct int fi_varcount; /* nr of variables in the list */ listwatch_T fi_lw; /* keep an eye on the item used. */ list_T *fi_list; /* list being used */ + int fi_bi; /* index of blob */ + blob_T *fi_blob; /* blob being used */ } forinfo_T; @@ -187,6 +189,7 @@ static struct vimvar {VV_NAME("t_none", VAR_NUMBER), VV_RO}, {VV_NAME("t_job", VAR_NUMBER), VV_RO}, {VV_NAME("t_channel", VAR_NUMBER), VV_RO}, + {VV_NAME("t_blob", VAR_NUMBER), VV_RO}, {VV_NAME("termrfgresp", VAR_STRING), VV_RO}, {VV_NAME("termrbgresp", VAR_STRING), VV_RO}, {VV_NAME("termu7resp", VAR_STRING), VV_RO}, @@ -202,6 +205,7 @@ static struct vimvar #define vv_str vv_di.di_tv.vval.v_string #define vv_list vv_di.di_tv.vval.v_list #define vv_dict vv_di.di_tv.vval.v_dict +#define vv_blob vv_di.di_tv.vval.v_blob #define vv_tv vv_di.di_tv static dictitem_T vimvars_var; /* variable used for v: */ @@ -338,6 +342,7 @@ eval_init(void) set_vim_var_nr(VV_TYPE_NONE, VAR_TYPE_NONE); set_vim_var_nr(VV_TYPE_JOB, VAR_TYPE_JOB); set_vim_var_nr(VV_TYPE_CHANNEL, VAR_TYPE_CHANNEL); + set_vim_var_nr(VV_TYPE_BLOB, VAR_TYPE_BLOB); set_reg_var(0); /* default for v:register is not 0 but '"' */ @@ -1918,10 +1923,12 @@ get_lval( { if (!(lp->ll_tv->v_type == VAR_LIST && lp->ll_tv->vval.v_list != NULL) && !(lp->ll_tv->v_type == VAR_DICT - && lp->ll_tv->vval.v_dict != NULL)) + && lp->ll_tv->vval.v_dict != NULL) + && !(lp->ll_tv->v_type == VAR_BLOB + && lp->ll_tv->vval.v_blob != NULL)) { if (!quiet) - EMSG(_("E689: Can only index a List or Dictionary")); + EMSG(_("E689: Can only index a List, Dictionary or Blob")); return NULL; } if (lp->ll_range) @@ -1974,11 +1981,14 @@ get_lval( clear_tv(&var1); return NULL; } - if (rettv != NULL && (rettv->v_type != VAR_LIST - || rettv->vval.v_list == NULL)) + if (rettv != NULL + && !(rettv->v_type == VAR_LIST + || rettv->vval.v_list != NULL) + && !(rettv->v_type == VAR_BLOB + || rettv->vval.v_blob != NULL)) { if (!quiet) - EMSG(_("E709: [:] requires a List value")); + EMSG(_("E709: [:] requires a List or Blob value")); clear_tv(&var1); return NULL; } @@ -2097,6 +2107,33 @@ get_lval( clear_tv(&var1); lp->ll_tv = &lp->ll_di->di_tv; } + else if (lp->ll_tv->v_type == VAR_BLOB) + { + /* + * Get the number and item for the only or first index of the List. + */ + if (empty1) + lp->ll_n1 = 0; + else + // is number or string + lp->ll_n1 = (long)tv_get_number(&var1); + clear_tv(&var1); + + if (lp->ll_n1 < 0 + || lp->ll_n1 > blob_len(lp->ll_tv->vval.v_blob)) + { + if (!quiet) + EMSGN(_(e_listidx), lp->ll_n1); + return NULL; + } + if (lp->ll_range && !lp->ll_empty2) + { + lp->ll_n2 = (long)tv_get_number(&var2); + clear_tv(&var2); + } + lp->ll_blob = lp->ll_tv->vval.v_blob; + lp->ll_tv = NULL; + } else { /* @@ -2201,7 +2238,52 @@ set_var_lval( { cc = *endp; *endp = NUL; - if (op != NULL && *op != '=') + if (lp->ll_blob != NULL) + { + int error = FALSE, val; + if (op != NULL && *op != '=') + { + EMSG2(_(e_letwrong), op); + return; + } + + if (lp->ll_range && rettv->v_type == VAR_BLOB) + { + int i; + + if (blob_len(rettv->vval.v_blob) != blob_len(lp->ll_blob)) + { + EMSG(_("E972: Blob value has more items than target")); + return; + } + + for (i = lp->ll_n1; i <= lp->ll_n2; i++) + blob_set(lp->ll_blob, i, + blob_get(rettv->vval.v_blob, i)); + } + else + { + val = (int)tv_get_number_chk(rettv, &error); + if (!error) + { + garray_T *gap = &lp->ll_blob->bv_ga; + + // Allow for appending a byte. Setting a byte beyond + // the end is an error otherwise. + if (lp->ll_n1 < gap->ga_len + || (lp->ll_n1 == gap->ga_len + && ga_grow(&lp->ll_blob->bv_ga, 1) == OK)) + { + blob_set(lp->ll_blob, lp->ll_n1, val); + if (lp->ll_n1 == gap->ga_len) + ++gap->ga_len; + } + else + EMSG(_(e_invrange)); + } + } + } + else if (op != NULL && *op != '=') { typval_T tv; @@ -2352,6 +2434,20 @@ tv_op(typval_T *tv1, typval_T *tv2, char case VAR_CHANNEL: break; + case VAR_BLOB: + if (*op != '+' || tv2->v_type != VAR_BLOB) + break; + // BLOB += BLOB + if (tv1->vval.v_blob != NULL && tv2->vval.v_blob != NULL) + { + blob_T *b1 = tv1->vval.v_blob; + blob_T *b2 = tv2->vval.v_blob; + int i, len = blob_len(b2); + for (i = 0; i < len; i++) + ga_append(&b1->bv_ga, blob_get(b2, i)); + } + return OK; + case VAR_LIST: if (*op != '+' || tv2->v_type != VAR_LIST) break; @@ -2451,6 +2547,7 @@ eval_for_line( char_u *expr; typval_T tv; list_T *l; + blob_T *b; *errp = TRUE; /* default: there is an error */ @@ -2476,25 +2573,39 @@ eval_for_line( *errp = FALSE; if (!skip) { - l = tv.vval.v_list; - if (tv.v_type != VAR_LIST) + if (tv.v_type == VAR_LIST) + { + l = tv.vval.v_list; + if (l == NULL) + { + // a null list is like an empty list: do nothing + clear_tv(&tv); + } + else + { + // No need to increment the refcount, it's already set for + // the list being used in "tv". + fi->fi_list = l; + list_add_watch(l, &fi->fi_lw); + fi->fi_lw.lw_item = l->lv_first; + } + } + else if (tv.v_type == VAR_BLOB) + { + b = tv.vval.v_blob; + if (b == NULL) + clear_tv(&tv); + else + { + fi->fi_blob = b; + fi->fi_bi = 0; + } + } + else { EMSG(_(e_listreq)); clear_tv(&tv); } - else if (l == NULL) - { - /* a null list is like an empty list: do nothing */ - clear_tv(&tv); - } - else - { - /* No need to increment the refcount, it's already set for the - * list being used in "tv". */ - fi->fi_list = l; - list_add_watch(l, &fi->fi_lw); - fi->fi_lw.lw_item = l->lv_first; - } } } if (skip) @@ -2516,6 +2627,20 @@ next_for_item(void *fi_void, char_u *arg int result; listitem_T *item; + if (fi->fi_blob != NULL) + { + typval_T tv; + + if (fi->fi_bi >= blob_len(fi->fi_blob)) + return FALSE; + tv.v_type = VAR_NUMBER; + tv.v_lock = VAR_FIXED; + tv.vval.v_number = blob_get(fi->fi_blob, fi->fi_bi); + ++fi->fi_bi; + return ex_let_vars(arg, &tv, TRUE, + fi->fi_semicolon, fi->fi_varcount, NULL) == OK; + } + item = fi->fi_lw.lw_item; if (item == NULL) result = FALSE; @@ -2955,6 +3080,7 @@ item_lock(typval_T *tv, int deep, int lo list_T *l; listitem_T *li; dict_T *d; + blob_T *b; hashitem_T *hi; int todo; @@ -2986,6 +3112,15 @@ item_lock(typval_T *tv, int deep, int lo case VAR_CHANNEL: break; + case VAR_BLOB: + if ((b = tv->vval.v_blob) != NULL) + { + if (lock) + b->bv_lock |= VAR_LOCKED; + else + b->bv_lock &= ~VAR_LOCKED; + } + break; case VAR_LIST: if ((l = tv->vval.v_list) != NULL) { @@ -3609,7 +3744,8 @@ eval5(char_u **arg, typval_T *rettv, int if (op != '+' && op != '-' && op != '.') break; - if ((op != '+' || rettv->v_type != VAR_LIST) + if ((op != '+' || (rettv->v_type != VAR_LIST + && rettv->v_type != VAR_BLOB)) #ifdef FEAT_FLOAT && (op == '.' || rettv->v_type != VAR_FLOAT) #endif @@ -3659,6 +3795,25 @@ eval5(char_u **arg, typval_T *rettv, int rettv->v_type = VAR_STRING; rettv->vval.v_string = p; } + else if (op == '+' && rettv->v_type == VAR_BLOB + && var2.v_type == VAR_BLOB) + { + blob_T *b1 = rettv->vval.v_blob; + blob_T *b2 = var2.vval.v_blob; + blob_T *b = blob_alloc(); + int i; + + if (b != NULL) + { + for (i = 0; i < blob_len(b1); i++) + ga_append(&b->bv_ga, blob_get(b1, i)); + for (i = 0; i < blob_len(b2); i++) + ga_append(&b->bv_ga, blob_get(b2, i)); + + clear_tv(rettv); + rettv_blob_set(rettv, b); + } + } else if (op == '+' && rettv->v_type == VAR_LIST && var2.v_type == VAR_LIST) { @@ -3921,6 +4076,7 @@ eval6( /* * Handle sixth level expression: * number number constant + * 0zFFFFFFFF Blob constant * "string" string constant * 'string' literal string constant * &option-name option value @@ -4027,7 +4183,38 @@ eval7( } else #endif - { + if (**arg == '0' && ((*arg)[1] == 'z' || (*arg)[1] == 'Z')) + { + char_u *bp; + blob_T *blob; + + // Blob constant: 0z0123456789abcdef + if (evaluate) + blob = blob_alloc(); + for (bp = *arg + 2; vim_isxdigit(bp[0]); bp += 2) + { + if (!vim_isxdigit(bp[1])) + { + EMSG(_("E973: Blob literal should have an even number of hex characters'")); + vim_free(blob); + ret = FAIL; + break; + } + if (blob != NULL) + ga_append(&blob->bv_ga, + (hex2nr(*bp) << 4) + hex2nr(*(bp+1))); + } + if (blob != NULL) + { + ++blob->bv_refcount; + rettv->v_type = VAR_BLOB; + rettv->vval.v_blob = blob; + } + *arg = bp; + } + else + { + // decimal, hex or octal number vim_str2nr(*arg, NULL, &len, STR2NR_ALL, &n, NULL, 0); *arg += len; if (evaluate) @@ -4263,6 +4450,7 @@ eval_index( { int empty1 = FALSE, empty2 = FALSE; typval_T var1, var2; + long i; long n1, n2 = 0; long len = -1; int range = FALSE; @@ -4297,6 +4485,7 @@ eval_index( case VAR_NUMBER: case VAR_LIST: case VAR_DICT: + case VAR_BLOB: break; } @@ -4439,6 +4628,67 @@ eval_index( rettv->vval.v_string = s; break; + case VAR_BLOB: + len = blob_len(rettv->vval.v_blob); + if (range) + { + // The resulting variable is a substring. If the indexes + // are out of range the result is empty. + if (n1 < 0) + { + n1 = len + n1; + if (n1 < 0) + n1 = 0; + } + if (n2 < 0) + n2 = len + n2; + else if (n2 >= len) + n2 = len - 1; + if (n1 >= len || n2 < 0 || n1 > n2) + { + clear_tv(rettv); + rettv->v_type = VAR_BLOB; + rettv->vval.v_blob = NULL; + } + else + { + blob_T *blob = blob_alloc(); + + if (blob != NULL) + { + if (ga_grow(&blob->bv_ga, n2 - n1 + 1) == FAIL) + { + blob_free(blob); + return FAIL; + } + blob->bv_ga.ga_len = n2 - n1 + 1; + for (i = n1; i <= n2; i++) + blob_set(blob, i - n1, + blob_get(rettv->vval.v_blob, i)); + + clear_tv(rettv); + rettv_blob_set(rettv, blob); + } + } + } + else + { + // The resulting variable is a string of a single + // character. If the index is too big or negative the + // result is empty. + if (n1 < len && n1 >= 0) + { + int v = (int)blob_get(rettv->vval.v_blob, n1); + + clear_tv(rettv); + rettv->v_type = VAR_NUMBER; + rettv->vval.v_number = v; + } + else + EMSGN(_(e_blobidx), n1); + } + break; + case VAR_LIST: len = list_len(rettv->vval.v_list); if (n1 < 0) @@ -4970,6 +5220,9 @@ tv_equal( --recursive_cnt; return r; + case VAR_BLOB: + return blob_equal(tv1->vval.v_blob, tv2->vval.v_blob); + case VAR_NUMBER: return tv1->vval.v_number == tv2->vval.v_number; @@ -5602,6 +5855,36 @@ echo_string_core( break; } + case VAR_BLOB: + if (tv->vval.v_blob == NULL) + { + *tofree = NULL; + r = (char_u *)"[]"; + } + else + { + blob_T *b; + int i; + garray_T ga; + + // Store bytes in the growarray. + ga_init2(&ga, 1, 4000); + b = tv->vval.v_blob; + ga_append(&ga, '['); + for (i = 0; i < blob_len(b); i++) + { + if (i > 0) + ga_concat(&ga, (char_u *)","); + vim_snprintf((char *)numbuf, NUMBUFLEN, "0x%02X", + (int)blob_get(b, i)); + ga_concat(&ga, numbuf); + } + ga_append(&ga, ']'); + *tofree = ga.ga_data; + r = *tofree; + } + break; + case VAR_LIST: if (tv->vval.v_list == NULL) { @@ -6841,6 +7124,9 @@ free_tv(typval_T *varp) case VAR_PARTIAL: partial_unref(varp->vval.v_partial); break; + case VAR_BLOB: + blob_unref(varp->vval.v_blob); + break; case VAR_LIST: list_unref(varp->vval.v_list); break; @@ -6887,6 +7173,10 @@ clear_tv(typval_T *varp) partial_unref(varp->vval.v_partial); varp->vval.v_partial = NULL; break; + case VAR_BLOB: + blob_unref(varp->vval.v_blob); + varp->vval.v_blob = NULL; + break; case VAR_LIST: list_unref(varp->vval.v_list); varp->vval.v_list = NULL; @@ -6990,6 +7280,9 @@ tv_get_number_chk(typval_T *varp, int *d EMSG(_("E913: Using a Channel as a Number")); break; #endif + case VAR_BLOB: + EMSG(_("E974: Using a Blob as a Number")); + break; case VAR_UNKNOWN: internal_error("tv_get_number(UNKNOWN)"); break; @@ -7037,6 +7330,9 @@ tv_get_float(typval_T *varp) EMSG(_("E914: Using a Channel as a Float")); break; # endif + case VAR_BLOB: + EMSG(_("E975: Using a Blob as a Float")); + break; case VAR_UNKNOWN: internal_error("tv_get_float(UNKNOWN)"); break; @@ -7113,6 +7409,9 @@ tv_get_string_buf_chk(typval_T *varp, ch case VAR_SPECIAL: STRCPY(buf, get_var_special_name(varp->vval.v_number)); return buf; + case VAR_BLOB: + EMSG(_("E976: using Blob as a String")); + break; case VAR_JOB: #ifdef FEAT_JOB_CHANNEL { @@ -7805,6 +8104,15 @@ copy_tv(typval_T *from, typval_T *to) ++to->vval.v_partial->pt_refcount; } break; + case VAR_BLOB: + if (from->vval.v_blob == NULL) + to->vval.v_blob = NULL; + else + { + to->vval.v_blob = from->vval.v_blob; + ++to->vval.v_blob->bv_refcount; + } + break; case VAR_LIST: if (from->vval.v_list == NULL) to->vval.v_list = NULL; @@ -7863,6 +8171,7 @@ item_copy( case VAR_SPECIAL: case VAR_JOB: case VAR_CHANNEL: + case VAR_BLOB: copy_tv(from, to); break; case VAR_LIST: @@ -8601,6 +8910,7 @@ read_viminfo_varlist(vir_T *virp, int wr #endif case 'D': type = VAR_DICT; break; case 'L': type = VAR_LIST; break; + case 'B': type = VAR_BLOB; break; case 'X': type = VAR_SPECIAL; break; } @@ -8608,7 +8918,8 @@ read_viminfo_varlist(vir_T *virp, int wr if (tab != NULL) { tv.v_type = type; - if (type == VAR_STRING || type == VAR_DICT || type == VAR_LIST) + if (type == VAR_STRING || type == VAR_DICT || + type == VAR_LIST || type == VAR_BLOB) tv.vval.v_string = viminfo_readstring(virp, (int)(tab - virp->vir_line + 1), TRUE); #ifdef FEAT_FLOAT @@ -8617,7 +8928,7 @@ read_viminfo_varlist(vir_T *virp, int wr #endif else tv.vval.v_number = atol((char *)tab + 1); - if (type == VAR_DICT || type == VAR_LIST) + if (type == VAR_DICT || type == VAR_LIST || type == VAR_BLOB) { typval_T *etv = eval_expr(tv.vval.v_string, NULL); @@ -8640,7 +8951,8 @@ read_viminfo_varlist(vir_T *virp, int wr if (tv.v_type == VAR_STRING) vim_free(tv.vval.v_string); - else if (tv.v_type == VAR_DICT || tv.v_type == VAR_LIST) + else if (tv.v_type == VAR_DICT || tv.v_type == VAR_LIST || + tv.v_type == VAR_BLOB) clear_tv(&tv); } } @@ -8684,6 +8996,7 @@ write_viminfo_varlist(FILE *fp) case VAR_FLOAT: s = "FLO"; break; case VAR_DICT: s = "DIC"; break; case VAR_LIST: s = "LIS"; break; + case VAR_BLOB: s = "BLO"; break; case VAR_SPECIAL: s = "XPL"; break; case VAR_UNKNOWN: @@ -9250,6 +9563,33 @@ typval_compare( * it means TRUE. */ n1 = (type == TYPE_NEQUAL); } + else if (typ1->v_type == VAR_BLOB || typ2->v_type == VAR_BLOB) + { + if (type_is) + { + n1 = (typ1->v_type == typ2->v_type + && typ1->vval.v_blob == typ2->vval.v_blob); + if (type == TYPE_NEQUAL) + n1 = !n1; + } + else if (typ1->v_type != typ2->v_type + || (type != TYPE_EQUAL && type != TYPE_NEQUAL)) + { + if (typ1->v_type != typ2->v_type) + EMSG(_("E977: Can only compare Blob with Blob")); + else + EMSG(_(e_invalblob)); + clear_tv(typ1); + return FAIL; + } + else + { + // Compare two Blobs for being equal or unequal. + n1 = blob_equal(typ1->vval.v_blob, typ2->vval.v_blob); + if (type == TYPE_NEQUAL) + n1 = !n1; + } + } else if (typ1->v_type == VAR_LIST || typ2->v_type == VAR_LIST) { if (type_is) @@ -10278,6 +10618,7 @@ filter_map(typval_T *argvars, typval_T * dict_T *d = NULL; typval_T save_val; typval_T save_key; + blob_T *b = NULL; int rem; int todo; char_u *ermsg = (char_u *)(map ? "map()" : "filter()"); @@ -10286,7 +10627,12 @@ filter_map(typval_T *argvars, typval_T * int save_did_emsg; int idx = 0; - if (argvars[0].v_type == VAR_LIST) + if (argvars[0].v_type == VAR_BLOB) + { + if ((b = argvars[0].vval.v_blob) == NULL) + return; + } + else if (argvars[0].v_type == VAR_LIST) { if ((l = argvars[0].vval.v_list) == NULL || (!map && tv_check_lock(l->lv_lock, arg_errmsg, TRUE))) @@ -10353,6 +10699,37 @@ filter_map(typval_T *argvars, typval_T * } hash_unlock(ht); } + else if (argvars[0].v_type == VAR_BLOB) + { + int i; + typval_T tv; + + vimvars[VV_KEY].vv_type = VAR_NUMBER; + for (i = 0; i < b->bv_ga.ga_len; i++) + { + tv.v_type = VAR_NUMBER; + tv.vval.v_number = blob_get(b, i); + vimvars[VV_KEY].vv_nr = idx; + if (filter_map_one(&tv, expr, map, &rem) == FAIL || did_emsg) + break; + if (tv.v_type != VAR_NUMBER) + { + EMSG(_(e_invalblob)); + return; + } + tv.v_type = VAR_NUMBER; + blob_set(b, i, tv.vval.v_number); + if (!map && rem) + { + char_u *p = (char_u *)argvars[0].vval.v_blob->bv_ga.ga_data; + + mch_memmove(p + idx, p + i + 1, + (size_t)b->bv_ga.ga_len - i - 1); + --b->bv_ga.ga_len; + --i; + } + } + } else { vimvars[VV_KEY].vv_type = VAR_NUMBER; diff --git a/src/evalfunc.c b/src/evalfunc.c --- a/src/evalfunc.c +++ b/src/evalfunc.c @@ -96,6 +96,7 @@ static void f_ch_log(typval_T *argvars, static void f_ch_logfile(typval_T *argvars, typval_T *rettv); static void f_ch_open(typval_T *argvars, typval_T *rettv); static void f_ch_read(typval_T *argvars, typval_T *rettv); +static void f_ch_readblob(typval_T *argvars, typval_T *rettv); static void f_ch_readraw(typval_T *argvars, typval_T *rettv); static void f_ch_sendexpr(typval_T *argvars, typval_T *rettv); static void f_ch_sendraw(typval_T *argvars, typval_T *rettv); @@ -570,6 +571,7 @@ static struct fst {"ch_logfile", 1, 2, f_ch_logfile}, {"ch_open", 1, 2, f_ch_open}, {"ch_read", 1, 2, f_ch_read}, + {"ch_readblob", 1, 2, f_ch_readblob}, {"ch_readraw", 1, 2, f_ch_readraw}, {"ch_sendexpr", 2, 3, f_ch_sendexpr}, {"ch_sendraw", 2, 3, f_ch_sendraw}, @@ -1237,6 +1239,7 @@ f_acos(typval_T *argvars, typval_T *rett f_add(typval_T *argvars, typval_T *rettv) { list_T *l; + blob_T *b; rettv->vval.v_number = 1; /* Default: Failed */ if (argvars[0].v_type == VAR_LIST) @@ -1247,6 +1250,16 @@ f_add(typval_T *argvars, typval_T *rettv && list_append_tv(l, &argvars[1]) == OK) copy_tv(&argvars[0], rettv); } + else if (argvars[0].v_type == VAR_BLOB) + { + if ((b = argvars[0].vval.v_blob) != NULL + && !tv_check_lock(b->bv_lock, + (char_u *)N_("add() argument"), TRUE)) + { + ga_append(&b->bv_ga, (char_u)tv_get_number(&argvars[1])); + copy_tv(&argvars[0], rettv); + } + } else EMSG(_(e_listreq)); } @@ -2309,7 +2322,16 @@ f_ch_open(typval_T *argvars, typval_T *r static void f_ch_read(typval_T *argvars, typval_T *rettv) { - common_channel_read(argvars, rettv, FALSE); + common_channel_read(argvars, rettv, FALSE, FALSE); +} + +/* + * "ch_readblob()" function + */ + static void +f_ch_readblob(typval_T *argvars, typval_T *rettv) +{ + common_channel_read(argvars, rettv, TRUE, TRUE); } /* @@ -2318,7 +2340,7 @@ f_ch_read(typval_T *argvars, typval_T *r static void f_ch_readraw(typval_T *argvars, typval_T *rettv) { - common_channel_read(argvars, rettv, TRUE); + common_channel_read(argvars, rettv, TRUE, FALSE); } /* @@ -3170,6 +3192,12 @@ f_empty(typval_T *argvars, typval_T *ret n = argvars[0].vval.v_number != VVAL_TRUE; break; + case VAR_BLOB: + n = argvars[0].vval.v_blob == NULL + || argvars[0].vval.v_blob->bv_ga.ga_data == NULL + || argvars[0].vval.v_blob->bv_ga.ga_len == 0; + break; + case VAR_JOB: #ifdef FEAT_JOB_CHANNEL n = argvars[0].vval.v_job == NULL @@ -4365,7 +4393,21 @@ f_get(typval_T *argvars, typval_T *rettv dict_T *d; typval_T *tv = NULL; - if (argvars[0].v_type == VAR_LIST) + if (argvars[0].v_type == VAR_BLOB) + { + int error = FALSE; + int idx = tv_get_number_chk(&argvars[1], &error); + + if (!error) + { + rettv->v_type = VAR_NUMBER; + if (idx >= blob_len(argvars[0].vval.v_blob)) + EMSGN(_(e_blobidx), idx); + else + rettv->vval.v_number = blob_get(argvars[0].vval.v_blob, idx); + } + } + else if (argvars[0].v_type == VAR_LIST) { if ((l = argvars[0].vval.v_list) != NULL) { @@ -6965,23 +7007,50 @@ f_index(typval_T *argvars, typval_T *ret { list_T *l; listitem_T *item; + blob_T *b; long idx = 0; int ic = FALSE; + int error = FALSE; rettv->vval.v_number = -1; - if (argvars[0].v_type != VAR_LIST) + if (argvars[0].v_type == VAR_BLOB) + { + typval_T tv; + int start = 0; + + if (argvars[2].v_type != VAR_UNKNOWN) + { + start = tv_get_number_chk(&argvars[2], &error); + if (error) + return; + } + b = argvars[0].vval.v_blob; + if (b == NULL) + return; + for (idx = start; idx < blob_len(b); ++idx) + { + tv.v_type = VAR_NUMBER; + tv.vval.v_number = blob_get(b, idx); + if (tv_equal(&tv, &argvars[1], ic, FALSE)) + { + rettv->vval.v_number = idx; + return; + } + } + return; + } + else if (argvars[0].v_type != VAR_LIST) { EMSG(_(e_listreq)); return; } + l = argvars[0].vval.v_list; if (l != NULL) { item = l->lv_first; if (argvars[2].v_type != VAR_UNKNOWN) { - int error = FALSE; - /* Start at specified item. Use the cached index that list_find() * sets, so that a negative number also works. */ item = list_find(l, (long)tv_get_number_chk(&argvars[2], &error)); @@ -7160,10 +7229,45 @@ f_insert(typval_T *argvars, typval_T *re list_T *l; int error = FALSE; - if (argvars[0].v_type != VAR_LIST) + if (argvars[0].v_type == VAR_BLOB) + { + int val, len; + char_u *p; + + len = blob_len(argvars[0].vval.v_blob); + if (argvars[2].v_type != VAR_UNKNOWN) + { + before = (long)tv_get_number_chk(&argvars[2], &error); + if (error) + return; // type error; errmsg already given + if (before < 0 || before > len) + { + EMSG2(_(e_invarg2), tv_get_string(&argvars[2])); + return; + } + } + val = tv_get_number_chk(&argvars[1], &error); + if (error) + return; + if (val < 0 || val > 255) + { + EMSG2(_(e_invarg2), tv_get_string(&argvars[1])); + return; + } + + if (ga_grow(&argvars[0].vval.v_blob->bv_ga, 1) == FAIL) + return; + p = (char_u *)argvars[0].vval.v_blob->bv_ga.ga_data; + mch_memmove(p + before + 1, p + before, (size_t)len - before); + *(p + before) = val; + ++argvars[0].vval.v_blob->bv_ga.ga_len; + + copy_tv(&argvars[0], rettv); + } + else if (argvars[0].v_type != VAR_LIST) EMSG2(_(e_listarg), "insert()"); - else if ((l = argvars[0].vval.v_list) != NULL - && !tv_check_lock(l->lv_lock, (char_u *)N_("insert() argument"), TRUE)) + else if ((l = argvars[0].vval.v_list) != NULL && !tv_check_lock(l->lv_lock, + (char_u *)N_("insert() argument"), TRUE)) { if (argvars[2].v_type != VAR_UNKNOWN) before = (long)tv_get_number_chk(&argvars[2], &error); @@ -7527,6 +7631,9 @@ f_len(typval_T *argvars, typval_T *rettv rettv->vval.v_number = (varnumber_T)STRLEN( tv_get_string(&argvars[0])); break; + case VAR_BLOB: + rettv->vval.v_number = blob_len(argvars[0].vval.v_blob); + break; case VAR_LIST: rettv->vval.v_number = list_len(argvars[0].vval.v_list); break; @@ -8926,6 +9033,7 @@ f_range(typval_T *argvars, typval_T *ret f_readfile(typval_T *argvars, typval_T *rettv) { int binary = FALSE; + int blob = FALSE; int failed = FALSE; char_u *fname; FILE *fd; @@ -8944,12 +9052,23 @@ f_readfile(typval_T *argvars, typval_T * { if (STRCMP(tv_get_string(&argvars[1]), "b") == 0) binary = TRUE; + if (STRCMP(tv_get_string(&argvars[1]), "B") == 0) + blob = TRUE; + if (argvars[2].v_type != VAR_UNKNOWN) maxline = (long)tv_get_number(&argvars[2]); } - if (rettv_list_alloc(rettv) == FAIL) - return; + if (blob) + { + if (rettv_blob_alloc(rettv) == FAIL) + return; + } + else + { + if (rettv_list_alloc(rettv) == FAIL) + return; + } /* Always open the file in binary mode, library functions have a mind of * their own about CR-LF conversion. */ @@ -8960,6 +9079,17 @@ f_readfile(typval_T *argvars, typval_T * return; } + if (blob) + { + if (read_blob(fd, rettv->vval.v_blob) == FAIL) + { + EMSG("cannot read file"); + blob_free(rettv->vval.v_blob); + } + fclose(fd); + return; + } + while (cnt < maxline || maxline < 0) { readlen = (int)fread(buf, 1, io_size, fd); @@ -9555,6 +9685,7 @@ f_remove(typval_T *argvars, typval_T *re dict_T *d; dictitem_T *di; char_u *arg_errmsg = (char_u *)N_("remove() argument"); + int error = FALSE; if (argvars[0].v_type == VAR_DICT) { @@ -9579,16 +9710,76 @@ f_remove(typval_T *argvars, typval_T *re } } } + else if (argvars[0].v_type == VAR_BLOB) + { + idx = (long)tv_get_number_chk(&argvars[1], &error); + if (!error) + { + blob_T *b = argvars[0].vval.v_blob; + int len = blob_len(b); + char_u *p; + + if (idx < 0) + // count from the end + idx = len + idx; + if (idx < 0 || idx >= len) + { + EMSGN(_(e_blobidx), idx); + return; + } + if (argvars[2].v_type == VAR_UNKNOWN) + { + // Remove one item, return its value. + p = (char_u *)b->bv_ga.ga_data; + rettv->vval.v_number = (varnumber_T) *(p + idx); + mch_memmove(p + idx, p + idx + 1, (size_t)len - idx - 1); + --b->bv_ga.ga_len; + } + else + { + blob_T *blob; + + // Remove range of items, return list with values. + end = (long)tv_get_number_chk(&argvars[2], &error); + if (error) + return; + if (end < 0) + // count from the end + end = len + end; + if (end >= len || idx > end) + { + EMSGN(_(e_blobidx), end); + return; + } + blob = blob_alloc(); + if (blob == NULL) + return; + blob->bv_ga.ga_len = end - idx + 1; + if (ga_grow(&blob->bv_ga, end - idx + 1) == FAIL) + { + vim_free(blob); + return; + } + p = (char_u *)b->bv_ga.ga_data; + mch_memmove((char_u *)blob->bv_ga.ga_data, p + idx, + (size_t)(end - idx + 1)); + ++blob->bv_refcount; + rettv->v_type = VAR_BLOB; + rettv->vval.v_blob = blob; + + mch_memmove(p + idx, p + end + 1, (size_t)(len - end)); + b->bv_ga.ga_len -= end - idx + 1; + } + } + } else if (argvars[0].v_type != VAR_LIST) EMSG2(_(e_listdictarg), "remove()"); else if ((l = argvars[0].vval.v_list) != NULL - && !tv_check_lock(l->lv_lock, arg_errmsg, TRUE)) - { - int error = FALSE; - + && !tv_check_lock(l->lv_lock, arg_errmsg, TRUE)) + { idx = (long)tv_get_number_chk(&argvars[1], &error); if (error) - ; /* type error: do nothing, errmsg already given */ + ; // type error: do nothing, errmsg already given else if ((item = list_find(l, idx)) == NULL) EMSGN(_(e_listidx), idx); else @@ -9602,10 +9793,10 @@ f_remove(typval_T *argvars, typval_T *re } else { - /* Remove range of items, return list with values. */ + // Remove range of items, return list with values. end = (long)tv_get_number_chk(&argvars[2], &error); if (error) - ; /* type error: do nothing */ + ; // type error: do nothing else if ((item2 = list_find(l, end)) == NULL) EMSGN(_(e_listidx), end); else @@ -9912,6 +10103,22 @@ f_reverse(typval_T *argvars, typval_T *r list_T *l; listitem_T *li, *ni; + if (argvars[0].v_type == VAR_BLOB) + { + blob_T *b = argvars[0].vval.v_blob; + int i, len = blob_len(b); + + for (i = 0; i < len / 2; i++) + { + int tmp = blob_get(b, i); + + blob_set(b, i, blob_get(b, len - i - 1)); + blob_set(b, len - i - 1, tmp); + } + rettv_blob_set(rettv, b); + return; + } + if (argvars[0].v_type != VAR_LIST) EMSG2(_(e_listarg), "reverse()"); else if ((l = argvars[0].vval.v_list) != NULL @@ -14198,6 +14405,7 @@ f_type(typval_T *argvars, typval_T *rett break; case VAR_JOB: n = VAR_TYPE_JOB; break; case VAR_CHANNEL: n = VAR_TYPE_CHANNEL; break; + case VAR_BLOB: n = VAR_TYPE_BLOB; break; case VAR_UNKNOWN: internal_error("f_type(UNKNOWN)"); n = -1; @@ -14556,23 +14764,33 @@ f_writefile(typval_T *argvars, typval_T FILE *fd; int ret = 0; listitem_T *li; - list_T *list; + list_T *list = NULL; + blob_T *blob = NULL; rettv->vval.v_number = -1; if (check_restricted() || check_secure()) return; - if (argvars[0].v_type != VAR_LIST) - { - EMSG2(_(e_listarg), "writefile()"); - return; - } - list = argvars[0].vval.v_list; - if (list == NULL) - return; - for (li = list->lv_first; li != NULL; li = li->li_next) - if (tv_get_string_chk(&li->li_tv) == NULL) + if (argvars[0].v_type == VAR_LIST) + { + list = argvars[0].vval.v_list; + if (list == NULL) return; + for (li = list->lv_first; li != NULL; li = li->li_next) + if (tv_get_string_chk(&li->li_tv) == NULL) + return; + } + else if (argvars[0].v_type == VAR_BLOB) + { + blob = argvars[0].vval.v_blob; + if (blob == NULL) + return; + } + else + { + EMSG2(_(e_invarg2), "writefile()"); + return; + } if (argvars[2].v_type != VAR_UNKNOWN) { @@ -14604,6 +14822,18 @@ f_writefile(typval_T *argvars, typval_T EMSG2(_(e_notcreate), *fname == NUL ? (char_u *)_("") : fname); ret = -1; } + else if (blob) + { + if (write_blob(fd, blob) == FAIL) + ret = -1; +#ifdef HAVE_FSYNC + else if (do_fsync) + // Ignore the error, the user wouldn't know what to do about it. + // May happen for a device. + vim_ignored = fsync(fileno(fd)); +#endif + fclose(fd); + } else { if (write_list(fd, list, binary) == FAIL) diff --git a/src/globals.h b/src/globals.h --- a/src/globals.h +++ b/src/globals.h @@ -1524,6 +1524,8 @@ EXTERN char_u e_readonlysbx[] INIT(= N_( EXTERN char_u e_emptykey[] INIT(= N_("E713: Cannot use empty key for Dictionary")); EXTERN char_u e_dictreq[] INIT(= N_("E715: Dictionary required")); EXTERN char_u e_listidx[] INIT(= N_("E684: list index out of range: %ld")); +EXTERN char_u e_blobidx[] INIT(= N_("E979: Blob index out of range: %ld")); +EXTERN char_u e_invalblob[] INIT(= N_("E978: Invalid operation for Blob")); EXTERN char_u e_toomanyarg[] INIT(= N_("E118: Too many arguments for function: %s")); EXTERN char_u e_dictkey[] INIT(= N_("E716: Key not present in Dictionary: %s")); EXTERN char_u e_listreq[] INIT(= N_("E714: List required")); diff --git a/src/if_perl.xs b/src/if_perl.xs --- a/src/if_perl.xs +++ b/src/if_perl.xs @@ -236,6 +236,7 @@ typedef int perl_key; # else # define Perl_sv_2pv dll_Perl_sv_2pv # endif +# define Perl_sv_2pvbyte dll_Perl_sv_2pvbyte # define Perl_sv_bless dll_Perl_sv_bless # if (PERL_REVISION == 5) && (PERL_VERSION >= 8) # define Perl_sv_catpvn_flags dll_Perl_sv_catpvn_flags @@ -388,6 +389,7 @@ static char* (*Perl_sv_2pv_nolen)(pTHX_ # else static char* (*Perl_sv_2pv)(pTHX_ SV*, STRLEN*); # endif +static char* (*Perl_sv_2pvbyte)(pTHX_ SV*, STRLEN*); static SV* (*Perl_sv_bless)(pTHX_ SV*, HV*); # if (PERL_REVISION == 5) && (PERL_VERSION >= 8) static void (*Perl_sv_catpvn_flags)(pTHX_ SV* , const char*, STRLEN, I32); @@ -543,6 +545,7 @@ static struct { # else {"Perl_sv_2pv", (PERL_PROC*)&Perl_sv_2pv}, # endif + {"Perl_sv_2pvbyte", (PERL_PROC*)&Perl_sv_2pvbyte}, # ifdef PERL589_OR_LATER {"Perl_sv_2iv_flags", (PERL_PROC*)&Perl_sv_2iv_flags}, {"Perl_newXS_flags", (PERL_PROC*)&Perl_newXS_flags}, @@ -1556,6 +1559,27 @@ Eval(str) vim_free(value); } +SV* +Blob(SV* sv) + PREINIT: + STRLEN len; + char *s; + int i; + char buf[3]; + SV* newsv; + + CODE: + s = SvPVbyte(sv, len); + newsv = newSVpv("0z", 2); + for (i = 0; i < len; i++) + { + sprintf(buf, "%02X", s[i]); + sv_catpvn(newsv, buf, 2); + } + RETVAL = newsv; + OUTPUT: + RETVAL + void Buffers(...) diff --git a/src/if_py_both.h b/src/if_py_both.h --- a/src/if_py_both.h +++ b/src/if_py_both.h @@ -867,6 +867,10 @@ VimToPython(typval_T *our_tv, int depth, } return ret; } + else if (our_tv->v_type == VAR_BLOB) + ret = PyBytes_FromStringAndSize( + (char*) our_tv->vval.v_blob->bv_ga.ga_data, + (Py_ssize_t) our_tv->vval.v_blob->bv_ga.ga_len); else { Py_INCREF(Py_None); @@ -6394,6 +6398,10 @@ ConvertToPyObject(typval_T *tv) tv->vval.v_partial->pt_argc, argv, tv->vval.v_partial->pt_dict, tv->vval.v_partial->pt_auto); + case VAR_BLOB: + return PyBytes_FromStringAndSize( + (char*) tv->vval.v_blob->bv_ga.ga_data, + (Py_ssize_t) tv->vval.v_blob->bv_ga.ga_len); case VAR_UNKNOWN: case VAR_CHANNEL: case VAR_JOB: diff --git a/src/if_python.c b/src/if_python.c --- a/src/if_python.c +++ b/src/if_python.c @@ -1575,6 +1575,7 @@ do_pyeval (char_u *str, typval_T *rettv) case VAR_SPECIAL: case VAR_JOB: case VAR_CHANNEL: + case VAR_BLOB: break; } } diff --git a/src/if_python3.c b/src/if_python3.c --- a/src/if_python3.c +++ b/src/if_python3.c @@ -232,6 +232,8 @@ typedef PySliceObject PySliceObject_T; # endif # undef PyBytes_FromString # define PyBytes_FromString py3_PyBytes_FromString +# undef PyBytes_FromStringAndSize +# define PyBytes_FromStringAndSize py3_PyBytes_FromStringAndSize # define PyFloat_FromDouble py3_PyFloat_FromDouble # define PyFloat_AsDouble py3_PyFloat_AsDouble # define PyObject_GenericGetAttr py3_PyObject_GenericGetAttr @@ -394,6 +396,7 @@ static PyObject* (*py3_PyUnicode_AsEncod static char* (*py3_PyBytes_AsString)(PyObject *bytes); static int (*py3_PyBytes_AsStringAndSize)(PyObject *bytes, char **buffer, Py_ssize_t *length); static PyObject* (*py3_PyBytes_FromString)(char *str); +static PyObject* (*py3_PyBytes_FromStringAndSize)(char *str, Py_ssize_t length); static PyObject* (*py3_PyFloat_FromDouble)(double num); static double (*py3_PyFloat_AsDouble)(PyObject *); static PyObject* (*py3_PyObject_GenericGetAttr)(PyObject *obj, PyObject *name); @@ -559,6 +562,7 @@ static struct {"PyBytes_AsString", (PYTHON_PROC*)&py3_PyBytes_AsString}, {"PyBytes_AsStringAndSize", (PYTHON_PROC*)&py3_PyBytes_AsStringAndSize}, {"PyBytes_FromString", (PYTHON_PROC*)&py3_PyBytes_FromString}, + {"PyBytes_FromStringAndSize", (PYTHON_PROC*)&py3_PyBytes_FromStringAndSize}, {"PyFloat_FromDouble", (PYTHON_PROC*)&py3_PyFloat_FromDouble}, {"PyFloat_AsDouble", (PYTHON_PROC*)&py3_PyFloat_AsDouble}, {"PyObject_GenericGetAttr", (PYTHON_PROC*)&py3_PyObject_GenericGetAttr}, @@ -1680,6 +1684,7 @@ do_py3eval (char_u *str, typval_T *rettv case VAR_SPECIAL: case VAR_JOB: case VAR_CHANNEL: + case VAR_BLOB: break; } } diff --git a/src/if_ruby.c b/src/if_ruby.c --- a/src/if_ruby.c +++ b/src/if_ruby.c @@ -51,6 +51,7 @@ # define rb_cFloat (*dll_rb_cFloat) # endif # define rb_cNilClass (*dll_rb_cNilClass) +# define rb_cString (*dll_rb_cString) # define rb_cSymbol (*dll_rb_cSymbol) # define rb_cTrueClass (*dll_rb_cTrueClass) # if defined(DYNAMIC_RUBY_VER) && DYNAMIC_RUBY_VER >= 18 @@ -219,6 +220,7 @@ static void ruby_vim_init(void); */ # define rb_assoc_new dll_rb_assoc_new # define rb_cObject (*dll_rb_cObject) +# define rb_class_new_instance dll_rb_class_new_instance # define rb_check_type dll_rb_check_type # ifdef USE_TYPEDDATA # define rb_check_typeddata dll_rb_check_typeddata @@ -365,8 +367,10 @@ VALUE *dll_rb_cFloat; # endif VALUE *dll_rb_cNilClass; static VALUE *dll_rb_cObject; +VALUE *dll_rb_cString; VALUE *dll_rb_cSymbol; VALUE *dll_rb_cTrueClass; +static VALUE (*dll_rb_class_new_instance) (int,VALUE*,VALUE); static void (*dll_rb_check_type) (VALUE,int); # ifdef USE_TYPEDDATA static void *(*dll_rb_check_typeddata) (VALUE,const rb_data_type_t *); @@ -579,8 +583,10 @@ static struct # endif {"rb_cNilClass", (RUBY_PROC*)&dll_rb_cNilClass}, {"rb_cObject", (RUBY_PROC*)&dll_rb_cObject}, + {"rb_cString", (RUBY_PROC*)&dll_rb_cString}, {"rb_cSymbol", (RUBY_PROC*)&dll_rb_cSymbol}, {"rb_cTrueClass", (RUBY_PROC*)&dll_rb_cTrueClass}, + {"rb_class_new_instance", (RUBY_PROC*)&dll_rb_class_new_instance}, {"rb_check_type", (RUBY_PROC*)&dll_rb_check_type}, # ifdef USE_TYPEDDATA {"rb_check_typeddata", (RUBY_PROC*)&dll_rb_check_typeddata}, @@ -1164,7 +1170,13 @@ static VALUE vim_to_ruby(typval_T *tv) result = Qtrue; else if (tv->vval.v_number == VVAL_FALSE) result = Qfalse; - } /* else return Qnil; */ + } + else if (tv->v_type == VAR_BLOB) + { + result = rb_str_new(tv->vval.v_blob->bv_ga.ga_data, + tv->vval.v_blob->bv_ga.ga_len); + } + /* else return Qnil; */ return result; } @@ -1242,6 +1254,19 @@ static buf_T *get_buf(VALUE obj) return buf; } +static VALUE vim_blob(VALUE self UNUSED, VALUE str) +{ + VALUE result = rb_str_new("0z", 2); + char buf[4]; + int i; + for (i = 0; i < RSTRING_LEN(str); i++) + { + sprintf(buf, "%02X", RSTRING_PTR(str)[i]); + rb_str_concat(result, rb_str_new_cstr(buf)); + } + return result; +} + static VALUE buffer_s_current(void) { return buffer_new(curbuf); @@ -1662,6 +1687,7 @@ static void ruby_vim_init(void) rb_define_module_function(mVIM, "set_option", vim_set_option, 1); rb_define_module_function(mVIM, "command", vim_command, 1); rb_define_module_function(mVIM, "evaluate", vim_evaluate, 1); + rb_define_module_function(mVIM, "blob", vim_blob, 1); eDeletedBufferError = rb_define_class_under(mVIM, "DeletedBufferError", rb_eStandardError); diff --git a/src/json.c b/src/json.c --- a/src/json.c +++ b/src/json.c @@ -195,8 +195,10 @@ json_encode_item(garray_T *gap, typval_T { char_u numbuf[NUMBUFLEN]; char_u *res; + blob_T *b; list_T *l; dict_T *d; + int i; switch (val->v_type) { @@ -233,6 +235,25 @@ json_encode_item(garray_T *gap, typval_T EMSG(_(e_invarg)); return FAIL; + case VAR_BLOB: + b = val->vval.v_blob; + if (b == NULL || b->bv_ga.ga_len == 0) + ga_concat(gap, (char_u *)"[]"); + else + { + ga_append(gap, '['); + for (i = 0; i < b->bv_ga.ga_len; i++) + { + if (i > 0) + ga_concat(gap, (char_u *)","); + vim_snprintf((char *)numbuf, NUMBUFLEN, "%d", + (int)blob_get(b, i)); + ga_concat(gap, numbuf); + } + ga_append(gap, ']'); + } + break; + case VAR_LIST: l = val->vval.v_list; if (l == NULL) diff --git a/src/netbeans.c b/src/netbeans.c --- a/src/netbeans.c +++ b/src/netbeans.c @@ -404,7 +404,7 @@ netbeans_parse_messages(void) if (*p == NUL) { own_node = TRUE; - buffer = channel_get(nb_channel, PART_SOCK); + buffer = channel_get(nb_channel, PART_SOCK, NULL); /* "node" is now invalid! */ } else diff --git a/src/proto.h b/src/proto.h --- a/src/proto.h +++ b/src/proto.h @@ -88,6 +88,7 @@ extern int _stricoll(char *a, char *b); # include "hashtab.pro" # include "json.pro" # include "list.pro" +# include "blob.pro" # include "main.pro" # include "mark.pro" # include "memfile.pro" diff --git a/src/proto/blob.pro b/src/proto/blob.pro new file mode 100644 --- /dev/null +++ b/src/proto/blob.pro @@ -0,0 +1,13 @@ +/* blob.c */ +blob_T *blob_alloc(void); +int rettv_blob_alloc(typval_T *rettv); +void rettv_blob_set(typval_T *rettv, blob_T *b); +void blob_free(blob_T *b); +void blob_unref(blob_T *b); +long blob_len(blob_T *b); +char_u blob_get(blob_T *b, int idx); +void blob_set(blob_T *b, int idx, char_u c); +int blob_equal(blob_T *b1, blob_T *b2); +int read_blob(FILE *fd, blob_T *blob); +int write_blob(FILE *fd, blob_T *blob); +/* vim: set ft=c : */ diff --git a/src/proto/channel.pro b/src/proto/channel.pro --- a/src/proto/channel.pro +++ b/src/proto/channel.pro @@ -18,7 +18,7 @@ void channel_write_any_lines(void); void channel_write_new_lines(buf_T *buf); readq_T *channel_peek(channel_T *channel, ch_part_T part); char_u *channel_first_nl(readq_T *node); -char_u *channel_get(channel_T *channel, ch_part_T part); +char_u *channel_get(channel_T *channel, ch_part_T part, int *outlen); void channel_consume(channel_T *channel, ch_part_T part, int len); int channel_collapse(channel_T *channel, ch_part_T part, int want_nl); int channel_can_write_to(channel_T *channel); @@ -30,7 +30,7 @@ void channel_close(channel_T *channel, i void channel_close_in(channel_T *channel); void channel_clear(channel_T *channel); void channel_free_all(void); -void common_channel_read(typval_T *argvars, typval_T *rettv, int raw); +void common_channel_read(typval_T *argvars, typval_T *rettv, int raw, int blob); channel_T *channel_fd2channel(sock_T fd, ch_part_T *partp); void channel_handle_events(int only_keep_open); int channel_any_keep_open(void); diff --git a/src/structs.h b/src/structs.h --- a/src/structs.h +++ b/src/structs.h @@ -1251,6 +1251,7 @@ typedef double float_T; typedef struct listvar_S list_T; typedef struct dictvar_S dict_T; typedef struct partial_S partial_T; +typedef struct blobvar_S blob_T; typedef struct jobvar_S job_T; typedef struct readq_S readq_T; @@ -1272,6 +1273,7 @@ typedef enum VAR_SPECIAL, // "v_number" is used VAR_JOB, // "v_job" is used VAR_CHANNEL, // "v_channel" is used + VAR_BLOB, // "v_blob" is used } vartype_T; /* @@ -1295,6 +1297,7 @@ typedef struct job_T *v_job; /* job value (can be NULL!) */ channel_T *v_channel; /* channel value (can be NULL!) */ #endif + blob_T *v_blob; /* blob value (can be NULL!) */ } vval; } typval_T; @@ -1401,6 +1404,16 @@ struct dictvar_S dict_T *dv_used_prev; /* previous dict in used dicts list */ }; +/* + * Structure to hold info about a blob. + */ +struct blobvar_S +{ + garray_T bv_ga; // growarray with the data + int bv_refcount; // reference count + char bv_lock; // zero, VAR_LOCKED, VAR_FIXED +}; + #if defined(FEAT_EVAL) || defined(PROTO) typedef struct funccall_S funccall_T; @@ -3526,6 +3539,7 @@ typedef struct lval_S dict_T *ll_dict; /* The Dictionary or NULL */ dictitem_T *ll_di; /* The dictitem or NULL */ char_u *ll_newkey; /* New key for Dict in alloc. mem or NULL. */ + blob_T *ll_blob; /* The Blob or NULL */ } lval_T; /* Structure used to save the current state. Used when executing Normal mode diff --git a/src/testdir/Make_all.mak b/src/testdir/Make_all.mak --- a/src/testdir/Make_all.mak +++ b/src/testdir/Make_all.mak @@ -73,6 +73,7 @@ NEW_TESTS = \ test_backspace_opt \ test_backup \ test_behave \ + test_blob \ test_blockedit \ test_breakindent \ test_bufline \ @@ -283,6 +284,7 @@ NEW_TESTS_RES = \ test_autocmd.res \ test_autoload.res \ test_backspace_opt.res \ + test_blob.res \ test_blockedit.res \ test_breakindent.res \ test_bufwintabinfo.res \ diff --git a/src/testdir/test_blob.vim b/src/testdir/test_blob.vim new file mode 100644 --- /dev/null +++ b/src/testdir/test_blob.vim @@ -0,0 +1,179 @@ +" Tests for the Blob types + +func TearDown() + " Run garbage collection after every test + call test_garbagecollect_now() +endfunc + +" Tests for Blob type + +" Blob creation from constant +func Test_blob_create() + let b = 0zDEADBEEF + call assert_equal(v:t_blob, type(b)) + call assert_equal(4, len(b)) + call assert_equal(0xDE, b[0]) + call assert_equal(0xAD, b[1]) + call assert_equal(0xBE, b[2]) + call assert_equal(0xEF, b[3]) + call assert_fails('let x = b[4]') + + call assert_equal(0xDE, get(b, 0)) + call assert_equal(0xEF, get(b, 3)) + call assert_fails('let x = get(b, 4)') +endfunc + +" assignment to a blob +func Test_blob_assign() + let b = 0zDEADBEEF + let b2 = b[1:2] + call assert_equal(0zADBE, b2) + + let bcopy = b[:] + call assert_equal(b, bcopy) + call assert_false(b is bcopy) +endfunc + +func Test_blob_to_string() + let b = 0zDEADBEEF + call assert_equal('[0xDE,0xAD,0xBE,0xEF]', string(b)) + call remove(b, 0, 3) + call assert_equal('[]', string(b)) +endfunc + +func Test_blob_compare() + let b1 = 0z0011 + let b2 = 0z1100 + call assert_false(b1 == b2) + call assert_true(b1 != b2) + call assert_true(b1 == 0z0011) + + call assert_false(b1 is b2) + let b2 = b1 + call assert_true(b1 is b2) + + call assert_fails('let x = b1 > b2') + call assert_fails('let x = b1 < b2') + call assert_fails('let x = b1 - b2') + call assert_fails('let x = b1 / b2') + call assert_fails('let x = b1 * b2') +endfunc + +" test for range assign +func Test_blob_range_assign() + let b = 0z00 + let b[1] = 0x11 + let b[2] = 0x22 + call assert_equal(0z001122, b) + call assert_fails('let b[4] = 0x33') +endfunc + +func Test_blob_for_loop() + let blob = 0z00010203 + let i = 0 + for byte in blob + call assert_equal(i, byte) + let i += 1 + endfor + + let blob = 0z00 + call remove(blob, 0) + call assert_equal(0, len(blob)) + for byte in blob + call assert_error('loop over empty blob') + endfor +endfunc + +func Test_blob_concatenate() + let b = 0z0011 + let b += 0z2233 + call assert_equal(0z00112233, b) + + call assert_fails('let b += "a"') + call assert_fails('let b += 88') + + let b = 0zDEAD + 0zBEEF + call assert_equal(0zDEADBEEF, b) +endfunc + +" Test removing items in blob +func Test_blob_func_remove() + " Test removing 1 element + let b = 0zDEADBEEF + call assert_equal(0xDE, remove(b, 0)) + call assert_equal(0zADBEEF, b) + + let b = 0zDEADBEEF + call assert_equal(0xEF, remove(b, -1)) + call assert_equal(0zDEADBE, b) + + let b = 0zDEADBEEF + call assert_equal(0xAD, remove(b, 1)) + call assert_equal(0zDEBEEF, b) + + " Test removing range of element(s) + let b = 0zDEADBEEF + call assert_equal(0zBE, remove(b, 2, 2)) + call assert_equal(0zDEADEF, b) + + let b = 0zDEADBEEF + call assert_equal(0zADBE, remove(b, 1, 2)) + call assert_equal(0zDEEF, b) + + " Test invalid cases + let b = 0zDEADBEEF + call assert_fails("call remove(b, 5)", 'E979:') + call assert_fails("call remove(b, 1, 5)", 'E979:') + call assert_fails("call remove(b, 3, 2)", 'E979:') + call assert_fails("call remove(1, 0)", 'E712:') + call assert_fails("call remove(b, b)", 'E974:') +endfunc + +func Test_blob_read_write() + let b = 0zDEADBEEF + call writefile(b, 'Xblob') + let br = readfile('Xblob', 'B') + call assert_equal(b, br) + call delete('Xblob') +endfunc + +" filter() item in blob +func Test_blob_filter() + let b = 0zDEADBEEF + call filter(b, 'v:val != 0xEF') + call assert_equal(0zDEADBE, b) +endfunc + +" map() item in blob +func Test_blob_map() + let b = 0zDEADBEEF + call map(b, 'v:val + 1') + call assert_equal(0zDFAEBFF0, b) +endfunc + +func Test_blob_index() + call assert_equal(2, index(0zDEADBEEF, 0xBE)) + call assert_equal(-1, index(0zDEADBEEF, 0)) +endfunc + +func Test_blob_insert() + let b = 0zDEADBEEF + call insert(b, 0x33) + call assert_equal(0z33DEADBEEF, b) + + let b = 0zDEADBEEF + call insert(b, 0x33, 2) + call assert_equal(0zDEAD33BEEF, b) +endfunc + +func Test_blob_reverse() + call assert_equal(0zEFBEADDE, reverse(0zDEADBEEF)) + call assert_equal(0zBEADDE, reverse(0zDEADBE)) + call assert_equal(0zADDE, reverse(0zDEAD)) + call assert_equal(0zDE, reverse(0zDE)) +endfunc + +func Test_blob_json_encode() + call assert_equal('[222,173,190,239]', json_encode(0zDEADBEEF)) + call assert_equal('[]', json_encode(0z)) +endfunc diff --git a/src/testdir/test_channel.vim b/src/testdir/test_channel.vim --- a/src/testdir/test_channel.vim +++ b/src/testdir/test_channel.vim @@ -516,6 +516,51 @@ func Test_raw_pipe() call assert_equal(1, found) endfunc +func Test_raw_pipe_blob() + if !has('job') + return + endif + call ch_log('Test_raw_pipe_blob()') + " Add a dummy close callback to avoid that messages are dropped when calling + " ch_canread(). + " Also test the non-blocking option. + let job = job_start(s:python . " test_channel_pipe.py", + \ {'mode': 'raw', 'drop': 'never', 'noblock': 1}) + call assert_equal(v:t_job, type(job)) + call assert_equal("run", job_status(job)) + + call assert_equal("open", ch_status(job)) + call assert_equal("open", ch_status(job), {"part": "out"}) + + try + " Create a blob with the echo command and write it. + let blob = 0z00 + let cmd = "echo something\n" + for i in range(0, len(cmd) - 1) + let blob[i] = char2nr(cmd[i]) + endfor + call assert_equal(len(cmd), len(blob)) + call ch_sendraw(job, blob) + + " Read a blob with the reply. + let msg = ch_readblob(job) + let expected = 'something' + for i in range(0, len(expected) - 1) + call assert_equal(char2nr(expected[i]), msg[i]) + endfor + + let reply = ch_evalraw(job, "quit\n", {'timeout': 100}) + call assert_equal("Goodbye!\n", substitute(reply, "\r", "", 'g')) + finally + call job_stop(job) + endtry + + let g:Ch_job = job + call WaitForAssert({-> assert_equal("dead", job_status(g:Ch_job))}) + let info = job_info(job) + call assert_equal("dead", info.status) +endfunc + func Test_nl_pipe() if !has('job') return diff --git a/src/version.c b/src/version.c --- a/src/version.c +++ b/src/version.c @@ -796,6 +796,8 @@ static char *(features[]) = static int included_patches[] = { /* Add new patch number below this line */ /**/ + 735, +/**/ 734, /**/ 733, diff --git a/src/vim.h b/src/vim.h --- a/src/vim.h +++ b/src/vim.h @@ -1994,13 +1994,14 @@ typedef int sock_T; #define VV_TYPE_NONE 78 #define VV_TYPE_JOB 79 #define VV_TYPE_CHANNEL 80 -#define VV_TERMRFGRESP 81 -#define VV_TERMRBGRESP 82 -#define VV_TERMU7RESP 83 -#define VV_TERMSTYLERESP 84 -#define VV_TERMBLINKRESP 85 -#define VV_EVENT 86 -#define VV_LEN 87 /* number of v: vars */ +#define VV_TYPE_BLOB 81 +#define VV_TERMRFGRESP 82 +#define VV_TERMRBGRESP 83 +#define VV_TERMU7RESP 84 +#define VV_TERMSTYLERESP 85 +#define VV_TERMBLINKRESP 86 +#define VV_EVENT 87 +#define VV_LEN 88 /* number of v: vars */ /* used for v_number in VAR_SPECIAL */ #define VVAL_FALSE 0L @@ -2019,6 +2020,7 @@ typedef int sock_T; #define VAR_TYPE_NONE 7 #define VAR_TYPE_JOB 8 #define VAR_TYPE_CHANNEL 9 +#define VAR_TYPE_BLOB 10 #ifdef FEAT_CLIPBOARD