Fix issues with LASTWIDGET caused by async suggestions

There are a number of widgets that rely on LASTWIDGET so that they can
chain their behavior together on subsequent invocations. When a
suggestion was fetched asynchronously, the widget showing the suggestion
would overwrite LASTWIDGET and break these widgets that rely on it.
Relatively recently, a flag was added to the `zle` builtin to opt out of
setting LASTWIDGET when calling a widget. We can use that flag to avoid
setting LASTWIDGET when displaying suggestions that were fetched
asynchronously.

.circleci/config.yml

@@ -1,15 +0,0 @@
-version: 2
-jobs:
-  build:
-    parallelism: 4
-    shell: /bin/bash --login
-    docker:
-    - image: ericfreese/zsh-autosuggestions-test:latest
-    steps:
-    - checkout
-    - run:
-        name: Running tests
-        command: |
-          for v in $(grep "^[^#]" ZSH_VERSIONS | awk "(NR + $CIRCLE_NODE_INDEX) % $CIRCLE_NODE_TOTAL == 0"); do
-            TEST_ZSH_BIN=zsh-$v make test || exit 1
-          done

.github/workflows/integration.yml

@@ -0,0 +1,51 @@
+on: [push, pull_request]
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+env:
+  IMAGE_CACHE_PATH: /tmp/.image-cache
+  IMAGE_CACHE_NAME: zsh-autosuggestions-test
+jobs:
+  determine-versions:
+    runs-on: ubuntu-22.04
+    outputs:
+      versions: ${{ steps.set-versions.outputs.versions }}
+    steps:
+    - uses: actions/checkout@v3
+    - id: set-versions
+      run: |
+        echo "versions=$(
+          grep "^[^#]" ZSH_VERSIONS \
+          | sed -E 's/(^|$)/"/g' \
+          | paste -sd ',' - \
+          | sed -e 's/^/[/' -e 's/$/]/'
+        )" >> $GITHUB_OUTPUT
+  test:
+    needs: determine-versions
+    runs-on: ubuntu-22.04
+    strategy:
+      matrix:
+        version: ${{ fromJson(needs.determine-versions.outputs.versions) }}
+    steps:
+    - uses: actions/checkout@v3
+    - name: Docker image cache
+      id: image-cache
+      uses: actions/cache@v3
+      with:
+        path: ${{ env.IMAGE_CACHE_PATH }}
+        key: image-cache-${{ matrix.version }}-${{ hashFiles('Dockerfile', 'install_test_zsh.sh', 'Gemfile.lock') }}
+    - name: Load cached docker image if available
+      if: ${{ steps.image-cache.outputs.cache-hit }}
+      run: gunzip < $IMAGE_CACHE_PATH/$IMAGE_CACHE_NAME.tar.gz | docker load
+    - name: Build the docker image if necessary
+      if: ${{ !steps.image-cache.outputs.cache-hit }}
+      run: |
+        docker build --build-arg TEST_ZSH_VERSION=${{ matrix.version }} -t $IMAGE_CACHE_NAME .
+        mkdir -p $IMAGE_CACHE_PATH
+        docker save $IMAGE_CACHE_NAME | gzip > $IMAGE_CACHE_PATH/$IMAGE_CACHE_NAME.tar.gz
+    - name: Run the tests
+      run: |
+        docker run --rm \
+          -v $PWD:/zsh-autosuggestions \
+          $IMAGE_CACHE_NAME \
+          make test

CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## v0.7.0
+- Enable asynchronous mode by default (#498)
+- No longer wrap user widgets starting with `autosuggest-` prefix (#496)
+- Fix a bug wrapping widgets that modify the buffer (#541)
+
+
 ## v0.6.4
 - Fix `vi-forward-char` triggering a bell when using it to accept a suggestion (#488)
 - New configuration option to skip completion suggestions when buffer matches a pattern (#487)

Dockerfile

@@ -1,5 +1,8 @@
 FROM ruby:2.5.3-alpine
 
+ARG TEST_ZSH_VERSION
+RUN : "${TEST_ZSH_VERSION:?}"
+
 RUN apk add --no-cache autoconf
 RUN apk add --no-cache libtool
 RUN apk add --no-cache libcap-dev
@@ -11,10 +14,8 @@ RUN apk add --no-cache tmux
 
 WORKDIR /zsh-autosuggestions
 
-ADD ZSH_VERSIONS /zsh-autosuggestions/ZSH_VERSIONS
-ADD install_test_zsh.sh /zsh-autosuggestions/install_test_zsh.sh
+ADD install_test_zsh.sh ./
 RUN ./install_test_zsh.sh
 
-ADD Gemfile /zsh-autosuggestions/Gemfile
-ADD Gemfile.lock /zsh-autosuggestions/Gemfile.lock
+ADD Gemfile Gemfile.lock ./
 RUN bundle install

INSTALL.md

@@ -39,7 +39,10 @@
 2. Add the plugin to the list of plugins for Oh My Zsh to load (inside `~/.zshrc`):
 
     ```sh
-    plugins=(zsh-autosuggestions)
+    plugins=( 
+        # other plugins...
+        zsh-autosuggestions
+    )
     ```
 
 3. Start a new terminal session.

LICENSE

@@ -1,5 +1,5 @@
 Copyright (c) 2013 Thiago de Arruda
-Copyright (c) 2016-2019 Eric Freese
+Copyright (c) 2016-2021 Eric Freese
 
 Permission is hereby granted, free of charge, to any person
 obtaining a copy of this software and associated documentation

README.md

@@ -170,18 +170,16 @@ Tests are written in ruby using the [`rspec`](http://rspec.info/) framework. The
 
 Test files live in `spec/`. To run the tests, run `make test`. To run a specific test, run `TESTS=spec/some_spec.rb make test`. You can also specify a `zsh` binary to use by setting the `TEST_ZSH_BIN` environment variable (ex: `TEST_ZSH_BIN=/bin/zsh make test`).
 
-A docker image for testing is available [on docker hub](https://hub.docker.com/r/ericfreese/zsh-autosuggestions-test). It comes with ruby, the bundler dependencies, and all supported versions of zsh installed.
-
-Pull the docker image with:
+It's possible to run the tests for any supported version of zsh in a Docker image by building an image from the provided Dockerfile. To build the docker image for a specific version of zsh (where `<version>` below is substituted with the contents of a line from the [`ZSH_VERSIONS`](ZSH_VERSIONS) file), run:
 
 ```sh
-docker pull ericfreese/zsh-autosuggestions-test
+docker build --build-arg TEST_ZSH_VERSION=<version> -t zsh-autosuggestions-test .
 ```
 
-To run the tests for a specific version of zsh (where `<version>` below is substituted with the contents of a line from the [`ZSH_VERSIONS`](ZSH_VERSIONS) file):
+After building the image, run the tests via:
 
 ```sh
-docker run -it -e TEST_ZSH_BIN=zsh-<version> -v $PWD:/zsh-autosuggestions zsh-autosuggestions-test make test
+docker run -it -v $PWD:/zsh-autosuggestions zsh-autosuggestions-test make test
 ```
 
 

VERSION

@@ -1 +1 @@
-v0.6.4
+v0.7.0

ZSH_VERSIONS

@@ -1,9 +1,5 @@
 # Zsh releases to run tests against
 # See https://github.com/zsh-users/zsh/releases
-#
-# When modifying this file, rebuild and push docker image:
-#   $ docker build -t ericfreese/zsh-autosuggestions-test .
-#   $ docker push ericfreese/zsh-autosuggestions-test
 4.3.11
 5.0.2
 5.0.8
@@ -14,3 +10,5 @@
 5.5.1
 5.6.2
 5.7.1
+5.8.1
+5.9

install_test_zsh.sh

@@ -2,25 +2,22 @@
 
 set -ex
 
-for v in $(grep "^[^#]" ZSH_VERSIONS); do
-  mkdir zsh-$v
-  cd zsh-$v
+mkdir zsh-build
+cd zsh-build
 
-  curl -L https://api.github.com/repos/zsh-users/zsh/tarball/zsh-$v | tar xz --strip=1
+curl -L https://api.github.com/repos/zsh-users/zsh/tarball/zsh-$TEST_ZSH_VERSION | tar xz --strip=1
 
-  ./Util/preconfig
-  ./configure --enable-pcre \
-              --enable-cap \
-              --enable-multibyte \
-              --with-term-lib='ncursesw tinfo' \
-              --with-tcsetpgrp \
-              --program-suffix="-$v"
+./Util/preconfig
+./configure --enable-pcre \
+            --enable-cap \
+            --enable-multibyte \
+            --with-term-lib='ncursesw tinfo' \
+            --with-tcsetpgrp
 
-  make install.bin
-  make install.modules
-  make install.fns
+make install.bin
+make install.modules
+make install.fns
 
-  cd ..
+cd ..
 
-  rm -rf zsh-$v
-done
+rm -rf zsh-build

spec/multi_line_spec.rb

@@ -1,11 +1,6 @@
 describe 'a multi-line suggestion' do
   it 'should be displayed on multiple lines' do
-    with_history(-> {
-      session.send_string('echo "')
-      session.send_keys('enter')
-      session.send_string('"')
-      session.send_keys('enter')
-    }) do
+    with_history("echo \"\n\"") do
       session.send_keys('e')
       wait_for { session.content }.to eq("echo \"\n\"")
     end

spec/spec_helper.rb

@@ -1,6 +1,7 @@
 require 'pry'
 require 'rspec/wait'
 require 'terminal_session'
+require 'tempfile'
 
 RSpec.shared_context 'terminal session' do
   let(:term_opts) { {} }
@@ -21,18 +22,20 @@ RSpec.shared_context 'terminal session' do
   end
 
   def with_history(*commands, &block)
-    session.run_command('fc -p')
+    Tempfile.create do |f|
+      f.write(commands.map{|c| c.gsub("\n", "\\\n")}.join("\n"))
+      f.flush
 
-    commands.each do |c|
-      c.respond_to?(:call) ? c.call : session.run_command(c)
+      session.run_command('fc -p')
+      session.run_command("fc -R #{f.path}")
+
+      session.clear_screen
+
+      yield block
+
+      session.send_keys('C-c')
+      session.run_command('fc -P')
     end
-
-    session.clear_screen
-
-    yield block
-
-    session.send_keys('C-c')
-    session.run_command('fc -P')
   end
 end
 

spec/strategies/special_characters_helper.rb

@@ -1,58 +1,71 @@
 shared_examples 'special characters' do
-  describe 'a special character in the buffer' do
-    it 'should be treated like any other character' do
+  describe 'a special character in the buffer should be treated like any other character' do
+    it 'asterisk' do
       with_history('echo "hello*"', 'echo "hello."') do
         session.send_string('echo "hello*')
         wait_for { session.content }.to eq('echo "hello*"')
       end
+    end
 
+    it 'question mark' do
       with_history('echo "hello?"', 'echo "hello."') do
         session.send_string('echo "hello?')
         wait_for { session.content }.to eq('echo "hello?"')
       end
+    end
 
+    it 'backslash' do
       with_history('echo "hello\nworld"') do
         session.send_string('echo "hello\\')
         wait_for { session.content }.to eq('echo "hello\nworld"')
       end
+    end
 
+    it 'double backslash' do
       with_history('echo "\\\\"') do
         session.send_string('echo "\\\\')
         wait_for { session.content }.to eq('echo "\\\\"')
       end
+    end
 
+    it 'tilde' do
       with_history('echo ~/foo') do
         session.send_string('echo ~')
         wait_for { session.content }.to eq('echo ~/foo')
       end
+    end
 
+    it 'parentheses' do
       with_history('echo "$(ls foo)"') do
         session.send_string('echo "$(')
         wait_for { session.content }.to eq('echo "$(ls foo)"')
       end
+    end
 
+    it 'square bracket' do
       with_history('echo "$history[123]"') do
         session.send_string('echo "$history[')
         wait_for { session.content }.to eq('echo "$history[123]"')
         session.send_string('123]')
         wait_for { session.content }.to eq('echo "$history[123]"')
       end
+    end
 
+    it 'octothorpe' do
       with_history('echo "#yolo"') do
         session.send_string('echo "#')
         wait_for { session.content }.to eq('echo "#yolo"')
       end
+    end
 
-      with_history('echo "#foo"', 'echo $#abc') do
-        session.send_string('echo "#')
-        wait_for { session.content }.to eq('echo "#foo"')
-      end
-
+    it 'caret' do
       with_history('echo "^A"', 'echo "^B"') do
         session.send_string('echo "^A')
         wait_for { session.content }.to eq('echo "^A"')
       end
+    end
 
+    it 'dash' do
       with_history('-foo() {}') do
         session.send_string('-')
         wait_for { session.content }.to eq('-foo() {}')

src/async.zsh

@@ -44,7 +44,7 @@ _zsh_autosuggest_async_request() {
 
 	# There's a weird bug here where ^C stops working unless we force a fork
 	# See https://github.com/zsh-users/zsh-autosuggestions/issues/364
-	command true
+	is-at-least 5.8 || command true
 
 	# Read the pid from the child process
 	read _ZSH_AUTOSUGGEST_CHILD_PID <&$_ZSH_AUTOSUGGEST_ASYNC_FD
@@ -64,7 +64,7 @@ _zsh_autosuggest_async_response() {
 	if [[ -z "$2" || "$2" == "hup" ]]; then
 		# Read everything from the fd and give it as a suggestion
 		IFS='' read -rd '' -u $1 suggestion
-		zle autosuggest-suggest -- "$suggestion"
+		_zsh_autosuggest_async_suggest "$suggestion"
 
 		# Close the fd
 		exec {1}<&-
@@ -73,3 +73,14 @@ _zsh_autosuggest_async_response() {
 	# Always remove the handler
 	zle -F "$1"
 }
+
+_zsh_autosuggest_async_suggest() {
+	# Before 5.9, async suggestions break widgets that rely on LASTWIDGET
+	# such as copy-earlier-word and {up,down}-line-or-beginning-search. In
+	# 5.9, a flag was added to `zle` that will skip setting LASTWIDGET so
+	# that those widgets that depend on it will continue to work
+	# See https://www.zsh.org/mla/workers/2020/msg00824.html
+	local nolast
+	is-at-least 5.9 && nolast=supported
+	zle autosuggest-suggest ${=nolast:+-f nolast} -- "$1"
+}

src/widgets.zsh

@@ -61,20 +61,9 @@ _zsh_autosuggest_modify() {
 		return $retval
 	fi
 
-	# Optimize if manually typing in the suggestion
-	if (( $#BUFFER > $#orig_buffer )); then
-		local added=${BUFFER#$orig_buffer}
-
-		# If the string added matches the beginning of the postdisplay
-		if [[ "$added" = "${orig_postdisplay:0:$#added}" ]]; then
-			POSTDISPLAY="${orig_postdisplay:$#added}"
-			return $retval
-		fi
-	fi
-
-	# Don't fetch a new suggestion if the buffer hasn't changed
-	if [[ "$BUFFER" = "$orig_buffer" ]]; then
-		POSTDISPLAY="$orig_postdisplay"
+	# Optimize if manually typing in the suggestion or if buffer hasn't changed
+	if [[ "$BUFFER" = "$orig_buffer"* && "$orig_postdisplay" = "${BUFFER:$#orig_buffer}"* ]]; then
+		POSTDISPLAY="${orig_postdisplay:$(($#BUFFER - $#orig_buffer))}"
 		return $retval
 	fi
 

zsh-autosuggestions.zsh

@@ -1,8 +1,8 @@
 # Fish-like fast/unobtrusive autosuggestions for zsh.
 # https://github.com/zsh-users/zsh-autosuggestions
-# v0.6.4
+# v0.7.0
 # Copyright (c) 2013 Thiago de Arruda
-# Copyright (c) 2016-2019 Eric Freese
+# Copyright (c) 2016-2021 Eric Freese
 # 
 # Permission is hereby granted, free of charge, to any person
 # obtaining a copy of this software and associated documentation
@@ -323,20 +323,9 @@ _zsh_autosuggest_modify() {
 		return $retval
 	fi
 
-	# Optimize if manually typing in the suggestion
-	if (( $#BUFFER > $#orig_buffer )); then
-		local added=${BUFFER#$orig_buffer}
-
-		# If the string added matches the beginning of the postdisplay
-		if [[ "$added" = "${orig_postdisplay:0:$#added}" ]]; then
-			POSTDISPLAY="${orig_postdisplay:$#added}"
-			return $retval
-		fi
-	fi
-
-	# Don't fetch a new suggestion if the buffer hasn't changed
-	if [[ "$BUFFER" = "$orig_buffer" ]]; then
-		POSTDISPLAY="$orig_postdisplay"
+	# Optimize if manually typing in the suggestion or if buffer hasn't changed
+	if [[ "$BUFFER" = "$orig_buffer"* && "$orig_postdisplay" = "${BUFFER:$#orig_buffer}"* ]]; then
+		POSTDISPLAY="${orig_postdisplay:$(($#BUFFER - $#orig_buffer))}"
 		return $retval
 	fi
 
@@ -810,7 +799,7 @@ _zsh_autosuggest_async_request() {
 
 	# There's a weird bug here where ^C stops working unless we force a fork
 	# See https://github.com/zsh-users/zsh-autosuggestions/issues/364
-	command true
+	is-at-least 5.8 || command true
 
 	# Read the pid from the child process
 	read _ZSH_AUTOSUGGEST_CHILD_PID <&$_ZSH_AUTOSUGGEST_ASYNC_FD
@@ -830,7 +819,7 @@ _zsh_autosuggest_async_response() {
 	if [[ -z "$2" || "$2" == "hup" ]]; then
 		# Read everything from the fd and give it as a suggestion
 		IFS='' read -rd '' -u $1 suggestion
-		zle autosuggest-suggest -- "$suggestion"
+		_zsh_autosuggest_async_suggest "$suggestion"
 
 		# Close the fd
 		exec {1}<&-
@@ -840,6 +829,17 @@ _zsh_autosuggest_async_response() {
 	zle -F "$1"
 }
 
+_zsh_autosuggest_async_suggest() {
+	# Before 5.9, async suggestions break widgets that rely on LASTWIDGET
+	# such as copy-earlier-word and {up,down}-line-or-beginning-search. In
+	# 5.9, a flag was added to `zle` that will skip setting LASTWIDGET so
+	# that those widgets that depend on it will continue to work
+	# See https://www.zsh.org/mla/workers/2020/msg00824.html
+	local nolast
+	is-at-least 5.9 && nolast=supported
+	zle autosuggest-suggest ${=nolast:+-f nolast} -- "$1"
+}
+
 #--------------------------------------------------------------------#
 # Start                                                              #
 #--------------------------------------------------------------------#