Clear async global variables on successful response

If _ZSH_AUTOSUGGEST_ASYNC_FD is not cleared, something else may open the
file descriptor pointed to by it. And then the next call into
_zsh_autosuggest_async_request will close it causing trouble.

It seems like good practice to clean up _ZSH_AUTOSUGGEST_CHILD_PID as
well, though it's not directly causing any known problems at the moment.

I was able to produce errors with ZSH_AUTOSUGGEST_MANUAL_REBIND active
and sourcing a file async-widget-setup.zsh after the first precmd with
the plugin active. If manual rebind is not active or if the widgets are
created before the first precmd, then zsh-autosuggestions wraps the
widgets and for some reason we don't seem to get any fd conflicts.

The .zshrc:

```
ZSH_AUTOSUGGEST_MANUAL_REBIND=true
source zsh-autosuggestions.zsh
```

and async-widget-setup.zsh:

```
function async-widget-fork() {
	exec {fd}< <(echo foo)
	zle -M "opened: $fd, autosuggest fd: $_ZSH_AUTOSUGGEST_ASYNC_FD"
}

function async-widget-read() {
	zle -M "reading from $fd: $(cat <&$fd)"
}

zle -N async-widget-fork
zle -N async-widget-read

bindkey ^A async-widget-fork
bindkey ^B async-widget-read
```

Then run `ZDOTDIR=$PWD zsh` and run `source async-widget-setup.zsh`. At
the next prompt, type one character e.g. "a" to trigger an async
request/response cycle. This leaves _ZSH_AUTOSUGGEST_ASYNC_FD set to the
stale file descriptor number. Then press ^A to activate the fork. This
will set the fd parameter to the same number as
_ZSH_AUTOSUGGEST_ASYNC_FD. Then type another character e.g. "a" to
trigger an async request. This will print a "No handler installed" error
and close the file descriptor pointed to by both
_ZSH_AUTOSUGGEST_ASYNC_FD and fd. Pressing ^B at this point will fail to
read with a "bad file descriptor" error.

.circleci/config.yml

@@ -1,15 +0,0 @@
-version: 2
-jobs:
-  build:
-    parallelism: 4
-    shell: /bin/bash --login
-    docker:
-    - image: ericfreese/zsh-autosuggestions-test:latest
-    steps:
-    - checkout
-    - run:
-        name: Running tests
-        command: |
-          for v in $(grep "^[^#]" ZSH_VERSIONS | awk "(NR + $CIRCLE_NODE_INDEX) % $CIRCLE_NODE_TOTAL == 0"); do
-            TEST_ZSH_BIN=zsh-$v make test || exit 1
-          done

.github/workflows/integration.yml

@@ -0,0 +1,51 @@
+on: [push, pull_request]
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+env:
+  IMAGE_CACHE_PATH: /tmp/.image-cache
+  IMAGE_CACHE_NAME: zsh-autosuggestions-test
+jobs:
+  determine-versions:
+    runs-on: ubuntu-22.04
+    outputs:
+      versions: ${{ steps.set-versions.outputs.versions }}
+    steps:
+    - uses: actions/checkout@v3
+    - id: set-versions
+      run: |
+        echo "versions=$(
+          grep "^[^#]" ZSH_VERSIONS \
+          | sed -E 's/(^|$)/"/g' \
+          | paste -sd ',' - \
+          | sed -e 's/^/[/' -e 's/$/]/'
+        )" >> $GITHUB_OUTPUT
+  test:
+    needs: determine-versions
+    runs-on: ubuntu-22.04
+    strategy:
+      matrix:
+        version: ${{ fromJson(needs.determine-versions.outputs.versions) }}
+    steps:
+    - uses: actions/checkout@v3
+    - name: Docker image cache
+      id: image-cache
+      uses: actions/cache@v3
+      with:
+        path: ${{ env.IMAGE_CACHE_PATH }}
+        key: image-cache-${{ matrix.version }}-${{ hashFiles('Dockerfile', 'install_test_zsh.sh', 'Gemfile.lock') }}
+    - name: Load cached docker image if available
+      if: ${{ steps.image-cache.outputs.cache-hit }}
+      run: gunzip < $IMAGE_CACHE_PATH/$IMAGE_CACHE_NAME.tar.gz | docker load
+    - name: Build the docker image if necessary
+      if: ${{ !steps.image-cache.outputs.cache-hit }}
+      run: |
+        docker build --build-arg TEST_ZSH_VERSION=${{ matrix.version }} -t $IMAGE_CACHE_NAME .
+        mkdir -p $IMAGE_CACHE_PATH
+        docker save $IMAGE_CACHE_NAME | gzip > $IMAGE_CACHE_PATH/$IMAGE_CACHE_NAME.tar.gz
+    - name: Run the tests
+      run: |
+        docker run --rm \
+          -v $PWD:/zsh-autosuggestions \
+          $IMAGE_CACHE_NAME \
+          make test

CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## v0.7.0
+- Enable asynchronous mode by default (#498)
+- No longer wrap user widgets starting with `autosuggest-` prefix (#496)
+- Fix a bug wrapping widgets that modify the buffer (#541)
+
+
 ## v0.6.4
 - Fix `vi-forward-char` triggering a bell when using it to accept a suggestion (#488)
 - New configuration option to skip completion suggestions when buffer matches a pattern (#487)

Dockerfile

@@ -1,5 +1,8 @@
 FROM ruby:2.5.3-alpine
 
+ARG TEST_ZSH_VERSION
+RUN : "${TEST_ZSH_VERSION:?}"
+
 RUN apk add --no-cache autoconf
 RUN apk add --no-cache libtool
 RUN apk add --no-cache libcap-dev
@@ -11,10 +14,8 @@ RUN apk add --no-cache tmux
 
 WORKDIR /zsh-autosuggestions
 
-ADD ZSH_VERSIONS /zsh-autosuggestions/ZSH_VERSIONS
-ADD install_test_zsh.sh /zsh-autosuggestions/install_test_zsh.sh
+ADD install_test_zsh.sh ./
 RUN ./install_test_zsh.sh
 
-ADD Gemfile /zsh-autosuggestions/Gemfile
-ADD Gemfile.lock /zsh-autosuggestions/Gemfile.lock
+ADD Gemfile Gemfile.lock ./
 RUN bundle install

INSTALL.md

@@ -39,7 +39,10 @@
 2. Add the plugin to the list of plugins for Oh My Zsh to load (inside `~/.zshrc`):
 
     ```sh
-    plugins=(zsh-autosuggestions)
+    plugins=( 
+        # other plugins...
+        zsh-autosuggestions
+    )
     ```
 
 3. Start a new terminal session.

LICENSE

@@ -1,5 +1,5 @@
 Copyright (c) 2013 Thiago de Arruda
-Copyright (c) 2016-2019 Eric Freese
+Copyright (c) 2016-2021 Eric Freese
 
 Permission is hereby granted, free of charge, to any person
 obtaining a copy of this software and associated documentation

README.md

@@ -170,18 +170,16 @@ Tests are written in ruby using the [`rspec`](http://rspec.info/) framework. The
 
 Test files live in `spec/`. To run the tests, run `make test`. To run a specific test, run `TESTS=spec/some_spec.rb make test`. You can also specify a `zsh` binary to use by setting the `TEST_ZSH_BIN` environment variable (ex: `TEST_ZSH_BIN=/bin/zsh make test`).
 
-A docker image for testing is available [on docker hub](https://hub.docker.com/r/ericfreese/zsh-autosuggestions-test). It comes with ruby, the bundler dependencies, and all supported versions of zsh installed.
-
-Pull the docker image with:
+It's possible to run the tests for any supported version of zsh in a Docker image by building an image from the provided Dockerfile. To build the docker image for a specific version of zsh (where `<version>` below is substituted with the contents of a line from the [`ZSH_VERSIONS`](ZSH_VERSIONS) file), run:
 
 ```sh
-docker pull ericfreese/zsh-autosuggestions-test
+docker build --build-arg TEST_ZSH_VERSION=<version> -t zsh-autosuggestions-test .
 ```
 
-To run the tests for a specific version of zsh (where `<version>` below is substituted with the contents of a line from the [`ZSH_VERSIONS`](ZSH_VERSIONS) file):
+After building the image, run the tests via:
 
 ```sh
-docker run -it -e TEST_ZSH_BIN=zsh-<version> -v $PWD:/zsh-autosuggestions zsh-autosuggestions-test make test
+docker run -it -v $PWD:/zsh-autosuggestions zsh-autosuggestions-test make test
 ```
 
 

VERSION

@@ -1 +1 @@
-v0.6.4
+v0.7.0

ZSH_VERSIONS

@@ -1,9 +1,5 @@
 # Zsh releases to run tests against
 # See https://github.com/zsh-users/zsh/releases
-#
-# When modifying this file, rebuild and push docker image:
-#   $ docker build -t ericfreese/zsh-autosuggestions-test .
-#   $ docker push ericfreese/zsh-autosuggestions-test
 4.3.11
 5.0.2
 5.0.8
@@ -14,3 +10,5 @@
 5.5.1
 5.6.2
 5.7.1
+5.8.1
+5.9

install_test_zsh.sh

@@ -2,25 +2,22 @@
 
 set -ex
 
-for v in $(grep "^[^#]" ZSH_VERSIONS); do
-  mkdir zsh-$v
-  cd zsh-$v
+mkdir zsh-build
+cd zsh-build
 
-  curl -L https://api.github.com/repos/zsh-users/zsh/tarball/zsh-$v | tar xz --strip=1
+curl -L https://api.github.com/repos/zsh-users/zsh/tarball/zsh-$TEST_ZSH_VERSION | tar xz --strip=1
 
-  ./Util/preconfig
-  ./configure --enable-pcre \
-              --enable-cap \
-              --enable-multibyte \
-              --with-term-lib='ncursesw tinfo' \
-              --with-tcsetpgrp \
-              --program-suffix="-$v"
+./Util/preconfig
+./configure --enable-pcre \
+            --enable-cap \
+            --enable-multibyte \
+            --with-term-lib='ncursesw tinfo' \
+            --with-tcsetpgrp
 
-  make install.bin
-  make install.modules
-  make install.fns
+make install.bin
+make install.modules
+make install.fns
 
-  cd ..
+cd ..
 
-  rm -rf zsh-$v
-done
+rm -rf zsh-build

spec/multi_line_spec.rb

@@ -1,11 +1,6 @@
 describe 'a multi-line suggestion' do
   it 'should be displayed on multiple lines' do
-    with_history(-> {
-      session.send_string('echo "')
-      session.send_keys('enter')
-      session.send_string('"')
-      session.send_keys('enter')
-    }) do
+    with_history("echo \"\n\"") do
       session.send_keys('e')
       wait_for { session.content }.to eq("echo \"\n\"")
     end

spec/spec_helper.rb

@@ -1,6 +1,7 @@
 require 'pry'
 require 'rspec/wait'
 require 'terminal_session'
+require 'tempfile'
 
 RSpec.shared_context 'terminal session' do
   let(:term_opts) { {} }
@@ -21,18 +22,20 @@ RSpec.shared_context 'terminal session' do
   end
 
   def with_history(*commands, &block)
-    session.run_command('fc -p')
+    Tempfile.create do |f|
+      f.write(commands.map{|c| c.gsub("\n", "\\\n")}.join("\n"))
+      f.flush
 
-    commands.each do |c|
-      c.respond_to?(:call) ? c.call : session.run_command(c)
+      session.run_command('fc -p')
+      session.run_command("fc -R #{f.path}")
+
+      session.clear_screen
+
+      yield block
+
+      session.send_keys('C-c')
+      session.run_command('fc -P')
     end
-
-    session.clear_screen
-
-    yield block
-
-    session.send_keys('C-c')
-    session.run_command('fc -P')
   end
 end
 

spec/strategies/special_characters_helper.rb

@@ -1,58 +1,71 @@
 shared_examples 'special characters' do
-  describe 'a special character in the buffer' do
-    it 'should be treated like any other character' do
+  describe 'a special character in the buffer should be treated like any other character' do
+    it 'asterisk' do
       with_history('echo "hello*"', 'echo "hello."') do
         session.send_string('echo "hello*')
         wait_for { session.content }.to eq('echo "hello*"')
       end
+    end
 
+    it 'question mark' do
       with_history('echo "hello?"', 'echo "hello."') do
         session.send_string('echo "hello?')
         wait_for { session.content }.to eq('echo "hello?"')
       end
+    end
 
+    it 'backslash' do
       with_history('echo "hello\nworld"') do
         session.send_string('echo "hello\\')
         wait_for { session.content }.to eq('echo "hello\nworld"')
       end
+    end
 
+    it 'double backslash' do
       with_history('echo "\\\\"') do
         session.send_string('echo "\\\\')
         wait_for { session.content }.to eq('echo "\\\\"')
       end
+    end
 
+    it 'tilde' do
       with_history('echo ~/foo') do
         session.send_string('echo ~')
         wait_for { session.content }.to eq('echo ~/foo')
       end
+    end
 
+    it 'parentheses' do
       with_history('echo "$(ls foo)"') do
         session.send_string('echo "$(')
         wait_for { session.content }.to eq('echo "$(ls foo)"')
       end
+    end
 
+    it 'square bracket' do
       with_history('echo "$history[123]"') do
         session.send_string('echo "$history[')
         wait_for { session.content }.to eq('echo "$history[123]"')
         session.send_string('123]')
         wait_for { session.content }.to eq('echo "$history[123]"')
       end
+    end
 
+    it 'octothorpe' do
       with_history('echo "#yolo"') do
         session.send_string('echo "#')
         wait_for { session.content }.to eq('echo "#yolo"')
       end
+    end
 
-      with_history('echo "#foo"', 'echo $#abc') do
-        session.send_string('echo "#')
-        wait_for { session.content }.to eq('echo "#foo"')
-      end
-
+    it 'caret' do
       with_history('echo "^A"', 'echo "^B"') do
         session.send_string('echo "^A')
         wait_for { session.content }.to eq('echo "^A"')
       end
+    end
 
+    it 'dash' do
       with_history('-foo() {}') do
         session.send_string('-')
         wait_for { session.content }.to eq('-foo() {}')

src/async.zsh

@@ -15,40 +15,39 @@ _zsh_autosuggest_async_request() {
 		zle -F $_ZSH_AUTOSUGGEST_ASYNC_FD
 
 		# We won't know the pid unless the user has zsh/system module installed
-		if [[ -n "$_ZSH_AUTOSUGGEST_CHILD_PID" ]]; then
-			# Zsh will make a new process group for the child process only if job
-			# control is enabled (MONITOR option)
-			if [[ -o MONITOR ]]; then
-				# Send the signal to the process group to kill any processes that may
-				# have been forked by the suggestion strategy
-				kill -TERM -$_ZSH_AUTOSUGGEST_CHILD_PID 2>/dev/null
-			else
-				# Kill just the child process since it wasn't placed in a new process
-				# group. If the suggestion strategy forked any child processes they may
-				# be orphaned and left behind.
-				kill -TERM $_ZSH_AUTOSUGGEST_CHILD_PID 2>/dev/null
-			fi
+		if (( _ZSH_AUTOSUGGEST_CHILD_PID )); then
+			kill -TERM -- $_ZSH_AUTOSUGGEST_CHILD_PID 2>/dev/null
 		fi
 	fi
 
 	# Fork a process to fetch a suggestion and open a pipe to read from it
 	exec {_ZSH_AUTOSUGGEST_ASYNC_FD}< <(
-		# Tell parent process our pid
-		echo $sysparams[pid]
+		# Tell parent process our pid if we can
+		echo ${sysparams[pid]:-}
 
 		# Fetch and print the suggestion
 		local suggestion
 		_zsh_autosuggest_fetch_suggestion "$1"
-		echo -nE "$suggestion"
+		echo -nE - "$suggestion"
 	)
 
 	# There's a weird bug here where ^C stops working unless we force a fork
 	# See https://github.com/zsh-users/zsh-autosuggestions/issues/364
-	command true
+	autoload -Uz is-at-least
+	is-at-least 5.8 || command true
 
 	# Read the pid from the child process
 	read _ZSH_AUTOSUGGEST_CHILD_PID <&$_ZSH_AUTOSUGGEST_ASYNC_FD
 
+	# Zsh will make a new process group for the child process only if job
+	# control is enabled (MONITOR option)
+	if [[ -o MONITOR ]]; then
+		# If we need to kill the background process in the future, we'll send
+		# SIGTERM to the process group to kill any processes that may have been
+		# forked by the suggestion strategy
+		_ZSH_AUTOSUGGEST_CHILD_PID=${_ZSH_AUTOSUGGEST_CHILD_PID:+-$_ZSH_AUTOSUGGEST_CHILD_PID}
+	fi
+
 	# When the fd is readable, call the response handler
 	zle -F "$_ZSH_AUTOSUGGEST_ASYNC_FD" _zsh_autosuggest_async_response
 }
@@ -59,17 +58,20 @@ _zsh_autosuggest_async_request() {
 _zsh_autosuggest_async_response() {
 	emulate -L zsh
 
+	typeset -g _ZSH_AUTOSUGGEST_ASYNC_FD _ZSH_AUTOSUGGEST_CHILD_PID
 	local suggestion
 
-	if [[ -z "$2" || "$2" == "hup" ]]; then
+	if [[ $# == 1 || "$2" == "hup" ]]; then
 		# Read everything from the fd and give it as a suggestion
 		IFS='' read -rd '' -u $1 suggestion
 		zle autosuggest-suggest -- "$suggestion"
 
 		# Close the fd
 		exec {1}<&-
+		_ZSH_AUTOSUGGEST_ASYNC_FD=
+		_ZSH_AUTOSUGGEST_ASYNC_PID=
 	fi
 
 	# Always remove the handler
-	zle -F "$1"
+	zle -F $1
 }

src/widgets.zsh

@@ -61,20 +61,9 @@ _zsh_autosuggest_modify() {
 		return $retval
 	fi
 
-	# Optimize if manually typing in the suggestion
-	if (( $#BUFFER > $#orig_buffer )); then
-		local added=${BUFFER#$orig_buffer}
-
-		# If the string added matches the beginning of the postdisplay
-		if [[ "$added" = "${orig_postdisplay:0:$#added}" ]]; then
-			POSTDISPLAY="${orig_postdisplay:$#added}"
-			return $retval
-		fi
-	fi
-
-	# Don't fetch a new suggestion if the buffer hasn't changed
-	if [[ "$BUFFER" = "$orig_buffer" ]]; then
-		POSTDISPLAY="$orig_postdisplay"
+	# Optimize if manually typing in the suggestion or if buffer hasn't changed
+	if [[ "$BUFFER" = "$orig_buffer"* && "$orig_postdisplay" = "${BUFFER:$#orig_buffer}"* ]]; then
+		POSTDISPLAY="${orig_postdisplay:$(($#BUFFER - $#orig_buffer))}"
 		return $retval
 	fi
 

zsh-autosuggestions.zsh

@@ -1,8 +1,8 @@
 # Fish-like fast/unobtrusive autosuggestions for zsh.
 # https://github.com/zsh-users/zsh-autosuggestions
-# v0.6.4
+# v0.7.0
 # Copyright (c) 2013 Thiago de Arruda
-# Copyright (c) 2016-2019 Eric Freese
+# Copyright (c) 2016-2021 Eric Freese
 # 
 # Permission is hereby granted, free of charge, to any person
 # obtaining a copy of this software and associated documentation
@@ -323,20 +323,9 @@ _zsh_autosuggest_modify() {
 		return $retval
 	fi
 
-	# Optimize if manually typing in the suggestion
-	if (( $#BUFFER > $#orig_buffer )); then
-		local added=${BUFFER#$orig_buffer}
-
-		# If the string added matches the beginning of the postdisplay
-		if [[ "$added" = "${orig_postdisplay:0:$#added}" ]]; then
-			POSTDISPLAY="${orig_postdisplay:$#added}"
-			return $retval
-		fi
-	fi
-
-	# Don't fetch a new suggestion if the buffer hasn't changed
-	if [[ "$BUFFER" = "$orig_buffer" ]]; then
-		POSTDISPLAY="$orig_postdisplay"
+	# Optimize if manually typing in the suggestion or if buffer hasn't changed
+	if [[ "$BUFFER" = "$orig_buffer"* && "$orig_postdisplay" = "${BUFFER:$#orig_buffer}"* ]]; then
+		POSTDISPLAY="${orig_postdisplay:$(($#BUFFER - $#orig_buffer))}"
 		return $retval
 	fi
 
@@ -781,40 +770,39 @@ _zsh_autosuggest_async_request() {
 		zle -F $_ZSH_AUTOSUGGEST_ASYNC_FD
 
 		# We won't know the pid unless the user has zsh/system module installed
-		if [[ -n "$_ZSH_AUTOSUGGEST_CHILD_PID" ]]; then
-			# Zsh will make a new process group for the child process only if job
-			# control is enabled (MONITOR option)
-			if [[ -o MONITOR ]]; then
-				# Send the signal to the process group to kill any processes that may
-				# have been forked by the suggestion strategy
-				kill -TERM -$_ZSH_AUTOSUGGEST_CHILD_PID 2>/dev/null
-			else
-				# Kill just the child process since it wasn't placed in a new process
-				# group. If the suggestion strategy forked any child processes they may
-				# be orphaned and left behind.
-				kill -TERM $_ZSH_AUTOSUGGEST_CHILD_PID 2>/dev/null
-			fi
+		if (( _ZSH_AUTOSUGGEST_CHILD_PID )); then
+			kill -TERM -- $_ZSH_AUTOSUGGEST_CHILD_PID 2>/dev/null
 		fi
 	fi
 
 	# Fork a process to fetch a suggestion and open a pipe to read from it
 	exec {_ZSH_AUTOSUGGEST_ASYNC_FD}< <(
-		# Tell parent process our pid
-		echo $sysparams[pid]
+		# Tell parent process our pid if we can
+		echo ${sysparams[pid]:-}
 
 		# Fetch and print the suggestion
 		local suggestion
 		_zsh_autosuggest_fetch_suggestion "$1"
-		echo -nE "$suggestion"
+		echo -nE - "$suggestion"
 	)
 
 	# There's a weird bug here where ^C stops working unless we force a fork
 	# See https://github.com/zsh-users/zsh-autosuggestions/issues/364
-	command true
+	autoload -Uz is-at-least
+	is-at-least 5.8 || command true
 
 	# Read the pid from the child process
 	read _ZSH_AUTOSUGGEST_CHILD_PID <&$_ZSH_AUTOSUGGEST_ASYNC_FD
 
+	# Zsh will make a new process group for the child process only if job
+	# control is enabled (MONITOR option)
+	if [[ -o MONITOR ]]; then
+		# If we need to kill the background process in the future, we'll send
+		# SIGTERM to the process group to kill any processes that may have been
+		# forked by the suggestion strategy
+		_ZSH_AUTOSUGGEST_CHILD_PID=${_ZSH_AUTOSUGGEST_CHILD_PID:+-$_ZSH_AUTOSUGGEST_CHILD_PID}
+	fi
+
 	# When the fd is readable, call the response handler
 	zle -F "$_ZSH_AUTOSUGGEST_ASYNC_FD" _zsh_autosuggest_async_response
 }
@@ -825,19 +813,22 @@ _zsh_autosuggest_async_request() {
 _zsh_autosuggest_async_response() {
 	emulate -L zsh
 
+	typeset -g _ZSH_AUTOSUGGEST_ASYNC_FD _ZSH_AUTOSUGGEST_CHILD_PID
 	local suggestion
 
-	if [[ -z "$2" || "$2" == "hup" ]]; then
+	if [[ $# == 1 || "$2" == "hup" ]]; then
 		# Read everything from the fd and give it as a suggestion
 		IFS='' read -rd '' -u $1 suggestion
 		zle autosuggest-suggest -- "$suggestion"
 
 		# Close the fd
 		exec {1}<&-
+		_ZSH_AUTOSUGGEST_ASYNC_FD=
+		_ZSH_AUTOSUGGEST_ASYNC_PID=
 	fi
 
 	# Always remove the handler
-	zle -F "$1"
+	zle -F $1
 }
 
 #--------------------------------------------------------------------#