Terminal Latency
================
Measuring Terminal Latency with Typometer
Posted on Mar 16 2024

Motivation
==========
I've been a long-time user of Xterm. I tried to switch to other
terminal emulators several times because of Xterm's broken Unicode
support, especially regarding glyphs/emojis and multi-font
substitution. These glyphs are part of many modern CLI tools and are
often printed as blank squares in Xterm. More recently, I attempted
to switch again, but every time I try, I'm discouraged by the
additional latency added during typing. I'm not a super-fast typist.
I average about 80 WPM for normal text with bursts for common
terminal commands of up to 120 WPM. The text appears in the terminal,
of course, but not as quickly as I would like. There is a noticeable
delay, especially when comparing something like Xterm to
xfce4-terminal. I've placed some hope in the recent development of
GPU-accelerated terminals, e.g., wezterm, but it still felt as slow
as xfce4-terminals. When I read the benchmarks, they often show how
fast it can print a gigabyte text file to stdout, but honestly, this
is something I'm not so interested in for everyday use. I found some
other interesting benchmarks regarding terminal latency, but there
were always some terminal emulators missing for which I would like to
know the result, or the results were slightly outdated.

Benchmark
=========
For the benchmark, I used Typometer [1], a tool designed to measure
and analyze the latency of various applications that have text input.
The test does not include keyboard latency or display latency, as
Typometer emulates the keystrokes in software, and the screen capture
is also in software, not via a physical camera in front of the
screen. Hence, you can expect additional latency from the hardware,
and these measurements represent only the latency that originates
from the software stack. All versions should be either the latest
stable version available via Arch Linux at the time of writing or the
latest master commit. All tests were conducted on the same machine, a
T14 Gen 1 (AMD Ryzen 7 PRO 4750U) with Arch Linux and Xorg (21.1.11-1).

Results
=======
xterm (389-1)
<https://beuke.org/images/xterm.jpg>

alacritty (0.13.1-1)
<https://beuke.org/images/alacritty.jpg>

kitty-tuned (0.31.0-1)
<https://beuke.org/images/kitty-tuned.jpg>

zutty (0.14-2)
<https://beuke.org/images/zutty.jpg>

st (master 95f22c5)
<https://beuke.org/images/st.jpg>

urxvt (9.31-4)
<https://beuke.org/images/urxvt.jpg>

konsole (24.02.0-1)
<https://beuke.org/images/konsole.jpg>

kitty (0.31.0-1)
<https://beuke.org/images/kitty.jpg>

wezterm (20230712.072601)
<https://beuke.org/images/wezterm.jpg>

gnome-terminal (3.50.1-1)
<https://beuke.org/images/gnome-terminal.jpg>

xfce4-terminal (1.1.1-2)
<https://beuke.org/images/xfce4-terminal.jpg>

terminator (2.1.3-3)
<https://beuke.org/images/terminator.jpg>

tilix (1.9.6-3)
<https://beuke.org/images/tilix.jpg>

hyper (v3.4.1)
<https://beuke.org/images/hyper.jpg>

Terminal Emulator Min Max Avg Stddev
---------------------- ---- ---- ---- ------
xterm (389-1) 2.8 9.8 5.3 1.1
alacritty (0.13.1-1) 5.2 17.8 6.9 1.8
kitty-tuned (0.31.0-1) 8.1 16.3 10.7 1.4
zutty (0.14-2) 7.4 16.4 11.2 1.6
st (master 95f22c5) 11.4 17.9 14.2 1.2
urxvt (9.31-4) 18.4 22.7 20.4 0.8
konsole (24.02.0-1) 16.4 26.8 20.7 2.2
kitty (0.31.0-1) 11.5 34.4 23.8 2.6
wezterm (20230712.072601) 11.3 40.9 26.1 7.2
gnome-terminal (3.50.1-1) 29.0 32.3 30.2 0.8
xfce4-terminal (1.1.1-2) 28.0 36.1 30.2 1.1
terminator (2.1.3-3) 28.7 48.0 30.5 2.0
tilix (1.9.6-3) 28.6 69.7 31.0 4.4
hyper (v3.4.1) 28.1 58.9 39.8 5.7

Xterm yields the best results, and Hyper (a web-based terminal) has
the worst results. This has met my expectations and matched the
results from other blog posts. Hyper, with about 40 ms latency, is
not as bad as I thought. However, anything with more than 20 ms I
consider a noticeable delay, and everything below 10 ms is fast
enough for my needs. I find it quite interesting that kitty can be
tuned to be more than twice as fast in terms latency. For kitty tuned
I used the following settings:

# 150 FPS
repaint_delay 8

# Remove artificial input delay
input_delay 0

# turn off vsync
sync_to_monitor no

I gathered these settings from another blog post about terminal
latency [2], which is worth reading. Please note that the results in
this blog post are not comparable with the results shown here because
the author used a camera to measure the latency, which also includes
the latency of the monitor.

I've also tested the following applications:

Application Min Max Avg Stddev
----------------------------- ---- ---- ---- ------
gvim (9.1.0000-1) 4.3 31.7 8.0 5.4
alacritty+tmux+neovim
(0.13.1-1+3.3_a-7+0.9.5-2) 5.4 12.9 8.3 1.4
chromium (120.0.6099.216-1) 9.1 28.6 19.6 6.2
firefox (121.0.1-1) 10.3 28.3 24.1 2.5
Visual Studio Code (1.87.2-1) 26.3 36.7 31.2 3.3

As we can see, the latency for Neovim inside tmux inside Alacritty
(8.3 ms) is not much higher than just Alacritty (6.9 ms). Hence, tmux
and Neovim add only about 1.4 ms of latency, which is quite
acceptable. We can also see that the latency of an HTML text area in
Chromium or Firefox is more than double the Alacritty latency. So, if
you often write in applications like Teams, then there is probably
not much you can do about it, other than accept about 20 ms of delay
for typing. And you are also out of luck in terms of latency if your
favorite code editor of choice is Visual Studio Code, as this editor
clocks in with a 31.2 ms delay before any hardware latency
considerations.

Conclusion
==========
I'm quite satisfied with the results, especially now that I have
found a decent alternative to Xterm, which has only 1.7 ms more
latency - Alacritty. I've seen benchmarks in the past that measured
higher values for Alacritty. Hence, I think the terminal latency has
improved over time due to complaints on GitHub [3] that caught some
attention from the maintainers (there's also my thumbs up on that
issue). For now, I will migrate my configs from Xterm to Alacritty
and report back in the form of another blog post in case there are
any issues.

From: <https://beuke.org/terminal-latency/>

[1]
<https://github.com/pavelfatin/typometer>

[2]
<https://www.lkhrs.com/blog/2022/07/terminal-latency/>

[3]
<https://github.com/alacritty/alacritty/issues/673>

Ben Collver <bencollver@tilde.pink> wrote at 01:53 this Friday (GMT):
> Terminal Latency
>================
> Measuring Terminal Latency with Typometer
> Posted on Mar 16 2024
>
> Motivation
>==========
> I've been a long-time user of Xterm. I tried to switch to other
> terminal emulators several times because of Xterm's broken Unicode
> support, especially regarding glyphs/emojis and multi-font
> substitution. These glyphs are part of many modern CLI tools and are
> often printed as blank squares in Xterm. More recently, I attempted
> to switch again, but every time I try, I'm discouraged by the
> additional latency added during typing. I'm not a super-fast typist.
> I average about 80 WPM for normal text with bursts for common
> terminal commands of up to 120 WPM. The text appears in the terminal,
> of course, but not as quickly as I would like. There is a noticeable
> delay, especially when comparing something like Xterm to
> xfce4-terminal. I've placed some hope in the recent development of
> GPU-accelerated terminals, e.g., wezterm, but it still felt as slow
> as xfce4-terminals. When I read the benchmarks, they often show how
> fast it can print a gigabyte text file to stdout, but honestly, this
> is something I'm not so interested in for everyday use. I found some
> other interesting benchmarks regarding terminal latency, but there
> were always some terminal emulators missing for which I would like to
> know the result, or the results were slightly outdated.
>
> Benchmark
>=========
> For the benchmark, I used Typometer [1], a tool designed to measure
> and analyze the latency of various applications that have text input.
> The test does not include keyboard latency or display latency, as
> Typometer emulates the keystrokes in software, and the screen capture
> is also in software, not via a physical camera in front of the
> screen. Hence, you can expect additional latency from the hardware,
> and these measurements represent only the latency that originates
> from the software stack. All versions should be either the latest
> stable version available via Arch Linux at the time of writing or the
> latest master commit. All tests were conducted on the same machine, a
> T14 Gen 1 (AMD Ryzen 7 PRO 4750U) with Arch Linux and Xorg (21.1.11-1).
>
> Results
>=======
> xterm (389-1)
><https://beuke.org/images/xterm.jpg>
>
> alacritty (0.13.1-1)
><https://beuke.org/images/alacritty.jpg>
>
> kitty-tuned (0.31.0-1)
><https://beuke.org/images/kitty-tuned.jpg>
>
> zutty (0.14-2)
><https://beuke.org/images/zutty.jpg>
>
> st (master 95f22c5)
><https://beuke.org/images/st.jpg>
>
> urxvt (9.31-4)
><https://beuke.org/images/urxvt.jpg>
>
> konsole (24.02.0-1)
><https://beuke.org/images/konsole.jpg>
>
> kitty (0.31.0-1)
><https://beuke.org/images/kitty.jpg>
>
> wezterm (20230712.072601)
><https://beuke.org/images/wezterm.jpg>
>
> gnome-terminal (3.50.1-1)
><https://beuke.org/images/gnome-terminal.jpg>
>
> xfce4-terminal (1.1.1-2)
><https://beuke.org/images/xfce4-terminal.jpg>
>
> terminator (2.1.3-3)
><https://beuke.org/images/terminator.jpg>
>
> tilix (1.9.6-3)
><https://beuke.org/images/tilix.jpg>
>
> hyper (v3.4.1)
><https://beuke.org/images/hyper.jpg>
>
> Terminal Emulator Min Max Avg Stddev
> ---------------------- ---- ---- ---- ------
> xterm (389-1) 2.8 9.8 5.3 1.1
> alacritty (0.13.1-1) 5.2 17.8 6.9 1.8
> kitty-tuned (0.31.0-1) 8.1 16.3 10.7 1.4
> zutty (0.14-2) 7.4 16.4 11.2 1.6
> st (master 95f22c5) 11.4 17.9 14.2 1.2
> urxvt (9.31-4) 18.4 22.7 20.4 0.8
> konsole (24.02.0-1) 16.4 26.8 20.7 2.2
> kitty (0.31.0-1) 11.5 34.4 23.8 2.6
> wezterm (20230712.072601) 11.3 40.9 26.1 7.2
> gnome-terminal (3.50.1-1) 29.0 32.3 30.2 0.8
> xfce4-terminal (1.1.1-2) 28.0 36.1 30.2 1.1
> terminator (2.1.3-3) 28.7 48.0 30.5 2.0
> tilix (1.9.6-3) 28.6 69.7 31.0 4.4
> hyper (v3.4.1) 28.1 58.9 39.8 5.7
>
> Xterm yields the best results, and Hyper (a web-based terminal) has
> the worst results. This has met my expectations and matched the
> results from other blog posts. Hyper, with about 40 ms latency, is
> not as bad as I thought. However, anything with more than 20 ms I
> consider a noticeable delay, and everything below 10 ms is fast
> enough for my needs. I find it quite interesting that kitty can be
> tuned to be more than twice as fast in terms latency. For kitty tuned
> I used the following settings:
>
> # 150 FPS
> repaint_delay 8
>
> # Remove artificial input delay
> input_delay 0
>
> # turn off vsync
> sync_to_monitor no
>
> I gathered these settings from another blog post about terminal
> latency [2], which is worth reading. Please note that the results in
> this blog post are not comparable with the results shown here because
> the author used a camera to measure the latency, which also includes
> the latency of the monitor.
>
> I've also tested the following applications:
>
> Application Min Max Avg Stddev
> ----------------------------- ---- ---- ---- ------
> gvim (9.1.0000-1) 4.3 31.7 8.0 5.4
> alacritty+tmux+neovim
> (0.13.1-1+3.3_a-7+0.9.5-2) 5.4 12.9 8.3 1.4
> chromium (120.0.6099.216-1) 9.1 28.6 19.6 6.2
> firefox (121.0.1-1) 10.3 28.3 24.1 2.5
> Visual Studio Code (1.87.2-1) 26.3 36.7 31.2 3.3
>
> As we can see, the latency for Neovim inside tmux inside Alacritty
> (8.3 ms) is not much higher than just Alacritty (6.9 ms). Hence, tmux
> and Neovim add only about 1.4 ms of latency, which is quite
> acceptable. We can also see that the latency of an HTML text area in
> Chromium or Firefox is more than double the Alacritty latency. So, if
> you often write in applications like Teams, then there is probably
> not much you can do about it, other than accept about 20 ms of delay
> for typing. And you are also out of luck in terms of latency if your
> favorite code editor of choice is Visual Studio Code, as this editor
> clocks in with a 31.2 ms delay before any hardware latency
> considerations.
>
> Conclusion
>==========
> I'm quite satisfied with the results, especially now that I have
> found a decent alternative to Xterm, which has only 1.7 ms more
> latency - Alacritty. I've seen benchmarks in the past that measured
> higher values for Alacritty. Hence, I think the terminal latency has
> improved over time due to complaints on GitHub [3] that caught some
> attention from the maintainers (there's also my thumbs up on that
> issue). For now, I will migrate my configs from Xterm to Alacritty
> and report back in the form of another blog post in case there are
> any issues.
>
> From: <https://beuke.org/terminal-latency/>
>
> [1]
><https://github.com/pavelfatin/typometer>
>
> [2]
><https://www.lkhrs.com/blog/2022/07/terminal-latency/>
>
> [3]
><https://github.com/alacritty/alacritty/issues/673>

Interesting. I'm glad my terminal (urxvt) isn't too much slower than the
alternative.
--
user <candycane> is generated from /dev/urandom

On 2024-05-03, candycanearter07 <candycanearter07@...> wrote:
> Interesting. I'm glad my terminal (urxvt) isn't too much slower than the
> alternative.

Glad urxvt is working for you!

I think acceptable response time is a matter of taste.

I've seen many comments from people who only use the terminal emulator
for system administration. Most of their usage, uncluding editing, is
done in the GUI. That's a whole different "use case" than running all
terminal-based apps and development tools.

I remember using 8-bit computers with video controllers and CRT's with
terrible refresh rates, flicker, etc. My perspective is that it's all
good, even the worst case is better than that.

Here's a comment from the original article:

> Generally speaking, there's a tradeoff between latency and
> throughput/flicker.
>
> The smaller the latency, the worse the throughput is (e.g. in cat
> huge.txt) because the terminal has to render more frequently, and the
> more flicker-prone it becomes, for instance when the terminal updates
> the screen before the application completed its “output batch” - which
> then requires another screen update once the output batch is complete,
> e.g. when holding page-down in auto-repeat in vim or less.

And here's a quote about latency from a different article:

> After thorough research on terminal emulators performance, I have come
> to the conclusion that its most important aspect is latency.
>
> But what is latency and why does it matter? In his article, Fatin
> defined latency as "a delay between the keystroke and corresponding
> screen update" and quoted the Handbook of Human-Computer Interaction
> which says: "Delay of visual feedback on a computer display have
> important effects on typist behavior and satisfaction."
>
> Fatin explained that latency has more profound effects than just
> satisfaction: "typing becomes slower, more errors occur, eye strain
> increases, and muscle strain increases". In other words, latency can
> lead to typos but also to lesser code quality as it imposes extra
> cognitive load on the brain. But worse, "eye and muscle strain increase"
> seems to imply that latency can also lead to physical repetitive strain
> injuries.
>
> Some of those effects have been known for a long time, with some results
> published in the Ergonomics journal in 1976 showing that a
> hundred-millisecond delay "significantly impairs the keying speed". More
> recently, the GNOME Human Interface Guidelines set the acceptable
> response time (archive.org link, new hig guide has no guidelines) at ten
> milliseconds and, pushing this limit down even further, this video from
> Microsoft Research shows that the ideal target might even be as low as
> one millisecond.

<https://pavelfatin.com/typing-with-pleasure/>

<https://books.google.com/books/about/Handbook_of_Human_Computer_
Interaction.html?id=WuQbERgXR10C&redir_esc=y>

<https://en.wikipedia.org/wiki/Repetitive_strain_injury>

<https://doi.org/10.1080/00140137608931531>

<https://web.archive.org/web/20210306065942/
https://developer.gnome.org/hig-book/3.12/
feedback-response-times.html.en>

<https://www.youtube.com/watch?v=vOvQCPLkPt4>

Ben Collver <bencollver@tilde.pink> writes:

> Terminal Latency
> ================
> Measuring Terminal Latency with Typometer
> Posted on Mar 16 2024
>
> Motivation
> ==========
> I've been a long-time user of Xterm. I tried to switch to other
> terminal emulators several times because of Xterm's broken Unicode
> support, especially regarding glyphs/emojis and multi-font
> substitution. These glyphs are part of many modern CLI tools and are
> often printed as blank squares in Xterm. More recently, I attempted
> to switch again, but every time I try, I'm discouraged by the
> additional latency added during typing.

Very cool research. Thanks for posting.

[...]

> Terminal Emulator Min Max Avg Stddev
> ---------------------- ---- ---- ---- ------
> xterm (389-1) 2.8 9.8 5.3 1.1
> alacritty (0.13.1-1) 5.2 17.8 6.9 1.8
> kitty-tuned (0.31.0-1) 8.1 16.3 10.7 1.4
> zutty (0.14-2) 7.4 16.4 11.2 1.6
> st (master 95f22c5) 11.4 17.9 14.2 1.2
> urxvt (9.31-4) 18.4 22.7 20.4 0.8
> konsole (24.02.0-1) 16.4 26.8 20.7 2.2
> kitty (0.31.0-1) 11.5 34.4 23.8 2.6
> wezterm (20230712.072601) 11.3 40.9 26.1 7.2
> gnome-terminal (3.50.1-1) 29.0 32.3 30.2 0.8
> xfce4-terminal (1.1.1-2) 28.0 36.1 30.2 1.1
> terminator (2.1.3-3) 28.7 48.0 30.5 2.0
> tilix (1.9.6-3) 28.6 69.7 31.0 4.4
> hyper (v3.4.1) 28.1 58.9 39.8 5.7
>
> Xterm yields the best results, and Hyper (a web-based terminal) has
> the worst results.

Hyper is an Electron-based application. I tried using it once, but
indeed it's very slow.

Ben Collver <bencollver@tilde.pink> wrote at 14:53 this Friday (GMT):
> On 2024-05-03, candycanearter07 <candycanearter07@...> wrote:
>> Interesting. I'm glad my terminal (urxvt) isn't too much slower than the
>> alternative.
>
> Glad urxvt is working for you!
>
> I think acceptable response time is a matter of taste.
>
> I've seen many comments from people who only use the terminal emulator
> for system administration. Most of their usage, uncluding editing, is
> done in the GUI. That's a whole different "use case" than running all
> terminal-based apps and development tools.

I use the terminal a fair bit.

> I remember using 8-bit computers with video controllers and CRT's with
> terrible refresh rates, flicker, etc. My perspective is that it's all
> good, even the worst case is better than that.
>
> Here's a comment from the original article:
>
>> Generally speaking, there's a tradeoff between latency and
>> throughput/flicker.
>>
>> The smaller the latency, the worse the throughput is (e.g. in cat
>> huge.txt) because the terminal has to render more frequently, and the
>> more flicker-prone it becomes, for instance when the terminal updates
>> the screen before the application completed its “output batch” - which
>> then requires another screen update once the output batch is complete,
>> e.g. when holding page-down in auto-repeat in vim or less.
>
> And here's a quote about latency from a different article:
>
>> After thorough research on terminal emulators performance, I have come
>> to the conclusion that its most important aspect is latency.
>>
>> But what is latency and why does it matter? In his article, Fatin
>> defined latency as "a delay between the keystroke and corresponding
>> screen update" and quoted the Handbook of Human-Computer Interaction
>> which says: "Delay of visual feedback on a computer display have
>> important effects on typist behavior and satisfaction."
>>
>> Fatin explained that latency has more profound effects than just
>> satisfaction: "typing becomes slower, more errors occur, eye strain
>> increases, and muscle strain increases". In other words, latency can
>> lead to typos but also to lesser code quality as it imposes extra
>> cognitive load on the brain. But worse, "eye and muscle strain increase"
>> seems to imply that latency can also lead to physical repetitive strain
>> injuries.
>>
>> Some of those effects have been known for a long time, with some results
>> published in the Ergonomics journal in 1976 showing that a
>> hundred-millisecond delay "significantly impairs the keying speed". More
>> recently, the GNOME Human Interface Guidelines set the acceptable
>> response time (archive.org link, new hig guide has no guidelines) at ten
>> milliseconds and, pushing this limit down even further, this video from
>> Microsoft Research shows that the ideal target might even be as low as
>> one millisecond.
>
><https://pavelfatin.com/typing-with-pleasure/>
>
><https://books.google.com/books/about/Handbook_of_Human_Computer_
> Interaction.html?id=WuQbERgXR10C&redir_esc=y>
>
><https://en.wikipedia.org/wiki/Repetitive_strain_injury>
>
><https://doi.org/10.1080/00140137608931531>
>
><https://web.archive.org/web/20210306065942/
> https://developer.gnome.org/hig-book/3.12/
> feedback-response-times.html.en>
>
><https://www.youtube.com/watch?v=vOvQCPLkPt4>

Huh, who knew that latency had such an effect?
--
user <candycane> is generated from /dev/urandom