Hi All,

Prompted by a Unix StackExchange question, I started a mini project
called mnpgr a few days ago. It's in a very usable state now.

This is glue code, written in TXR Lisp which turns Vim into a
page for reading man pages.

Plus, a small Vim syntax filee.

https://www.kylheku.com/cgit/mnpgr/about/

The benefits are

- Remembers your last position in any man page. This is useful for
working with long man pages.

- Leverages your Vim skills and environment.

- Has better rendering of backspace overstrikes than the "less" utility.
For instance, it correctly renders the French accented characters
exemplified in the GNU grep man page.
- bold+italic is recognized and assigned to its own highlight
category.

The program translates man's overstrikes into its own markup notation;
for instance, bold text is {B{bold text}B}. The Vim syntax file hides
the markup and colorizes the payload.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On 15.10.2023 02:46, Kaz Kylheku wrote:
> Hi All,
>
> Prompted by a Unix StackExchange question, I started a mini project
> called mnpgr a few days ago. It's in a very usable state now.
>
> This is glue code, written in TXR Lisp which turns Vim into a
> page for reading man pages.
>
> Plus, a small Vim syntax filee.
>
> https://www.kylheku.com/cgit/mnpgr/about/
>
> The benefits are
>
> - Remembers your last position in any man page. This is useful for
> working with long man pages.
>
> - Leverages your Vim skills and environment.
>
> - Has better rendering of backspace overstrikes than the "less" utility.
> For instance, it correctly renders the French accented characters
> exemplified in the GNU grep man page.
> - bold+italic is recognized and assigned to its own highlight
> category.
>
> The program translates man's overstrikes into its own markup notation;
> for instance, bold text is {B{bold text}B}. The Vim syntax file hides
> the markup and colorizes the payload.

I don't think I'll use 'txr', but your post made me think about the
task...

If 'vim' is your pager you could do

man ... | vim -

:set syntax=man will also do some (predefined) highlighting. While in
Vim you can of course not only remember the "last position" (there's
the default mark, a single quote, for that) but you can also define
arbitrary marks and, of course, randomly jump between them. Or split
the screen to see more than one man page context. You can also (e.g.)
extract portions of the man page. Do whatever an editor supports.

A reason against using an editor is probably the size consideration.

What doesn't seem to work nicely is setting the pager to vim

PAGER='vim -' man ...

that will (as opposed to the pipe above) show all the overstrikes you
mention above. Simply adding a pipe for col(1) (to remove that junk)
isn't straightforward it seems.

Janis

In article <ugh237$j6fg$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
....
>I don't think I'll use 'txr', but your post made me think about the
>task...

Indeed. It seems unlikely that anyone would take on TXR just for this
purpose.

Anyway, just to take your idea a bit firther, I'm actually quite pleased
with the effect of:

$ /usr/bin/man ls | vi -c 'set syntax=man nomod|split' -

--
Elect a clown, expect a circus.

On 15/10/2023 17:23, Kenny McCormack wrote:
> In article <ugh237$j6fg$1@dont-email.me>,
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> ...
>> I don't think I'll use 'txr', but your post made me think about the
>> task...
>
> Indeed. It seems unlikely that anyone would take on TXR just for this
> purpose.
>
> Anyway, just to take your idea a bit firther, I'm actually quite pleased
> with the effect of:
>
> $ /usr/bin/man ls | vi -c 'set syntax=man nomod|split' -
>

I use a function (to save typing):

vman ()
{ COLUMNS=$((COLUMNS-5)) /usr/bin/man ${@-man} | col -bpx | iconv -c |
view -c 'set ft=man nomod' -c 'set noeb vb t_vb=' -c 'map q
<Esc>:q!<cr>' -c 'map i <nop>' -
}

and then alias man to vman

--
Chris Elvidge, England
I WILL NOT YELL "FIRE" IN A CROWDED CLASSROOM

On 2023-10-15, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> On 15.10.2023 02:46, Kaz Kylheku wrote:
>> Hi All,
>>
>> Prompted by a Unix StackExchange question, I started a mini project
>> called mnpgr a few days ago. It's in a very usable state now.
>>
>> This is glue code, written in TXR Lisp which turns Vim into a
>> page for reading man pages.
>>
>> Plus, a small Vim syntax filee.
>>
>> https://www.kylheku.com/cgit/mnpgr/about/
>>
>> The benefits are
>>
>> - Remembers your last position in any man page. This is useful for
>> working with long man pages.
>>
>> - Leverages your Vim skills and environment.
>>
>> - Has better rendering of backspace overstrikes than the "less" utility.
>> For instance, it correctly renders the French accented characters
>> exemplified in the GNU grep man page.
>> - bold+italic is recognized and assigned to its own highlight
>> category.
>>
>> The program translates man's overstrikes into its own markup notation;
>> for instance, bold text is {B{bold text}B}. The Vim syntax file hides
>> the markup and colorizes the payload.
>
> I don't think I'll use 'txr', but your post made me think about the
> task...
>
> If 'vim' is your pager you could do
>
> man ... | vim -
>
>:set syntax=man will also do some (predefined) highlighting. While in

The problem is that vim will not remember the position in a buffer that
had been read from a pipe. This whole thing was prompted by someone
trying to solve that problem. They wanted to remember the position in
each individual man page. That resonated with me because there have been
times when I wanted to revisit something I recently read in a man page.

(One such situation is when I'm working *on* a man page, making
adjustments to some section and wanting to see their effect!)

Secondly, the "man" syntax highlighting in Vim is a poor hack. It only
highlights some salient features like headings. It has no idea where the
keywords are that were supposed to be bolded and italicized.

> Vim you can of course not only remember the "last position" (there's
> the default mark, a single quote, for that) but you can also define
> arbitrary marks and, of course, randomly jump between them. Or split
> the screen to see more than one man page context. You can also (e.g.)
> extract portions of the man page. Do whatever an editor supports.
>
> A reason against using an editor is probably the size consideration.

I tried it on a 13-year-old Core i5, slow machine, on the txr man page,
which is 65000 lines. There is a bit of a delay, but otherwise it's
fine.

I don't know of a larger man page; it makes a PDF document over 930
pages with the default troff margins.

(That man page also uses text that is both bold and italic, which
provides a test case for the third color.)

> What doesn't seem to work nicely is setting the pager to vim
>
> PAGER='vim -' man ...

For that you want:

MANPAGER='vim +MANPAGER' man

(The MANPAGER variable is a red herring here; I'm using it in
this example instead of PAGER because PAGER will have no effect on man
if MANPAGER is already defined.)

The :MANPAGER vim command referenced by +MANPAGER brings in some
packaged support for reading man pages, and does render the overstrikes.

But:

1. I was not able to use +MANPAGER as the basis of a solution that
can remember the last position.

Here is why, I think: vim +MANPAGER doesn't actually read the man page
input; it seems to work by analyzing the MAN_PN environment variable and
itself fetching the man page.

2. For that reason, it also doesn't work with "man -l <file>",
which is a dealbreaker for people who work on man pages or view
man pages inside source trees, not only installed pages.

3. While the overstrike codes are rendered, the underlying data
has the interspersed backspaces. This means you cannot search for
those words. If foo is highlighted, /foo won't find it because it's
actually f^Hfo^Hoo^Ho.

So I gave up on the vim +MANPAGER approach.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On 2023-10-15, Kenny McCormack <gazelle@shell.xmission.com> wrote:
> In article <ugh237$j6fg$1@dont-email.me>,
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> ...
>>I don't think I'll use 'txr', but your post made me think about the
>>task...
>
> Indeed. It seems unlikely that anyone would take on TXR just for this
> purpose.

Anyway, the code captures the requirements of a particular working
solution; it would be easier to clone what that program is doing, than
to research and develop this from scratch.

The syntax highlighting file (mnpgr.vim) can be reused in a compatible
solution written in another scripting language.

If someone writes mnpgr.tl in another language, and is willing to put it
under the same BSD-1 license, I will add it to the tree.

> Anyway, just to take your idea a bit firther, I'm actually quite pleased
> with the effect of:
>
> $ /usr/bin/man ls | vi -c 'set syntax=man nomod|split' -

Won't remember your most recent position in that man page;
won't accurately colorize all the words that are marked up in the
original man page.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

In comp.unix.shell, Kaz Kylheku <864-117-4973@kylheku.com> wrote:
> Prompted by a Unix StackExchange question, I started a mini project
> called mnpgr a few days ago. It's in a very usable state now.

mnpgr: 8270e33c0a820ccb14c1cc6b97460db0a5748f0a
txr: bf5a4cbe020b6726e8f9cef4cd935861442541cf

$ mnpgr perl
/home/username/bin/mnpgr:72: syntax error

> This is glue code, written in TXR Lisp which turns Vim into a
> page for reading man pages.

Turns out txr is a lot more work to install than mnpgr.

> - Leverages your Vim skills and environment.
>
> - Has better rendering of backspace overstrikes than the "less" utility.

These two features sound compelling, hence giving it a chance.

Elijah
------
is not about to try to debug lisp

On 2023-10-15, Eli the Bearded <*@eli.users.panix.com> wrote:
> In comp.unix.shell, Kaz Kylheku <864-117-4973@kylheku.com> wrote:
>> Prompted by a Unix StackExchange question, I started a mini project
>> called mnpgr a few days ago. It's in a very usable state now.
>
> mnpgr: 8270e33c0a820ccb14c1cc6b97460db0a5748f0a
> txr: bf5a4cbe020b6726e8f9cef4cd935861442541cf
>
> $ mnpgr perl
> /home/username/bin/mnpgr:72: syntax error

If you remove the .tl suffix, you have to use "txr --lisp" on it,
or else the "txrlisp" alternative executable name (hard link).
Otherwise it's treated as different language.

The mnpgr.tl program takes no arguments and is not meant to be invoked
directly by the user (so you don't achieve any ergonomics by dropping
the suffix).

It is invoked by man, and expects to find the MAN_PN environment
variable set up by man.

See README.md file.

$ export MANPAGER=/path/to/mnpgr.tl

or

$ txr --compile=/path/to/mnpgr.tl
$ export MANPAGER=/path/to/mnpgr.tlo

Then just

$ man perl

It only works particular man implementation used on GNU/Linux systems;
the one maintained by Colin Watson.

>> This is glue code, written in TXR Lisp which turns Vim into a
>> page for reading man pages.
>
> Turns out txr is a lot more work to install than mnpgr.

If you're building from source, it should be just:

../configure && make && make install # "make tests" is recommended.

The dependencies are low and optional. If the configure script deosn't
detect header files for libz and libffi, you get a build without them.

In comp.unix.shell, Kaz Kylheku <864-117-4973@kylheku.com> wrote:
> On 2023-10-15, Eli the Bearded <*@eli.users.panix.com> wrote:
> > $ mnpgr perl
> > /home/username/bin/mnpgr:72: syntax error
> If you remove the .tl suffix, you have to use "txr --lisp" on it,
> or else the "txrlisp" alternative executable name (hard link).
> Otherwise it's treated as different language.

Hmmm. Okay.

I changed the #! line and:

$ MANPAGER=mnpgr man man
MAN(1) Manual pager utils
MAN(1)

{B{NAME}B}
man - an interface to the system reference manuals

{B{SYNOPSIS}B}
{B{man}B} [{I{man options}I}] [[{I{section}I}] {I{page}I} ...] ...

$

No highlighting, just your markup. Vim version 8.1.3741, vimrc:
$ grep -v '^"' ~/.vimrc
set mouse=
set exrc
set nohlsearch
set noincsearch
map * "yyy@y
map S F r
map = y p
set t_Co=
set sw=2 ai ruler
set shortmess=filnxtToOs
set nobackup
set backspace=indent,eol
set pastetoggle=<Insert>
map #F /^sub /<CR>w"pywf{%"ppF}a # end &<Esc>
$ find $HOME/.vim
/home/username/.vim
/home/username/.vim/syntax
/home/username/.vim/syntax/mnpgr.vim
$

Also:

$ MANPAGER=mnpgr man ./txr.1

Hangs for ~30 seconds before showing me the 62828L manpage.

> The mnpgr.tl program takes no arguments and is not meant to be invoked
> directly by the user (so you don't achieve any ergonomics by dropping
> the suffix).
>
> It is invoked by man, and expects to find the MAN_PN environment
> variable set up by man.

Also: MAN_PN is a poorly documented feature that exists only in some
implementations of 'man'. I don't get it set on NetBSD, for example.

> It only works particular man implementation used on GNU/Linux systems;
> the one maintained by Colin Watson.

Yes, apparently.

> ./configure && make && make install # "make tests" is recommended.

You forgot the 25M git clone. Also I set a prefix for the install. It
took a while to build, I didn't time out, and make install errored out
(harmlessly it seems):

$ make install
INSTALL txr -> /home/username/usr/txr/bin
HARDLINK /home/username/usr/txr/bin/txrlisp -> /home/username/usr/txr/bin/txr
HARDLINK /home/username/usr/txr/bin/txrvm -> /home/username/usr/txr/bin/txr
INSTALL LICENSE -> /home/username/usr/txr/share/txr
INSTALL METALICENSE -> /home/username/usr/txr/share/txr
INSTALL txr.1 -> /home/username/usr/txr/share/man/man1
INSTALL stdlib/*.txr stdlib/*.tl stdlib/*.tlo -> /home/username/usr/txr/share/txr/stdlib
cp: cannot stat 'stdlib/*.tlo': No such file or directory
make: failing command:
cp -f stdlib/*.txr stdlib/*.tl stdlib/*.tlo /home/username/usr/txr/share/txr/stdlib
make: *** [Makefile:536: install] Error 1
$

> The dependencies are low and optional. If the configure script deosn't
> detect header files for libz and libffi, you get a build without them.

I had them installed, didn't even notice the dependency there.

Elijah
------
contemplating attempting to rewrite the .tl file in perl

On 2023-10-16, Eli the Bearded <*@eli.users.panix.com> wrote:
> {B{SYNOPSIS}B}
> {B{man}B} [{I{man options}I}] [[{I{section}I}] {I{page}I} ...] ...

OK, so that part of it is working fine. We know man is invoking
mnpgr, which is doing the filtering and running Vim

If you type ":syntax on" does anything happen, or still no?

Do you uget syntax highlighting automatically when opening files?

I tried installing nothing but your configuration as my ~/.vimrc.

I still get the syntax highlighting.

(The background is white---my eyes!---and word wrap is on, which
screws up some lines: :set nowrap is needed.)

> $ MANPAGER=mnpgr man ./txr.1
>
> Hangs for ~30 seconds before showing me the 62828L manpage.

Compiling mnpgr.tl makes a big difference.

Another possible contributing factor to this is that the the compiled
stdlib didn't get built (see below).

For me, it comes up in about 7 seconds on a really slow, low-end Core
i5-3570 machine from 2012.

On the same machine:

$ time txr --compile=mnpgr.tl

real 0m0.192s
user 0m0.184s
sys 0m0.009s

If the compiler isn't compiled, that will be a lot slower.

> Also: MAN_PN is a poorly documented feature that exists only in some
> implementations of 'man'. I don't get it set on NetBSD, for example.

Yep. This manpgr thing is not a portable project. At least not in
its present state.

(Not to mention that it won't work with NetBSD's vi implementation, ha!)

> You forgot the 25M git clone.

I think that could be smaller if you don't need the history going back
to 2009, using "git clone --depth <number-of-commits>". There is a way
to pull a tarball also from the cgit.

(That goes for any project out there.)

> Also I set a prefix for the install. It
> took a while to build, I didn't time out, and make install errored out
> (harmlessly it seems):
>
> $ make install

Unfortunately, "make install" does not build everything. You have to run
"make" to build the default target.

> cp: cannot stat 'stdlib/*.tlo': No such file or directory

That is not exactly harmless; it means that the compiled stdlib didn't
get installed (since the .tlo files weren't built). It will fall back on
the interpreted one. Programs that rely on that library will run
slower. Possibly a lot slower.

> contemplating attempting to rewrite the .tl file in perl

Great; if you get stuck on anything, let me know.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

In comp.unix.shell, Kaz Kylheku <864-117-4973@kylheku.com> wrote:
> On 2023-10-16, Eli the Bearded <*@eli.users.panix.com> wrote:
> > {B{SYNOPSIS}B}
> > {B{man}B} [{I{man options}I}] [[{I{section}I}] {I{page}I} ...] ...
>
> OK, so that part of it is working fine. We know man is invoking
> mnpgr, which is doing the filtering and running Vim
>
> If you type ":syntax on" does anything happen, or still no?

E484: Can't open file /usr/local/share/vim/syntax/syntax.vim

Weird thing is, some files show syntax, eg, the mnpgr.vim file and
mnpgr.tl. Using ":set syntax?" I see the "mnpgr" value.

> Do you uget syntax highlighting automatically when opening files?

Usually I run ":syntax off" but I removed that line from my vimrc for
these experiments.

> (The background is white---my eyes!---and word wrap is on, which

All my xterms are white text on black.

> screws up some lines: :set nowrap is needed.)

I use that setting sparingly.

> For me, it comes up in about 7 seconds on a really slow, low-end Core
> i5-3570 machine from 2012.

I'm (presently) testing this on my Surface Go 2.

> (Not to mention that it won't work with NetBSD's vi implementation, ha!)

You won't get nvi with this:

90 (sh `vim +'@{vim-commands}' '@{rendered-file}' < /dev/tty`))))

> > You forgot the 25M git clone.
> I think that could be smaller if you don't need the history going back
> to 2009, using "git clone --depth <number-of-commits>".

That's true, and maybe I should default to that, but for other things I
do want history.

> There is a way to pull a tarball also from the cgit.

Github tarballs do not give last commit hash that I have noticed, which
makes them useful as point in time snapshots, but harder to report
issues with. I typically avoid the tarballs for that reason.

> > $ make install
>
> Unfortunately, "make install" does not build everything. You have to run
> "make" to build the default target.

The 'configure' tool told me to "make tests and "make install" so that's
what I did. I thought that the tests target would include all of
components of the default target. Is that not so?

I ran "make -k install" after the first install failed, then ran "make
install" again to capture the output for the post.

Elijah
------
wrote a much cruder "man in vi" in the 1990s

On 2023-10-16, Eli the Bearded <*@eli.users.panix.com> wrote:
> In comp.unix.shell, Kaz Kylheku <864-117-4973@kylheku.com> wrote:
>> If you type ":syntax on" does anything happen, or still no?
>
> E484: Can't open file /usr/local/share/vim/syntax/syntax.vim

That is odd. Is that because it doesn't exist, or the perms
are wrong?

/usr/local: you built Vim yourself?

It's interesting that there is no Vim version in the path, similar to
/usr/local/share/vim/vim82/syntax/syntax.vim.

I see that version-specific directory in distro Vim installations
like Ubuntu. (Is that a Vim thing, or courtesy of the distro, I
wonder.)

> The 'configure' tool told me to "make tests and "make install" so that's
> what I did. I thought that the tests target would include all of
> components of the default target. Is that not so?

It actually says the following, which lends itself to unwanted
interpretations:

The next step is to build the program with make.

If that is successful, please follow the INSTALL guide.

Usually, most users just need to "make tests" and "make install",
possibly switching to superuser for "make install" if the prefix
points to a privileged location like /usr/local/.

I will revise it.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On 15.10.2023 19:18, Kaz Kylheku wrote:
> On 2023-10-15, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>>
>> If 'vim' is your pager you could do
>>
>> man ... | vim -
>>
>> :set syntax=man will also do some (predefined) highlighting. While in
>
> The problem is that vim will not remember the position in a buffer that
> had been read from a pipe. This whole thing was prompted by someone
> trying to solve that problem. They wanted to remember the position in
> each individual man page. That resonated with me because there have been
> times when I wanted to revisit something I recently read in a man page.

Ah, I see; you meant memorizing marks across sessions.

>
> (One such situation is when I'm working *on* a man page, making
> adjustments to some section and wanting to see their effect!)

Isn't that then a _new_ (manpage-) document? - I wouldn't expect that
a mark from a previous version would find some identical match in the
new version; should that be by line number or by pattern match? - I'd
say we can't tell generally.

But I think it's also about how we work in such cases (and with Vim).

What one can do is searching the section you are working on, say
/DESCRIPTION
And if you leave and restart the Vim-session - also a piped one! -,
you'd just type
n
to get again to the DESCRIPTION in your document.

Though usually I work with two windows, one for the editing and one
to see and test the results. Another method is shell escaping (:sh)
from the editor to run and test a change. (It depends.)

>>
>> A reason against using an editor is probably the size consideration.
>
> I tried it on a 13-year-old Core i5, slow machine, on the txr man page,
> which is 65000 lines. There is a bit of a delay, but otherwise it's
> fine.

Oh, I meant size of executables (vim: ~2400 kB vs. less: ~150 kB);
i.e. what the reason is to *re-implement* a very small subset of
editor functions (search, highlight, ...) in the pager software
(instead of just using an editor). Executable size matters maybe.

But (in case of Vim) data sizes matter as well, I observed. - But
(despite insanely large man pages in some cases) probably not for
man pages of reasonable size.

Janis

On 15.10.2023 18:52, Chris Elvidge wrote:
> On 15/10/2023 17:23, Kenny McCormack wrote:
>> Anyway, just to take your idea a bit firther, I'm actually quite pleased
>> with the effect of:
>>
>> $ /usr/bin/man ls | vi -c 'set syntax=man nomod|split' -
>
> I use a function (to save typing):
>
> vman ()
> {
> COLUMNS=$((COLUMNS-5)) /usr/bin/man ${@-man} | col -bpx | iconv -c |
> view -c 'set ft=man nomod' -c 'set noeb vb t_vb=' -c 'map q
> <Esc>:q!<cr>' -c 'map i <nop>' -
> }
>
> and then alias man to vman

Yes, having that in a function (or in a local script) is recommended.
(Might make it also possible to make the PAGER variables work better.)

I'd maybe also add the :split - it's useful when you want the synopsis
visible and skim through the options in the other section - OTOH, it's
also simply typed when needed; how often do I want it?

Anyway, having an editor as pager allows a lot interesting functions.

Janis

On 2023-10-15, Kaz Kylheku <864-117-4973@kylheku.com> wrote:
> Hi All,
>
> Prompted by a Unix StackExchange question, I started a mini project
> called mnpgr a few days ago. It's in a very usable state now.

I just discovered and addressed the following issue.

If you view the same man page using different widths (number of
columns) the posititions remembered by Vim don't make sense.

The solution is to determine what the width is by examining
the COLUMNS and MANWIDTH variables, falling back on 80.
Then this column value is interpolated into the name of the
temporary file.

Thus Vim remembers the position separately for different
width renderings of a man page.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.