Webpresence of Philip Kaludercic

What I’d like to see done in Emacs

09 September, 2021

There are a few things I think would be good to see implemented in and around Emacs, but I don’t know if I will find any time soon to do so properly. There is no pride in hiding ideas one thinks might be worthwhile, so I’d like to share them here:

More deriving from prog-mode

Major modes can “derive” functionality from other modes, similar to how classes can inherit from one another in OOP.

For example: Customizing prog-mode-hook should affect all programming languages. I think this is a good thing, that makes maintaining Emacs easier: I want flyspell-prog-mode, abbrev-mode, display-line-numbers-mode, etc. to be enabled in every programming mode by default, but I don’t have to add it to every mode I might end up using manually.

The issue is this: The concept of deriving could go a lot further, and defining new major modes for programming languages could be a lot less repetitive.

So how about this: Instead of just deriving-prog-mode directly, further “abstract” modes are defined for different types of programming languages: interpreted-prog-mode, compiled-prog-mode, interactive-prog-mode. These define their own commands, such as complied-prog-mode-compile, interactive-prog-mode-eval, etc. that are used instead of custom commands for each actual mode. Their behaviour is generic, and depends on the actual mode deriving these intermediate modes.

The immediate advantage is that the user has less to learn. All interactive modes open a REPL the same way via C-c C-z, all compiled modes compile a project using C-c C-k. Every programming mode has C-c C-f bound to a formatting command. Of course, if a functionality is not available for a certain language, a warning is issued.

It might even be possible to provide a macro that makes defining these modes even easier:

(define-interpreted-prog-mode python-mode
  :eval #'python-mode-eval
  :format #'python-mode-format

that then expands to a define-derived-mode and finally a regular major mode definition.

Of course such a change might be very intrusive, especially when changing the keybindings, as no compromise can be found. I can imagine two mitigation strategies, assuming such a feature should even be included into the core:

  1. Generic major modes are enabled manually, either all at once or one-by-one. They will not interfere with existing definitions, unless the user expressly permits this.
  2. Different maps are provided, emulating exiting conventions in popular major modes. By default these might emulate the old mode, but this should be easily reconfigurable.

Something along these lines could also be done for markup languages.

(See also)

A forwards-compatibility library

Previously, I have expressed my lack of enthusiasm towards libraries such as dash, s, f, ht, …

Despite stylistic and other disagreements, one has to admit that even just a few years ago these libraries met a real need to simplify programming with Elisp. However, for a while now code Emacs has improved, and provides many functionalities it previously lacked.

The issue is that these have been added in successive versions of Emacs, and it is understandable that a package maintainer might not want to depend on the newest version, especially when older versions are still in circulation, e.g. by being packaged by Debian and CentOS.

So how about this: A library that provides at least strict subsets of functions and variables that have been defined in a more recent version of Emacs.

So while subr-x defines string-clean-whitespace in version 28.1, and the same version provides length< is written in C the motivation to use this code in a package is limited by the fact that it is not worth excluding most Emacs users for a little convenience.

Instead, one could depend on compat.el, the compatibility library. This would provide a copy of string-clean-whitespace and a reimplementation of length< that would only be loaded for versions older than 28.1.

The effect would hopefully be that packages can turn around: Instead of looking backwards at what was supported in version 24.1, they can look ahead and use what is new, without fragmenting the language.

(See also)

Local packages with package.el

An issue with package.el is that it is not easy to safely modify packages installed from an package archive. With each update, all your local changes can be overridden, and there is no way to store these changes in a VCS.

Most packages on the other hand are managed by some VCS (git, hg, CVS, …), but it requires some work OOTB to make that work. My workaround is a mix of this and the thread linked to below.

So how about this: package.el is added support for managing and initializing local packages. Maybe this means new packages can be installed by giving M-x package-install a URL, maybe this means packages can be installed by cloning them into ~/.emacs.d/elpa/. Either way, a local package should be automatically byte-compiled and autoloaded.

M-x list-packages might also be able to update these packages, by fetching new commits, and if possible merging them with local customizations.

The intention is to provide a low-maintenance method of managing packages the user might want to work on, and perhaps contribute their improvement upstream.

(See also; as far as I know a library called straight.el is comparable to what I am describing, but it rejects package.el instead of extending it. I have not tried it for this reason, so I cannot say.)

Semantic faces

Someone in my local Emacs user-group suggested this once, and I think it would be nice to see. The issue is this: Do M-x list-faces-display, and a buffer will appear. This buffer documents all the faces Emacs knows of, and shows you how they look like.

On closer inspection, especially the more packages a user might have installed, one might notice duplication. A lot of faces that indicate “good” things are green, a lot of faces that indicate “bad” things are red. Most markup modes also define similar faces, for similar concepts, that go beyond the basic bold, italic, bold-italic, …

So how about this: A package consisting mainly of face declarations, and uncontroversial/uncreative defaults. That’s it.

The effect would be that packages do not have to define as many faces, and instead can reuse the ones defined in such a package. My hope would be to see two effects:

  1. More consistent visual hints and design
  2. Less effort in defining new themes: Instead of adding support for every package, a theme just has to modify the semantic faces.

It could be discussed whether or not these faces should be used as they are, or if they should be derived. If I had to decide, I think the former would be preferable from a performance and consistency standpoint.

Package installation hints

I think it is good for packages to minimize their UI. Provide as little as possible, let something else do as much work as possible: Be lazy! For example: If implementing a syntax checker for some language Blub, don’t implement a minor mode that generates overlays, defines commands for introspection, etc. Instead, implement a Flymake backend. That way you have less to deal with, while users have less to configure.

But there might still be something to configure. add-hook, add-to-list, global-set-key, etc. might have to be used.

So how about this: A package can provide machine-readable hints, on how a package can be configured. It might for example want to express that all you need is to add

My idea (see below) was to use a list pacakge-configuration-suggestions like this

   (key                             ; type of suggestion
     "Jump to a specific character" ; explanation
     ,(kbd "C-:")                   ; key to bind
     avy-goto-char)                 ; command to bind
   (key                             ; etc.
     "Jump to a specific word"
     ,(kbd "C-'")

but it might also make sense to reuse the customize interface, and add an attribute to user options with hints. package.el could find these and inform the user about possible options when installing a package (or later when requested manually).

The question then remains if this should just set an option and hide what is being done, or if actual code should be generated? The first way is easier, The latter is preferable if you want to teach users how to configure Emacs. It might even be able to generate use-package and setup.el code from these hints all together.

(See also)

(Persistent and caducous) selection interface

I remember being charmed by completing-read and related functions. It was the missing abstraction that distinguished Emacs as a “shell” from the classical Unix shells. And it is true: completing-read provides an interface from both a programmer and user perspective, that can be used for a wide variety of cases. The popularity of “completion frameworks” such as Ivy, Helm, Vertico, … prove this point.

But the abstraction might have turned out to be leaky. Consider the “type” of completing-read: It maps a set (list, alist, hash table, …) of strings to a single string. OOTB this is clear, as pressing TAB just completes the user input as bash might do in a terminal session. The “completion frameworks” take the liberty of interpreting the interface differently. The focus is placed on narrowing and selecting elements of the set. Technically, both are interpretations are acceptable.

Issues emerge when developers assume more than the interface provides, and rely on specific front-ends. For example: “selection” makes it easier to choose in non-ascii text, “completion” handles multiple selections well.

From a developer perspective, the issue of being returned a string can be inconvenient. Especially when the intention is to select an item from a set, the text representation has to be mapped back on to the actual object. This is cumbersome, but also restrictive, because multiple objects might share a common representation (e.g. two buffer might have the same name).

So how about this: If we want selection, we do selection. A function, selecting-read, choose-selection, select-ephemeraly or whatever is created that maps a list to an element of the list. Each element has to be able to generate a text representation, that is used by the selecting function to represent it to the user.

I have tried implementing this, and it looked something like this

Selecting a city

As I was designing a new interface, I decided to add the ability to define hierarchies and attributes to items via cl-generic. The idea was that you could filter items by attributes, fold and show children, etc. In this example is also displays a buffer on the side, instead of the bottom of a frame as most completion frameworks do.

This might also be extended to a select-persistently function that doesn’t return an object, but attaches actions to objects that are executed on selection. Maybe select-consistently could be updated based on what the user is doing, and provide context sensitive options to suggest (In that case M-number key could be bound to easily select an option).

An older version of the code can be found here. I continued working on it, but my implementation was too slow and unstable.

(See also)

A cleaner Gnus, a more powerful Rmail

Gnus can be… unintuitive. But after using Rmail for a while, I still prefer it when it comes to functionality

Gnus it is not a mail client, it is a very extensible news reader. Rmail felt nicer when handling mail, not only because it is more light weight. But it lacks too many features I enjoy using, and the code is difficult to work on.

So how about this: A Emacs MUA that explicitly is written for Mail, yet extensible to handle multiple backends, while keeping configurations simple.

Again, I have sketched out a rough design based on cl-lib. Ideally the configuration should just consist of giving a list of mail sources:

(setq nmail-sources '((maildir "~/Mail/Personal")
                      (maildir "~/Mail/Professional"
                               :address "john@job.com")
                      (news "emacs.devel" :server "news.gmane.io")))

Where “Nmail” is a reference to “Rmail”, as I would like it to have a similar UI (message first, list second).

The user would then enter M-x nmail, select a inbox, and just do what they want to do.

Admittedly, this is not that creative of an idea. “What we have now, just better and a new name” is standard industry practice. Not only that, but there are a lot of MUAs in Emacs, and I haven’t tried them all. It might be that what I want already exists, at least to a close enough approximation.

Regular expression objects

(Note: This is the idea I am least sure about, because it involves the C-part of Emacs, and my experience with that part of the code is limited.)

I am not even sure if this is an issue, but I always wondered why regular expressions are always handled as strings. Couldn’t they be compiled to increase performance? Especially considering that they currently are used for a lot of highlighting.

So how about this: A new reader syntax for regular expressions.

#r<GNU \([eE]macs?\)*\'>

On the one hand, this should make quoting easier, and improve the highlighting of the syntax. But a the same time, they represent a compiled and perhaps optimised matcher object. All the functions that accept regular expressions (search-forward-regexp, string-match, looking-at, …) would have to support this addition, and handle these matcher objects properly.

Maybe this is premature optimisation, maybe might speed things up. I don’t really know.

This turned out to be longer than I initially though. I’m just glad I don’t have to keep remembering these ideas, and noticing that I still haven’t done anything.

My hope is that someone might pick up on these ideas up before I do, in which case I would be glad to discuss, collaborate and contribute. Just send me a message!