Web processing with Gnus


Categories: Emacs Gnus Tags: Emacs Gnus

This is a post about reading news using an Emacs package called Gnus instead of a traditional web browser. Gnus gets a lot of bad press from the Emacs community, but I think it is a very effective reader, and has some distinct advantages over browsing.

Last year, I started using the venerable and inscrutable newsreader Gnus to process most of the web sites that I read in my free time. Gnus is a newsreader that started out as an Emacs module to read newsgroups on USENET. These days, it is now distributed with Emacs itself, and it does a whole lot more than read USENET.

Wikipedia describes Gnus as a web processor. That’s a term I have not heard before, but when I use Gnus it feels apt, because I feel that I am processing the web rather than browsing it. There is some focus to it. Gnus is sort of like an Emacs version of Thunderbird. Thunderbird can read email, RSS, and NNTP. Gnus can read from all those sources as well, but it can do much more, and with more user control.

There are a few things are made me want to take a look at Gnus. I noticed that I can easily sync my Gnus profile between computers, and that Gnus has a load of features that help me triage my reading. I expound more on this below. And if I run into a problem with Gnus, I have a chance of fixing it with some Emacs-Lisp. I can’t say the same for Thunderbird. I love Thunderbird, but it’s a struggle just to get the fonts configured to my liking.

Gnus’s concepts that sold me

There are a few features that really helped me sync with the Gnus mindset. Specifically,

  1. Gnus abstracts different types of web content into groups, which can be used and customized uniformly.
  2. Gnus has precise levels of control over group subscribedness and article prioritization.
  3. The configuration is self contained, extensible, and share-able.
  4. There is specific attention to reading ergonomics.1
  5. Gnus switches web browsing for web processing.

All news sources in Gnus are handled together and in a uniform way. Whether you are reading email, RSS, or a mailing list, we are just reading articles from a particular source, called a group in Gnus-speak. Gnus implements a lot of backends that handle this abstraction for a user, but after the backends are set up properly, the user interaction with all groups is essentially the same. Gnus has many backends, but I find that I only need a few of them to cover a lot of my web processing.

The major feature that sold me is the highly precise level of control over group “subscribedness”. In Gnus, news sources are not just subscribed, but can have varying degrees of subscribedness. Gnus allows the user to assign a level from 1 to 9 to every group. Groups with level 1-5 are subscribed. Levels 6-7 are unsubscribed, and levels 8-9 are “dead”. Unsubscribed groups still exist, but they are not automatically shown to the user.

Having levels of subscribedness allows me to triage the news that I find on the internet. Some of my Email groups have a high level, so I get these emails at the top of my Gnus feed. Some emails (shipping updates, term of service change, etc) have a low group level. I can read these after I get to the most interesting or priority content. I can unsubscribe from some of my IMAP folders so that I only see updates if I need them. This is how I treat most mailing lists. It’s the kind of reading that I want to see sometimes, but I don’t want Emacs-devel to be mixed in with my usual stuff unless manually toggled on.

Setting subscribedness can be automatic or manual. I personally manually set the group level (priority level) of each group. This type of abstraction is missing from Thunderbird, which handles my news and email separately.

Further, Gnus can also be configured to prioritize articles based on keywords and user actions. This means that Gnus can prioritize both groups and articles within groups! This is called scoring. Effectively, users can make their own news recommendation algorithm by setting up score files, rules that Gnus reads to prioritize articles. I still prefer the usual chronological article view, but it’s good to know that I can roll my own news recommendation engine when I reach that level of the Emacs iceberg.

The Gnus IMAP backend is great, and requires little configuration. I can read emails from my email accounts without downloading very much, and I don’t need to install anything outside of Emacs. People write about how Gnus is slow compared to notmuch, but I don’t really notice much delay. I use fastmail, and imap requests are really fast. I use native compilation on Emacs, so Gnus’s Elisp code is also pretty fast. So, I think Gnus’s reputation as a slow program can be set aside due to advances in hardware and software.2

Lastly, Gnus is set up to help you process web content as efficiently as possible. When you open Gnus, you press the space bar to read the first article in the top group. Pressing space again pages the article. At the end of the article, space moves to the next article. At the end of a group, space moves you to the next group. That’s good ergonomics. However, I mostly use a mouse and the toolbar to navigate Gnus.

The Zen of reading with Gnus

With Gnus, I’m not browsing the web very much anymore. Instead, you sort of just let the web come to you. I read my email, mailing lists, and web feeds. I save things that I want to read later, and I get reminders sent to my inbox. Once I complete my reading, I usually just close my laptop lid and do something else. It’s a freeing way to interact with the web. If I still have free time when I’m done reading Gnus, I open some unsubscribed groups. When so much of the internet is designed to keep me scrolling, I can buck the trend with Emacs. I program my newsreader to search the internet for the content that I want to read, rather than the content that someone else wants me to read. Once I have that content, I filter and sort it according to rules that are totally transparent to me, not black box AI. (See previous post on all search engines are now recommendation engines.) I get to be the signal processor that ensures the signal to noise ratio of my reading is high.

Emacs can’t render javascript, so the only stuff that Gnus can show me is plain text and HTML. It is a pretty austere reading experience that reminds me of my Kindle. Gnus has lot of options for displaying HTML (called article washing). But the gist of it is that I get to read my news in the same format for every site, and it’s my preferred format, and there are no ads or injected content.

Of course, I still use a web browser. I need one for shopping and some other stuff. But most of what I’m doing at a computer is reading, writing, and coding. And that’s all Emacs now.

Starting to use Gnus

Despite being an Emacser for 8 years, the thought of using Gnus to read the web never occurred to me until now. This is because Gnus gets so much bad press. And most Google searches about Gnus will lead you to believe that the user is better served by either mu4e or notmuch for email, and elfeed for reading web feeds. Those three programs are great and effective, but they don’t stick for me.

But Thunderbird was driving me nuts with a bug that was causing it to crash, and I was looking for a “thunderbird”-ish experience in Emacs. I also wanted an email client that didn’t require external programs or downloading all of my inbox in one shot, and I wanted to be able to backup my reader settings. So I gave Gnus a shot.

Gnus unfortunately suffers from the problem that it is totally inscrutable. The Gnus manual is about 100 pages long, and one of the few guides to Gnus that one can find on the internet cautions the reader that they’d better know Emacs pretty well before using Gnus. Besides the manual, there are few entry points to Gnus on the internet. But if you spend enough time reading the github guide and John Weigley’s Gnus configuration, eventually things start to make sense. I had a pretty workable Gnus configuration after a few hours of fiddling.

NNTP Gateways

While Gnus tries to treat all news the same, the truth is that Gnus backends are of differing levels of quality. New users will probably try to use the nnrss backend to read web feeds. This will be a disaster because Gnus will try to sequentially download articles from each feed. Users that want to read RSS in Emacs are better served by Elfeed. But users that want to read Email+RSS+Mailing Lists in Emacs are better served by Gnus.

The trick is to use an NNTP gateway to read RSS (and mailing lists). I use Feedbase for many of my feeds. It works better than Gwene for some of my feeds. Users that need to read mailing lists should look into Gmane.

I have always wanted an RSS reader that I can rely on for most of my web processing, but the scope of RSS never seemed big enough: there was just enough of the web that was not RSS-able that I needed to yet browse the web. So, in the same way that some people try to move all computing activities into Emacs, I tried to move my web reading habits into Gnus. But there’s a growing list of Gnus backends that can handle Twitter, Reddit, Hacker News, and other sources without the nntp go-around. I decided I’m not ready to experiment with them yet.

The advantage of using a nntp gateway is that all news can be downloaded from connecting to one server. This is easier on Emacs’s feeble, sequential network reading capabilities.

But I don’t recommend that you use Gnus

There is no Emacs package that requires more configuration just to get past step one than Gnus. You probably do need to skim some of the manual in order to start using it.

The other issue is that Gnus is a very deeply 90s era Emacs package. It saves state in an odd way. A lot of its configuration is not obviously programmable, like the rest of Emacs.

Gnus’s approach to news reading is arguably a leaky abstraction. I know enough about Emacs’s rough edges that I can fight my way through, but I don’t recommend this to anyone else, not even most Emacsers. For example, there is one primary source, and several secondary sources. The distinction is only historical and confusing. Some group data can be a set using gnus-parameters, other information cannot be set in that variable. Some data structures mix “fast” (such as unread counts) and “slow” (such as group subs) data together when they should be separate.

You need to be willing to accept that Gnus does things a certain way. Going against that grain makes configuration and usage more difficult.

Critically, Gnus doesn’t replace Thunderbird’s calendar for me. Gnus does have a reminder system, but it is hard to use, and it doesn’t work with Caldav.

Getting started with Gnus

I read the manual and gnus configuration files until it started to make sense. But a good introduction can also be had on youtube.

  1. You mostly can read with just one button, and the mouse controls are well designed.↩︎

  2. I use a 5 year old Dell laptop.↩︎