Newsboat
Newsboat is an open source news aggregator licensed under the MIT License.
From newsboat(1) § DESCRIPTION:
- Newsboat is an RSS/Atom feed reader for text terminals. RSS and Atom are a number of widely-used XML formats to transmit, publish and syndicate articles, for example news or blog articles. Newsboat is designed to be used on text terminals on Unix or Unix-like systems such as GNU/Linux, BSD or macOS.
Newsboat is a fork of abandoned Newsbeuter.
Installation
Install the newsboat package. For the development version, install the newsboat-gitAUR package.
Usage
Newsboat cannot start without any configured feeds. Feeds can be configured in ~/.newsboat/urls. Newsboat can be started from the command line with
$ newsboat
Press the ? key to see a list of all keybindings. Keybindings can be rebound, see #Configuration.
Managing feeds
Adding, removing, and tagging feeds is done by editing the urls file. By default that is either ~/.newsboat/urls or
~/.config/newsboat/urls. We will use ~/.config/newsboat/urls for the rest of the section.
RSS or ATOM sources
To add URLs, open ~/.config/newsboat/urls with your favorite text editor and add the URLs, one per line:
~/.config/newsboat/urls
https://archlinux.org/feeds/news/ https://lwn.net/headlines/newrss
To add URLs that have restricted access via username/password, simply provide the username/password in the following way:
~/.config/newsboat/urls
http://username:password@hostname.domain.tld/feed.rss
In order to protect username and password, make sure that ~/.config/newsboat/urls has the appropriate permissions. Newsboat also makes sure that usernames and passwords within URLs aren’t displayed in its user interface. In case there is a @ in the username, type it as %40 instead so that it can be distinguished from the @ that separates the username/password part from the hostname part.
To configure local files as feeds, simply prefix the local path with file://:
~/.config/newsboat/urls
file:///var/log/rss_eventlog.xml
HTML as a source
If HTML sources can be converted to RSS via a user supplied script, newsboat can both execute this script and consume the output.
This is accomplished by using the filter: prefix in the configuration file. For example, write a script that can read HTML from stdin and print out an RSS or Atom feed to stdout. Save that script to ~/.config/newsboat/html2rss.sh and add the following feed:
~/.config/newsboat/urls
filter:~/.config/newsboat/html2rss.sh:https://myurl.com
- For several examples using this strategy, see graysky's newsboat repo.
- For more discussion, see Possible to config newsboat to look at a web exposed directory and report changes #1157.
Tagging feeds
Every feed can be assigned 0 or more tags. This makes it easy to categorize your feeds as well as the ability to easily apply commands to multiple feeds at once.
Usually, the ~/.config/newsboat/urls file contains one RSS feed URL per line. To assign a tag to an RSS feed, simply attach it as a single word, separated by blanks such as space or tab. If the tag needs to contain spaces, use quotes (") around the tag (see example below). An example may look like this:
~/.config/newsboat/urls
http://blog.fefe.de/rss.xml?html interesting conspiracy news "cool stuff" http://rss.orf.at/news.xml news orf http://www.heise.de/newsticker/heise.rdf news interesting
Starting Newsboat with this configuration, one can press t to select a tag. Upon selecting the tag "news", one will see all three RSS feeds. Pressing t again and e.g. selecting the "conspiracy" tag, will only show the http://blog.fefe.de/rss.xml?html RSS feed. Pressing Ctrl-T clears the current tag,
and again shows all RSS feeds, regardless of their assigned tags.
"Special tags
Custom feed names
The name of a feed can be defined with a special tag in your urls file. Simply prefix the tag name with the ~ character and the tag name will become the feed name.
For example:
~/.config/newsboat/urls
http://rss.cnn.com/rss/cnn_topstories.rss "~CNN Top stories"
will define the feed with feed-name "CNN Top stories"
Hidden feeds
A feed can be hidden from the regular list of feeds by prefixing the tag name with an !.
For example:
~/.config/newsboat/urls
http://rss.orf.at/news.xml "!ORF News (hidden)"
The content of a hidden feed can only be found through a query feed.
Query feeds
Custom feeds can be created from existing feeds by creating what is called a query feed: A feed generated by matching articles against a query. Query feeds are defined in your ~/.config/newsboat/urls file like a normal feed.
They have the following syntax:
query:FEEDNAME:FILTER_EXPRESSION [TAG ...]
Like a normal feed, a query feed has a name, FEEDNAME, which is displayed in the feedlist.
The filter expression language for FILTER_EXPRESSION is defined in the newsboat manual.
Query feeds can be optionally tagged, just like a normal feed. See #Tagging feeds.
Basic examples
~/.config/newsboat/urls
"query:Example1:age > 1" "query:Example2:(tags # \"example2\") and (title =~ \"example2\")" "query:Example3:(rssurl =~ \"example3\") or (total_count > 3)"
The first example defines a feed named Example1 that will contain all articles that are older than a day
The second example defines a feed named Example2 that will contain all articles whose feed was tagged example2
and whose article title matches example2. Here the # is the "contains" operator.
The third example defines a feed named Example3 that will contain all articles whose feed url matches the
regular expression example3 or all articles that are apart of a feed that contain more than 3 articles.
Unread articles
To create a feed named Unread Articles that contains all unread articles, define the following:
~/.config/newsboat/urls
"query:Unread Articles:unread = \"yes\""
Amalgamating multiple feeds into one
A good use of query feeds is when you are following a creator whose content is spread across multiple feeds. You can use a query feed to amalgamate all their content into one place.
~/.config/newsboat/urls
"query:Metafeed:tags # \"metafeed\"" https://feed.url/1 metafeed ! https://feed.url/2 metafeed ! https://feed.url/3 metafeed !
The trick is to assign each individual feed a unique tag - here we used metafeed - so that we can match against it in the filter expression. We also make use of #Hidden feeds - the ! at the end of each feed - so that the individual feeds do not show up in the feedlist.
Filtering a specific feed
If a feed contains content you do not want to see, you can filter it by using a query feed. For example, to monitor package updates you are interested in, you can use:
~/.config/newsboat/urls
"query:Package Updates:(tags # \"packageupdates\") and (title =~ \"linux [0-9]|newsboat\")" https://archlinux.org/feeds/packages/ packageupdates !
This feed will only display updates to linux and newsboat.
Ignoring articles
As with query feeds, another way to filter feeds is by using the ignore-article command in your ~/.config/newsboat/config. It has the following syntax
ignore-article FEEDURL FILTER_EXPRESSION
Examples
~/.config/newsboat/config
ignore-article "https://feed/url1" "content =~ \"tele(phone|communications)\"" ignore-article "https://feed/url2" "title =~ \"text_1|text_2\""
The first example will ignore all articles in https://feed/url1 whose content matches
tele(phone|communications)
The second example will ignore all articles in https://feed/url2 whose title matches text_1|text_2
Ignore modes
In contrast to filtering with query feeds, Newsboat allows you to decide what to do with articles that match an ignore-article command: you can delete them or not show them.
~/.config/newsboat/config
# ignore-mode "download" # Default: Delete them. # ignore-mode "display" # Do not show them.
Comparison with query feeds
In terms of pure filtering, whatever you can do with ignore-article you can also do with a query feed.
Query feeds are defined in your ~/.config/newsboat/urls file while ignore-article commands are placed in your ~/.config/newsboat/config. So if you prefer having all feed related things in one place, you might want to stick
to using query feeds.
However if you want to save space and prevent newsboat from downloading certain articles, then
using ignore-article is your only choice.
Configuration
Several aspects of Newsboat’s behaviour can be configured via a configuration file which is located, by default, in ~/.newsboat/config or ~/.config/newsboat/config. This configuration file contains lines of the form:
<config-command> <arg1> ...
Color schemes
The package ships with a number of schemes in /usr/share/doc/newsboat/contrib/colorschemes/ which can be loaded with the include command like so:
include /usr/share/doc/newsboat/contrib/colorschemes/inkpot
Additional options
The configuration file can also contain comments, which start with the # character and go as far as the end of line. To enter a configuration argument that contains spaces, use quotes " around the whole argument. It is even possible to integrate the output of external commands into the configuration. The text between two backticks ` is evaluated as shell command, and its output is put on its place instead. This works like backtick evaluation in Bourne-compatible shells and allows users to use external information from the system within the configuration.
See [1] and [2] for example configurations.
Tips and tricks
Automatic feed reloads
Newsboat can automatically reload all feeds on startup with the following configuration:
auto-reload yes
With this setting, Newsboat also runs periodic auto-reloads – by default, every 60 minutes. The number of minutes between automatic reloads can be configured like so:
reload-time <desired number of minutes>
Alternatively, use cron or systemd to automatically reload feeds. Just add a line in your crontab, or create a systemd service/timer unit combo that issues the following command:
/usr/bin/newsboat -x reload
Loading of feeds in parallel for faster updates
By default, Newsboat will attempt to update entries in ~/.newsboat/urls in sequence; however, the following setting will update them n at a time, in parallel. Adjust n to your liking.
reload-threads 100
Pass article URL to external command
A clever little hack allows one to pass the URL of an article to an external command. The idea is to use a macro to set the browser that Newsboat opens the article with to the path of some other command and then change it back afterwards.
For example, subscribers of a particular youtube channel can open the video with mpv, by doing the following:
macro y set browser "mpv %u" ; open-in-browser ; set browser "elinks %u"
, key, followed by the keybind to use the macro. In the example above, type , + y Subheadings
There is another little hack that allows the user to add subheadings to sort feeds more effectively. This hack involves adding fake feeds to the urls file like so:
---Heading---
--- characters are optional but having some sort of character identifying the headings is useful for styling.Then in ~/.config/newsboat/config, add the following two lines:
highlight feedlist "---.*---" COLOR_OF_YOUR_CHOOSING default bold highlight feedlist ".*0/0.." default default invis
Of course, replace COLOR_OF_YOUR_CHOOSING with a newsboat supported color.
This trick can also interact with Tags by replacing ---Heading--- with:
"query:---Heading---:tags # \"Tag\""
Then tagging all the feeds below the heading with Tag.
Troubleshooting
Newsboat will not start
Newsboat cannot start without any configured feeds! To add URLs, see #Managing feeds. If that is not the problem, check to see if another instance of Newsboat is running. Newsboat issues a lock on its database so that only one instance can access it at a time; thus, attempting to open a second instance will fail.