minimalist reddit2nntp gateway

Overview

nntpit

This is a simple reddit2nntp gateway server that lets you use a newsreader to follow discussions on reddit. The intention is for you to run it locally, tell your newsreader to connect to localhost, and subreddits will appear as newsgroups!

Slrn Headers

Slrn Article

This software is in an early experimental state: you will encounter bugs.

Building

You need libev-dev, libglib2.0-dev, libjson-c-dev, and libcurl4-openssl-dev.

To make the configure script:

$ autoreconf -i

Now type the usual ./configure, and make.

Usage

$ ./nntpit -D -p 8119

Now connect your newsreader; if you use slrn, you would do this:

$ NNTPSERVER=news://localhost:8119 slrn -f reddit --create

The groups list will be empty at first, but don't worry: that's normal.

Press a to add a group, and type the name of the subreddit you want to subscribe to.

Adding a subreddit

Don't include the /r/!

It should populate a list of articles and comments for you to read.

Reporting Bugs

If you're using slrn, please include the --debug and the nntpit -D log.

If you're not using slrn, please include as much detail as you can, and include any debugging logs your software can generate. Please be willing to help test!

Credit

The NNTP code is based on nntpsink by Felicity Tarnell (@unixwitch).

All other code is by @taviso.

Why?

I prefer to read discussions in a newsreader. There are many powerful features that we never got back when the world moved away from usenet.

There are the simple things like keeping track of which posts you've read or haven't seen yet, and customizing how threads are sorted and displayed.

There's more advanced features like tagging, watching, ignoring, and so on. Have you ever been following a discussion, but it's wandered off topic and people are now discussing something you're not interested in? Well, newsreaders let you ignore subthreads, but continue reading the rest without the clutter.

Then there are poweruser features, like scoring and macros.

Scoring

Most newsreaders will let you adjust the score of an article based on rules. A post with a high score will be higlighted, low scores might be dimmed, and very low scores might be completely hidden. You could also sort by score, so the best comments are at the top, and you read those first.

The simplest example would be if there's a user whose posts you always enjoy you can score their posts higher. Their comments will be hilighted and you'll never miss them.

You can chain complicated sets of rules together, really customizing your experience. Let's say you want to highlight all the cookie recipes posted by cookielover22, unless they contain the word "chocolate" because you're allergic. No problem: you can do that too.

How about assigning scores for different ingredients you like, then sorting all the comments by score?

Keyword Score
Chocolate -10
Raisins +20
Nuts +15

Now a post for "New Recipe for Chocolate Raisin Brownies!!" will be scored -10+20+15 = 25, and so on.

If you want to read about scoring, the slrn documentation is here.

How?

I have only tested this with the newsreader I use, slrn.

It might work with other newsreaders, but most likely I will need to make some changes. If you're willing to help with testing to get it working, then file an issue and let's see if we can make it work!

What about...

Gwene

Gwene is an rss2nntp gateway, reddit supports rss so why not use that?

Rss doesn't support threading, and makes each story appear as a new group.

You need to subscribe to every single story, and miss out on half the features, so it just doesn't work.

Posting

Can I post replies?

No, reading only. Maybe one day.

Issues
  • nntpit: Add XOVER and LIST OVERVIEW.FMT.

    nntpit: Add XOVER and LIST OVERVIEW.FMT.

    This adds support for XOVER and some initial support for LIST OVERVIEW.FMT. The latter is specified by the RFC as "should implement when implementing XOVER".

    This almostly makes nntpit work with the Pan newsreader--the message ids are still not to its liking (missing "@"). I've also got a patch for the latter already. Then it works just fine.

    opened by daym 7
  • Add an “X-Reddit-URL” header

    Add an “X-Reddit-URL” header

    Sometimes while reading a post I’d like to view the “real thing” on Reddit. This commit adds an X-Reddit-URL header to all stories and comments that gives the content’s permalink.

    opened by bdesham 2
  • Many warnings for /r/wallstreetbets

    Many warnings for /r/wallstreetbets

    When https://reddit.com/r/wallstreetbets is select as group, nntpit will report a lot of warnings, like:

    [15] <- [GROUP wallstreetbets]
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    ...
    

    but when you go to /r/programming, it will print a lot less warnings:

    [15] <- [GROUP programming]
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    
    ** (process:2154): WARNING **: failed to parse subreddit json
    [15] <- [HEAD 0]
    [15] <- [HEAD 1]
    

    I presume this has to do with number for posts in each thread? /r/wallstreetbets is overcrowded these days...

    nntpit is started with: ./nntpit -D -t 3 -p 8119.

    opened by sanel 2
  • Support for offline reading via slrnpull

    Support for offline reading via slrnpull

    slrnpull [1] is a small tool that is used to pull news for offline read. Is there any way to get it working with nntpit?

    Here is my slrnpull.conf sample:

    emacs   100   0   0
    linux   100   0   0
    

    When I start nntpit with ./nntpit -D -t 3 -p 8119 and run slrnpull with slrnpull -h localhost:8119, I'm getting this in slrnpull output:

    02/08/2021 19:59:05 slrnpull started.
    02/08/2021 19:59:05 Reading /var/spool/slrnpull/slrnpull.conf
    02/08/2021 19:59:05 Created dir /var/spool/slrnpull/news/linux.
    02/08/2021 19:59:05 Connecting to host localhost ...
    02/08/2021 19:59:05 Connected to host.  Posting ok.
    02/08/2021 19:59:05 Fetching articles for emacs.
    02/08/2021 19:59:06 ***listgroup emacs failed: Command not recognized
    02/08/2021 19:59:06 Fetching articles for linux.
    02/08/2021 19:59:22 ***listgroup linux failed: Command not recognized
    02/08/2021 19:59:22 A total of 157 bytes received, 66 bytes sent in 17 seconds.
    

    No articles were fetched.

    [1] https://linux.die.net/man/1/slrnpull

    opened by sanel 2
  • Add support for 'tin', Thunderbird and force-adding of subreddits

    Add support for 'tin', Thunderbird and force-adding of subreddits

    The 'tin' newsreader requires the server to advertise READER mode. It also can't select groups it hasn't previously seen in the LIST ACTIVE response, even if you type in their names with the 'subscribe pattern' command. For this reason this commit adds a command line option to add new subreddits to the database. It only needs doing once.

    LIST ACTIVE support also seems to make Thunderbird work.

    Tested with tin 2.4.6 and Thunderbird 91.7.0

    opened by caliston 1
  • Add

    Add "@reddit" to Message-Id in order to make nntpit work in Pan newsr…

    …eader.

    The Pan newsreader has a sanity check that the message ids have chars out of <>@ in there.

    nntpit didn't have @.

    This patch adds @reddit to the message ids.

    opened by daym 1
  • Use Automake

    Use Automake

    Thanks for making this, it works fine with Gnus too btw. Not sure how you feel about using Automake directly rather than a hand-written Makefile.in, but it avoids the awkward automake --add-missing # ignore errors step that's required at the moment to get install-sh.

    opened by nickg 1
  • compile error on Debian 8

    compile error on Debian 8

    A user on twitter reports that there is a compile error on Debian:

    ‘reddit_spool_store’: 
    spool.c:70:9: error: void value not ignored as it ought to be
    if (json_object_object_add(spool, id, object) == -1) {
    

    I can't really figure out what it's warning about, and can't reproduce that problem. That value isn't void,

    User says: gcc version 8.3.0 (Debian 8.3.0-6)

    opened by taviso 1
  • Update README.md

    Update README.md

    • 12: comma splice
    • 16: oxford comma for clarity
    • 24: bad subclause
    • 30: comma splice
    • 34: comma splice
    • 46: removed oxford comma to join objects
    • 48: add linking word to balance clauses
    • 53: capitalize initialism
    • 66: oxford comma to delineate
    • 68: comma after introductory adverb - maybe ditch the intro
    • 71: comma for flow
    • 80: who's/whose
    • 85: spelling
    • 87: colon for sentence intro
    opened by bby-bishopclark 0
  • Use IMAP instead of NNTP?

    Use IMAP instead of NNTP?

    Not sure what is a better place for such topic, so I guess I'll try here.

    I did a similar project in the past, a reddit-to-nntp and lobsters-to-nntp gateway with posting support, and I've encountered two issues:

    • Some NNTP software will actually slow down to a crawl when there is large number of messages returned by the NNTP server (I'm looking at Thunderbird, no such issues with slrn/gnus),
    • There is an issue with read flag synchronization in case the user uses multiple machines. So, when I read a message, or catch up with a news group on one machine, I have to do the same on another machine.

    I'm wondering if anyone has been considering using IMAP instead of NNTP as a gateway target? So reddit2nntp would become reddit2imap. That would definitely solve issue 2, but I'm wondering about 1. Some informative reddit threads are massive (e.g. r/OutOfTheLoop) and I wonder if e-mail clients would be prepared for such amounts of messages.

    So, did anyone consider IMAP instead of NNTP on projects like these? Or have experience with IMAP with massive amount of e-mails? My IMAP inbox counts ~50.000 e-mails and it's fine, but 50.000 is peanuts when we're talking about mirroring interesting threads from reddit ;)

    opened by antekone 0
  • Bringing back the ideology that brought us this project into the world

    Bringing back the ideology that brought us this project into the world

    I'm a big fan of this project for the very reason described in the readme under the "Why?" section.

    On a broader scope the Internet lost touch with itself on technical grounds now that the common population is thriving on using it.

    I've been asking myself a question you may or may not care to think about but since you are prone to understanding my previous statement having made this tool I would like to throw it in the air around it; how do we bring back the good parts of the Internet that are becoming forgotten?

    Let me know if you would be open to discuss this further, here or in private, my email is [email protected]

    opened by ncouture 0
  • Parse markdown/and html escape sequences.

    Parse markdown/and html escape sequences.

    I sometimes see escaped html in comments, e.g. &gt; it seems like we should be cleaning this up?

    There are also markdown-style URLs, maybe we could be parsing that too.

    Just making a note to investigate, it's mostly still readable.

    opened by taviso 0
  • Older newsreaders see new posts in GROUP, but can't actually list any

    Older newsreaders see new posts in GROUP, but can't actually list any

    This is neat; been meaning to write something like this for older systems. Curious how vintage NNTP clients handle it (since I'm curious about the retro applications), and so far, none of them like it:

    • old Mozilla (but probably modern Thunderbird too) won't let you subscribe to groups that aren't listed already; if i use slrn to subscribe and add it to newsrc, it appears that it can see the metadata from GROUP, but can't request anything more; I don't see any further commands other than GROUP. No posts.
    • Outlook Express suffers the same fate
    • Xnews can see the group, but when it tries to get posts, it issues an XOVER command which nntpit rejects. It can at least add groups w/o them being listed.
    opened by NattyNarwhal 15
Owner
Tavis Ormandy
Tavis Ormandy
An open source Zstack3 gateway powered by ESP8266 and CC2652 modules

OpenZ3Gateway An open source Zstack3 gateway powered by ESP8266 and CC2652P modules. One costs less than 60 CNY in China. This repository includes Alt

Chen Yijun 27 May 9, 2022
ZigStar LAN Gateway - Firmware

ZigStar LAN Gateway - Firmware This firmware delevoped to use with ZigStar LAN Gateway Functions Zigbee UART port forwarding using socket LAN connecti

null 30 Jul 3, 2022
M17 Analog Gateway by ESP32

M17 Analog Hotspot Gateway Project How to Make digital voice radio (M17 Digital Voice) analog gateway yourself over the internet width ESP32 NodeMCU/D

Atten 10 May 10, 2022
Bluetooth Gateway for Phantom Remote Control based on ESP32

Phantom remote control Bluetooth gateway An ESP3232 firmware for the gateway of Phantom remote control, which can push the temperature and humidity data of Phantom remote control through LAN, and also support to control Phantom remote control to send and receive IR data through LAN.

George Zhao 13 Jan 23, 2022
Use an esp32 as gateway for the Eqiva Bluetooth smart lock to integrate it in Home Assistant as MQTT lock

esp32-keyble-homeassistant Use an esp32 as gateway for the Eqiva Bluetooth smart lock to integrate it in Home Assistant as MQTT lock Based on the grea

null 7 Apr 28, 2022
16 Channel Current Meter to MQTT Gateway

16 Channel Current Meter to MQTT Gateway This sketch runs on an ESP8266 and reads data from 16 Channel Current Measurement Module over RS485 Modbus an

null 4 Jan 11, 2022
Argh! A minimalist argument handler.

Frustration-free command line processing So many different command line processing libraries out there and none of them just work! Some bring their wh

Adi Shavit 1k Jun 27, 2022
A minimalist andf platform-agnostic application layer for writing graphical applications, with a strong emphasis on simplicity and ease of use.

SlimApp A minimalist(*) and platform-agnostic application layer for writing graphical applications. Available as either a single header file or a dire

Arnon Marcus 33 Apr 22, 2022
A minimalist library with basic facilities for developing interactive real-time 3D applications, with a strong emphasis on simplicity and ease of use.

SlimEngine A minimalist and platform-agnostic base project for interactive graphical applications (2D/3D) with a strong emphasis on simplicity, ease o

Arnon Marcus 66 May 10, 2022
Data compression utility for minimalist demoscene programs.

bzpack Bzpack is a data compression utility which targets retrocomputing and demoscene enthusiasts. Given the artificially imposed size limits on prog

Milos Bazelides 17 Apr 8, 2022
kvm-host is a minimalist type 2 hypervisor using Linux Kernel-based Virtual Machine (KVM), capable of running Linux kernel partially.

kvm-host kvm-host is a minimalist type 2 hypervisor using Linux Kernel-based Virtual Machine (KVM), capable of running Linux kernel partially. Build a

null 68 Jul 3, 2022
A minimalist and mundane scripting language.

Drift Script A minimalist and mundane scripting language. I like all simple things, simple and beautiful, simple and strong. I know that all developme

Drift 11 Apr 3, 2022
Minimalist video maker -- simplify your music score video making process!

VisualScores 极简视频制作程序,简化你的乐谱视频制作! 如果需要编译,请解压 lib 文件夹中压缩包。 使用前请参考 manual 文件夹中的用户手册。 请勿修改、移动或删除 resource 文件夹中的任何文件。 VisualScores Minimalist video maker

Chen and Sim 5 Jan 11, 2022
isabel - Simple, minimalist note manager.

isabel isabel - Simple, minimalist note manager. Usage Type name and body of note and press Ctrl+s for save note. Press Tab to open notes list. Press

null 1 Oct 2, 2021
SimPle SHell - minimalist Unix interactive shell written in a single C file

SimPle SHell - minimalist Unix interactive shell written in a single C file. The shell does not support scripting yet and is in an early stage of development. If you notice any bug, please open an issue on github.

sewe2000 2 Oct 24, 2021
Second life for famous JPEGView - fast and tiny viewer/editor for JPEG, BMP, PNG, WEBP, TGA, GIF and TIFF images with a minimalist GUI and base image processing.

JPEGView-Image-Viewer-and-Editor Updated Dec 07 2021. Version 1.1.1.0 has been released. Download link1, link2 added. Second life for famous JPEGView

Ann Hatt 22 Jul 3, 2022
Gfx - A minimalist and easy to use graphics API.

gfx gfx is a minimalist and easy to use graphics API built on top of Direct3D12/HLSL intended for rapid prototyping. It supports: Full shader reflecti

Guillaume Boissé 236 Jun 24, 2022
Legacy stepper motor analyzer - A DYI minimalist hardware stepper motor analyzer with graphical touch screen.

Simple Stepper Motor Analyzer NOTE: This is the legacy STM32 based design which was replaced by the single board, Raspberry Pi Pico design at https://

Zapta 158 Mar 21, 2022
Minimalist protocol buffer decoder and encoder in C++

protozero Minimalistic protocol buffer decoder and encoder in C++. Designed for high performance. Suitable for writing zero copy parsers and encoders

Mapbox 217 Jun 28, 2022
F3D - Fast and minimalist 3D viewer

F3D - Fast and minimalist 3D viewer By Michael Migliore and Mathieu Westphal. F3D (pronounced /fɛd/) is a VTK-based 3D viewer following the KISS princ

null 515 Jun 25, 2022
Dohd is a minimalist DNS-over-HTTPS daemon that redirects all DoH queries to a local DNS server running on localhost:53 (UDP)

dohd Dohd (pron. doh-dee) is a minimalist DNS-over-HTTPS daemon that redirects all DoH queries to a local DNS server running on localhost:53 (UDP). Fe

Dyne.org 14 Jun 26, 2022
A Sol-inspired minimalist Lua binding for Zig.

zoltan A Sol-inspired minimalist Lua binding for Zig. Features Supports Zig 0.9.0 Lua tables table creation from Zig get/set/create methods possible k

null 66 Jun 20, 2022