Nit (maybe minor, maybe not): The EPUB standard requires[1] that the `mimetype` file be the first file stored in the ZIP container, and also that it be stored without being compressed.
It doesn't look like the author's code is conforming to this part of the standard.
If that is the case, then their Kindle may be able to tolerate the deviation. But using this code with an e-reader that adheres closely to the EPUB spec may produce broken results.
Yeah, understandable given that the "sample-epub-minimal" repo you were going off of effectively says "just zip it all up and you're good to go", which is incorrect. Not sure what the best remedy is there, maybe I'll open a pull request.
Constantly pushing stuff to Kindle, which is inherently extremely limited device (gotta love walled gardens!) feels like a bit too much friction to me.
Boox devices are shipped with Android 11. Yes, recently. And their customer service is not great. As much as I like their devices, I can't imagine putting $300 in them.
I hope these alternatives catch on. But until they do, I paid $100 for my kindle. The cheapest one that isn’t sold out on the boox site is $170. Nearly everything is in the $200+ range.
Very neat. I've been doing this with Calibre (https://calibre-ebook.com/), which involves plugging it into your PC via USB. Simple RSS feeds work with little configuration, and more complicated news sites require writing a custom python "recipe".
This project uses Amazon's email gateway, which I think is limited to 25 articles per month (don't quote me on this).
FWIW to avoid the plugging in (which I hated) Calibre can be configured to send emails as well, which works well with the email to kindle feature, and with a little fiddling you can bridge the gap to have Calibre auto-email you. (https://www.mobileread.com/forums/showthread.php?t=314401)
Could Calibre (or any other software) generate an OPDS feed? I know that at least KOReader has support for OPDS[0], maybe vanilla Kindle firmware has it too. That would let you forget the part of connecting your device to a PC.
[0] KOReader also has an RSS reader, but I'm not sure how good is it and what it can support in terms of feeds "complicatedness"
I use KOReader's RSS reader, I haven't run into any issues yet. It supports RSS and Atom, supports pulling the full article text if it's in the feed, supports images, and puts each feed entry into its own epub.
I'm using Singlefile firefox extension on computer and phone to save article to html. Then it gets synced to kobo via syncthing (through koreader). Works pretty well.
How do you configure wifi connectivity in koreader to make this useful? koreader seems to prefer keeping wifi off unless you do an action that needs the internet. Do you leave wifi on, or just periodically turn it on manually to sync?
I've been doing a similar thing for my Kobo with an ITTT action that fetches RSS feed URLs and adds them to my Pocket account, which automatically syncs.
I'm obviously not as in-control as this method, but it's been surprisingly stable.
The Pocket integration with Kobo is really a godsend and why I end up using it way more than my kindle (despite prefering how the kindle Oasis feels in my hand)
Getting things onto the Kobo is handled by Pocket if you go this route, so the challenge becomes to add things to your Pocket account automatically. RSS seems easier than email, but I think it should be doable.
Seems like you can forward emails to add@getpocket.com, but I'm not sure if it works with body text or only if you just send a link[1].
I have run into similar problems, and I haven't found a perfect solution yet. I am curious about what people here are doing.
First, specifically about this "RSS to kindle" thing: it's definitely nice, but it means I need to push all articles to kindle, including those I'm not interested in by reading the headline. When I use feedly, I probably scroll past more than half of the articles in my feed. I wish there was a native RSS reader on kindle.
And I have similar needs to articles that I find on Hacker News or elsewhere. Most of the time I push them to Pocket and deal with them later. I haven't tried all pocket integrations (or other similar "push to kindle services), but p2k isn't great as it compresses images to the degree that they are not legible any more. And there are always some articles that either don't render at all, or things break (e.g. table not showing). Finally, it's painful to clean up my kindle library, and I wish I don't need to do any organization work.
I considered boox but they are really bad at keeping Android updated. Customer service is also bad -- I have seen horror stories.
Scribe's browser is actually much better than in previous generations, but nowhere near what I find usable.
Perhaps the best solution is to have an iPad mini. The device is very responsive, the screen size is perfect, there are lots of apps for almost everything I want to read, and the article mode in Safari works well. But I really don't want to spend $400 to have another tablet just for reading, and I really want things to just work on my two kindles. Unfortunately there doesn't seem to be a great way to do that.
I use maybe little more complicated workflow, but for now, I couldn't find better.
I subscribe RSS to Omnivore and use tags for those I read later. I have tag "logseq" which imports those read later articles to logseq (via omnivore plugin). Then I sync by syncthing to Onyx Boox ereader and read them here. Omnivore let me save articles from various sources offline, which works great for ereader.
It would be faster to read read Omnivore right in browser, however Im using Onyx Boox mostly offline because I dont trust their apps. Also, Omnivore let me highlight notes right in text which gives me another link in logseq database.
hacking around the limitations of proprietary soft-/hardware has been very interesting for me through project such as this one.
personally though, i am awaiting the new generations of eink displays on android tablets to become more accessible to avoid this level of complexity or reliance.
I've built something similar for myself (generating 2x-per-day pdfs for my reMarkable 2) and it's so nice to catch up on internet news on this kind of calm device instead of a laptop or phone.
I used a traditional web-based RSS reader for many years but found that I wounded up checking it just as compulsively as the sites it ostensibly replaced.
I also did something similar 14 years ago. It was a php website that allowed you to subscribe to online newspapers and get the news sent to your Kindle, in MOBI format. It worked but it was basically calling calibre under the hood. I never made it public (and I remember a similar website existed already at that time that did not work well)
How do you add an RSS feed to your own blog? Maybe my google-fu is failing me these days but all I'm getting is how to create one for your podcast. I'd just like people with an rss reader to be able to access my blog as a feed
Depends on how your blog is generated. Static generators like jekyll have plugins for this[1], but it’s also easy enough to generate it by hand with an xml template [2]. Wordpress, substack and other platforms generate it by default afaik.
I looked into this in 2013. At that point there was a "My Clippings.txt" file stored on the kindle that was accessible as a USB storage volume when plugged in. This file stored each annotation in plain text, along with the document ID and the start and ending location of the annotation.
Trouble was, the location was in the Kindle's "Loc" format which is nontrivial (at least to me at the time) to connect to specific text in the document.
I'm sure someone's probably worked this out by now?
Sorry I failed to mention I'm aware of the "clippings.txt" setup, but I would like to be able to have it automated via the web.
Amazon doesn't expose a direct API for highlighting, and Readwise for example does a little "hack" where you use their browser extension to redirect to the Kindle highlights page and I think they just slurp up the authenticated API requests.
There are some books where if you exceed a certain number of highlights, they are not saved on the file anymore, and I’m pretty sure it also affects the web version too.
This is set as part of the DRM, so be careful that you are really saving the data you want. I went deep into a book highlighting things before I noticed this limitation.
Sorry I failed to mention I'm aware of the "clippings.txt" setup, but I would like to be able to have it automated via background API process on the web.
You can achieve your share option in iOS using shortcuts. If you make a shortcut that sends the content of the share (like the URL of the article) to an arbitrary URL.
I did this with a tiny service I run on my home network that just calls yt-dlp on the passed URL and puts the result in a samba share that Emby uses, works amazingly well with all the services that yt-dlp understands.
Nit (maybe minor, maybe not): The EPUB standard requires[1] that the `mimetype` file be the first file stored in the ZIP container, and also that it be stored without being compressed.
It doesn't look like the author's code is conforming to this part of the standard.
If that is the case, then their Kindle may be able to tolerate the deviation. But using this code with an e-reader that adheres closely to the EPUB spec may produce broken results.
[1]: https://www.w3.org/TR/epub-33/#sec-zip-container-mime
TIL thanks for commenting, I'll make a note to fix this.
My crusty Python code implementing EPUB, with the conforming uncompressed mimetype file as first member in the archive:
https://github.com/davidwtbuxton/ucflib/blob/e576fb97fc2a2ea...
Robin Sloan has a good Ruby project that builds good-looking web pages alongside EPUB files: https://github.com/robinsloan/perfect-edition
It was in porting that to Racket [1] that I learned about some of these EPUB standard details.
But spoiler alert, both projects just shell out to `zip` to do the actual compression.
[1]: https://github.com/otherjoel/perfect-edition-pollen/blob/mas...
Yeah, understandable given that the "sample-epub-minimal" repo you were going off of effectively says "just zip it all up and you're good to go", which is incorrect. Not sure what the best remedy is there, maybe I'll open a pull request.
[1] https://github.com/thansen0/sample-epub-minimal
I feel like a new generation of Android powered e-ink tablets / phones is the answer here: https://shop.boox.com/collections/eink-tablet.
Or maybe even that TCL tablet that looks kinda like e-ink, but is actually OLED: https://www.tcl.com/uk/en/tablets/tcl-nxtpaper-10s.
Constantly pushing stuff to Kindle, which is inherently extremely limited device (gotta love walled gardens!) feels like a bit too much friction to me.
Boox devices are shipped with Android 11. Yes, recently. And their customer service is not great. As much as I like their devices, I can't imagine putting $300 in them.
I hope these alternatives catch on. But until they do, I paid $100 for my kindle. The cheapest one that isn’t sold out on the boox site is $170. Nearly everything is in the $200+ range.
Very neat. I've been doing this with Calibre (https://calibre-ebook.com/), which involves plugging it into your PC via USB. Simple RSS feeds work with little configuration, and more complicated news sites require writing a custom python "recipe".
This project uses Amazon's email gateway, which I think is limited to 25 articles per month (don't quote me on this).
FWIW to avoid the plugging in (which I hated) Calibre can be configured to send emails as well, which works well with the email to kindle feature, and with a little fiddling you can bridge the gap to have Calibre auto-email you. (https://www.mobileread.com/forums/showthread.php?t=314401)
Could Calibre (or any other software) generate an OPDS feed? I know that at least KOReader has support for OPDS[0], maybe vanilla Kindle firmware has it too. That would let you forget the part of connecting your device to a PC.
[0] KOReader also has an RSS reader, but I'm not sure how good is it and what it can support in terms of feeds "complicatedness"
Calibre-web can, I use it with an app on an iPad. It’s not immediately obvious how to access it, but here’s a GitHub issue with good info: https://github.com/janeczku/calibre-web/issues/2103
I use KOReader's RSS reader, I haven't run into any issues yet. It supports RSS and Atom, supports pulling the full article text if it's in the feed, supports images, and puts each feed entry into its own epub.
Indeed. I used to this in Calibre all the way back in 2010.
I'm using Singlefile firefox extension on computer and phone to save article to html. Then it gets synced to kobo via syncthing (through koreader). Works pretty well.
Interesting, I've never thought of running syncthing behind koreader. Do you use any particular plugin like [this](https://github.com/jasonchoimtt/koreader-syncthing)?
How do you configure wifi connectivity in koreader to make this useful? koreader seems to prefer keeping wifi off unless you do an action that needs the internet. Do you leave wifi on, or just periodically turn it on manually to sync?
>Do you use any particular plugin like [this](https://github.com/jasonchoimtt/koreader-syncthing)?
Yes that's the syncthing implementation I'm using. I keep wifi off to save battery and manually sync when I have enough of a backlog .
I've been doing a similar thing for my Kobo with an ITTT action that fetches RSS feed URLs and adds them to my Pocket account, which automatically syncs. I'm obviously not as in-control as this method, but it's been surprisingly stable.
The Pocket integration with Kobo is really a godsend and why I end up using it way more than my kindle (despite prefering how the kindle Oasis feels in my hand)
Agreed! I subscribe to a couple serialized novels, and having chapters appear automagically on my ereader is a really nice experience.
I've been wanting to do the same, haven't explored ITTT yet. Is it possible to fwd an email (say a newsletter) and have it on the kobo?
Getting things onto the Kobo is handled by Pocket if you go this route, so the challenge becomes to add things to your Pocket account automatically. RSS seems easier than email, but I think it should be doable.
Seems like you can forward emails to add@getpocket.com, but I'm not sure if it works with body text or only if you just send a link[1].
[1]: https://blog.getpocket.com/2009/04/2-new-ways-to-save-to-rea...
I have run into similar problems, and I haven't found a perfect solution yet. I am curious about what people here are doing.
First, specifically about this "RSS to kindle" thing: it's definitely nice, but it means I need to push all articles to kindle, including those I'm not interested in by reading the headline. When I use feedly, I probably scroll past more than half of the articles in my feed. I wish there was a native RSS reader on kindle.
And I have similar needs to articles that I find on Hacker News or elsewhere. Most of the time I push them to Pocket and deal with them later. I haven't tried all pocket integrations (or other similar "push to kindle services), but p2k isn't great as it compresses images to the degree that they are not legible any more. And there are always some articles that either don't render at all, or things break (e.g. table not showing). Finally, it's painful to clean up my kindle library, and I wish I don't need to do any organization work.
I considered boox but they are really bad at keeping Android updated. Customer service is also bad -- I have seen horror stories.
Scribe's browser is actually much better than in previous generations, but nowhere near what I find usable.
Perhaps the best solution is to have an iPad mini. The device is very responsive, the screen size is perfect, there are lots of apps for almost everything I want to read, and the article mode in Safari works well. But I really don't want to spend $400 to have another tablet just for reading, and I really want things to just work on my two kindles. Unfortunately there doesn't seem to be a great way to do that.
Exact same problems and I went the iPad mini route. Honestly, it was worth it. I bought an older one to save cash and it works great.
I use maybe little more complicated workflow, but for now, I couldn't find better.
I subscribe RSS to Omnivore and use tags for those I read later. I have tag "logseq" which imports those read later articles to logseq (via omnivore plugin). Then I sync by syncthing to Onyx Boox ereader and read them here. Omnivore let me save articles from various sources offline, which works great for ereader.
It would be faster to read read Omnivore right in browser, however Im using Onyx Boox mostly offline because I dont trust their apps. Also, Omnivore let me highlight notes right in text which gives me another link in logseq database.
https://omnivore.app/
https://logseq.com/
https://syncthing.net/
Not much to contribute other than "this is great." It's one of those things I didn't know I wanted until I read your article.
hacking around the limitations of proprietary soft-/hardware has been very interesting for me through project such as this one.
personally though, i am awaiting the new generations of eink displays on android tablets to become more accessible to avoid this level of complexity or reliance.
I've built something similar for myself (generating 2x-per-day pdfs for my reMarkable 2) and it's so nice to catch up on internet news on this kind of calm device instead of a laptop or phone.
I used a traditional web-based RSS reader for many years but found that I wounded up checking it just as compulsively as the sites it ostensibly replaced.
https://www.instapaper.com requires selecting individual articles, but it's how I store and read.
Instapaper has "Kindle Automatic Delivery" in settings. No need to select individual articles if you want all of them.
Many years ago, I built a Bash script to do something similar.
https://gist.github.com/leoherzog/1dcffe776af200cd9117
Very cool!
I also did something similar 14 years ago. It was a php website that allowed you to subscribe to online newspapers and get the news sent to your Kindle, in MOBI format. It worked but it was basically calling calibre under the hood. I never made it public (and I remember a similar website existed already at that time that did not work well)
How do you add an RSS feed to your own blog? Maybe my google-fu is failing me these days but all I'm getting is how to create one for your podcast. I'd just like people with an rss reader to be able to access my blog as a feed
Depends on how your blog is generated. Static generators like jekyll have plugins for this[1], but it’s also easy enough to generate it by hand with an xml template [2]. Wordpress, substack and other platforms generate it by default afaik.
[1] https://github.com/jekyll/jekyll-feed
[2] This is mine: https://github.com/facundoolano/olano.dev/blob/main/src/feed...
I just roll my own website with mostly hand-written html/css. Ill look into just dropping a handwritten xml into the root
Make sure to also include a link tag in all the website headers, so the feed is discoverable by readers:
Semi related, but have you found a way to get kindle highlights out of amazon?
I've been using Readwise (and Reader) which have been great, but it's rather expensive.
I'd love to be able to send an arbitrary article to Feedi like I can with Reader, but also would love to sync highlights back from my kindle.
Going to lurk the repo, cool stuff!
I looked into this in 2013. At that point there was a "My Clippings.txt" file stored on the kindle that was accessible as a USB storage volume when plugged in. This file stored each annotation in plain text, along with the document ID and the start and ending location of the annotation.
Trouble was, the location was in the Kindle's "Loc" format which is nontrivial (at least to me at the time) to connect to specific text in the document.
I'm sure someone's probably worked this out by now?
Update: yes, at least 160 projects
https://github.com/search?q=%22my%20clippings.txt%22&type=re...
Sorry I failed to mention I'm aware of the "clippings.txt" setup, but I would like to be able to have it automated via the web.
Amazon doesn't expose a direct API for highlighting, and Readwise for example does a little "hack" where you use their browser extension to redirect to the Kindle highlights page and I think they just slurp up the authenticated API requests.
There are some books where if you exceed a certain number of highlights, they are not saved on the file anymore, and I’m pretty sure it also affects the web version too.
This is set as part of the DRM, so be careful that you are really saving the data you want. I went deep into a book highlighting things before I noticed this limitation.
It obviously does not affect files without DRM.
Obsidian has a plugin to get the hilights to Obsidian notes: https://github.com/hadynz/obsidian-kindle-plugin
clippings.io has a browser extension that scrapes all your highlights from Amazon's website and lets you download them in various formats.
nice, hadn't heard about them I'll check it out!
My kindle has a document called clippings with all the highlights in it.
Sorry I failed to mention I'm aware of the "clippings.txt" setup, but I would like to be able to have it automated via background API process on the web.
See my other comment above.
You can achieve your share option in iOS using shortcuts. If you make a shortcut that sends the content of the share (like the URL of the article) to an arbitrary URL.
I did this with a tiny service I run on my home network that just calls yt-dlp on the passed URL and puts the result in a samba share that Emby uses, works amazingly well with all the services that yt-dlp understands.
I subscribe to QiReader for exactly this feature. Works great. https://www.qireader.com/
I made https://epubkit.app for myself to do this.