automated rss torrent downloads with a ReadyNAS at hex for life

Or any linux, for that matter.

I’ve owned a ReadyNAS NV for quite a while now, I’ve always thought it was a great product.

After they allowed ssh access to the box (with raidiator 4.0) they made it even better. Low power consumption, debian linux, and a metric ton of diskspace make it a really hackable device.

It has a bittorrent client addon available, the problem is that it is only accessible via an http interface, and cannot import torrents automagically from a specified location like a drop-in folder or an rss feed, unlike some of my favorite torrent clients.

I did a bit of reading and put together a few pieces of script-fu that already existed on the internet.

Auto importing torrents from drop-in folder

First piece to the puzzle is chirpa’s auto-import script as it appeared in the readynas forums, with a tiny modification to fix the whitespace problem with torrent names.

You can see the original thread here:

http://www.readynas.com/forum/viewtopic.php?f=35&t=19126

#!/bin/sh
#     What: Import .torrents into the NETGEAR ReadyNAS BitTorrent
#              Download Manager
#      Who: Quickly slapped together by chirpa (readynas.com/forum)
#    Usage: Crontab it and walk away.
# Requires: Curl (maybe I'll make it work with Wget at some point)
#  Version: 0.2a
#     Todo: Check for returned status: apiTorrentAddFinishedOk /
#           apiTorrentAddFailed

INCOMING_DIR=/c/torrent
DONE_DIR=/c/torrent/done
BTURL="http://localhost:8080/api/torrent-add?start=yes"
ORIG_IFS=$IFS
IFS=$'\n'
cd $INCOMING_DIR
for f in `ls -1 *.torrent 2>/dev/null`
do
echo Loading file: $f
curl -s -F fileEl=@$f $BTURL >/dev/null
mv $f $DONE_DIR
done
IFS=$ORIG_IFS

You can name this file “import_torrents” and run it every N minutes, adding it to your crontab. I run mine every 5 minutes.

Auto downloading torrents from an RSS feed

Lets suppose we want to download torrents from a mininova.org feed, this script coded by Abhi Yerra will get you started.

Note you will need to install both the python and python-feedparser packages via apt-get and also, you will need to modify DownloadPath, DownloadLog and FeedUrl to fit your needs.

#!/usr/bin/python
import feedparser
import urllib2

DownloadPath = '/c/torrent/'
DownloadLog = '/c/torrent/.tvdownloads'
FeedUrl = 'http://www.mininova.org/rss.xml?cat=6'

def is_downloaded(link):
        return link in open(DownloadLog, 'r+').read()

def write_log(link):
        open(DownloadLog, 'a+').write('%s\n' % link)

def get_tvtorrents():
        parser = feedparser.parse(FeedUrl)
        for item in parser['items']:
                if not is_downloaded(item['link']):
                        torrentfile = DownloadPath + item['title'].replace(' ', '_') + '.torrent'
                        torrentdata = urllib2.urlopen(item['link']).read()
                        print "Writing to: %s" % torrentfile
                        open(torrentfile, 'wb').write(torrentdata)
                        write_log(item['link'])

if __name__=='__main__':
    get_tvtorrents()

Crontab it, every 10 minutes or so, and you’re dandy.

You can find the original script here:

http://abhiyerra.wordpress.com/2007/02/17/python-vs-ruby-for-a-simple-script/

You may or may not know about yahoo pipes, which is an awesome tool for processing all sorts of information from the interweb, especially rss feeds although it will work with other formats too. The posibilities are virtually limitless.

Enjoy!

November 2008
M	T	W	T	F	S	S
		Feb »
	1	2
3	4	5	6	7	8	9
10	11	12	13	14	15	16
17	18	19	20	21	22	23
24	25	26	27	28	29	30

November 2008

Feb »

10 Responses to “automated rss torrent downloads with a ReadyNAS”

Feed for this Entry Trackback Address View blog reactions

1 TheUssef Dec 2nd, 2008 at 12:09 am

Nice infos :)
keep it up man:)
thank you
2 Inty Dec 2nd, 2008 at 3:29 pm

Hi jc,

I’m trying to get your first example working, but I’m falling down with getting Curl to install. I get the following error:

~/bin# apt-get install curl
Reading Package Lists… Done
Building Dependency Tree… Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.

Since you only requested a single operation it is extremely likely that
the package is simply not installable and a bug report against
that package should be filed.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
curl: Depends: libc6 (>= 2.3.5-1) but 2.3.2.ds1-22sarge6.infrant1 is to be installed
Depends: libcurl3 (>= 7.15.5-1) but it is not going to be installed
Depends: libidn11 (>= 0.5.18) but it is not going to be installed
E: Broken packages

I’m using Radiator 4.1.4 which is the latest, I’m guessing that might be the problem. Any Ideas?

thanks,
Inty
3 jc Dec 3rd, 2008 at 12:31 am

it ocurrs to me that maybe your sources.list isn’t totally up to date, since debian changed the url for it.

this is what i have.

bender:~# cat /etc/apt/sources.list
deb http://www.readynas.com/packages readynas/

deb http://www.infrant.com/packages readynas/

deb http://archive.debian.org/debian-security sarge/updates main contrib non-free
deb-src http://archive.debian.org/debian-security sarge/updates main contrib non-free

deb http://archive.debian.org/backports.org sarge-backports main contrib non-free
deb-src http://archive.debian.org/backports.org sarge-backports main contrib non-free

deb http://archive.debian.org/debian sarge main contrib non-free
deb-src http://archive.debian.org/debian sarge main contrib non-free

do an ‘apt-get update’ then try to install curl again.

if after that it still doesn’t work, maybe there’s something wrong with your libc package.

I’d try ‘apt-get remove libc6-dev’
and after that, ‘apt-get install build-essentials’

if the problem isn’t in there, i’m out of ideas!, maybe reinstalling the apt addon? did you do an ‘apt-get upgrade’ by any chance? it’s known to break stuff.
4 jc Dec 3rd, 2008 at 12:32 am

um, i supposed you already had ‘apt-get install build-essentials’, check that one first ;P
5 Inty Dec 3rd, 2008 at 12:46 pm

Excellent it worked…

Firstly I used your /etc/apt/sources.list (I think mine was horribly out of date).

Then I ran ‘apt-get update’ (and funnily enough this time I didn’t get loads of errors).

Finally I ran the install command again ‘apt-get install curl’

And hey presto, worked like a charm! I’ve now added my first torrent over ssh while at work, my god I love my little readynas, such a great piece of kit.

Thanks again JC! brilliant work
6 Michael Mar 4th, 2009 at 8:37 am

Thank you so much for this tutorial. It was a godsend. I had to tweak my sources.list per the comments above, and I added catching of the try/catch if there was an invalid torrent URL in the RSS feed to the python script, but it all worked smoothly. Thank you for demonstrating so many great things in one concise blog - you got me up to speed on many concepts in one place.
7 delhux May 30th, 2009 at 6:25 am

I don’t suppose there is some way to make this work as an add-on for the ReadyNas? Reason being I’m not quite confident enough to go through all these steps, but a GUI in FrontView that accepts RSS feeds and allows me to set an interval on how often it checks the feed would be ideal.

Just a thought…
8 Choppy Dec 5th, 2009 at 3:18 pm

Great stuff. Been struggling something shocking with the lack of features in this debian cutdown but the sources and build-seesntials has brought me forward finally.
Currently I am trying to install PPTPD without any success. here is a dump.

nas-DF-7A-45:~# apt-get install pptpd
Reading Package Lists… Done
Building Dependency Tree… Done
pptpd is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 68 not upgraded.
2 not fully installed or removed.
Need to get 0B of archives.
After unpacking 0B of additional disk space will be used.
Setting up ppp (2.4.3-20050321 2sarge1) …
install: invalid group `dip’
dpkg: error processing ppp (–configure):
subprocess post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of pptpd:
pptpd depends on ppp (>= 2.4.3-20041231); however:
Package ppp is not configured yet.
dpkg: error processing pptpd (–configure):
dependency problems - leaving unconfigured
Errors were encountered while processing:
ppp
pptpd
E: Sub-process /usr/bin/dpkg returned an error code (1)
nas-DF-7A-45:~#

Help would be greatly appreciated
9 Jonathan Hartley Jan 27th, 2010 at 11:44 pm

Hi. Thanks for this.

I want something slightly different, but don’t quite have the script-fu - maybe you can help?

My ISP has a daytime bandwidth quota. I want to add commands to my readynas root crontab which start and stop my bittorrent daemon at particular times of day. If this sounds like a reasonable approach, can you help me figure out what some commands to start and stop bittorrent would be?

Thanks for any pointers,

Jonathan
tartley at the domain tartley dot com
10 Jonathan Hartley Jan 27th, 2010 at 11:50 pm

Ah, think I found it, here:
http://www.readynas.com/forum/viewtopic.php?f=36&t=16866&st=0&sk=t&sd=a&start=30

Thanks anyway. Let me know if I’m doin it wrong.

hex for life