25
Nov
08

automated rss torrent downloads with a ReadyNAS

Or any linux, for that matter.

I’ve owned a ReadyNAS NV for quite a while now, I’ve always thought it was a great product.

After they allowed ssh access to the box (with raidiator 4.0) they made it even better. Low power consumption, debian linux, and a metric ton of diskspace make it a really hackable device.

It has a bittorrent client addon available, the problem is that it is only accessible via an http interface, and cannot import torrents automagically from a specified location like a drop-in folder or an rss feed, unlike some of my favorite torrent clients.

I did a bit of reading and put together a few pieces of script-fu that already existed on the internet.

Auto importing torrents from drop-in folder

First piece to the puzzle is chirpa’s auto-import script as it appeared in the readynas forums, with a tiny modification to fix the whitespace problem with torrent names.

You can see the original thread here:

http://www.readynas.com/forum/viewtopic.php?f=35&t=19126

#!/bin/sh
#     What: Import .torrents into the NETGEAR ReadyNAS BitTorrent
#              Download Manager
#      Who: Quickly slapped together by chirpa (readynas.com/forum)
#    Usage: Crontab it and walk away.
# Requires: Curl (maybe I'll make it work with Wget at some point)
#  Version: 0.2a
#     Todo: Check for returned status: apiTorrentAddFinishedOk /
#           apiTorrentAddFailed

INCOMING_DIR=/c/torrent
DONE_DIR=/c/torrent/done
BTURL="http://localhost:8080/api/torrent-add?start=yes"
ORIG_IFS=$IFS
IFS=$'\n'
cd $INCOMING_DIR
for f in `ls -1 *.torrent 2>/dev/null`
do
echo Loading file: $f
curl -s -F fileEl=@$f $BTURL >/dev/null
mv $f $DONE_DIR
done
IFS=$ORIG_IFS

You can name this file “import_torrents” and run it every N minutes, adding it to your crontab. I run mine every 5 minutes.

Auto downloading torrents from an RSS feed

Lets suppose we want to download torrents from a mininova.org feed, this script coded by Abhi Yerra will get you started.

Note you will need to install both the python and python-feedparser packages via apt-get and also, you will need to modify DownloadPath, DownloadLog and FeedUrl to fit your needs.

#!/usr/bin/python
import feedparser
import urllib2

DownloadPath = '/c/torrent/'
DownloadLog = '/c/torrent/.tvdownloads'
FeedUrl = 'http://www.mininova.org/rss.xml?cat=6'

def is_downloaded(link):
        return link in open(DownloadLog, 'r+').read()

def write_log(link):
        open(DownloadLog, 'a+').write('%s\n' % link)

def get_tvtorrents():
        parser = feedparser.parse(FeedUrl)
        for item in parser['items']:
                if not is_downloaded(item['link']):
                        torrentfile = DownloadPath + item['title'].replace(' ', '_') + '.torrent'
                        torrentdata = urllib2.urlopen(item['link']).read()
                        print "Writing to: %s" % torrentfile
                        open(torrentfile, 'wb').write(torrentdata)
                        write_log(item['link'])

if __name__=='__main__':
    get_tvtorrents()

Crontab it, every 10 minutes or so, and you’re dandy.

You can find the original script here:

http://abhiyerra.wordpress.com/2007/02/17/python-vs-ruby-for-a-simple-script/

You may or may not know about yahoo pipes, which is an awesome tool for processing all sorts of information from the interweb, especially rss feeds although it will work with other formats too. The posibilities are virtually limitless.

Enjoy!


8 Responses to “automated rss torrent downloads with a ReadyNAS”


  1. 1 TheUssef Dec 2nd, 2008 at 12:09 am

    Nice infos :)
    keep it up man:)
    thank you

  2. 2 Inty Dec 2nd, 2008 at 3:29 pm

    Hi jc,

    I’m trying to get your first example working, but I’m falling down with getting Curl to install. I get the following error:

    ~/bin# apt-get install curl
    Reading Package Lists… Done
    Building Dependency Tree… Done
    Some packages could not be installed. This may mean that you have
    requested an impossible situation or if you are using the unstable
    distribution that some required packages have not yet been created
    or been moved out of Incoming.

    Since you only requested a single operation it is extremely likely that
    the package is simply not installable and a bug report against
    that package should be filed.
    The following information may help to resolve the situation:

    The following packages have unmet dependencies:
    curl: Depends: libc6 (>= 2.3.5-1) but 2.3.2.ds1-22sarge6.infrant1 is to be installed
    Depends: libcurl3 (>= 7.15.5-1) but it is not going to be installed
    Depends: libidn11 (>= 0.5.18) but it is not going to be installed
    E: Broken packages

    I’m using Radiator 4.1.4 which is the latest, I’m guessing that might be the problem. Any Ideas?

    thanks,
    Inty

  3. 3 jc Dec 3rd, 2008 at 12:31 am

    it ocurrs to me that maybe your sources.list isn’t totally up to date, since debian changed the url for it.

    this is what i have.

    bender:~# cat /etc/apt/sources.list
    deb http://www.readynas.com/packages readynas/

    deb http://www.infrant.com/packages readynas/

    deb http://archive.debian.org/debian-security sarge/updates main contrib non-free
    deb-src http://archive.debian.org/debian-security sarge/updates main contrib non-free

    deb http://archive.debian.org/backports.org sarge-backports main contrib non-free
    deb-src http://archive.debian.org/backports.org sarge-backports main contrib non-free

    deb http://archive.debian.org/debian sarge main contrib non-free
    deb-src http://archive.debian.org/debian sarge main contrib non-free

    do an ‘apt-get update’ then try to install curl again.

    if after that it still doesn’t work, maybe there’s something wrong with your libc package.

    I’d try ‘apt-get remove libc6-dev’
    and after that, ‘apt-get install build-essentials’

    if the problem isn’t in there, i’m out of ideas!, maybe reinstalling the apt addon? did you do an ‘apt-get upgrade’ by any chance? it’s known to break stuff.

  4. 4 jc Dec 3rd, 2008 at 12:32 am

    um, i supposed you already had ‘apt-get install build-essentials’, check that one first ;P

  5. 5 Inty Dec 3rd, 2008 at 12:46 pm

    Excellent it worked…

    Firstly I used your /etc/apt/sources.list (I think mine was horribly out of date).

    Then I ran ‘apt-get update’ (and funnily enough this time I didn’t get loads of errors).

    Finally I ran the install command again ‘apt-get install curl’

    And hey presto, worked like a charm! I’ve now added my first torrent over ssh while at work, my god I love my little readynas, such a great piece of kit.

    Thanks again JC! brilliant work

  6. 6 Michael Mar 4th, 2009 at 8:37 am

    Thank you so much for this tutorial. It was a godsend. I had to tweak my sources.list per the comments above, and I added catching of the try/catch if there was an invalid torrent URL in the RSS feed to the python script, but it all worked smoothly. Thank you for demonstrating so many great things in one concise blog - you got me up to speed on many concepts in one place.

  7. 7 delhux May 30th, 2009 at 6:25 am

    I don’t suppose there is some way to make this work as an add-on for the ReadyNas? Reason being I’m not quite confident enough to go through all these steps, but a GUI in FrontView that accepts RSS feeds and allows me to set an interval on how often it checks the feed would be ideal.

    Just a thought…

  8. 8 Jonathan Hartley Jan 27th, 2010 at 11:50 pm

    Ah, think I found it, here:
    http://www.readynas.com/forum/viewtopic.php?f=36&t=16866&st=0&sk=t&sd=a&start=30

    Thanks anyway. Let me know if I’m doin it wrong.

Leave a Reply