For my photography news site, I want to use Twitter as a data source for popular links. This involves getting tweets out of Twitter and into MongoDB.
The first step is to be able to read the tweets from an account using Clojure. The library that holds the key is clojure-twitter.
To try this out, I created a new lein project, and added the following to project.clj:
(defproject twitter "1.0.0-SNAPSHOT" :description "Sample project to read in a tweet stream" :dependencies [[org.clojure/clojure "1.2.1"] [clojure-twitter "1.2.5"]] :dev-dependencies [[swank-clojure "1.3.0"]])
lein deps and start working out the code.
The example is a good start, however it the various authentication keys didn’t quite make sense to me on first look.
- Consumer key - identifier for your application, found with your app details.
- Consumer secret - shown with your consumer key and almost twice as long. Keep this secret.
- Access Token (oauth_token) - identifier for your user, found under the “My Access Token” button next to your application details.
- Access Token Secret (oauth_token_secret) - shown with your Access Token
The access token took me a little while to find. Yours will be at a URL similar to:
Where 123456 is the identifier of your application.
Armed with these four pieces of information, the code is quite simple. First include the API:
(ns twitter.core (:require [twitter :as twitter] [oauth.client :as oauth]))
Then setup the various keys and secret phrases:
;; Make a OAuth consumer (def oauth-consumer (oauth/make-consumer "N0tr3a11ym1k3y" ;; consumer key "R23mnasdfphsfjas08ujhasf08aslfkjasfd234asfd" ;; consumer secret "https://api.twitter.com/oauth/request_token" "https://api.twitter.com/oauth/access_token" "https://api.twitter.com/oauth/authorize" :hmac-sha1)) (def oauth-access-token "1234456789-asdfjklkjasdfjkllASDFASDF") (def oauth-access-token-secret "CompletelyRandomStringWithNumbersAndLetters")
Obviously, include your own keys and tokens.
The Clojure API is easy to read from the source and follows closely to Twitter’s API.
I chose to use the https version of the API and also scope the authentication. To read the most recent tweets from my followers:
(defn latest-tweets [count] (twitter/with-https (twitter/with-oauth oauth-consumer oauth-access-token oauth-access-token-secret (twitter/home-timeline :count (str count)))))
I’m not sure why the limit parameter only takes a string. Now after
lein swank and connecting up Emacs, I can run:
user> (use :reload 'twitter.core) nil user> (first (map :text (latest-tweets 1))) "OK, that's 20 videos this weekend alone I posted at http://t.co/Vm2sVQV Hope you like (please subscribe if you do!)"
There is quite a lot of information returned for each tweet, such as conversation threads, user info and retweet data. Now I just need to decide how much of it to store for later analysis.