The initial error you were getting with the Daemon class is because you're not passing the correct parameters to the constructor. The contructor takes a string and a hash.
Moving on from that , the insert failed because:
- parsing status.datetime throws an exception (its already a Time object).
- status.coordinate throws an exception if there's no coordinate.
The following code works for me (note : I added growl so you can see the tweets):
#!/usr/bin/env ruby
require "tweetstream"
require "mongo"
require "time"
require 'growl'
DESIRED = %w{created_at text geo coordinates id id_str}
host= ENV["MONGO_HOST"] || 'localhost'
port = ENV["MONGO_PORT"] || 27017
username = ENV["MONGO_USERNAME"]
password = ENV["MONGO_PASSWORD"]
term = ARGV[1] || 'TERM'
begin
TweetStream.configure do |config|
config.consumer_key = ENV["TWEET_CONSUMER_KEY"]
config.consumer_secret = ENV["TWEET_CONSUMER_SECRET"]
config.oauth_token = ENV["TWEET_OAUTH_TOKEN"]
config.oauth_token_secret = ENV["TWEET_OAUTH_TOKEN_SECRET"]
config.auth_method = :oauth
end
db = Mongo::Connection.new(host, port).db("tweets")
db.authenticate(username, password)
tweets = db.collection("tweetdata")
puts "about to start tracking term #{term}"
TweetStream::Daemon.new('tracker').track(term) do |status|
Growl.notify status.text, :title => status.user.screen_name
#
# filter out nil values
# filter out all keys not in the desired array
#
data = status.attrs.select{|k,v| !v.nil? && DESIRED.include?(k.to_s)}
tweets.insert({"data" => data});
end
rescue Mongo::ConnectionFailure
puts "Connection Error : #{$!}"
rescue Mongo::AuthenticationError
puts "Auth Error : #{$!}"
rescue Mongo::MongoDBError
puts "Unexpected Error : #{$!}"
end
You'll need to setup your environment with the following correct values :
export MONGO_USERNAME="..."
export MONGO_PASSWORD="..."
export TWEET_CONSUMER_KEY="..."
export TWEET_CONSUMER_SECRET="..."
export TWEET_OAUTH_TOKEN="..."
export TWEET_OAUTH_TOKEN_SECRET="..."
Then you can start the daemon with something like (in this case we'll search for yankees):
ruby tweetscrape.rb start yankees