Twisted server, consume from Twitter stream api and serve static content at the same time

StackOverflow https://stackoverflow.com/questions/22909499

Question

I'm a bit new to twisted servers and servers in general. I need to have an asynchronous server running on my local retrieving data from twitter tweets (writing in a local file the data) and at the same time, I want to server some simple content such as "Hello world" at "local_ip:8880" The problem is that the stream api creates a http connection and I dont know how to do the two tasks at the same time. My code look like this:

"imports and connection to twitter api keys"
def pp(o):
   return json.dumps(o, indent=3)
class listener(StreamListener):

   def on_data(self,data):
      #Convert to JSON the input string
      data_json = json.loads(data)
      #print pp(data_json['user'])

      #We get the html from embedly
      tweet_id = data_json['id_str']
      tweet_user = data_json['user']['screen_name']
      tweet_url = "https://twitter.com/"+tweet_user+"/status/"+tweet_id
      html="<a class=\"embedly-card\" href=\""+tweet_url+"\">Prueba</a>\n"
      print "Datos recibidos en el listener."
      #Write the results in the file
      f = open('myfile.html','a')
      f.write(html) # python will convert \n to os.linesep
      f.close()
      return True

   def on_error(self,status):
      print status


class Root(resource.Resource,StreamListener):
   isLeaf = True
   def __init__(self):
      print "Server iniciado"
      auth = OAuthHandler(CONSUMER_KEY,CONSUMER_SECRET)
      auth.set_access_token(OAUTH_TOKEN,OAUTH_TOKEN_SECRET)
      twitterStream = Stream(auth, listener())

      twitterStream.filter(track=["query"])

   def render_GET(self, request):
         print "Recibida peticion"
         return '<html><body>Hello world<body></html>'

root = Root()
site = server.Site(root)
reactor.listenTCP(8880, site)
reactor.run()

When I run this, i got stuck in the console, it receives the stream data, but when I access my "local_ip:8880" it cant load the "Hello world". Is it possible to make this? Thanks in advance for attention and sorry for my English.

Was it helpful?

Solution

If you use blocking APIs - for example, a function that does network I/O using blocking sockets - then you will prevent your Twisted application from performing operations concurrently. Only one blocking operation can be in progress at a time.

Switch to a non-blocking Twitter API - for example, https://github.com/fiorix/twisted-twitter-stream

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top