سؤال

I'm writing a webspider and allocating it on Heroku and after the main actor (the one that allocates the router) reaches 512Mb it crashes.

Basically every URL parsed is sent to the FetchRouter. The list of messages of the queue is to big to get in memory.

Is there any way to store this list in the Disk ?

fetcherRouter = context.actorOf(Props[URLFetcher].withRouter(RandomRouter(30)), name = "fetcherRouter")
هل كانت مفيدة؟

المحلول

Indeed.

There is FileBasedMailbox. Look in section 7.1, "Durable Mailboxes," of the Akka 2.0.x PDF (or corresponding Web page).

مرخصة بموجب: CC-BY-SA مع الإسناد
لا تنتمي إلى StackOverflow
scroll top