문제

I'm writing a webspider and allocating it on Heroku and after the main actor (the one that allocates the router) reaches 512Mb it crashes.

Basically every URL parsed is sent to the FetchRouter. The list of messages of the queue is to big to get in memory.

Is there any way to store this list in the Disk ?

fetcherRouter = context.actorOf(Props[URLFetcher].withRouter(RandomRouter(30)), name = "fetcherRouter")
도움이 되었습니까?

해결책

Indeed.

There is FileBasedMailbox. Look in section 7.1, "Durable Mailboxes," of the Akka 2.0.x PDF (or corresponding Web page).

라이센스 : CC-BY-SA ~와 함께 속성
제휴하지 않습니다 StackOverflow
scroll top