Question

From what I understand, you have to enter in all of your usernames and passwords into Mint, so I assume they are actually logging into your bank account and scraping the resulting screen to put this data into a form that Mint and others use.

How do they actually simulate the keypresses and mouse clicks? I assume banks don't like it when they do this - how do their scrapers avoid detection?

Was it helpful?

Solution

I'm pretty sure they don't simulate clicks, etc. In the end, any data that ends up on a user's page is transmitted in a response to a request. If you can figure out how to construct a valid request and then how to parse the response, you'll have the data you want.

As far as I could gather after using Yodlee for quite a while, they deal with sites in two major ways: the sites they have official agreements to work with and the sites they don't have official agreements with. For the first category of sites they, most often, have agreed upon APIs for getting the data. For the sites in the second category they reverse-engineer layer 7 communication protocols and data structures (a.k.a. screen/html scraping).

OTHER TIPS

The way I understand it, Yodlee uses the OFX specification to access banks' financial information.

http://www.ofx.net/

For the banks that don't implement OFX, they use custom screen scrapers, which must constantly be updated when banks change the information that's displayed on their site.

I don't know Yodlee so i simply assume it's like "sofortüberweisung.de" where you give a 3rd party your bank login data (and depending on what you do even a valid TAN) and thus trust them not to abuse it and additionally break your bank's security regulations ("NEVER GIVE YOUR YOUR PIN/TAN").

They most likely simulate what a browser would do. As web-based banking interfaces are usually just HTML/JavaScript everyone can look at the client-side code and do whatever it does with a custom program. Since those actions are not done in a malicious way, actions which require e.g. a TAN or a CAPTCHA to be solved can be simply forwarded to the legit user who will then enter the necessary TAN or solve the CAPTCHA.

Nonetheless to say, it is really bad to use services like that. While they most likely won't do anything bad you cannot know it for sure. And your bank is damn right if they don't refund you anything if you ever get scammed by such a service.

Another solution which would be perfectly safe (as long as you are not concerned about a 3rd party knowing about your financial status etc.) would be the yodlee company making contracts with major banks allowing them to access your data after you've authorized it through some way (you can already do that on pages like Twitter - I'd never do that for bankign though but technically it wouldn't be hard to realize something like that). That would be clean and secure as it would not involve "screen-scraping" or customers entering their banking login data anywhere but on their bank's website. But I believe no bank does something like that and in my opinion that's good as there are way too many people out there who are far too trustworthy and we all know how many information they give out on Facebook & Co. Now imagine a facebook<->bank integration... M.Zuck.'s wet dreams which hopefully never become true... And even if it's not Facebook.. There'll always be companies who want people's personal data and enough people giving them out; especially if it's easy and looks secure ("I have to confirm it on MY BANK's page. so it MUST be safe - it's supported by MY BANK").

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top