I would think scraping these pages for content would be a very inefficient way of getting email into your database, and one that would be prone to a lot of error and maintenance. If they change the look of their page, the structure of their html, then you would need to update your calculations to scrape all the time.
There are several plugins that allow you to check email, either POP or IMAP. I would bypass the screen scraping (which would require user interaction to log into each site) and just check email directly.
Would that work better?
We don´t want to use any plugin, because
- they don´t work in FM GO
- they force the user to change their email-system, which most users don´t want to do
- you get all the spam in your database
- they are expensive
- do they work in Webdirect ?
I admit that screen scraping definetely might lead to compatibility problems when mail sites are changed.
My first thought then, would be to look into a php imap solution that could check mail and get it into a filemaker database somehow. If your solution is hosted on filemaker server, the Filemaker php api would be a method to get those in. If not, then your web script could make the emails available to a local web viewer in a more reliable format to scrape.
the intention is, that it works for single users too.