11 Replies Latest reply on Feb 8, 2015 9:03 PM by steve_ssh

    GetLayoutObjectAttribute   content    Field Data

    taylorsharpe

      You can web scrape data easily with a calculation such as:

       

      GetLayoutObjectAttribute ( "WebViewerObjectName" ; "content" )

       

      I was wanting to scrape a page that a user had filled out several fields in the FM web viewer interactively.  Is there a way to web scrape that also copies the information typed into the fields in the web viewer?

        • 1. Re: GetLayoutObjectAttribute   content    Field Data
          matthew_odell

          The best way to do this isn't to scrape the HTML, but to write javascript on the page to pass the data back as a parameter or variable to a script using an fmp: URL. I know that means you need to have some control over the page, but that's currently the way to do it. That data isn't accessible as HTML, but needs to be called from something like JavaScript or submitted as a form to another page.

          • 2. Re: GetLayoutObjectAttribute   content    Field Data
            taylorsharpe

            Well, you hit the nail with the hammer... I don't have control of the page or this wouldn't be an issue.  Actually, if it was me, I'd just have them fill it out in a WebDirect page and have full control <grin>.  Oh well, thanks for the input. 

            • 3. Re: GetLayoutObjectAttribute   content    Field Data
              user19752

              On Windows, you can run javascript on almost any page with running FMscript "set web viewer[go to URL]" like

              "javascript:location.href='fmp://$/" & GetAsURLEncoded ( Get ( FileName ) )

              & "?script=setf&param=' + encodeURIComponent(encodeURIComponent(autosearch.value))"

               

              (autosearch is an input element, top right corner in this page)

              • 4. Re: GetLayoutObjectAttribute   content    Field Data
                taylorsharpe

                That sounds interesting.  I'll have to find a Windows machine to try it out.  I assume from your comments that it does not run on a Mac.  But this is the type of answer I was looking for.  Thank you user19752. 

                • 5. Re: GetLayoutObjectAttribute   content    Field Data
                  Benjamin Fehr

                  Would be great to accomplish this without a PlugIn.

                  If not, It's TROI-URL, 360Works-WebAssistent or MBS PlugIns which would do that job

                  • 6. Re: GetLayoutObjectAttribute   content    Field Data
                    beverly

                    It would be great to accomplish this NATIVELY in FMP (read filled-in form fields as seen in Web Viewer). But I don't think this is possible except with Javascript (only if you have control over the web form), including user19752 method.

                     

                    The "submit" button must be able to control where the form is submitted ( javascript or otherwise) and that means control of the web form. I don't know if the plug-ins can do this without control of the web form, as well.

                     

                    But the wizards may have found another way around this...

                     

                    Beverly

                    • 7. Re: GetLayoutObjectAttribute   content    Field Data
                      taylorsharpe

                      Ahhhh.... efficientbizz, you have helped me the most.  I did not realize MBS had WebView functions and I can make use of that since I have a license of that plugin.  Pretty slick!  I know it won't work for those  without it, but that fixes this particular problem for me.  Thanks!

                      • 8. Re: GetLayoutObjectAttribute   content    Field Data
                        Benjamin Fehr

                        I've already sent this thread to Christian Schmitz to encourage him to log in. Wouldn't be surprised if next post is him, telling you which MBS-function you have to trigger.

                        (Probably another member of this species who catalyzes caffeine into great IT-solutions)

                        • 9. Re: GetLayoutObjectAttribute   content    Field Data
                          Benjamin Fehr

                          Nevertheless. A thrilling topic has been addressed: Using Javascripts in WV source-code's to get plugin-like functions for FMGo. I did a sketch recently to zip-compress some content by JS.

                          • 10. Re: GetLayoutObjectAttribute   content    Field Data
                            beverly

                            Absolutely, eb! there are cool JS things in WV (FMGo and FMPro) - when you have the _control_ over the form (and the JavaScript)!

                             

                            scraping-the-web, however is often a need with forms (and response pages) that are not under our control. Remember also that scraping can be difficult when you are landing on a result that is changed by the content manager anytime between the time you create your "routine" and the next time you need it.

                             

                            A request that can return xml or json results, can be much easier to work with when permission to do so is arranged with the content providers, if at all possible.

                             

                            Just sayin' be aware, & I know as I've been providing content (and scraping) since the beginning of the web...

                            beverly

                            • 11. Re: GetLayoutObjectAttribute   content    Field Data
                              steve_ssh

                              On a related note, for anyone trying to accomplish this sort of thing on OSX when the user is using Safari as a web browser:

                               

                              AppleScript, Safari, and Javascript play nicely enough together such that FMP can fill in and/or dynamically read forms which the user is browsing in Safari.  This includes reading, setting, and returning form/element values that have changed after the initial page load.  The technique seems to work fine without having any back-end control of the page content.

                               

                              Doing this is a matter of having FM trigger AppleScript, which in turn tells Safari to run JavaScript code.  Such Javascript code has a stunning (and somewhat disconcerting) range of control over the content in any open window/tab in Safari.

                               

                               

                              Seems to work:

                               

                              I tested the idea, and it seems to work just fine on a Mavericks machine, though all of beverly's cautions about how easily a scrape situation can break would certainly apply.  Another thing to keep in mind is that screen scrapes get harder to set up as the pages make greater use of client side UI scripting.

                               

                               

                              Finally:

                               

                              I realize that Taylor already has a good working solution, but I could see how this might help someone with a use case of automating the filling in of web forms in an OSX context, and figured someone looking to do so might turn up this post in a search.

                               

                              HTH and kind regards,

                               

                              -steve