1 2 3 Previous Next 31 Replies Latest reply on Jul 1, 2017 7:46 AM by fmpdude

    URL Validation Script

    steve_nanz

      I have a tabe with a few thousand records. There is a field named "URL". Some of the URLs are no longer good. I'd like to create script steps which tests if the URLs are still valid. I'd like to set a field called "validation" to "good" or "bad" depending on whether the URL is still working. I'm on a Mac using Safari, in case the solution requires applescript. Thanks in advance.

        • 1. Re: URL Validation Script
          beverly

          What determines "good" vs "bad"? An error returned? No content? ??

          Thanks,

          Beverly

           

          Sent from miPhone

          • 2. Re: URL Validation Script
            fmpdude

            URL validation is extremely complex. Trying to code complex pattern matching will be tedious at best.

             

            Here's a site I saw that shows the seemingly endless permutations with multiple Regular Expressions to choose from.

             

            In search of the perfect URL validation regex

             

            (Note: it's easy to add RegEx to FMP either by a little (free) programming or by using a plug-in).

            • 3. Re: URL Validation Script
              coherentkris

              You may be able to use Insert From URL / HTTP GET and do some detection on what is returned to validate the url.

               

              Another possible option is to put the url into a web viewer and evaluate the webviewer contents that are returned.

               

              Id rather hit the resource and see what is returned rather than trying to regex validate every possible permutation of a url for correctness.

              • 4. Re: URL Validation Script
                steve_nanz

                Ideally, if a web page successfully loads and the URL exactly matches the contents of my "URL" field, then the test would be  would be valid. (There are website which will load a deafault page if the full address is no longer good.) Since I can't imagine Filmaker can do that on its own, I'd settle for knowing if any page successfully loads.

                 

                I did add a web viewer, which is what coherentkris suggested. That may end up being the best solution. Unfortunately I am sharing the work load with others using Web Direct and web viewer doesn't doesn't appear to be supported. Maybe I'm doing something wrong. It works fine in FileMaker.

                 

                No worreis if this is not feasible. I was just hoping I migh be lucky and someone had already come up against this. I inherrited a pretty old website with lots of dead links. I'm just trying to find a way to quickly identify which are the bad ones. Thanks!

                • 5. Re: URL Validation Script
                  beverly

                  Web Direct with a web viewer is going to run into the CORS

                  Re: REST API CORS Headers

                  Trying to access another URL from within a site may throw security flags and error out.

                   

                  Don't try this inside a Web Direct session?

                  Beverly

                  • 6. Re: URL Validation Script
                    ChrisJohnston

                    steve_nanz wrote:

                    Ideally, if a web page successfully loads and the URL exactly matches the contents of my "URL" field, then the test would be would be valid.

                    With Spectrum as my internet provider every time I load an invalid url I get something like this. I also tried this in an colleagues office in our building who has Verizon and I get this. Notice both of them contain the silly url that is invalid. That url being "www.nikeuwc.com". So that makes everything previously said by Beverly, fmpdude, and coherentkris really worth something. When a page that loads a website properly inserts from url it will not have your internet providers 'cant find it page' url in the address. If you see this in the inserted text from the script step result, the page did not load. I would say a script to validate a page is loading is very feasible. The thing is it will be different for users not in the same location. But since there are only a handful of internet providers, some logic could be built into a script that checks for internet provider just like FileMaker can check an operating system platform. Do you want me to show you something (script)?

                    • 7. Re: URL Validation Script
                      coherentkris

                      this will validate links on a website

                      W3C Link Checker

                       

                      Its probably easier doing outside of FM as there are many tools to check for broken links on a website

                      • 8. Re: URL Validation Script
                        philmodjunk

                        We use Insert from URL to validate URL's in our solution. Have to include a bit of a pause or a loop to be sure to give system time to properly get a response inserted into the field, but it does work.

                        • 9. Re: URL Validation Script
                          fmpdude

                          Using a little code, I can validate a URL right in FMP.

                           

                          Here's the RegEx:

                           

                          (https?:\/\/(?:www\.|(?!www))[a-zA-Z0-9][a-zA-Z0-9-]+[a-zA-Z0-9]\.[^\s]{2,}|www\.[a-zA-Z0-9][a-zA-Z0-9-]+[a-zA-Z0-9]\.[^\s]{2,}|https?:\/\/(?:www\.|(?!www))[a-zA-Z0-9]\.[^\s]{2,}|www\.[a-zA-Z0-9]\.[^\s]{2,})

                           

                          javascript - What is a good regular expression to match a URL? - Stack Overflow

                           

                          ---

                           

                          Here's a Java micro-service and FMP that validates a relatively simple URL using this Regular Expression.

                           

                          CLICK ON THE IMAGE BELOW IF IT DOESN'T FULLY SHOW.

                           

                          If you have particular URLs you'd like me to test, just post them or send to me in a PM, OK?

                           

                          • 10. Re: URL Validation Script
                            bigtom

                            Insert from URL with cURL "-X GET -I" along with Get (LastError) should give you the tools you need to decide if there is a connection error before a redirect and decide if any redirect URL is valid.

                            1 of 1 people found this helpful
                            • 11. Re: URL Validation Script
                              ChrisJohnston

                              Cool... That sounds like something I want to know how to do. Can you show the actual script step setup? Thanks

                              • 12. Re: URL Validation Script
                                ChrisJohnston

                                This seems like a good way also. This page shows "Error: 500 Can't connect to www.nikeuwc.com:80" in the output. If you copy the address after checking you can use it in the Insert from URL. So I was thing about something like this. "https://validator.w3.org/checklink?uri=" & Resp::url & ":80\&hide_type=all&depth=\&check=Check" in an Insert from URL script step. When I test it gives me a result that contains the string "Error: 500 Can't connect to...". I keep trying with with good and bad, when good never Error 500, when bad always. Thanks

                                • 13. Re: URL Validation Script
                                  philmodjunk

                                  Interesting use of Java and RegEx, but aren't you just validating the URL syntax? A URL can have perfect syntax, but still be invalid simply because the URL does not take you to a web site that actually exists. The insert from URL step validates issues due to bad syntax and missing sites so it would seem a better option, though your tool would still be very useful as a way to tell the user why the URL is bad.

                                  • 14. Re: URL Validation Script
                                    bigtom

                                    Any 5xx is likely bad along with 404 and maybe 301. You might consider 302 to be valid. Depends on your use as http://google.com will redirect to https://www.google.com.

                                    1 2 3 Previous Next