AnsweredAssumed Answered

Getting Response Code 405 using Base Elements Curl but not FM's Web Viewer

Question asked by jeffest on Sep 15, 2018
Latest reply on Sep 17, 2018 by user19752

Filemaker 13

 

I'm working on an app that goes through a series of URL's that the user has saved and pulls in the html code to check for certain elements (nothing nefarious). Using FM's built-in web viewer was using too much memory despite clearing the url each time so I started using Base Elements Curl options which worked great until I hit one website that continues to give me a response code of 405 (Method Not Allowed). Using FM's web viewer however does pull the website in without problems but the inherent memory leak makes this method unusable over time. So my question is, what is it in FM's web viewer that allows the html to be pulled in but is apparently missing in my Curl options? Incidentally, Insert From URL also produces the 405 response code.

 

This is the code I'm using in my script:

 

Set Variable [ $Clear; Value:BE_Curl_Set_Option ]

Set Variable [ $Content_Type; Value:BE_HTTP_Set_Custom_Header ( "Content-Type" ; "text/html;charset=UTF-8" ) ]

Set Variable [ $Header_Accept; Value:BE_HTTP_Set_Custom_Header ( "Accept" ; "text/html;charset=UTF-8" ) ]

Set Variable [ $User_Agent; Value:BE_Curl_Set_Option ( "CURLOPT_USERAGENT"; "Mozilla/5.0" ) ]

Set Variable [ $Timeout; Value:BE_Curl_Set_Option ( "CURLOPT_TIMEOUT"; "5" ) ]

Set Variable [ $Follow; Value:BE_Curl_Set_Option ( "CURLOPT_FOLLOWLOCATION"; "1" ) ]

Set Variable [ $Response_Code; Value:BE_HTTP_Response_Code ]

Set Variable [ $Web_View; Value:BE_HTTP_GET ($Web_Link ) ]

 

And this is a sample of the response I'm getting back.

 

<!DOCTYPE html>

<html>

<head>

<title>Pardon Our Interruption</title>

<link rel="stylesheet" type="text/css" href="//cdn.distilnetworks.com/css/distil.css" media="all">

<meta http-equiv="content-type" content="text/html; charset=UTF-8" />

<meta name="viewport" content="width=1000" />

<META NAME="robots" CONTENT="noindex, nofollow">

<meta http-equiv="cache-control" content="max-age=0" />

<meta http-equiv="cache-control" content="no-cache" />

<meta http-equiv="expires" content="0" />

<meta http-equiv="expires" content="Tue, 01 Jan 1980 1:00:00 GMT" />

<meta http-equiv="pragma" content="no-cache" />

<script type="text/javascript"  defer></script><style type="text/css">#d__fFH{position:absolute;top:-5000px;left:-5000px}#d__fF{font-family:serif;font-size:200px;visibility:hidden}#zzvzcxzwdvswcfvrfxzsvvuesqcuswbscy{display:none!important}</style></head>

 

<body class='block-page'>

<div class='container'>

<div class='row'>

<div class='sidebar col-lg-4 col-sm-5'>

<img  alt="0">

</div>

<div class='content col-lg-8 col-sm-7'>

<h1>Pardon Our Interruption...</h1>

<p>

As you were browsing <strong>www.teepublic.com</strong> something about your browser made us think you were a bot. There are a few reasons this might happen:

</p>

<ul>

<li>You're a power user moving through this website with super-human speed.</li>

<li>You've disabled JavaScript in your web browser.</li>

<li>A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this <a title='Third party browser plugins that block javascript' href='http://ds.tl/help-third-party-plugins' target='_blank'>support article</a>.</li>

</ul>

<p>After completing the CAPTCHA below, you will immediately regain access to www.teepublic.com.</p>

           

When accessing the url through a regular browser there is no captcha required to view the page and FM's web viewer doesn't encounter that either, so there must be something I'm missing in the CURL options. Any help would be greatly appreciated. TIA  - Jeff

 

P.S. This is the URL I was testing but I just sent them a DMCA so it might not be there for long (that's what my app is checking for btw)

https://www.teepublic.com/onesie/1145003-black-holes-suck

Outcomes