4 Replies Latest reply on Oct 23, 2015 3:02 PM by TSGal

    HTTP POST with payload larger than 1024 bytes fails with squid proxies

    jbarnum

      Squid 2.7 proxy server, which is a very popular proxy software used in almost every major hotel, has a bug which prevents FileMaker Pro from being able to send POST requests with more than 1024 bytes.

       

      This is a major limitation, as it prevents many solution workflows from working properly when used with a public internet provider such as a hotel or business center.

       

      FileMaker uses curl as the underlying library for sending HTTP POSTs, and curl automatically adds an 100-Expect header when the request length exceeds 1024 bytes, as seen here:

       

      POST / HTTP/1.1

      User-Agent: FileMaker/14.0

      Host: sync.360works.com

      Accept: */*

      Accept-Encoding: deflate, gzip

      Content-Length: 1031

      Content-Type: application/x-www-form-urlencoded

      Expect: 100-continue

       

      This is perfectly valid behavior, but unfortunately Squid 2.7 (which is the version in popular use) does not handle it correctly, and the request fails.

       

      I propose adding this command line param to curl:

       

      -H "Expect:"

       

      That should disable the 100 HTTP header, as documented here: Curl: Disabling the Expect 100 HTTP Header

       

      To reproduce the problem, go to someplace that uses the Squid proxy server. For instance, the Hyatt Regency right next to FileMaker headquarters uses this. From there, try using the Insert from URL script step to get the contents of this URL:

       

      httppost://www.google.com/?param=thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs

       

      That gives the following error from squid:

       

      HTTP/1.0 417 Expectation failed

      Server: squid/2.7.STABLE3

      Date: Fri, 16 Oct 2015 04:52:35 GMT

      Content-Type: text/html

      Content-Length: 1373

      Expires: Fri, 16 Oct 2015 04:52:35 GMT

      X-Squid-Error: ERR_INVALID_REQ 0

      X-Cache: MISS from localhost

      X-Cache-Lookup: NONE from localhost:3128

      Via: 1.0 localhost:3128 (squid/2.7.STABLE3)

      Connection: close

       

       

      <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

      <HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">

      <TITLE>ERROR: The requested URL could not be retrieved</TITLE>

      <STYLE type="text/css"><!--BODY{background-color:#ffffff;font-family:verdana,sans-serif}PRE{font-family:sans-serif}--></STYLE>

      </HEAD><BODY>

      <H1>ERROR</H1>

      <H2>The requested URL could not be retrieved</H2>

      <HR noshade size="1px">

      <P>

      While trying to process the request:

      <PRE>

      POST / HTTP/1.1

      User-Agent: FileMaker/14.0

      Host: sync.360works.com

      Accept: */*

      Accept-Encoding: deflate, gzip

      Content-Length: 1031

      Content-Type: application/x-www-form-urlencoded

      Expect: 100-continue

       

       

      </PRE>

      <P>

      The following error was encountered:

      <UL>

      <LI>

      <STRONG>

      Invalid Request

      </STRONG>

      </UL>

       

       

      <P>

      Some aspect of the HTTP Request is invalid.  Possible problems:

      <UL>

      <LI>Missing or unknown request method

      <LI>Missing URL

      <LI>Missing HTTP Identifier (HTTP/1.0)

      <LI>Request is too large

      <LI>Content-Length missing for POST or PUT requests

      <LI>Illegal character in hostname; underscores are not allowed

      </UL>

      <P>Your cache administrator is <A HREF="mailto:webmaster">webmaster</A>.

       

       

      <BR clear="all">

      <HR noshade size="1px">

      <ADDRESS>

      Generated Fri, 16 Oct 2015 04:52:35 GMT by localhost (squid/2.7.STABLE3)

      </ADDRESS>

      </BODY></HTML>

       

       

      If the URL is shortened so that the parameter length is < 1024 bytes, you get this Google message (which is the desired behavior):

       

      <!DOCTYPE html>

      <html lang=en>

        <meta charset=utf-8>

        <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">

        <title>Error 405 (Method Not Allowed)!!1</title>

        <style>

          *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}

        </style>

        <a href=//www.google.com/><span id=logo aria-label=Google></span></a>

        <p><b>405.</b> <ins>That’s an error.</ins>

        <p>The request method <code>POST</code> is inappropriate for the URL <code>/</code>.  <ins>That’s all we know.</ins>