jbarnum

HTTP POST with payload larger than 1024 bytes fails with squid proxies

Discussion created by jbarnum on Oct 15, 2015
Latest reply on Oct 23, 2015 by TSGal

Squid 2.7 proxy server, which is a very popular proxy software used in almost every major hotel, has a bug which prevents FileMaker Pro from being able to send POST requests with more than 1024 bytes.

 

This is a major limitation, as it prevents many solution workflows from working properly when used with a public internet provider such as a hotel or business center.

 

FileMaker uses curl as the underlying library for sending HTTP POSTs, and curl automatically adds an 100-Expect header when the request length exceeds 1024 bytes, as seen here:

 

POST / HTTP/1.1

User-Agent: FileMaker/14.0

Host: sync.360works.com

Accept: */*

Accept-Encoding: deflate, gzip

Content-Length: 1031

Content-Type: application/x-www-form-urlencoded

Expect: 100-continue

 

This is perfectly valid behavior, but unfortunately Squid 2.7 (which is the version in popular use) does not handle it correctly, and the request fails.

 

I propose adding this command line param to curl:

 

-H "Expect:"

 

That should disable the 100 HTTP header, as documented here: Curl: Disabling the Expect 100 HTTP Header

 

To reproduce the problem, go to someplace that uses the Squid proxy server. For instance, the Hyatt Regency right next to FileMaker headquarters uses this. From there, try using the Insert from URL script step to get the contents of this URL:

 

httppost://www.google.com/?param=thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs_thequickbrownfoxjumpedoverthelazydogs

 

That gives the following error from squid:

 

HTTP/1.0 417 Expectation failed

Server: squid/2.7.STABLE3

Date: Fri, 16 Oct 2015 04:52:35 GMT

Content-Type: text/html

Content-Length: 1373

Expires: Fri, 16 Oct 2015 04:52:35 GMT

X-Squid-Error: ERR_INVALID_REQ 0

X-Cache: MISS from localhost

X-Cache-Lookup: NONE from localhost:3128

Via: 1.0 localhost:3128 (squid/2.7.STABLE3)

Connection: close

 

 

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">

<HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">

<TITLE>ERROR: The requested URL could not be retrieved</TITLE>

<STYLE type="text/css"><!--BODY{background-color:#ffffff;font-family:verdana,sans-serif}PRE{font-family:sans-serif}--></STYLE>

</HEAD><BODY>

<H1>ERROR</H1>

<H2>The requested URL could not be retrieved</H2>

<HR noshade size="1px">

<P>

While trying to process the request:

<PRE>

POST / HTTP/1.1

User-Agent: FileMaker/14.0

Host: sync.360works.com

Accept: */*

Accept-Encoding: deflate, gzip

Content-Length: 1031

Content-Type: application/x-www-form-urlencoded

Expect: 100-continue

 

 

</PRE>

<P>

The following error was encountered:

<UL>

<LI>

<STRONG>

Invalid Request

</STRONG>

</UL>

 

 

<P>

Some aspect of the HTTP Request is invalid.  Possible problems:

<UL>

<LI>Missing or unknown request method

<LI>Missing URL

<LI>Missing HTTP Identifier (HTTP/1.0)

<LI>Request is too large

<LI>Content-Length missing for POST or PUT requests

<LI>Illegal character in hostname; underscores are not allowed

</UL>

<P>Your cache administrator is <A HREF="mailto:webmaster">webmaster</A>.

 

 

<BR clear="all">

<HR noshade size="1px">

<ADDRESS>

Generated Fri, 16 Oct 2015 04:52:35 GMT by localhost (squid/2.7.STABLE3)

</ADDRESS>

</BODY></HTML>

 

 

If the URL is shortened so that the parameter length is < 1024 bytes, you get this Google message (which is the desired behavior):

 

<!DOCTYPE html>

<html lang=en>

  <meta charset=utf-8>

  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">

  <title>Error 405 (Method Not Allowed)!!1</title>

  <style>

    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}

  </style>

  <a href=//www.google.com/><span id=logo aria-label=Google></span></a>

  <p><b>405.</b> <ins>That’s an error.</ins>

  <p>The request method <code>POST</code> is inappropriate for the URL <code>/</code>.  <ins>That’s all we know.</ins>



Outcomes