Sometimes the easiest way to reverse engineer these things is via careful analysis of each response that the server returns to the browser, i.e. the HTML and any http response headers.
In the easiest scenario:
One only needs to examine the form and input elements within the HTML. Using the names of the form elements, and knowing the proper values to supply, one can use the BE plugin to send HTTP request content to the server which simulates the user logging in.
If the HTML contained the following form:
<form action="http://myserver.com/some/url.php" method="post">
<input type="text" name="user_name"/>
<input type="password" name="secret_password"/>
<input type="submit" name="submit" value="Submit"/>
Then the http request content that you would send via the BE plugin would be:
MY_USERNAME is replaced with the url-encoded username value
- and -
MY_PASSWORD is replaced with the url-encoded password value
The above request would be sent via http post to the action url ( in this example: http://myserver.com/some/url.php ).
Some advanced considerations:
1) Oftentimes http cookies are being passed back and forth (between the server and the browser). This is done via http headers. You will want to make sure to properly capture and send any needed cookie values.
Reference for HTTP cookies: http://en.wikipedia.org/wiki/HTTP_cookie
2) Login processes can sometimes be a sequence of multiple client/server exchanges. These sequences can be invisible to the general browser user, because they are driven by redirect commands sent by the server of which the user is generally unaware. If the server returns a status of 302 "found", as a response to your http request, just keep in mind that this probably means that the server is redirecting you to a new url (specified in a header).
Lastly -- A caution:
All of the effort put into this sort of endeavor can be lost and the system can suddenly break if there comes a time when whoever maintains the target web page updates/modifies their server code. As I believe you know, the safest approach is to see if there is a maintained API that you could use to meet your needs -- something that you can rely upon to be consistent in terms of the interface.
HTH & Best,
Message was edited by: steve_ssh
( Edited message to try to better answer the question/concern. )
In general, I tend to replicate every call and get it working, then try to cut out unneccesary calls to images, etc (I've seen calls to images that are redirected to scripts that set cookies then return the image) one at a time.