Using python to login to a webpage using a cgi script

Question

Logging into web servers is not my area of expertise, and I am looking to automate a task of logging into a web server, but I am not sure how I need to format my code to achieve this goal. I have looked at other pages, but no matter what I did I got the same response back.

web page snippets:

  <form name="form1" action="login.cgi" method="POST" onsubmit="encrypt();">
    <input type="hidden" name="Token" value="16">
    <tr height="15">
      <td><img src="/images/spacer.gif" alt="" height="15"></td>
    </tr>
    <tr height="32">
      <td valign="top">
        <table width="100%" border="0" cellspacing="0" cellpadding="0">
          <tr>
            <td width="19"><img src="/images/spacer.gif" alt="" width="19"></td>
            <td width="100%"> </td>
          </tr>
        </table>
      </td>
    </tr>
    <tr height="12">
      <td><img src="/images/spacer.gif" alt="" height="12"></td>
    </tr>
    <tr>
      <td height="1" valign="top">
        <table border="0" cellspacing="0" cellpadding="0">
          <tr>
            <td width="20"><img src="/images/spacer.gif" alt="" width="20"></td>
            <td valign="top">
              <table border="0" cellspacing="0" cellpadding="0">
                <tr>
                  <td><img src="/images/spacer.gif" alt="" width="10" height="8"></td>
                  <td><img src="/images/spacer.gif" alt="" width="400" height="8"></td>
                </tr>
                <tr>
                  <td nowrap>
                    <b>Login:</b>
                  </td>
                  <td>
                    <input name="userid_w" type="text" size="50">
                    <input name="userid" type="hidden" value=""><img src="/images/spacer.gif" alt="" width="10"></td>
                </tr>
                <tr>
                  <td colspan="2" height="3"><img src="/images/spacer.gif" alt="" height="3"></td>
                </tr>
                <tr>
                  <td nowrap>
                    <b>Login Password:</b>
                  </td>
                  <td>
                    <input name="password_w" type="password" size="50">
                    <input name="password" type="hidden" value="">
                  </td>
                </tr>
                <tr height="3">
                  <td colspan="2"><img src="/images/spacer.gif" alt="" height="3"></td>
                </tr>
                <tr>
                  <td></td>
                  <td>
                    <input type="submit" value="Login">

....

    <input type="hidden" name="open" value="">
  </form>
</table>

The response I am getting from the server, no matter what userid_w or password_w value I give, returns:

<html>  <head></head>  <body onload='document.form1.submit()'>    <form name='form1' method='POST' action='message.cgi' target='_top'>      <input type='hidden' name='title' value='MSG_TTL_COOKIEOFF'/>      <input type='hidden' name='messageID' value='MSG_COOKIEOFF'/>      <input type='hidden' name='buttonURL' value='Frame.cgi'/>      <input type='hidden' name='screen' value='all'/>   </form>  </body></html>

Any help would be appreciated.

Edit: Running curl -l on the page gives:

<html>  <head></head>  <body onload='document.form1.submit()'>    <form name='form1' method='POST' action='/web/guest/en/websys/webArch/message.cgi' target='_top'>      <input type='hidden' name='title' value='MSG_TTL_COOKIEOFF'/>      <input type='hidden' name='messageID' value='MSG_COOKIEOFF'/>      <input type='hidden' name='buttonURL' value='Frame.cgi'/>      <input type='hidden' name='screen' value='all'/>  </form>  </body></html

Which looks like what my pythons script outputs...

What is it you tried? If you show some code, it's a lot easier to help. — Thomas Fenzl, May 17 '13 at 05:46
modified this code: http://stackoverflow.com/questions/13925983/login-to-website-using-urllib2-python-2-7 , changed the user, passwd, and op field to userid_w, password_w. — Jim, May 17 '13 at 05:53
I did at some point, but it was not changing the response, so I figured my error was elsewhere.... — Jim, May 17 '13 at 06:01
it could be a one-time-token, so I suggest you get the login page, read the token and send it. What status code do you get? It's hard to help with so little information — Thomas Fenzl, May 17 '13 at 06:08
when I used page = urllib2.urlopen(url) the page, I guess because it is a cgi script, the only thing returned on page.read() is essentially what I posted above, with the difference being it gives me back a different .cgi location, but I dont see the token value in there. — Jim, May 17 '13 at 06:24
would you please show us all the headers (eg. by hitting the login page from curl command like eg. curl -I http://example.com/login/) also are you sure there is no JS code. — Muayyad Alsadi, May 20 '13 at 06:30
added the curl output, looks like the same output as what my script was giving back.... — Jim, May 20 '13 at 06:43

score 5 · Accepted Answer · answered May 20 '13 at 08:00

Looking at <form name="form1" action="login.cgi" method="POST" onsubmit="encrypt();"> one could assume data is modified by a javascript snippet before being sent to the login.cgi. If that is the case, you have two options to follow:

Use selenium to pass credentials and proceed with login, then get the session cookie and copy it over to the http client of your choices (e.g. requests)
Run a javascript interpreter (e.g. python-spidermonkey) using provided credentials to get the real POST values that get sent, and use them to make a call with http client of your choice

Using python to login to a webpage using a cgi script

1 Answers1