401 error after successful login using browser

Question

Using my browser I point to a URL and I am prompted with a username/password dialog. I enter my username/password and I get my webpage.

I get a 401 error, however, when using curl:

curl --anyauth --user "$USERNAME:$PASSWORD" $URL

wget:

wget --http-user=$USERNAME --http-password=$PASSWORD $URL

Python:

response = requests.get(url, auth=requests.auth.HTTPBasicAuth(username, password))
response = requests.get(url, auth=requests.auth.HTTPDigestAuth(username, password))

The verbose (sanitized) output is below for curl:

* About to connect() to application.intranet.net port 443 (#0)
*   Trying 10.10.10.139...
* Connected to application.intranet.net (10.10.10.139) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
*       subject: CN=application.intranet.net,OU=COMPANY - Web Hosting,O=Com Pany Inc.,STREET=address,L=city,ST=state,postalCode=12345,C=US
*       start date: Apr 06 00:00:00 2020 GMT
*       expire date: Apr 06 23:59:59 2022 GMT
*       common name: application.intranet.net
*       issuer: CN=COMODO RSA Organization Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB
> GET /appname/Reporting/ReportListStart.aspx HTTP/1.1
> User-Agent: curl/7.29.0
> Host: application.intranet.net
> Accept: */*
> 
< HTTP/1.1 401 Unauthorized
< Cache-Control: private
< Content-Type: text/html
< Server: application Server
< WWW-Authenticate: Negotiate
< WWW-Authenticate: NTLM
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Date: Wed, 23 Dec 2020 16:17:22 GMT
< Content-Length: 1293
< 
* Ignoring the response-body
* Connection #0 to host application.intranet.net left intact
* Issue another request to this URL: 'https://application.intranet.net/appname/Reporting/ReportListStart.aspx'
* Found bundle for host application.intranet.net: 0x1f4b050
* Re-using existing connection! (#0) with host application.intranet.net
* Connected to application.intranet.net (10.10.10.139) port 443 (#0)
> GET /appname/Reporting/ReportListStart.aspx HTTP/1.1
> User-Agent: curl/7.29.0
> Host: application.intranet.net
> Accept: */*
> 
< HTTP/1.1 401 Unauthorized
< Cache-Control: private
< Content-Type: text/html
< Server: application Server
* gss_init_sec_context() failed: : No Kerberos credentials available (default cache: KEYRING:persistent:9013)
< WWW-Authenticate: Negotiate
< WWW-Authenticate: NTLM
< X-Frame-Options: SAMEORIGIN
< X-Content-Type-Options: nosniff
< Date: Wed, 23 Dec 2020 16:17:22 GMT
< Content-Length: 1293
< 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>
<title>401 - Unauthorized: Access is denied due to invalid credentials.</title>
<style type="text/css">
<!--
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}
fieldset{padding:0 15px 10px 15px;} 
h1{font-size:2.4em;margin:0;color:#FFF;}
h2{font-size:1.7em;margin:0;color:#CC0000;} 
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;} 
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;
background-color:#555555;}
#content{margin:0 0 0 2%;position:relative;}
.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}
-->
</style>
</head>
<body>
<div id="header"><h1>Server Error</h1></div>
<div id="content">
 <div class="content-container"><fieldset>
  <h2>401 - Unauthorized: Access is denied due to invalid credentials.</h2>
  <h3>You do not have permission to view this directory or page using the credentials that you supplied.</h3>
 </fieldset></div>
</div>
</body>
</html>
* Connection #0 to host application.intranet.net left intact

wget:

--2020-12-23 11:18:14--  https://application.intranet.net/appname/Reporting/ReportListStart.aspx
Resolving application.intranet.net (application.intranet.net)... 10.10.10.139, 10.10.10.10
Connecting to application.intranet.net (application.intranet.net)|10.10.10.139|:443... connected.
HTTP request sent, awaiting response... 401 Unauthorized
Reusing existing connection to application.intranet.net:443.
HTTP request sent, awaiting response... 401 Unauthorized
Reusing existing connection to application.intranet.net:443.
HTTP request sent, awaiting response... 401 Unauthorized
Authorization failed.

Python:

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): application.intranet.net:443
send: b'GET /appname/Reporting/ReportListStart.aspx HTTP/1.1\r\nHost: application.intranet.net\r\nUser-Agent: python-requests/2.25.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nAuthorization: Basic U19KaXJhX0ludGVybmFsQXVkaXQ6R2l4X0lLdzFqTEYtMld0cw==\r\n\r\n'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Cache-Control: private
header: Content-Type: text/html
header: Server: application Server
header: WWW-Authenticate: Negotiate
header: WWW-Authenticate: NTLM
header: X-Frame-Options: SAMEORIGIN
header: X-Content-Type-Options: nosniff
header: Date: Wed, 23 Dec 2020 17:01:10 GMT
header: Content-Length: 1293
DEBUG:urllib3.connectionpool:https://application.intranet.net:443 "GET /appname/Reporting/ReportListStart.aspx HTTP/1.1" 401 1293
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): application.intranet.net:443
send: b'GET /appname/Reporting/ReportListStart.aspx HTTP/1.1\r\nHost: application.intranet.net\r\nUser-Agent: python-requests/2.25.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Cache-Control: private
header: Content-Type: text/html
header: Server: application Server
header: WWW-Authenticate: Negotiate
header: WWW-Authenticate: NTLM
header: X-Frame-Options: SAMEORIGIN
header: X-Content-Type-Options: nosniff
header: Date: Wed, 23 Dec 2020 17:01:10 GMT
header: Content-Length: 1293
DEBUG:urllib3.connectionpool:https://application.intranet.net:443 "GET /appname/Reporting/ReportListStart.aspx HTTP/1.1" 401 1293

From my browser there is the initial request that returns a 302:

Request URL: https://application.wuintranet.net/appname/Reporting/ReportListStart.aspx
Request Method: GET
Status Code: 302 Found
Remote Address: 10.10.10.123:443
Referrer Policy: strict-origin-when-cross-origin
Cache-Control: private
Content-Length: 160
Content-Type: text/html; charset=utf-8
Date: Wed, 23 Dec 2020 17:14:54 GMT
Location: /appname/Reporting/ReportListStart.aspx
Persistent-Auth: true
Server: application Server
Set-Cookie: ASP.NET_SessionId=dy2rr35onasw5ctumhuqb4af; path=/; secure; HttpOnly; SameSite=Lax
Set-Cookie: appname_Cookie=ConnectionTitle=DELwLGx+KbrtS0gKvmretg==&IsConnectionTitleSet=True&IsLogOff=False&CurrentOrganization=ELx658BVmiesDFQg7w5RtA==&IsOrganizationRequired=YBfC/taoB3Ll19UPqF9IEA==; path=/; secure; HttpOnly
Set-Cookie: .application_SSO_Cookie=ConnectionTitle=DELwLGx+KbrtS0gKvmretg==&IsConnectionTitleSet=True&IsLogOff=True&CurrentOrganization=ELx658BVmiesDFQg7w5RtA==&IsOrganizationRequired=YBfC/taoB3Ll19UPqF9IEA==; path=/; secure; HttpOnly
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Host: application.wuintranet.net
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36

and then the browser-generated followup that returns 200:

Request URL: https://application.wuintranet.net/appname/Reporting/ReportListStart.aspx
Request Method: GET
Status Code: 200 OK
Remote Address: 10.10.10.123:443
Referrer Policy: strict-origin-when-cross-origin
Cache-Control: private
Content-Encoding: gzip
Content-Length: 32914
Content-Type: text/html; charset=utf-8
Date: Wed, 23 Dec 2020 17:14:54 GMT
Persistent-Auth: true
Server: application Server
Set-Cookie: appname_Cookie=ConnectionTitle=DELwLGx+KbrtS0gKvmretg==&IsConnectionTitleSet=True&IsLogOff=False&CurrentOrganization=ELx658BVmiesDFQg7w5RtA==&IsOrganizationRequired=YBfC/taoB3Ll19UPqF9IEA==; path=/; secure; HttpOnly
Vary: Accept-Encoding
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cache-Control: max-age=0
Connection: keep-alive
Cookie: ASP.NET_SessionId=dy2rr35onasw5ctumhuqb4af; appname_Cookie=ConnectionTitle=DELwLGx+KbrtS0gKvmretg==&IsConnectionTitleSet=True&IsLogOff=False&CurrentOrganization=ELx658BVmiesDFQg7w5RtA==&IsOrganizationRequired=YBfC/taoB3Ll19UPqF9IEA==; .application_SSO_Cookie=ConnectionTitle=DELwLGx+KbrtS0gKvmretg==&IsConnectionTitleSet=True&IsLogOff=True&CurrentOrganization=ELx658BVmiesDFQg7w5RtA==&IsOrganizationRequired=YBfC/taoB3Ll19UPqF9IEA==
Host: application.wuintranet.net
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36

Please [edit] your question to include the actual HTTP requests you send and the HTTP responses you get when you use all four connection approaches (browser, curl, wget and python). — Progman, Dec 23 '20 at 15:24
Did not know what you meant, Progman. Had to ask a friend. Now I understand. For others reading this (on Chrome at least) one can inspect network traffic via the developer tools (https://developers.google.com/web/tools/chrome-devtools/network). wget by default is verbose ... could not figure out a way to make it more verbose. curl has a "--verbose" option. For Python I captured network details by following the instructions at https://stackoverflow.com/questions/10588644/how-can-i-see-the-entire-http-request-thats-being-sent-by-my-python-application. — jsf80238, Dec 23 '20 at 18:35
It is weird that not all shown requests have the `Authorization: Basic <...>` header, even though the correct parameters are used. Also, it looks like the HTTP request from the browsers are missing. — Progman, Dec 23 '20 at 18:45
SPNego covers both Kerberos (strong) and NTLM (not so strong) authentication. In this case the server requests explicitly NTLM. But `curl` ignores that request and tries Kerberos auth. _Question:_ did you check whether this version / build of `curl` supports NTLM? It's an old Microsoft thing, some Open Source solutions simply don't want to mess with that... — Samson Scharfrichter, Dec 26 '20 at 10:07
In other words, what happens if you replace `curl --anyauth` with an explicit `curl --ntlm`? cf. https://curl.se/docs/manpage.html#--ntlm — Samson Scharfrichter, Dec 26 '20 at 10:10
The --ntlm option definitely has an impact. I'm getting a 401, then a 302. I tried using the --location option to get past the 302, but it just keeps redirecting. I'm inclined at this point, @Samson, to mark this post as solved and open a new one for the 302. Thank you. If you want to add your comment as a solution I will mark it as correct. — jsf80238, Dec 27 '20 at 17:05
New, related post: https://stackoverflow.com/questions/65468536/how-to-get-curl-to-narrow-to-correct-redirect — jsf80238, Dec 27 '20 at 17:36

score 1 · Answer 1 · answered Dec 31 '20 at 16:30

1

replace

curl --anyauth

with

curl --ntlm

answered Dec 31 '20 at 16:30

jsf80238

1,577
2
11
24

401 error after successful login using browser

1 Answers1