PHP curl login to url with password and username and hidden token

Question

I'm having troubles to login on a site using curl that has an hidden input field with an token. Now i think its possible but i cant find an solution. Anyone an idea on how i can fix it?

$username = '2142019677';
$password = 'Vercautp1'; //no secrets here
        $link = 'https://www.autoscout24.be/dealer-statistics/api/listing/overview/d42ea89e-b717-4ab4-85ee-2b5e7bff959c';
        $html = file_get_contents($link);
        preg_match_all("'VerificationToken\" type=\"hidden\" value=\"(.*?)\"'si", $html, $match);
        $hidden = $match[1][0];

        preg_match_all("'action=\"/(.*?)\" method=\"post\"'si", $html, $match);
        $url = $match[1][0];

$path = "/ctemp";
$postinfo = "Username=".$username."&Password=".$password.'__RequestVerificationToken='.$hidden;

$cookie_file_path = $path."/cookie.txt";

$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, 'https://www.autoscout24.be'.$url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);

curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
    "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);

curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_exec($ch);

curl_setopt($ch, CURLOPT_URL, "https://www.autoscout24.be/dealer-statistics/api/listing/overview/d42ea89e-b717-4ab4-85ee-2b5e7bff959c");
$html = curl_exec($ch);
echo $html;
curl_close($ch);

Where come from `$html`? You need to send the same cookie used to get the HTML to send the POST request. — Syscall, Feb 24 '22 at 19:12
and how do i get the token that is hidden. thats my problem that i dont know how to get it in with curl. I am only used to work with file_get_contents :-( — Vercauteren Kevin, Feb 24 '22 at 19:25
Use cURL to get the HTML first. Then use a XML parser instead of preg_match to get the hidden value. Then, send POST using cURL with the cookie received from the first cURL call. — Syscall, Feb 24 '22 at 19:30
what do you think will happen if your password contains & ? you need ulencode() / http_build_query() ^^ — hanshenrik, Feb 24 '22 at 20:20

score 0 · Answer 1 · answered Feb 24 '22 at 19:35

0

Form data is being sent to the server:

So you need code that looks similar to this:

$ch = curl_init ( 'https://www.autoscout24.be/dealer-statistics/api/listing/overview/d42ea89e-b717-4ab4-85ee-2b5e7bff959c' );
curl_setopt_array ( $ch, array (
        CURLOPT_POST => 1,
        CURLOPT_POSTFIELDS => array (
           '__RequestVerificationToken` => 'Ikt8rjIySPHhjsx48MYwrHBDOjhEBdKDkGR6PIzkPHs9oDBfzO8M1O2ZEAlXt3ARCgJq-8UvjhUEMbAxZMffCOQTQ7AfTmxWK785T5TL18k1'
           'Username' => '2142019677',
           'Password' => 'Vercautp1',
           'RememberMeCheckBox' => 'on',
           'RememberMe' => true
        ) 
) );
curl_exec ( $ch );

The tricky part will be generating the __RequestVerificationToken which is likely a hash of the user name and password, but without analyzing the JS that creates this it's unknown how this is generated.

See also: https://stackoverflow.com/a/46872809/4323201

answered Feb 24 '22 at 19:35

RyanNerd

3,059
1
22
28

Thanks for the reply the Verification code is an completly random token and changed on every new connection. – Vercauteren Kevin Feb 24 '22 at 19:42
that won't work, the token is tied to the cookie session, you need to first create a cookie session, then fetch the verification code and parse it out from the html - try again ^^ – hanshenrik Feb 24 '22 at 20:20
also curl doesn't enable cookies by default, set `CURLOPT_COOKIEFILE` to emptystring to enable cookie support (and this login page, as almost all, use cookies) – hanshenrik Feb 24 '22 at 21:01
so first do a GET request to the login page, with cookies enabled, to fetch the token that is valid for your new cookie session, extract that from the HTML, then finally do your POST request – hanshenrik Feb 24 '22 at 21:02
and keep in mind, file_get_contents does *not* share cookies with curl_ , so you must do the html fetching *and* posting with curl. (that's 1 of the things OP does wrong, he does fetching with file_get_contents and post with curl, they have separate cookies and thus different tokens) – hanshenrik Feb 24 '22 at 21:03

score 0 · Answer 2 · answered Feb 24 '22 at 20:58

lots of issues here, where to start..

first, this won't work:

$link = 'https://www.autoscout24.be/dealer-statistics/api/listing/overview/d42ea89e-b717-4ab4-85ee-2b5e7bff959c';
$html = file_get_contents($link);
preg_match_all("'VerificationToken\" type=\"hidden\" value=\"(.*?)\"'si", $html, $match);

the verificationToken you're trying to extract here is tied to the cookie session recived by file_get_contents(), and file_get_contents does not share cookies with curl (in fact file_get_contents has no cookie support at all, practically speaking, but not literally speaking, elaborating would go off track here..)

so you must fetch it with curl, and curl must have cookies enabled when fetching it. also your regex extraction should be replaced with DOMDocument+DOMXPath extraction, but since you don't you must run the code you extract through html_entity_decode() , for example if your regex extract a&b then the token isn't a&b, it is a&b html-encoded (so you need to html-decode it)

also your $postinfo = "Username=".$username."&Password=".$password.'__RequestVerificationToken='.$hidden; will only work if there's no special characters in your username/password, for example if your password is Password&999, you must send it as Password%26999 - because & needs to be urlencoded, so you need to do it like

$postinfo = "Username=".urlencode($username)."&Password=".urlencode($password).'&__RequestVerificationToken='.urlencode($hidden);

or better yet

$postinfo = http_build_query(array(
    "Username" => $username,
    "Password" => $password,
    '__RequestVerificationToken' => $hidden
));

also do not use CURLOPT_CUSTOMREQUEST for POST, quoting the libcurl doc https://curl.se/libcurl/c/CURLOPT_CUSTOMREQUEST.html :

Many people have wrongly used this option to replace the entire request with their own, including multiple headers and POST contents. While that might work in many cases, it will cause libcurl to send invalid requests and it could possibly confuse the remote server badly. Use CURLOPT_POST and CURLOPT_POSTFIELDS to set POST data.

... and one last piece, i see you forgot to put an & in front of the __RequestVerificationToken - http_build_query would do that for you too

Thanks for the reply gonna try to make it work with you comments (wish me luck) ;-) — Vercauteren Kevin, Feb 24 '22 at 21:12

PHP curl login to url with password and username and hidden token

2 Answers2