curl login to website and then navigate trough different links while logged in

Question

I'm using curl to login to a website:

$credentials = [
    'user' => 'username',
    'password' => 'passowrd',
];

curl_setopt($ch, CURLOPT_URL, 'https://shop.com/login.php');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($credentials));

$response = curl_exec($ch);

It works to log in but once i'm logged in i need to go trough a few urls and crawl some data(which is available only after log in):

$products = ['1', '2', '3'];
foreach ($products as $id) {
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, 'https://shop.com/product.php?id=' . $id);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_MAXREDIRS, 1);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    $response = curl_exec($ch);

    //do something with that response

    curl_close($ch);
}

Is there a way to maintain that session started in the first curl request so i can use it in the foreach loop - so i can crawl my data?

Thank you!

[CURLOPT_COOKIEFILE](https://stackoverflow.com/questions/13020404/keeping-session-alive-with-curl-and-php) ? — Madhan Varadhodiyil, Aug 22 '18 at 11:15
You could use [Guzzle](https://github.com/guzzle/guzzle) and pass `['cookies' => true]` to your `Client` constructor. It will automatically maintain session/cookies and is generally much easier to use than curl. — Tobias K., Aug 22 '18 at 17:54

score 0 · Answer 1 · answered Aug 22 '18 at 11:29

in your first request, you should save the cookie in a variable and use it in other requests. like this :

$credentials = [
    'user' => 'username',
    'password' => 'passowrd',
];

curl_setopt($ch, CURLOPT_URL, 'https://shop.com/login.php');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($credentials));

$response = curl_exec($ch);

get cookie and save in in $cookie variable :

prog_match('/^Set-Cookie:\s*([^;]*)/mi',$response,$m);
parse_str($m[1],$cookie);

and in other request you can use it like this:

$products = ['1', '2', '3'];
foreach ($products as $id) {
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, 'https://shop.com/product.php?id=' . $id);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_MAXREDIRS, 1);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_COOKIE, 'PHPSESSID='.$cookie['PHPSESSID']);
    $response = curl_exec($ch);

    //do something with that response

    curl_close($ch);
}

Hey @MohammadRajabloo, can you please tell me why do i have to set CURLOPT_HEADER? I've read the documentation but i don't get it :( — emma, Aug 22 '18 at 14:45
@emma. because we need to get the cookie from response header. set-cookie row contains cookie. — Mohammad Rajabloo, Aug 23 '18 at 15:42

curl login to website and then navigate trough different links while logged in

1 Answers1