0

I have a device Stiebel Eltron heat pump home and I would like to use Linux shell curl (not php) curl to login (POST) and retrieve (GET) data once logged

Here is how my curl login POST call looks like (I used Firebug + persist option to copy/paste the below):

curl --data-urlencode 'userName=tutu&password=xx' \
'https://thesite/api/login?noCacheDummyValue=1459356436185' -X POST \
 -H 'Host: thesite' \
 -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0' \
 -H 'Accept: application/json, text/plain, */*' \
 -H 'Accept-Language: en-US,en;q=0.7,fr-FR;q=0.3' \
 -H 'Accept-Encoding: gzip, deflate, br' \
 -H 'X-Requested-With: XMLHttpRequest' \
 -H 'Content-Type: application/json;charset=utf-8' \
 -H 'Referer: https://thesite/mobile/app/app.html' \
 -H 'Content-Length: 44' \
 -H 'Cookie: JSESSIONID=qhs02mfeip2p1n5n4t2rj1huu'

But this gives me nothing in the output:

 sh myproj.sh 
curl: (52) Empty reply from server

Q1: What should I collect from this first POST, and how to do it?

I tried to get the cookie with --cookie-jar cookie.txt but nothing showed up in my current directory. However I sent cookies with the header... shouldn't I get them back?

After that I would like to continue the session I (think I) opened so to collect data from a GET that is (same method as above):

curl 'https://thesite/api/data/1036493/heatEnergy/today?noCacheDummyValue=1459356438440' \
     -H 'Host: thesite' \
     -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0' \
     -H 'Accept: application/json, text/plain, */*' \
     -H 'Accept-Language: en-US,en;q=0.7,fr-FR;q=0.3' \
     -H 'Accept-Encoding: gzip, deflate, br' \
     -H 'X-Requested-With: XMLHttpRequest' \
     -H 'Referer: https://thesite/mobile/dashboard/dash.html' \
     -H 'Cookie: JSESSIONID=qhs02mfeip2p1n5n4t2rj1huu'

Q2: How do I keep the session opened to perform the GETabove?

I tried --next option idea from here to call the GET after the POST; without success, --next option is not recognized on my Ubuntu Linux.

If it can help understanding, the page I should get is full of javascript (angularjs I think).

Cœur
  • 37,241
  • 25
  • 195
  • 267
J. Chomel
  • 8,193
  • 15
  • 41
  • 69
  • 1
    I did a similar thing back in the day where I wrote a script that connects to my Scottrade account and display my profit or loss in my tmux status bar. There was a lot of trial and error and inspection of the requests with Chrome dev tools and right click copy as Curl. (What you are doing already). I think the reply you expect after the POST is just a cookie that you can pass along with the following GET request. I also had to use -L for follow redirect. Sorry can't help much without a working request. – neric Mar 30 '16 at 18:13
  • I tried with `-L`, it is accepted by `curl`, but I only get another `curl: (52) Empty reply from server` . – J. Chomel Apr 03 '16 at 07:58
  • 1
    Re Q1, for more diagnostic output add **--trace-ascii debugdump.txt** to the **curl** command. – agc Apr 04 '16 at 06:51
  • I tried with `--trace-ascii`, but didn't get any clue. I saw on http://stackoverflow.com/a/10634492/6019417 (c'mon, don't dup me guys) that `curl` doesn't do `javascript`:(. Is there a way for me to know if cookies are set with header or javascript? I see cookies in header, however... – J. Chomel Apr 04 '16 at 20:18

2 Answers2

3

From your POST call it is obvious that you are trying to do an ajax XMLHttpRequest. Likely this is expecting json encoded data. However you are providing url-encoded data.

Probably you should try using something along 

... -X POST -H "Content-Type: application/json" -d '{"userName":"tutu","password":"xx"}' 'https://thesite/api/login?noCacheDummyValue=1459356436185'

To make the login work, you need to match the way the server is maintaining the session. If session info is referred to via a cookie yo may just extract the cookie from the POST call (as you said you tried already, but on an error related response)

If the server does not use a straight way of referencing the session you might not succeed in getting your GET call working. If you are lucky maybe you then just need to parse a string from the page (fragment) or json data returned.

So monitor what a working handshake (e.g using a browser) is exchanging over the wire. And try identifing the important pieces of data. Then, put those together to form your GET call.

rpy
  • 3,953
  • 2
  • 20
  • 31
  • This is what I could not figure out alone. Thanks. I will write a full answer in case someone has same device and want to play with it. – J. Chomel Apr 06 '16 at 20:01
0

Thanks to rpy, I was able to do what I wanted:

Collect the session cookie:

curl -a -X POST -d '{"userName":"tutu","password":"xx"}'                  \
 --trace-ascii debugdump.txt --cookie-jar cookies.txt                     \
-H 'Content-Type: application/json;charset=utf-8'                         \
-H 'Referer: https://thesite/mobile/app/app.html'                         \
   'https://thesite/api/login?noCacheDummyValue=1459356436185' -o my.post
  • todo: check result is "ok" in my.post
  • todo: get content of cookies.txt into ${COOKIE}

Use it to retrieve the raw data I wanted:

curl -a \
-H 'Referer: https://thesite/mobile/dashboard/dash.html'   \
-H 'Cookie:JSESSIONID='${COOKIE}                           \
   'https://thesite/api/data/1036493/outTemp/lastMonth?noCacheDummyValue=1459972646687'   \
   -o lm_outTemp.get \
-L 'https://thesite/api/data/1036493/heatEnergy/lastMonth?noCacheDummyValue=1459972646694' \
   -o lm_heatEnergy.get

And I already started working on how to process the data here: Firefox - Collecting the data used by LAB.min.js

Community
  • 1
  • 1
J. Chomel
  • 8,193
  • 15
  • 41
  • 69