0

Need some help with fetching some data from a website.

Previously , we had following code in our application and it used to fetch the required data. We just used to read the required fields by forming a URL by passing username , password and search parameter (DEA number). The same URL (with parameters ) could also be hit from browser directly to see the results. It was a simple GET request:

{URL url = new URL(
                    "http://www.deanumber.com/Websvc/deaWebsvc.asmx/GetQuery?UserName="+getUsername()+"&Password="+getPassword()+"&DEA="
                            + deaNumber
                            + "&BAC=&BASC=&ExpirationDate=&Company=&Zip=&State=&PI=&MaxRows=");

Document document = parser.parse(url.toExternalForm());
// Ask the document for a list of all <sect1> tags it contains
NodeList sections = document.getElementsByTagName("DEA");
//Followed by a loop code to get each element by using sections.item(index).getFirstChild() etc.
}

Now, the website URL has got changed to following:

https://www.deanumber.com/RelId/33637/ISvars/default/Home.htm

I am able to login to the URL with credentials , go to the search page , enter the DEA number and search. The login page comes as a pop-up once I click 'Login' link on home page. Also, the final result comes as a pop-up. This is a POST request so I am unable to form the complete URL which I could use in my code.

I am not an expert in Web Services , but I think I need a web service URL like the one mentioned in the code above. Not sure how to get that !! Even if I get the URL , I am not sure how to perform the login through Java code and search the DEA number.

Also, it would be great if I could validate the URL manually before using in Java. Let me know if there is any way.

Or, in case there is any alternate approach in Java; kindly suggest.

Thanks in advance.

Satej Koli
  • 79
  • 2
  • 15
  • Have you look at the page "WebService" from https://deanumber.com ? there seems to be a link to your requested information: https://www.deanumber.com/Websvc/deaWebsvc.asmx?op=GetQuery – Loic Mouchard Sep 28 '17 at 13:24
  • Thanks a lot Loic M . BTW, how did you reach that link? – Satej Koli Sep 28 '17 at 14:05
  • Simply going on the website and looking for any documentation... – Loic Mouchard Sep 29 '17 at 14:54
  • Thanks a lot @Loic !! Can you please help me with my revised question : https://stackoverflow.com/questions/46485704/exceptions-after-changing-a-webservice-url-from-http-to-https-java-net-protoco – Satej Koli Sep 29 '17 at 16:26

1 Answers1

1

First of all, the previous approach provided by the website was completely wrong and insecure, because it passes the username and password as querystring parameters in plain text. I think, they would have realized this thing and changed their way of authentication.

Also, it looks like that they have restricted the direct URL based requests from the client applications like yours. For such requests from clients, they have published the web services. Check this link. They also have mentioned the rates for web service request counts.

So, you may need to open a formal communication channel to get authentication and other details to access their web services for this purpose. Depends on what they use for web service client authentication, you may code your client to access the web services.

I hope this helps.

ACloudRoamer
  • 1,043
  • 2
  • 11
  • 19
  • Thanks a lot AJavaCoder . Your answer has given me a starting point. The link also has a sample code. I will start coding and post here in case I face more issues. BTW, how did you reach that link? – Satej Koli Sep 28 '17 at 13:45
  • You are welcome. I found an option named "Web Service" under "Products & Services" menu. Also, could you please mark my answer accepted if that is what you feel? – ACloudRoamer Sep 28 '17 at 16:30
  • Thanks again @AJavaCoder !! I found the webservice URL and also http get request on their code examples page. The URL is same as the one used in my code except the protocol. So, I just changed http to https in my code , but I am getting following exception: java.net.UnknownHostException: www.deanumber.com. The get URL is working when hit using a browser. – Satej Koli Sep 29 '17 at 08:18
  • I am marking you answer as accepted as I have got the required URL. I will ask a separate question for the unknown host issue. Please guide for the new question as well if possible :). Thanks. – Satej Koli Sep 29 '17 at 08:50
  • Link to my revised question : https://stackoverflow.com/questions/46485704/exceptions-after-changing-a-webservice-url-from-http-to-https-java-net-protoco – Satej Koli Sep 29 '17 at 11:39
  • OK. I will try to answer that as well. – ACloudRoamer Sep 29 '17 at 16:31