0

I'm trying to scrape this Website using RSelenium. Sending text to other elements works but the login field drives me nuts. I think it might be because it is in Java (at least the css selectors in SelectorGadget say .j_username & .j_password) but I can't seem to figure out how to deal with it.

I wouldn't mind logging in manually but I also can't figure out how that works =).

remDr <- RSelenium::remoteDriver(remoteServerAddr = "localhost",
                                     port = 4445L,
                                     browserName = "chrome")
    remDr$open()

remDr$navigate("https://smd.ch/SMDView/log/index.jsp")
login_name <- remDr$findElement(using = "xpath", "//*[contains(concat( ' ', @class, ' ' ), concat( ' ', 'j_username', ' ' ))]")
login_name$clickElement()
login_name$clearElement()
login_name$clickElement()
login_name$sendKeysToElement(list("xxxxxxx"))
login_pw <- remDr$findElement(using = "xpath", "//*[contains(concat( ' ', @class, ' ' ), concat( ' ', 'j_password', ' ' ))]")
login_pw$click()
login_pw$clearElement()
login_name$click()
login_pw$sendKeysToElement(list("xxxxxxx", "\uE007"))
remDr$screenshot(display = TRUE)
Nico Saameli
  • 321
  • 2
  • 11

3 Answers3

1

The page contains multiple logins, at least one for mobile and one for desktop. Depending on the simulated screen size / user agent only one login is visible the other is invisible. To get the login for desktop use the following xpath expressions:

//*/div[@class='login_screen']//*[contains(@class,'j_username')
//*/div[@class='login_screen']//*[contains(@class,'j_password')

General tip for testing xPath expressions: Open the web page you want to scrape in Google Chrome, open the developer tools, switch to Console tab. Then enter: $x("XPATH"), replace XPATH with your XPath expression you want to test and hit enter. Chrome will show you the results for the active web page and you can even click on them and Chrome will highligh the results or not (if they are invisible). Testing XPath expressions this way, before you use them in code saves a lot of time ;-)

rmunge
  • 3,653
  • 5
  • 19
1

It Seems xpath problem.Try following xpath for username and password.

//div[@class='login_screen']//input[@name='j_username']

//div[@class='login_screen']//input[@name='j_password']
KunduK
  • 32,888
  • 5
  • 17
  • 41
1

Seems you were pretty close enough. Possibly the Locator Strategies which you were using identifies multiple elements and hence invoking clearElement(), clickElement() and sendKeysToElement() was trying to interact with the first matching element within

<div class="login_mobile">

which isn't your desired element. Instead you want to interact with the elements within:

<div class="login_screen">

Solution

To interact with the login fields you can use either of the following Locator Strategies:

  • cssSelector

    • Benutzername field:

      div.login_screen input.inputfield.j_username[name='j_username']
      
    • Passwort field:

      div.login_screen input.inputfield.j_password[name='j_password']
      
  • xpath :

    • Benutzername field:

      //div[@class='login_screen']//input[@class='inputfield j_username' and @name='j_username']
      
    • Passwort field:

      //div[@class='login_screen']//input[@class='inputfield j_password' and @name='j_password']
      
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352