2

Im writing a python 2.7 script using selenium webdriver on OS X to login to Yahoo fantasy sports and automate some actions.

The script works fine with webDriver Firefox and Chromedriver. I've recently started using the PhantomJS (GhostDriver) and I've found I can't get the PhantomJS Selenium Driver (GhostDriver) to log into Yahoo login forms.

#!/usr/bin/python
import time
from selenium import webdriver
from selenium.webdriver import PhantomJS
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from sys import argv
import click

@click.command()
@click.option('--days', type=int, prompt='Number of days to set active lineup', help='Number of days to set active lineup')
@click.option('--username', prompt='Your Yahoo username:', help='Your Yahoo account username')
@click.option('--password', prompt='Your Yahoo passwordname:', help='Your Yahoo account password')
def start_active_players(days, username, password):
    """Simple python program that sets your active players for the next number DAYS."""
    print("Logging in as: " + username)

    dcap = DesiredCapabilities.PHANTOMJS.copy()
    dcap['javascriptEnabled'] = True 
    dcap['browserConnectionEnabled'] = True 
    dcap['acceptSslCerts'] = True
    dcap['localToRemoteUrlAccessEnabled'] = True 
    dcap['webSecurityEnabled'] = True 
    dcap['version'] = ''


    driver = webdriver.PhantomJS(executable_path='/Users/devin.mancuso/node_modules/phantomjs/bin/phantomjs', desired_capabilities=dcap)

    driver.get('https://login.yahoo.com/config/login?.src=spt&.intl=us&.done=http%3A%2F%2Fbasketball.fantasysports.yahoo.com%2Fnba')

    with open('jquery-2.1.3.min.js', 'r') as jquery_js: jquery = jquery_js.read() #read the jquery from a file
    driver.execute_script(jquery) #active the jquery lib

    driver.find_element_by_id('login-username').send_keys(username) 
    driver.find_element_by_id('login-passwd').send_keys(password)
    driver.implicitly_wait(8) # 8 seconds
    driver.find_element_by_name('signin').click()
    #form1 = driver.find_element_by_id('mbr-login-form')
    #form1.submit()
    driver.implicitly_wait(8) # 8 seconds
    driver.save_screenshot('screenshot.png')

    driver.find_element_by_xpath("//a[text() = 'My Team ']").click()
    driver.implicitly_wait(8) # 8 seconds

    for x in range(0, days):

        driver.find_element_by_xpath("//a[text() = 'Start Active Players']").click()
        driver.implicitly_wait(2) # 2 seconds
        date_text = driver.find_element_by_xpath("//span[@class='flyout-title']").text
        print("Starting active players for: " + date_text)
        driver.find_element_by_xpath("//a[contains(@class, 'Js-next')]").click()
        driver.implicitly_wait(2) # 2 seconds

    driver.quit()

if __name__ == '__main__':
    start_active_players()

The script fails on line 47,

driver.find_element_by_xpath("//a[text() = 'My Team ']").click()

when it attemps to find the link with the text My Team. A screenshot dump shows that it never makes it past the login form. An on-screen error message above the form states

Please reload the page and try again or use another browser

I saw in this post and thus included the execute_script command to load in Jquery locally, but that didn't solve it. I'm not sure if it's a Yahoo security issue that is stopping PhantomJS. But why would it only fail on a headless browser and not FF or Chrome?

I also found this question and have tried to submit the form itself instead of clicking the button, but that made no difference. I've commented out the code in the example above.

PhantomJS version: 2.0.0

Community
  • 1
  • 1
Devin
  • 205
  • 2
  • 13
  • Maybe try using the Firefox or Chrome driver, see if that works, and then you'll know if it's a browser-specific problem? – Andrew Magee Feb 16 '15 at 06:28
  • Code executes with no issues using the webDriver (FF) and ChomeDriver (Chrome) drivers in Selenium. It seems Yahoo has an issue with how the PhantomJS driver is attempting to submit the form. – Devin Feb 16 '15 at 06:31
  • 1
    Maybe try fiddling with the `userAgent`? http://phantomjs.org/api/webpage/property/settings.html – Andrew Magee Feb 16 '15 at 06:32
  • I'm not 100% sure, but I think that only works when you are using PhantomJS by itself, as opposed to the PhantomJS ghostdriver driver for Selenium. Here is the [ghostdriver source](https://github.com/ariya/phantomjs/blob/056aa50c19b31de967ec7c30e59c6cb16680e48d/src/ghostdriver/session.js) and the [Selenium PhantomJS desired capabilities](https://selenium.googlecode.com/svn/trunk/docs/api/py/webdriver/selenium.webdriver.common.desired_capabilities.html?highlight=desired#selenium.webdriver.common.desired_capabilities.DesiredCapabilities), and it seems userAgent isn't in either. – Devin Feb 16 '15 at 06:43
  • 1
    I'm not sure either but from looking at the source you might be able to put `'phantomjs.page.settings.userAgent': '...'` in your `desiredCapabilities`? (Looking at lines 114-124.) – Andrew Magee Feb 16 '15 at 06:53
  • I have added `dcap['phantomjs.settings.userAgent'] = 'Chrome/37.0.2062.120'` to the desired capabilities but it does not seem to have made a difference. hmm. – Devin Feb 16 '15 at 06:59
  • Sorry you must have done that before I edited my comment which was initially wrong, could you try again? – Andrew Magee Feb 16 '15 at 07:00
  • Interesting. Adding the user agent produced this error `selenium.common.exceptions.WebDriverException: Message: {"errorMessage":"'undefined' is not an object (evaluating '(C(a)?xb(C(a)):window).getComputedStyle(a,null).MozTransform.match')"` for this line `driver.find_element_by_id('login-passwd').send_keys(password)` - maybe it doesn't like the UA being declared? – Devin Feb 16 '15 at 07:06
  • Ok initially out of the box, python bindings don't support setting that UA. But seems someone from selenium has posted a [solution](https://github.com/detro/ghostdriver/issues/134). Will update question with own answer – Devin Feb 16 '15 at 07:16
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/71005/discussion-between-andrew-magee-and-devin). – Andrew Magee Feb 16 '15 at 09:40

1 Answers1

5

Solution was to set the PhantomJS userAgent using the python bindings. Discovered through Andrew Magee's recommendations in comments and via this conversation on the ghostdriver github.

DesiredCapabilities.PHANTOMJS['phantomjs.page.settings.userAgent'] = 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:16.0) Gecko/20121026 Firefox/16.0'

driver = webdriver.PhantomJS(executable_path='/Users/devin.mancuso/node_modules/phantomjs/bin/phantomjs')
Devin
  • 205
  • 2
  • 13