Dynamically Generated Element -nosuchelementexception: Message: No Such Element: Unable To Locate Element?

I am having trouble selecting a load more button on a Linkedin page. I receive this error in finding the xpath: selenium.common.exceptions.NoSuchElementException: Message: no such

Solution 1:

After playing around with it, I seem to have figured out where the problem is stemming from. The error

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//button[@class="pv-profile-section__card-action-bar pv-skills-section__additional-skills artdeco-container-card-action-bar artdeco-button artdeco-button--tertiary artdeco-button--3 artdeco-button--fluid"]"} (Session info: chrome=81.0.4044.113)

always correctly states the problem its encountering and as such it's not able to find the element. The possible causes of this include:

• Element not present at the time of execution
• Dynamically generated
• content Conflicting names

In your case, it was the second point. As the content that is displayed is loaded dynamically as you scroll down. So When it first loads your profile the skills sections aren't actually present in the DOM. So to solve this, you simply have to scroll to the section so that it gets applied in the DOM.

This line is the trick here. It will position it to the correct panel and thus loading and applying the data to the DOM.

driver.execute_script("window.scrollTo(0, 1800)")

Here's my code (Please change it as necessary)

from time import sleep

# import parametersfrom selenium.webdriver.common.action_chains import ActionChains
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait

ChromeOptions = webdriver.ChromeOptions()
driver = webdriver.Chrome('../chromedriver.exe')

driver.get('https://www.linkedin.com/login?fromSignIn=true&trk=guest_homepage-basic_nav-header-signin')
sleep(0.5)

username = driver.find_element_by_name('session_key')

username.send_keys('')
sleep(0.5)

password = driver.find_element_by_name('session_password')
password.send_keys('')
sleep(0.5)

sign_in_button = driver.find_element_by_xpath('//button[@class="btn__primary--large from__button--floating"]')
sign_in_button.click()


driver.get('https://www.linkedin.com/in/kate-yun-yi-wang-054977127/?originalSubdomain=hk')

sleep(3)
# driver.execute_script("window.scrollTo(0, 1800)")
sleep(3)
loadmore_skills=driver.find_element_by_xpath('//button[@class="pv-profile-section__card-action-bar pv-skills-section__additional-skills artdeco-container-card-action-bar artdeco-button artdeco-button--tertiary artdeco-button--3 artdeco-button--fluid"]')


actions = ActionChains(driver)
actions.move_to_element(loadmore_skills).perform()
#actions.move_to_element_with_offset(loadmore_skills, 0, 0).perform()
loadmore_skills.click()

Output

Update

In concerns to your newer problem, you need to implement a continuous scroll method that would enable you to dynamically update the skills section. This requires a lot of change and should ideally be asked as a another question.

I have also found a simple solution by setting the scroll to the correct threshold. For y=3200 seems to work fine for all the profiles I've checked including yours, mine and few others.

driver.execute_script("window.scrollTo(0, 3200)")

Solution 2:

If the button is not visible on the page at the time of loading then use the until method to delay the execution

try:
    myElem = WebDriverWait(browser, delay).until(EC.presence_of_element_located((By.ID, 'IdOfMyElement')))
    print"Button is rdy!"except TimeoutException:
    print"Loading took too much time!"

Example is taken from here

To get the exact location of the element, you can use the following method to do so.

element = driver.find_element_by_id('some_id')
element.location_once_scrolled_into_view

This actually intends to return you the coordinates (x, y) of the element on-page, but also scroll down right to target element. You can then use the coordinates to make a click on the button. You can read more on that here.

Solution 3:

You are getting NoSuchElementException error when the locators (i.e. id / xpath/name/class_name/css selectors etc) we mentioned in the selenium program code is unable to find the web element on the web page.

How to resolve NoSuchElementException:

1. Apply WebDriverWait : allow webdriver to wait for a specific time
2. Try catch block

so before performing action on webelement you need to take web element into view, I have removed unwated code and also avoided use of hardcoded waits as its not good practice to deal with synchronization issue. Also while clicking on show more button you have to scroll down otherwise it will not work.

from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import Byfrom selenium.webdriver.common.action_chains import ActionChains


driver = webdriver.Chrome(executable_path="path of chromedriver.exe")

driver.get('https://www.linkedin.com/login?fromSignIn=true&trk=guest_homepage-basic_nav-header-signin')
driver.maximize_window()

WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.NAME, "session_key"))).send_keys("email id")
WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.NAME, "session_password"))).send_keys("password ")
WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, "//button[@class='btn__primary--large from__button--floating']"))).click()

driver.get("https://www.linkedin.com/in/kate-yun-yi-wang-054977127/?originalSubdomain=hk")
driver.maximize_window()

driver.execute_script("scroll(0, 250);")
buttonClick = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, "//span[text()='Show more']")))
ActionChains(driver).move_to_element(buttonClick).click().perform()

Output: