I wrote a python script to query a website and check to see if an item is in stock. If it finds the product in question it will email me a report with a hyperlink so that I can click the link and place and order straight from my email. This script runs as a cron job (once every hour) and saves me the time of having to constantly check their website, wondering when they will get the next delivery. <\/p>\n\n\n\n
It was a great exercise in using the BeautifulSoup4 python library as well as using selenium for the first time which was needed to flesh out the javascript that is creating the dynamic content. <\/p>\n\n\n
\nfrom selenium import webdriver\nfrom bs4 import BeautifulSoup as bs\nfrom selenium.webdriver.firefox.options import Options\nfrom selenium.common.exceptions import NoSuchElementException\nimport lxml\nimport smtplib\nimport time\n\noptions = Options()\noptions.headless = True\n\nmylist = []\nnot_found = ''\n\ndriver = webdriver.Firefox(options=options)\ndriver.get("https:\/\/www.website.com)\n\n# Try and fix the random timing errors --> better way is with selenium waitfor\ntime.sleep(5)\n\n# Look for the state "no product in stock" --> "0 matches, that stinks"\ntry:\n not_found = driver.find_element_by_class_name("css-1ctldcn.ew1p50q2")\n not_found_html = not_found.get_attribute('innerHTML')\n\n# handle the exception of product actually being found.\nexcept NoSuchElementException as e:\n print (str(e))\n\n# print "not found message" and exit program\nif(not_found):\n print (not_found_html)\n driver.close()\n exit()\n\ntry: \n\n # They have stock; now find how many products they have at runtime.\n products = driver.find_element_by_class_name("css-hecap1.ettsl931")\n total_products = products.get_attribute('innerHTML')\n\n# handle the exception of product elements not being found and exit program\nexcept NoSuchElementException as e:\n print (str(e))\n driver.close()\n exit()\n\n# Find all of the Grid Elements, or all of the products available - all products use the same grid ID\nelement = driver.find_element_by_class_name("css-19ofktj.e29d1tf2")\nhtml = element.get_attribute('innerHTML')\nsoup = bs(html, "lxml")\n\nprint (total_products)\n\nfor a in soup.find_all('a', href=True):\n mylist.append("Found the URL: https:\/\/www.website.com" + a['href'])\n\n# Python 3 only\nprint (*mylist, sep="\\n")\n\n# See the whole tree with price and description for each item\n# prettyHTML = soup.prettify()\n# print (prettyHTML)\n\nport = 587 \nsender_email = "SENDER@gmail.com"\nreceiver_email = "RECEIVER@email.com"\n\nmessage = """\\\nSubject: new products have arrived!\n\n{}. """ .format(total_products) + str(mylist)\n\nserver = smtplib.SMTP('smtp.gmail.com', 587)\nserver.ehlo()\nserver.starttls()\nserver.ehlo()\nserver.login(sender_email, "SENDERPASSWORD")\nserver.sendmail(sender_email, receiver_email, message)\nserver.quit()\n\ndriver.close()\n<\/pre><\/div>","protected":false},"excerpt":{"rendered":"I wrote a python script to query a website and check to see if an item is in stock. If it finds the product in question it will email me a report with a hyperlink so that I can click … Continue reading