Skip to main content

​
Introduction

Selenium is a powerful tool for automating web browsers. This guide covers deploying Selenium scripts on Shard Cloud using Python.

​
Creating Your Project

Ensure you have Python and pip installed.

​
Installing Dependencies

pip install selenium

​
Basic Selenium Script

Create a main.py file:
main.py
import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options

# Chrome options for headless operation
options = Options()
options.add_argument("--headless")
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--disable-gpu")

# Initialize the driver
service = Service('/usr/bin/chromedriver')
driver = webdriver.Chrome(service=service, options=options)

# Set window size
driver.set_window_size(1920, 1080)

try:
    # Navigate to a page
    driver.get('https://www.google.com')
    print(f"Page title: {driver.title}")

    # Take a screenshot
    driver.save_screenshot('screenshot.png')
    print("Screenshot saved!")

    # Keep running and take screenshots periodically
    while True:
        driver.save_screenshot('page.png')
        print("Screenshot updated")
        time.sleep(60)

except Exception as e:
    print(f"Error: {e}")
finally:
    driver.quit()

​
Requirements File

Create a requirements.txt:
requirements.txt
selenium
The Chrome options --no-sandbox and --disable-dev-shm-usage are required for Selenium to work in containerized environments.

​
Chrome/Chromium Path

On Shard Cloud, Chromium is installed at /usr/bin/chromium and ChromeDriver at /usr/bin/chromedriver. The WebDriver can locate these automatically.

​
Shard Cloud Configuration

Create a .shardcloud file:
.shardcloud
DISPLAY_NAME=Selenium Bot
DESCRIPTION=Browser Automation Script
MAIN=main.py
MEMORY=1024
VERSION=recommended
Selenium applications typically require at least 512MB-1024MB of memory due to the browser overhead.

​
Advanced Example: Web Scraping

scraper.py
import time
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

def setup_driver():
    options = Options()
    options.add_argument("--headless")
    options.add_argument("--no-sandbox")
    options.add_argument("--disable-dev-shm-usage")
    options.add_argument("--disable-gpu")
    options.add_argument("--window-size=1920,1080")

    service = Service('/usr/bin/chromedriver')
    return webdriver.Chrome(service=service, options=options)

def scrape_page(url):
    driver = setup_driver()

    try:
        driver.get(url)

        # Wait for an element to load
        wait = WebDriverWait(driver, 10)
        element = wait.until(
            EC.presence_of_element_located((By.TAG_NAME, "body"))
        )

        # Get page content
        title = driver.title
        content = driver.page_source

        print(f"Title: {title}")
        print(f"Content length: {len(content)} characters")

        return content

    except Exception as e:
        print(f"Error: {e}")
        return None
    finally:
        driver.quit()

if __name__ == "__main__":
    url = "https://example.com"
    result = scrape_page(url)

​
Deploying

1

Prepare Your Files

Ensure you have: - main.py (or your script) - requirements.txt - .shardcloud
2

Exclude Unnecessary Files

Remove: __pycache__/, venv/, *.pyc
3

Create ZIP Archive

Compress your project folder.
4

Upload to Shard Cloud

Go to Shard Cloud Dashboard and upload.

​
Accessing Screenshots

Screenshots saved by your script can be downloaded from Shard Cloud’s file manager in the dashboard.

​
Additional Resources

​
Troubleshooting

  • Ensure you’re using headless mode - Add --disable-dev-shm-usage flag - Increase memory allocation
  • Use WebDriverWait for dynamic content - Verify the selector is correct - Take a screenshot to debug the page state
Close the driver after each task and increase the MEMORY value in your config.