MLX · Python
Web Scraping with Selenium 101
A 101-level scraping pattern that respects profile isolation—Multilogin for the browser, Selenium for the control plane.
Principles
- One profile → one identity story. Do not reuse profiles across unrelated targets.
- Attach, don’t launch random Chrome. Use Multilogin-started browsers (Selenium example).
- Fail closed. Quarantine on repeated blocks; do not rotate forever in silence.
- Respect the law & ToS. This is engineering hygiene, not a bypass tutorial for prohibited targets.
Minimal scrape loop
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# driver = attached to Multilogin profile (see Selenium example)
wait = WebDriverWait(driver, 20)
driver.get("https://example.com/list")
items = wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, ".item")))
rows = []
for el in items:
rows.append({
"title": el.find_element(By.CSS_SELECTOR, "h2").text.strip(),
"url": el.find_element(By.CSS_SELECTOR, "a").get_attribute("href"),
})
# persist rows + profile_id + trace_id, then stop profile
Stability checklist
| Check | Why |
|---|---|
| Explicit waits | Reduce flaky empty DOM reads |
| Bounded pagination | Prevent runaway job cost |
| Proxy + fingerprint notes in evidence pack | Makes failures debuggable |
Stop profile in finally |
Controls Minutes / machine load |
Scale gate
Before widening concurrency, run the fingerprint readiness score. Pilot-Only or worse → fix drift before adding workers.
Need Multilogin capacity? Partner codes 50MAM (browser) and MAMOFF (Minutes) are on the homepage.