I’m trying to get product details from an online store using Python and BeautifulSoup. Here’s what I’ve got so far:
import requests
from bs4 import BeautifulSoup
url = 'https://example-ecommerce.com'
headers = {'User-Agent': 'Mozilla/5.0'}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, 'html.parser')
product_container = soup.find('div', class_='product-list')
if product_container:
product_items = product_container.find_all('li', class_='product-item')
for item in product_items:
product_name = item.find('h3', class_='product-name')
product_price = item.find('span', class_='product-price')
if product_name and product_price:
print(f'Name: {product_name.text.strip()}, Price: {product_price.text.strip()}')
else:
print('Product container not found')
This code finds the main product list and tries to get the names and prices. But I’m not sure if I’m doing it correctly. How can I capture all the necessary product details, including their links? I’d appreciate any advice on improving this script. Thanks!