How to Monitor Websites Automatically with Python

Websites change constantly. Prices are updated, product pages are modified, news headlines appear, job listings are added, and important announcements can be published without warning. Checking these changes manually can be time-consuming.

Python makes it possible to build a simple website monitoring system that regularly checks a webpage and alerts you when something changes. With a few libraries, even beginners can create a useful monitoring script.

What Is Website Monitoring?

Website monitoring means automatically checking a website or webpage at regular intervals and detecting changes.

For example, you could monitor:

A product price
A job listing
A news page
A blog post
A website's availability
A particular piece of text
A public announcement
Changes to HTML content

Instead of opening the website repeatedly, Python can perform these checks for you.

How Python Can Monitor a Website

A basic monitoring program follows this process:

Website → Download page → Extract information → Compare with previous result → Detect change → Send notification

Python provides several libraries that make this process relatively straightforward.

For a simple monitor, you can use:

requests — downloads webpage content
BeautifulSoup — extracts information from HTML
hashlib — creates a fingerprint of content
time — waits between checks
smtplib or a notification service — sends alerts

Installing the Required Libraries

First, install the two commonly used packages:

pip install requests beautifulsoup4

Then create a Python file such as:

website_monitor.py

A Simple Website Change Monitor

Here is a basic example that checks whether the content of a webpage has changed:

import requests
import hashlib
import time

URL = "https://example.com"

def get_page_hash():
    response = requests.get(
        URL,
        timeout=10,
        headers={"User-Agent": "Mozilla/5.0"}
    )

    response.raise_for_status()

    content = response.text
    return hashlib.sha256(content.encode("utf-8")).hexdigest()


old_hash = get_page_hash()

print("Monitoring started...")

while True:
    time.sleep(300)  # Check every 5 minutes

    try:
        new_hash = get_page_hash()

        if new_hash != old_hash:
            print("Website has changed!")
            old_hash = new_hash
        else:
            print("No changes detected.")

    except requests.RequestException as error:
        print("Unable to check website:", error)

The program downloads the webpage and creates a SHA-256 hash from its HTML. If the hash changes during the next check, the program reports that the page has changed.

Why Use a Hash?

Comparing an entire webpage every time can be inconvenient. A hash provides a compact representation of the content.

For example:

Webpage content
       ↓
SHA-256
       ↓
a8f4...9c21

If even a small part of the input changes, the resulting hash will normally be different.

This makes hashes useful for detecting whether a downloaded document or webpage has changed.

Monitoring Only a Specific Part of a Website

Sometimes you don't care about the entire webpage.

Imagine you want to monitor only a product price:

<div class="price">₹49,999</div>

Using BeautifulSoup, you can extract that particular element.

import requests
from bs4 import BeautifulSoup

URL = "https://example.com/product"

response = requests.get(
    URL,
    timeout=10,
    headers={"User-Agent": "Mozilla/5.0"}
)

response.raise_for_status()

soup = BeautifulSoup(response.text, "html.parser")

price = soup.select_one(".price")

if price:
    print("Current price:", price.get_text(strip=True))

Now the monitoring system can compare the price rather than the complete webpage.

Building a Price Change Detector

The next step is to remember the previous value.

import requests
from bs4 import BeautifulSoup
import time

URL = "https://example.com/product"

def get_price():
    response = requests.get(
        URL,
        timeout=10,
        headers={"User-Agent": "Mozilla/5.0"}
    )

    response.raise_for_status()

    soup = BeautifulSoup(response.text, "html.parser")
    element = soup.select_one(".price")

    return element.get_text(strip=True) if element else None


old_price = get_price()

print("Initial price:", old_price)

while True:
    time.sleep(300)

    try:
        new_price = get_price()

        if new_price != old_price:
            print("Price changed!")
            print("Old:", old_price)
            print("New:", new_price)

            old_price = new_price
        else:
            print("Price has not changed.")

    except requests.RequestException as error:
        print("Error:", error)

This example can form the foundation of a much larger monitoring application.

Sending an Alert

Printing a message in the terminal is useful for testing, but a monitoring system becomes much more useful when it sends a notification.

For example, you could connect your Python program to:

Email
Telegram
Discord
Slack
A custom notification API

The basic workflow becomes:

Website changes
      ↓
Python detects change
      ↓
Notification function
      ↓
Your phone/email

A notification function might look like this:

def send_alert(message):
    print("ALERT:", message)

You can later replace this function with an email or messaging API.

Monitoring Multiple Websites

Python can also monitor several pages.

websites = {
    "News": "https://example.com/news",
    "Blog": "https://example.com/blog",
    "Jobs": "https://example.com/jobs"
}

for name, url in websites.items():
    print("Checking:", name)

    response = requests.get(
        url,
        timeout=10,
        headers={"User-Agent": "Mozilla/5.0"}
    )

    print(response.status_code)

For a larger application, you could store the websites in a JSON file or database.

For example:

[
    {
        "name": "News",
        "url": "https://example.com/news"
    },
    {
        "name": "Jobs",
        "url": "https://example.com/jobs"
    }
]

This makes adding new websites easier without changing the Python program.

Using SQLite for Persistent Monitoring

A more advanced monitor should remember previous results even after the program closes.

Python includes SQLite support through the built-in sqlite3 module.

You could store:

Website URL
Last checked time
Previous content hash
Last detected change
Monitoring status

A simplified database structure might look like:

websites
-------------------------
id
name
url
content_hash
last_checked

This turns a small script into the foundation of a real monitoring service.

Handling Websites That Use JavaScript

One important limitation is that requests downloads the server's response but does not behave like a normal browser.

Some websites generate their content using JavaScript.

In such cases, BeautifulSoup may not find the information you're looking for because the desired content isn't present in the initial HTML.

For websites that require browser rendering, tools such as Playwright or Selenium can be used.

A browser-based workflow looks like:

Python
  ↓
Automated browser
  ↓
Load JavaScript
  ↓
Rendered webpage
  ↓
Extract information
  ↓
Compare with previous result

However, browser automation requires more resources than a simple requests-based monitor.

Important Monitoring Practices

Website monitoring should be performed responsibly.

Before monitoring a website, check its terms of service and robots.txt where applicable. Avoid sending excessive requests because frequent automated requests can put unnecessary load on a server.

A good monitoring application should:

Use reasonable intervals
Set request timeouts
Handle errors
Identify itself appropriately where appropriate
Avoid bypassing authentication or access controls
Respect website terms and applicable laws
Cache information when possible

For many tasks, checking every few minutes or even every hour is sufficient.

Turning the Script Into a Background Service

Once your monitor works, you don't necessarily need to keep a terminal window open manually.

On Linux, you can use:

cron

On Windows, you can use:

Task Scheduler

You can also deploy the monitoring program to a server or cloud environment.

Another approach is to use Python's scheduling libraries to execute monitoring jobs at predefined intervals.

Taking the Project Further

A basic website monitor can evolve into a complete application.

You could build a dashboard using Flask, FastAPI, or Django.

The dashboard could display:

Website Monitor
--------------------------------
Website       Status     Last Check

News          Changed    10:30 PM
Jobs          Stable     10:29 PM
Product       Changed    10:28 PM
Blog          Stable     10:27 PM

You could also add:

User accounts
Website categories
Custom CSS selectors
Change history
Email notifications
Telegram notifications
Scheduling
Database storage
Dashboard charts
Automatic screenshots
Failure alerts
Multiple monitoring frequencies

Conclusion

Python provides a practical way to automate website monitoring without building a complicated system from scratch. A simple combination of requests, BeautifulSoup and hashing can detect basic webpage changes, while tools such as Playwright can handle more dynamic websites.

The most important idea is to monitor only the information you actually need. Instead of repeatedly comparing an entire webpage, extracting a specific price, headline, status, or section usually produces cleaner and more useful results.

With persistent storage and notifications, a small Python script can eventually become a powerful website monitoring platform that works continuously in the background.

TechnologiesInternetz

Tuesday, September 15, 2026