Log Monitoring in Watchlog

Learn how to set up Watchlog Agent and configure your logs for monitoring — now with automatic log level detection and live updates!

Step 1: Install the Watchlog Agent

Before you can monitor your logs, you need to install the Watchlog Agent on your server. Follow the installation guide here:

Install Watchlog Agent

Step 2: Configure Log Monitoring

Once the agent is installed, navigate to its directory and edit the configuration file to specify which logs you want to monitor. 

# Open the log-watchlist.json file for editing
nano /opt/watchlog/agent/src/log-watchlist.json

Inside log-watchlist.json, define the log files you want to monitor: 

{
    "logs": [
        {
            "name": "Nginx Access Logs",
            "path": "/var/log/nginx/access.log",
            "service": "nginx",
            "format": "auto"
        },
        {
            "name": "Redis Logs",
            "path": "/var/log/redis/redis.log",
            "service": "redis",
            "format": "custom",
            "pattern": "^(\\d{2} \\w{3} \\d{2}:\\d{2}:\\d{2}) (\\w+): (.*)$"
        }
    ]
}

After saving your changes, you need to reload the Watchlog Agent so it can pick up the new configuration. For example, if you're using PM2: 

pm2 reload watchlog-agent

Understanding format and pattern

The format field defines how the log file should be processed:

  • auto: Watchlog automatically detects the format (recommended for common services like Nginx, Redis, Docker, etc.).
  • custom: You need to provide a specific pattern (regular expression) to parse the logs.

If you set format to custom, you must define a pattern. The pattern should be a valid regex that extracts:

  • The timestamp of the log.
  • The log level (e.g., INFO, ERROR, WARNING, DEBUG, etc.).
  • The message content.

For example, the regex: 

"^(\\d{2} \\w{3} \\d{2}:\\d{2}:\\d{2}) (\\w+): (.*)$"

Will extract: 

"08 Mar 13:45:10 INFO: Server started"

Into:

  • date: 08 Mar 13:45:10
  • level: INFO
  • message: Server started

Step 3: Automatic Log Levels

Watchlog automatically detects the log level from your logs, as long as you capture it in a field named level (for custom formats) or rely on our built-in auto parsers for common services.

There is no limit to the log levels you can use. For example: INFO, WARN, ERROR, DEBUG, NOTICE, CRITICAL, SUCCESS, or any other string recognized in the level group.

In the Watchlog dashboard, each recognized level can be filtered or highlighted automatically. If a log line does not provide a recognizable level, Watchlog will display it under a default category (e.g. UNKNOWN).

Step 4: Searching and Filtering Logs

Watchlog allows you to **search logs** and apply filters in near real-time.

  • 🔍 Search logs by **keywords** in the message.
  • 📅 Filter logs by **time range**.
  • ⚙️ Filter logs by **log level** (any level that has been extracted, such as INFO, ERROR, DEBUG, etc.).

You can access the logs in the **Log Monitoring** section of the Watchlog dashboard: http://app.watchlog.io/logs

Step 5: Real-Time Monitoring (Live Mode)

Watchlog provides a Live Mode feature that updates your log view instantly as new entries arrive. When Live Mode is enabled:

  • Your log table and any charts or statistics are refreshed in real-time.
  • You can continue to apply filters (e.g., show only ERROR logs) while receiving new data.
  • This is especially useful for rapid troubleshooting and debugging sessions.

Simply toggle Live Mode in the dashboard to start or stop real-time updates.