Week 4, Day 4 — Nginx Log-Based Troubleshooting

Lab 4D — Log format deep dive, 301 redirect verification, four deliberate faults and log-based diagnosis

Part 1 — Log format reference (15 min)

• Watch both log files simultaneously with two terminal windows:
  Window 1: sudo tail -f /var/log/nginx/yourname.net.access.log
  Window 2: sudo tail -f /var/log/nginx/yourname.net.error.log
  Generate requests from the Windows browser and watch them appear.
• Complete the access.log field reference table in the lab sheet. For each field, identify what it contains in a real log line. Pay attention to:
  Status codes: 200 (OK), 301 (redirect), 304 (not modified/cached), 400 (bad request), 403 (forbidden), 404 (not found), 500 (server error), 502 (bad gateway).
  Request method: GET (retrieve), POST (submit), HEAD (headers only).
  User-agent string: identifies the browser and OS. Useful for distinguishing human traffic from crawlers and scanners.
• Verify the 301 redirect in the log: browse to http://yourname.net. The access log should show two lines — a 301 for the HTTP request, then a 200 for the HTTPS request that followed the redirect. Record both entries.

Part 2 — Four deliberate faults and log diagnosis (70 min)

Introduce each fault, observe the symptoms in the browser, then find and interpret the log entry that explains the failure. Fix each fault before introducing the next. Do not introduce multiple faults simultaneously.

Fault 1 — Wrong document root (403/404) (15 min)

• Edit the HTTPS server block and change the root to a path that doesn't exist:
  root /var/www/yourname.net/html_broken;
  Run nginx -t — it passes (nginx doesn't verify paths at test time). Reload nginx. Browse to https://yourname.net.
• Symptom: Browser shows 404 Not Found. Check the error log — it shows something like: [error] ... open() "/var/www/yourname.net/html_broken/index.html" failed (2: No such file or directory). The error.log tells you exactly which file it tried to open and why it failed.
• If the directory exists but the index.html is missing, you may get a 403 instead (forbidden — directory listing is disabled by default). The error.log in that case shows: [error] ... directory index of "/var/www/yourname.net/html_broken/" is forbidden. Distinct from a 404 — the directory exists but there's no index file.
• Fix: restore the correct path. Reload nginx. Verify 200 in access log.

Fault 2 — Removed symlink (502/connection refused) (15 min)

• Remove the symlink from sites-enabled:
  sudo rm /etc/nginx/sites-enabled/yourname.net
  Reload nginx: sudo systemctl reload nginx. Browse to https://yourname.net.
• Symptom: The browser may show a different site (the Nginx default), or a "connection refused" if port 443 has no matching server block. The access log may show requests going to the default server. The error log shows nothing specific — this is a configuration problem, not a runtime error.
• Key diagnostic: sudo nginx -T | grep server_name — list all active server_name directives. Notice yourname.net is absent. ls /etc/nginx/sites-enabled/ — symlink is gone.
• Fix: recreate the symlink. Reload nginx. Verify the site is back.

Fault 3 — Wrong certificate path (SSL handshake failure) (20 min)

• Edit /etc/nginx/snippets/self-signed.conf and change the certificate path to a non-existent file:
  ssl_certificate /etc/ssl/certs/nginx-selfsigned-broken.crt;
  Run nginx -t — this time it fails! nginx -t checks SSL file paths. Record the exact error message.
• Symptom from nginx -t: SSL_CTX_use_certificate_file("/etc/ssl/certs/nginx-selfsigned-broken.crt") failed (SSL: error:02001002:system library:fopen:No such file or directory). The config test fails — this is the first fault type where nginx -t catches the problem before the service even restarts.
• This demonstrates why nginx -t is always run before restarting: if this fault had been deployed, the nginx restart would fail, taking down the entire web server, not just this site.
• Fix: restore the correct path. nginx -t passes. Reload nginx.

Fault 4 — server_name mismatch (wrong site served / 404) (20 min)

• Edit the HTTPS server block and change server_name to a different hostname:
  server_name notmydomain.net www.notmydomain.net;
  nginx -t passes. Reload nginx. Browse to https://yourname.net.
• Symptom: The default Nginx page appears instead of the yourname.net content. The access log shows requests arriving with status 200, but coming from the wrong server block (the default). The Host header doesn't match any server_name, so Nginx falls back to the default server.
• Diagnostic: curl -v https://yourname.net 2>&1 | grep -E "Host|server" — see which server is responding. The curl verbose output shows the server header and certificate CN, revealing it's the default server, not the yourname.net block.
• Fix: restore server_name to yourname.net. Reload nginx. Verify yourname.net content returns.

Part 3 — Log analysis summary (15 min)

• Complete the fault diagnosis table in the lab sheet — for each fault, record: the symptom observed in the browser, the status code in the access log, the key error.log entry (if any), and the diagnostic command that identified the root cause.
• Answer the lab question: "A colleague reports the website is showing 403. Before touching any config files, what two log commands would you run first, and what information would you look for in each?"

Nginx reverse proxy preview

• Start a background Python HTTP server: python3 -m http.server 8080 &
• Add to the HTTPS server block: location /proxy/ { proxy_pass http://127.0.0.1:8080/; }. Reload nginx.
• Browse to https://yourname.net/proxy/ — Nginx forwards the request to the Python server. This is the same pattern used in the Week 6 Docker capstone, at a larger scale.