1. Purpose & RFB Rules
The Daily RFB process makes the morning operational state visible and testable even when the primary DevOps engineer is unavailable. The flow covers server access/resource health, Docker, domain/Nginx/HTTPS, PostgreSQL/MongoDB, application/API smoke testing, logs, controlled cleanup and written handover.
Critical services healthy and business smoke test passes. Business can proceed.
Non-critical degradation, expected maintenance or incomplete test. Record and notify owner.
Critical application, database, network, TLS, Docker or domain path unavailable. Escalate immediately.
2. Service Inventory
The service inventory is the source of truth for expected domains, containers, ports, database dependencies, owners and criticality. Always compare the live server with the approved inventory before declaring RFB complete.
Production Service InventoryGoogle Drive spreadsheet — production domain/container/service mapping. Development Service InventoryGoogle Drive spreadsheet — development domain/container/service mapping.| Required inventory fields | Why |
|---|---|
| Domain / HTTPS URL | External reachability and TLS validation. |
| Reverse-proxy path / upstream host:port | Confirms Nginx routing. |
| Docker container + image/tag | Confirms deployed service/version. |
| Database dependency | Helps trace application failures. |
| Criticality + owner | Determines escalation and RFB impact. |
3. Infrastructure Flow
Typical MR2K public-service flow: the client reaches the domain, DNS points to the server/load-balancer entry point, Nginx terminates HTTPS and routes traffic to the correct Docker service, and the application communicates with its approved database.
5. Daily RFB Checklist
Run the checks in this order. Commands below are intentionally mostly read-only. Replace placeholders with values from the approved service inventory.
| # | Area | Minimum check | PASS / concern |
|---|---|---|---|
| 1 | Server identity | hostnamectl --static, whoami, uptime | Correct server/operator and normal uptime/load. |
| 2 | Resources | free -h, nproc, df -hT, df -ih | No severe memory/CPU pressure or filesystem/inode exhaustion; investigate >80–85% capacity. |
| 3 | System services | sudo systemctl --failed | No unexpected failed critical services. |
| 4 | Docker | sudo systemctl is-active docker, docker info | Docker active and responding. |
| 5 | Containers | docker ps -a | Expected critical containers Up; no restart loops/unexpected exits/unhealthy services. |
| 6 | Port mapping | docker ps --format ..., sudo ss -lntp | Host/container ports match approved inventory; no unexpected public listener. |
| 7 | Nginx | sudo systemctl is-active nginx, sudo nginx -t | Active and configuration test successful. |
| 8 | DNS/HTTPS | getent hosts <domain>, curl -I https://<domain>/ | Resolves correctly; expected HTTP status and valid TLS. |
| 9 | PostgreSQL | pg_isready plus read-only DB checks | Accepting connections; no unusual sessions, growth, blocking or autovacuum concerns. |
| 10 | MongoDB | mongosh ... ping | Ping succeeds if MongoDB is used. |
| 11 | Logs | Docker/Nginx/system/PostgreSQL/Mongo logs | No unexplained fatal errors, restart loops, upstream failures or OOM events. |
| 12 | Application/API | HTTPS health endpoint + approved smoke test | Critical business flow works; protected API accepts valid token and rejects invalid/missing token. |
| 13 | Capacity | docker system df, journalctl --disk-usage | Storage growth understood; no blind cleanup. |
| 14 | Handover | Record status, evidence, ticket/owner | GREEN / AMBER / RED communicated. |
6. Daily RFB Commands
6.1 Server baseline
# Identity, uptime, memory and filesystem
hostnamectl --static
whoami
uptime
free -h
nproc
df -hT
df -ih
cat /etc/os-release
# Failed services
sudo systemctl --failed
# Optional combined baseline
hostnamectl --static && uptime && free -h && df -hT
6.2 Docker health
# Docker daemon
sudo systemctl is-active docker
docker info
# All containers: name, image, status and ports
docker ps -a --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}'
# Running containers
docker ps
# Restart loops / exited containers
docker ps -a --filter status=restarting
docker ps -a --filter status=exited
# Health status where healthchecks exist
docker ps --format 'table {{.Names}}\t{{.Status}}'
# Networks and images
docker network ls
docker images --format 'table {{.Repository}}\t{{.Tag}}\t{{.CreatedSince}}'
# Docker disk usage
docker system df
6.3 Ports, Nginx and domain routing
# Listening ports
sudo ss -lntp
# Nginx state and syntax
sudo systemctl is-active nginx
sudo nginx -t
# Enabled sites
ls -la /etc/nginx/sites-enabled/
# DNS resolution
getent hosts <domain>
# HTTPS status and remote IP
curl -sS -o /dev/null -w 'HTTP=%{http_code} REMOTE_IP=%{remote_ip}\n' https://<domain>/
# Headers only
curl -I https://<domain>/
6.4 PostgreSQL daily commands
# Service / readiness
sudo systemctl is-active postgresql
psql --version
pg_isready
# Databases
sudo -u postgres psql -c '\l'
# Database sizes
sudo -u postgres psql -c "SELECT datname, pg_size_pretty(pg_database_size(datname)) FROM pg_database WHERE datistemplate=false ORDER BY pg_database_size(datname) DESC;"
# Sessions by database/user/state
sudo -u postgres psql -c "SELECT datname, usename, state, count(*) FROM pg_stat_activity GROUP BY datname, usename, state ORDER BY count(*) DESC;"
# Long-running active queries (>10 minutes)
sudo -u postgres psql -c "SELECT pid, datname, usename, now()-query_start AS duration, state, left(query,120) FROM pg_stat_activity WHERE query_start IS NOT NULL AND now()-query_start > interval '10 minutes' AND state <> 'idle' ORDER BY query_start;"
# Dead tuples / autovacuum / analyze
sudo -u postgres psql -c "SELECT schemaname, relname, n_live_tup, n_dead_tup, last_autovacuum, last_autoanalyze FROM pg_stat_user_tables ORDER BY n_dead_tup DESC LIMIT 20;"
# Replication, only if applicable
sudo -u postgres psql -c 'SELECT * FROM pg_stat_replication;'
# PostgreSQL storage
sudo du -sh /var/lib/postgresql /var/log/postgresql 2>/dev/null
# Recent PostgreSQL errors
sudo journalctl -u postgresql --since 'today' --no-pager | tail -n 100
6.5 MongoDB daily commands
# Host-installed MongoDB, if applicable
sudo systemctl is-active mongod
mongosh --version
# Safe connectivity test
mongosh --quiet --eval 'db.adminCommand({ ping: 1 })'
# Server status check
mongosh --quiet --eval 'db.serverStatus().ok'
# Recent errors
sudo journalctl -u mongod --since 'today' --no-pager | tail -n 100
If the service is named differently, use the approved unit name from the service inventory. If MongoDB is containerized, use the Docker/container healthcheck instead of assuming a host service.
6.6 Logs and incident indicators
# Recent container log sample
docker logs --tail 200 <container>
# Container activity during the last 30 minutes
docker events --since 30m --until 0s
# Nginx errors
sudo tail -n 200 /var/log/nginx/error.log
# System errors since today
sudo journalctl --since 'today' -p err..alert --no-pager
# OOM indicators
sudo dmesg -T | grep -i -E 'oom|out of memory' | tail -n 50
Look for repeated ERROR/FATAL messages, database connection failures, upstream timeout/refused errors, 5xx responses, unexpected container restarts and OOM events.
7. PostgreSQL Health & Maintenance Readiness
| Check | Expected | Action |
|---|---|---|
| Service/readiness | Active; pg_isready accepting connections. | Investigate service/startup/connectivity failures. |
| Database list/size | Expected databases present; no unexplained rapid growth. | Record unusual growth; involve DB owner. |
| Sessions | No unexpected surge/excessive connections. | Investigate before any termination. |
| Long queries | Normally no unexplained active query >10 minutes. | Report; do not terminate without authorization. |
| Dead tuples/autovacuum | No concerning stale autovacuum/analyze state. | Record maintenance candidates. |
| Blocking | No unexplained blocking of critical application queries. | Investigate with application/DB owner. |
| Replication | Expected replicas connected, if applicable. | Investigate lag/disconnects. |
| Storage/logs | Within approved capacity. | Use approved retention/maintenance procedure. |
8. MongoDB Health
For MongoDB services, validate service state, client version, ping/server status and recent logs. The normal RFB action is read-only validation. Cleanup or data modification requires the approved maintenance/change process.
9. Logs & Incident Indicators
| Indicator | Command | Escalation trigger |
|---|---|---|
| Container logs | docker logs --tail 200 <container> | Repeated ERROR/FATAL, startup failure, DB connection failure. |
| Container events | docker events --since 30m --until 0s | Unexpected restart/stop activity. |
| Nginx errors | sudo tail -n 200 /var/log/nginx/error.log | Repeated upstream timeout/refused/5xx. |
| System journal | sudo journalctl --since 'today' -p err..alert --no-pager | New critical system errors. |
| Disk | df -hT | Critical filesystem near capacity. |
| Memory/OOM | free -h and dmesg check | OOM events or severe memory pressure. |
10. Application / API Full-Flow Smoke Test
The goal is not to execute every test case. Prove the complete business path from internet entry point through reverse proxy, container, application and database.
- Open the primary application domain and confirm the login page loads.
- Confirm HTTPS certificate is valid and there is no browser security warning.
- Use an approved non-production/test account where available; never store passwords here.
- Confirm authentication succeeds and the dashboard/landing page loads.
- Open one core business screen and verify data can be read.
- Execute one safe read-only or reversible business action.
- Check browser/network requests for unexpected repeated 401/403/404/5xx responses.
- Verify a core database-backed read succeeds.
- If event/history/notification features exist, open one representative record.
- Logout and confirm the session is invalidated as expected.
- Record the exact function/endpoint tested and result.
HTTP/API checks
curl -I https://<domain>/
curl -sS -o /dev/null -w 'HTTP=%{http_code} TIME=%{time_total}\n' https://<domain>/health
curl -sS https://<domain>/health
curl -vk --max-time 15 https://<domain>/api/<safe-read-only-endpoint>
Do not use -k for the normal PASS test. It disables certificate verification and should only be used for troubleshooting.
Protected API token check
# Approved test token only
curl -sS -o /dev/null -w 'HTTP=%{http_code} TIME=%{time_total}\n' \
-H 'Authorization: Bearer <APPROVED_TEST_TOKEN>' \
https://<domain>/api/<protected-endpoint>
# Missing/invalid token should be rejected according to application policy
curl -sS -o /dev/null -w 'HTTP=%{http_code}\n' \
https://<domain>/api/<protected-endpoint>
11. Capacity & Controlled Cleanup
First inspect. Then confirm authorization for any change. Do not use broad destructive cleanup simply to make the server look clean.
| Area | Inspect | Cleanup rule |
|---|---|---|
| Disk / inodes | df -hT, df -ih | Locate consumers before deletion. |
| Filesystem usage | sudo du -xhd1 / 2>/dev/null | sort -h | Delete only approved application/temp data. |
| Docker disk | docker system df | Do not routinely run broad prune commands; retain rollback images as required. |
| Stopped containers | docker ps -a --filter status=exited | Remove only verified obsolete containers. |
| Nginx logs | sudo du -sh /var/log/nginx | Use logrotate/approved retention, not manual active-log deletion. |
| Journal | sudo journalctl --disk-usage | Use approved journal retention policy. |
| APT cache | sudo du -sh /var/cache/apt 2>/dev/null | Package-manager cleanup only when approved. |
| Temporary files | sudo du -sh /tmp /var/tmp 2>/dev/null | Delete only known-safe stale temporary files. |
12. Deployment Guide — Container to Public Service
- Confirm image/tag. Pull the approved image from the registry and confirm the intended version.
- Run the container. Publish only the required host port; inject approved environment/secrets through the normal secure method.
- Check container locally. Confirm it is Up and the application responds on its container port through the host mapping.
- Record the mapping. Update the service inventory with container name, image/tag, host port, container port, domain, owner and criticality.
- Point DNS. Create/update the approved DNS record to the public entry point.
- Configure Nginx. Add the domain server block and proxy the domain to the approved local host port.
- Validate Nginx. Run
sudo nginx -t; reload only after a successful test. - Test HTTP/HTTPS. Verify DNS, routing, application response and logs.
- Add SSL. Use Certbot for the approved domain and verify renewal readiness.
- Run RFB smoke test. Validate health endpoint, authentication, protected API and a core business flow.
Example container start
docker pull <registry>/<image>:<tag>
docker run -d \
--name <container-name> \
-p <HOST_PORT>:<CONTAINER_PORT> \
--restart unless-stopped \
<registry>/<image>:<tag>
docker ps --filter name=<container-name>
curl -sS http://127.0.0.1:<HOST_PORT>/health
13. Assign Domain & Configure Nginx
13.1 Create the site configuration
sudo nano /etc/nginx/sites-available/<domain>
Example reverse-proxy configuration:
server {
listen 80;
listen [::]:80;
server_name <domain>;
location / {
proxy_pass http://127.0.0.1:<HOST_PORT>;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 30s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
}
}
13.2 Enable and test
sudo ln -s /etc/nginx/sites-available/<domain> /etc/nginx/sites-enabled/<domain>
sudo nginx -t
sudo systemctl reload nginx
# Verify local routing through Host header
curl -sS -H 'Host: <domain>' http://127.0.0.1/
nginx -t before reload.14. SSL / HTTPS
After DNS and HTTP routing work, issue the certificate for the approved domain.
# Confirm Certbot is available
sudo certbot --version
# Issue/install certificate using Nginx integration
sudo certbot --nginx -d <domain>
# Verify HTTPS
curl -I https://<domain>/
# Check certificate details
openssl s_client -connect <domain>:443 -servername <domain>
15. Evidence, Daily Record & Handover
| Field | Entry |
|---|---|
| Date / Time | ____________________________ |
| Operator | ____________________________ |
| Server hostname | ____________________________ |
| Overall status | GREEN / AMBER / RED |
| Docker | PASS / WARN / FAIL |
| Domain / HTTPS | PASS / WARN / FAIL |
| PostgreSQL | PASS / WARN / FAIL / N/A |
| MongoDB | PASS / WARN / FAIL / N/A |
| Application smoke test | PASS / WARN / FAIL |
| Critical issue / ticket | ____________________________ |
| Escalated to | ____________________________ |
| Business notified | YES / NO |
| Operator sign-off | ____________________________ |
Evidence standard
- Record the exact time of the failed test.
- Capture command output that demonstrates the issue.
- Record affected domain, container, endpoint and database if known.
- Never paste passwords, API keys, JWTs, private keys or database credentials into tickets/chats.
- Use ticket/incident IDs to correlate repeated failures.
- After restoration, repeat the original failed test and record the recovery result.
16. Safety & Change Rules
Allowed during normal RFB
- Read-only server/resource checks
- Docker status/log inspection
- Nginx syntax validation
- DNS/HTTPS/API checks
- Read-only PostgreSQL/MongoDB checks
- Application smoke testing
- Evidence collection
Do not do without authorization
- Restart/stop production services as a routine check
- Edit Nginx or environment files
- Rotate certificates without procedure
- Delete containers/images or broad Docker prune
- Terminate PostgreSQL sessions
- Run destructive SQL
- Delete/modify server or database data
docker ps -a to inventory, run nginx -t where permitted, run domain/database connectivity checks and perform the documented smoke test. They should capture evidence and escalate rather than make production changes.17. One-Page Quick Run
For experienced operators, this is the first-pass readiness sequence. It does not replace the full checklist when a check fails.
# Server
hostnamectl --static && uptime && free -h && df -hT
sudo systemctl --failed
sudo systemctl is-active docker nginx postgresql mongod 2>/dev/null || true
# Docker
docker ps -a --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'
docker ps -a --filter status=restarting
docker ps -a --filter status=exited
docker system df
# Network / Nginx
sudo nginx -t
sudo ss -lntp
# PostgreSQL
pg_isready
sudo -u postgres psql -c "SELECT datname, pg_size_pretty(pg_database_size(datname)) FROM pg_database WHERE datistemplate=false ORDER BY pg_database_size(datname) DESC;"
sudo -u postgres psql -c "SELECT schemaname, relname, n_dead_tup, last_autovacuum FROM pg_stat_user_tables ORDER BY n_dead_tup DESC LIMIT 10;"
# MongoDB, if applicable
mongosh --quiet --eval 'db.adminCommand({ ping: 1 })' 2>/dev/null || true
# Logs / capacity
sudo journalctl --disk-usage
sudo journalctl --since 'today' -p err..alert --no-pager | tail -n 100
# Critical HTTPS endpoint
curl -sS -o /dev/null -w 'HTTPS=%{http_code} TIME=%{time_total}\n' https://<critical-domain>/
Governance
Update this guide whenever a production domain, Docker container, port mapping, reverse-proxy route, database architecture, critical business flow, authorization mechanism, cleanup/retention policy or ownership changes. Review at least quarterly and after major infrastructure changes.
Version 2.1 • MR2K SOLUTIONS • Internal Company Operations