Buildbot’s master/worker architecture requires strict worker trust boundaries, secure transport/authentication between components, and careful isolation of build execution environments. Security focuses on protecting the master, securing worker communications, and preventing build escape attacks.
- Enable mTLS: Use mutual TLS authentication between master and workers (Buildbot 4.3.0+)
- Certificate management: Use valid certificates from trusted CAs or internal PKI
- Strong cipher suites: Configure TLS 1.2+ with modern cipher suites only
- Certificate rotation: Implement automated certificate renewal
# Example worker configuration in master.cfg
from buildbot.plugins import worker
c['workers'] = [
worker.Worker('worker1', 'secure-password-here'),
worker.Worker('worker2', 'another-secure-password'),
]
# Use mTLS for worker connections (Buildbot 4.3.0+)
c['protocols'] = {
'pb': {
'port': 'ssl:9989:privateKey=key.pem:certKey=cert.pem:requireClientCertificate=True',
},
}
- Strong worker passwords: Use complex, randomly generated passwords
- Unique credentials: Never reuse passwords across workers
- Rotate credentials: Change worker passwords after host re-provisioning
- Remove stale workers: Promptly remove decommissioned worker registrations
- Restrict listener ports: Bind master ports (8010, 9989) to private interfaces
- Firewall rules: Allow worker connections only from known IP ranges
- VLAN isolation: Separate build infrastructure from production networks
- Jump host access: Require bastion host for administrative access
- Dedicated workers per trust level: Separate workers for trusted/untrusted builds
- Non-root execution: Run workers under unprivileged system accounts
- Ephemeral environments: Use containers or VMs for build execution
- Resource limits: Apply CPU, memory, and disk quotas per build
# Configure build environment restrictions
from buildbot.plugins import util
c['buildbotURL'] = "https://buildbot.example.com/"
c['www'] = {
'port': 8010,
'auth': util.Auth(),
'authz': util.Authz(
allowRules=[util.AnyEndpointMatcher()],
roleMatchers=[util.RolesFromUsername(rolesFromUsername={
'admins': ['admin-user'],
'developers': ['dev1', 'dev2'],
})],
),
}
- Filesystem isolation: Use separate build directories per worker
- Network restrictions: Block outbound access from build environments
- Host socket protection: Prevent access to Docker socket, systemd, etc.
- No privileged builds: Block
sudo and setuid binaries in builds
- Separate worker pools: Isolate PR builds from main branch builds
- Ephemeral workers: Destroy workers after untrusted builds complete
- Resource quotas: Limit resource consumption per build
- Audit build logs: Monitor for suspicious build behavior
¶ 3) Harden Web UI and API
¶ Authentication and Authorization
- Reverse proxy authentication: Put Buildbot UI behind SSO/OAuth proxy
- Role-based access: Define granular roles (admin, developer, viewer)
- Restrict admin actions: Limit force build, stop build, reconfig to admins
- API token management: Use scoped API tokens with expiration
# Enable HTTPS on web UI
c['www'] = {
'port': 'ssl:443:privateKey=key.pem:certKey=cert.pem',
'auth': util.RemoteUserAuth(),
}
# Configure change sources securely
c['change_source'] = [
changes.GitPoller(
'https://github.com/example/repo.git',
branches=['main'],
pollInterval=300,
),
]
- Enforce HTTPS: Require TLS for all web UI and API access
- HSTS headers: Enable HTTP Strict Transport Security
- Valid certificates: Use certificates from trusted CAs
- Redirect HTTP to HTTPS: Force secure connections
- Webhook validation: Validate webhook signatures from Git providers
- SSH key protection: Use deploy keys with minimum permissions
- Branch filters: Restrict which branches can trigger builds
- Audit changes: Log all change source events
¶ 4) Secrets and Credential Management
- Secret managers: Use HashiCorp Vault, AWS Secrets Manager, or similar
- Buildbot secrets: Leverage Buildbot’s secret management (Interpolate)
- Encryption at rest: Encrypt secret storage backends
- No plaintext in config: Never store secrets in master.cfg directly
# Use secret interpolation for credentials
from buildbot.plugins import util
factory = util.BuildFactory()
factory.addStep(
ShellCommand(
command=["deploy"],
env={
'API_KEY': util.Interpolate("%(secret:api_key)s"),
'DATABASE_URL': util.Interpolate("%(secret:db_url)s"),
},
),
)
- Scoped credentials: Use service accounts with minimum permissions
- Credential rotation: Implement regular rotation schedules
- Audit credential use: Log when secrets are accessed in builds
- Mask secrets: Redact secrets from build logs
¶ 5) Monitoring and Incident Response
- Centralized logging: Forward logs to SIEM or log management platform
- Structured logs: Use JSON format for easier analysis
- Log retention: Define retention policies for compliance
- Audit logging: Log all administrative actions and config changes
# Check Buildbot master status
buildbot checkconfig /path/to/master
# Review worker connections
curl -s https://buildbot.example.com/api/v2/workers | jq
# Monitor build activity
tail -f /var/log/buildbot/master.log | grep -E "build|worker|auth"
- Alert on auth failures: Detect brute force attempts
- Monitor build anomalies: Watch for unusual build patterns
- Track config changes: Audit master configuration modifications
- Worker health: Monitor worker connectivity and build queue
- Document procedures: Create runbooks for security incidents
- Isolation procedures: Know how to isolate compromised workers
- Credential rotation: Have emergency rotation procedures
- Backup and recovery: Maintain secure backups of master configuration
¶ Verification Commands
# Check Buildbot version
buildbot --version
# Verify master configuration
buildbot checkconfig /opt/buildbot/master
# Review master configuration
grep -R "protocols\|port\|auth\|www\|ssl" /opt/buildbot/master /etc/buildbot 2>/dev/null | head -30
# Check listening ports
sudo ss -tulpn | grep -E ':8010|:9989'
# Review worker registrations
curl -s http://localhost:8010/api/v2/workers | jq '.workers[] | {name, connected_to}'
# Check for TLS configuration
openssl s_client -connect localhost:8010 -tls1_2 </dev/null 2>/dev/null | head -10
# Review recent build activity
curl -s http://localhost:8010/api/v2/builds?limit=20 | jq '.builds[] | {builderid, number, complete}'
- Buildbot Documentation: https://docs.buildbot.net/current/manual/
- Buildbot Security Advisories: https://github.com/buildbot/buildbot/security
- Buildbot Release Notes: https://github.com/buildbot/buildbot/releases
- Buildbot mTLS Guide: https://docs.buildbot.net/current/manual/installation/misc.html#ssl
- OWASP CI/CD Security: https://owasp.org/www-project-devsecops-guideline/