fix: Move SQL connection limits to initialization (#4981)

The connection limit wasn't being applied to pubsub, which would
overload the server if a ton of logs were being published at once.

This should fix it, and improve scale a lot!
This commit is contained in:
Kyle Carberry
2022-11-09 10:25:25 -06:00
committed by GitHub
parent 90c34b74de
commit 089659ffb1
2 changed files with 18 additions and 19 deletions

View File

@ -42,25 +42,6 @@ type DBTX interface {
// New creates a new database store using a SQL database connection.
func New(sdb *sql.DB) Store {
dbx := sqlx.NewDb(sdb, "postgres")
// The default is 0 but the request will fail with a 500 if the DB
// cannot accept new connections, so we try to limit that here.
// Requests will wait for a new connection instead of a hard error
// if a limit is set.
dbx.SetMaxOpenConns(40)
// Allow a max of 3 idle connections at a time. Lower values end up
// creating a lot of connection churn. Since each connection uses about
// 10MB of memory, we're allocating 30MB to Postgres connections per
// replica, but is better than causing Postgres to spawn a thread 15-20
// times/sec. PGBouncer's transaction pooling is not the greatest so
// it's not optimal for us to deploy.
//
// This was set to 10 before we started doing HA deployments, but 3 was
// later determined to be a better middle ground as to not use up all
// of PGs default connection limit while simultaneously avoiding a lot
// of connection churn.
dbx.SetMaxIdleConns(3)
return &sqlQuerier{
db: dbx,
sdb: dbx,