Self-Hosted Convex on AWS with SST using EC2

The last post I did was using SST to deploy and run locally, Convex on AWS. After finishing the post and going on my run, I realized I should likely have used EC2 instead. This post is my “Round 2”attempt to self-host Convex on AWS using EC2. Hopefully this will be more helpful and easier to follow.

Prerequisites

Node.js and npm/pnpm/bun installed
Basic familiarity with SST and Convex
An AWS account

Project Setup

Let’s dive into Round 2 with a fresh project setup.

1
bun create vite@latest convex-self-hosted-ec2 --template react-ts
2
cd convex-self-hosted-ec2
3

4
# Setup SST
5
bunx sst init
6

7
# Install dependencies
8
bun install
9
bun sst install

Note: Vite’s tsconfig.node.json needs to include the sst.config.ts file.

Convex Setup

Install Convex.

1
bun add convex

SST Setup

1
/// <reference path="./.sst/platform/config.d.ts" />
2

3
export default $config({
4
  app(input) {
5
    return {
6
      name: "convex-self-hosted",
7
      removal: input?.stage === "production" ? "retain" : "remove",
8
      protect: ["production"].includes(input?.stage),
9
      home: "aws",
10
      providers: {
11
        aws: {
12
          region: "us-east-1", // set to your desired region
13
          profile: "<YOUR_AWS_PROFILE>", // set to your desired AWS profile
14
        },
15
      },
16
    };
17
  },
18
  async run() {},
19
});

Local Development

My thinking clarity always comes to me when I’m on my run. And after finishing the last post and thinking about it some more, I realized I could have made the local setup a bit easier. So for Round 2, we are going to start with the Convex docker-compose.yml file and go from there. We will add in a MySQL database and then use the SST DevCommand to start everything up.

Note: I found, unfortunately, that even though I was using docker compose up without the -d flag, it was still running in the background when I stopped the SST dev environment. Therefore, to stop the containers we will need to use the docker compose down command manually. However, restarting the SST dev environment will still work even though the container were previously running.

1
services:
2
  mysql:
3
    image: mysql:8.0
4
    ports:
5
      - "3306:3306"
6
    environment:
7
      - MYSQL_ROOT_PASSWORD=root
8
      - MYSQL_DATABASE=convex_self_hosted
9
    volumes:
10
      - mysql-data:/var/lib/mysql
11
  backend:
40 collapsed lines
12
    image: ghcr.io/get-convex/convex-backend:5143fec81f146ca67495c12c6b7a15c5802c37e2
13
    stop_grace_period: 10s
14
    stop_signal: SIGINT
15
    ports:
16
      - "${PORT:-3210}:3210"
17
      - "${SITE_PROXY_PORT:-3211}:3211"
18
    volumes:
19
      - data:/convex/data
20
    environment:
21
      - INSTANCE_NAME=${INSTANCE_NAME:-}
22
      - INSTANCE_SECRET=${INSTANCE_SECRET:-}
23
      - CONVEX_RELEASE_VERSION_DEV=${CONVEX_RELEASE_VERSION_DEV:-}
24
      - ACTIONS_USER_TIMEOUT_SECS=${ACTIONS_USER_TIMEOUT_SECS:-}
25
      - CONVEX_CLOUD_ORIGIN=http://127.0.0.1:${PORT:-3210}
26
      - CONVEX_SITE_ORIGIN=http://127.0.0.1:${SITE_PROXY_PORT:-3211}
27
      - DATABASE_URL=${DATABASE_URL:-}
28
      - DISABLE_BEACON=${DISABLE_BEACON:-}
29
      - REDACT_LOGS_TO_CLIENT=${REDACT_LOGS_TO_CLIENT:-}
30
      - DO_NOT_REQUIRE_SSL=${DO_NOT_REQUIRE_SSL:-}
31
      - POSTGRES_URL=${POSTGRES_URL:-}
32
      - MYSQL_URL=${MYSQL_URL:-}
33
      - RUST_LOG=${RUST_LOG:-info}
34
      - RUST_BACKTRACE=${RUST_BACKTRACE:-}
35
      - AWS_REGION=${AWS_REGION:-}
36
      - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID:-}
37
      - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY:-}
38
      - AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN:-}
39
      - S3_STORAGE_EXPORTS_BUCKET=${S3_STORAGE_EXPORTS_BUCKET:-}
40
      - S3_STORAGE_SNAPSHOT_IMPORTS_BUCKET=${S3_STORAGE_SNAPSHOT_IMPORTS_BUCKET:-}
41
      - S3_STORAGE_MODULES_BUCKET=${S3_STORAGE_MODULES_BUCKET:-}
42
      - S3_STORAGE_FILES_BUCKET=${S3_STORAGE_FILES_BUCKET:-}
43
      - S3_STORAGE_SEARCH_BUCKET=${S3_STORAGE_SEARCH_BUCKET:-}
44
      - S3_ENDPOINT_URL=${S3_ENDPOINT_URL:-}
45
    depends_on:
46
      mysql:
47
        condition: service_started
48
    healthcheck:
49
      test: curl -f http://localhost:3210/version
50
      interval: 5s
51
      start_period: 10s
52

53
  dashboard:
10 collapsed lines
54
    image: ghcr.io/get-convex/convex-dashboard:5143fec81f146ca67495c12c6b7a15c5802c37e2
55
    stop_grace_period: 10s
56
    stop_signal: SIGINT
57
    ports:
58
      - "${DASHBOARD_PORT:-6791}:6791"
59
    environment:
60
      - NEXT_PUBLIC_DEPLOYMENT_URL=http://127.0.0.1:${PORT:-3210}
61
    depends_on:
62
      backend:
63
        condition: service_healthy
64

65
volumes:
66
  data:
67
  mysql-data:

I could use the script trick from the last post to start and stop the docker containers, but I think it is ok to manually stop them. Plus, restarting won’t be an issue. This way when we start the SST dev environment, the containers will start up.

Initial Setup

1
// in the async run function
2
const exportsBucket = new sst.aws.Bucket("ExportsBucket", {});
3
const snapshotImportsBucket = new sst.aws.Bucket("SnapshotImportsBucket", {});
4
const modulesBucket = new sst.aws.Bucket("ModulesBucket", {});
5
const filesBucket = new sst.aws.Bucket("FilesBucket", {});
6
const searchBucket = new sst.aws.Bucket("SearchBucket", {});
7

8
const convexUser = new aws.iam.User("ConvexUser");
9
const convexUserPolicy = new aws.iam.UserPolicy("ConvexUserPolicy", {
10
  user: convexUser.name,
11
  policy: {
12
    Version: "2012-10-17",
13
    Statement: [
14
      {
15
        Sid: "ExportsBucketAccess",
16
        Effect: "Allow",
17
        Action: ["s3:*"],
18
        Resource: [exportsBucket.arn, $interpolate`${exportsBucket.arn}/*`],
19
      },
20
      {
21
        Sid: "SnapshotImportsBucketAccess",
22
        Effect: "Allow",
23
        Action: ["s3:*"],
24
        Resource: [
25
          snapshotImportsBucket.arn,
26
          $interpolate`${snapshotImportsBucket.arn}/*`,
27
        ],
28
      },
29
      {
30
        Sid: "ModulesBucketAccess",
31
        Effect: "Allow",
32
        Action: ["s3:*"],
33
        Resource: [modulesBucket.arn, $interpolate`${modulesBucket.arn}/*`],
34
      },
35
      {
36
        Sid: "FilesBucketAccess",
37
        Effect: "Allow",
38
        Action: ["s3:*"],
39
        Resource: [filesBucket.arn, $interpolate`${filesBucket.arn}/*`],
40
      },
41
      {
42
        Sid: "SearchBucketAccess",
43
        Effect: "Allow",
44
        Action: ["s3:*"],
45
        Resource: [searchBucket.arn, $interpolate`${searchBucket.arn}/*`],
46
      },
47
    ],
48
  },
49
});
50
const convexUserAccessKey = new aws.iam.AccessKey("ConvexUserAccessKey", {
51
  user: convexUser.name,
52
});
53

54
const dockerCompose = new sst.x.DevCommand("DockerCompose", {
55
  dev: {
56
    command: "docker compose up",
57
  },
58
  environment: {
59
    MYSQL_URL: $interpolate`mysql://root:root@mysql:3306`,
60
    S3_STORAGE_EXPORTS_BUCKET: exportsBucket.name,
61
    S3_STORAGE_SNAPSHOT_IMPORTS_BUCKET: snapshotImportsBucket.name,
62
    S3_STORAGE_MODULES_BUCKET: modulesBucket.name,
63
    S3_STORAGE_FILES_BUCKET: filesBucket.name,
64
    S3_STORAGE_SEARCH_BUCKET: searchBucket.name,
65
    S3_ENDPOINT_URL: "https://s3.us-east-1.amazonaws.com",
66
    AWS_ACCESS_KEY_ID: convexUserAccessKey.id,
67
    AWS_SECRET_ACCESS_KEY: convexUserAccessKey.secret,
68
    AWS_REGION: "us-east-1",
69
    CONVEX_CLOUD_ORIGIN: "http://127.0.0.1:3210",
70
    CONVEX_SITE_ORIGIN: "http://127.0.0.1:3211",
71
    DO_NOT_REQUIRE_SSL: "true",
72
    DISABLE_BEACON: "true",
73
    REDACT_LOGS_TO_CLIENT: "true",
74
  },
75
});

Notice that we use mysql://root:root@mysql:3306 for the MYSQL_URL. This is because we are using the database in the same docker compose file and can reference the service name as the host.

I also had to add the S3_ENDPOINT_URL to the environment variables because I was getting an error about the empty string. Not sure why it did not come up the last time. 🤷

And, I left off the Vpc and Mysql resources because we can actually get away locally without them for now. They will be needed shortly to deploy to AWS though.

Adding the Convex Frontend and Tutorial Demo Code

Next we will setup Convex for the frontend using the code from their tutorial. I copied over the src/ directory from my last post and then added the following. And copy the convex/chat.ts file to the new convex/ directory.

1
bun add convex @faker-js/faker

1
import { v } from "convex/values";
2
import { mutation, query } from "./_generated/server";
3

4
export const sendMessage = mutation({
11 collapsed lines
5
  args: {
6
    user: v.string(),
7
    body: v.string(),
8
  },
9
  handler: async (ctx, args) => {
10
    console.log("This TypeScript function is running on the server.");
11
    await ctx.db.insert("messages", {
12
      user: args.user,
13
      body: args.body,
14
    });
15
  },
16
});
17

18
export const getMessages = query({
5 collapsed lines
19
  args: {},
20
  handler: async (ctx) => {
21
    const messages = await ctx.db.query("messages").order("desc").take(50);
22
    return messages.reverse();
23
  },
24
});

Setup the .env.local file with the following.

1
CONVEX_SELF_HOSTED_URL='http://127.0.0.1:3210'
2
CONVEX_SELF_HOSTED_ADMIN_KEY='<your admin key>'

Update `sst.config.ts`

To start Convex dev and the demo react app we can add the following to the sst.config.ts file.

1
// ... dockerCompose DevCommand above ...
2

3
const convexDev = new sst.x.DevCommand(
4
  "ConvexDevServer",
5
  { dev: { command: "bun run convex dev" } },
6
  { dependsOn: [dockerCompose] }
7
);
8

9
const site = new sst.aws.StaticSite("Site", {
10
  dev: {
11
    command: "bun run dev",
12
    url: "http://127.0.0.1:5173",
13
  },
14
  build: {
15
    command: "bun run build",
16
    output: "./dist",
17
  },
18
  environment: {
19
    VITE_CONVEX_URL: "http://127.0.0.1:3210",
20
  },
21
});

SST Dev

We can now start the SST dev environment and have it start the docker containers and the Convex dev server.

1
bun sst dev

When you stop the SST dev environment, the docker containers will still be running. You can stop them with the following command.

1
docker compose down

Deploying to AWS EC2

We will first need to create a new SSH key pair so we can connect to the EC2 instance we will be using. As simple as this is, I always go to the GitHub Docs on generating a new SSH key for the instructions.

I like to give the key an explicit name so I can easily differentiate it from other keys.

You can then set the public key as a secret in SST. I’m using the --fallback flag to ensure the secret is used on any stage unless it is overridden in that stage.

1
ssh-keygen -C "your-email@example.com" -f ~/.ssh/convex-self-hosted-ec2-blog
2
# you can also use the following command to copy the public key to the clipboard
3
#pbcopy < ~/.ssh/convex-self-hosted-ec2-blog.pub
4

5
bun sst secret set --fallback PublicKeySecret < ~/.ssh/convex-self-hosted-ec2-blog.pub

Create the EC2 Instance

We will update the sst.config.ts file to include the VPC, database, EC2 instance, and route 53 records. This is a lot of code, and I will highlight the important parts.

User Data Script

The userData script is a dense block of code that that will be run on the EC2 instance as the root user. It can be used to setup the EC2 instance, and run commands. I apologize for the size, but I wanted to try and get as much of the setup done in the sst.config.ts file as possible.

At a high level, we need to do the following:

Install Docker and Docker Compose
Setup the environment variables for the Convex backend
Setup the basic nginx configuration
Install Certbot and the Route53 plugin
Set the AWS credentials for the root user to the IAM user we created in the sst.config.ts file
Generate the SSL certificates
Add the SSL certificates to the nginx configuration
Restart the nginx service

I’ll go through the script in more detail after the code block.

1
const vpc = new sst.aws.Vpc("Vpc", { bastion: true });
2

3
const database = new sst.aws.Mysql("Database", {
9 collapsed lines
4
  vpc,
5
  database: "convex_self_hosted",
6
  dev: {
7
    database: "convex_self_hosted",
8
    host: "localhost",
9
    port: 3306,
10
    username: "root",
11
    password: "root",
12
  },
13
});
14

15
const zone = aws.route53.Zone.get("Zone", "<YOUR_ROUTE53_ZONE_ID>");
16
const baseDomain = "<YOUR_BASE_DOMAIN>";
17
// you can set the domain however you want, I just did this for this demo.
18
const domain = $dev ? `${$app.stage}.${baseDomain}` : baseDomain;
19

7 collapsed lines
20
const exportsBucket = new sst.aws.Bucket("ExportsBucket", {});
21
const snapshotImportsBucket = new sst.aws.Bucket("SnapshotImportsBucket", {});
22
const modulesBucket = new sst.aws.Bucket("ModulesBucket", {});
23
const filesBucket = new sst.aws.Bucket("FilesBucket", {});
24
const searchBucket = new sst.aws.Bucket("SearchBucket", {});
25

26
const convexUser = new aws.iam.User("ConvexUser");
27
const convexUserPolicy = new aws.iam.UserPolicy("ConvexUserPolicy", {
28
  user: convexUser.name,
29
  policy: {
30
    Version: "2012-10-17",
31
    Statement: [
33 collapsed lines
32
      {
33
        Sid: "ExportsBucketAccess",
34
        Effect: "Allow",
35
        Action: ["s3:*"],
36
        Resource: [exportsBucket.arn, $interpolate`${exportsBucket.arn}/*`],
37
      },
38
      {
39
        Sid: "SnapshotImportsBucketAccess",
40
        Effect: "Allow",
41
        Action: ["s3:*"],
42
        Resource: [
43
          snapshotImportsBucket.arn,
44
          $interpolate`${snapshotImportsBucket.arn}/*`,
45
        ],
46
      },
47
      {
48
        Sid: "ModulesBucketAccess",
49
        Effect: "Allow",
50
        Action: ["s3:*"],
51
        Resource: [modulesBucket.arn, $interpolate`${modulesBucket.arn}/*`],
52
      },
53
      {
54
        Sid: "FilesBucketAccess",
55
        Effect: "Allow",
56
        Action: ["s3:*"],
57
        Resource: [filesBucket.arn, $interpolate`${filesBucket.arn}/*`],
58
      },
59
      {
60
        Sid: "SearchBucketAccess",
61
        Effect: "Allow",
62
        Action: ["s3:*"],
63
        Resource: [searchBucket.arn, $interpolate`${searchBucket.arn}/*`],
64
      },
65
      {
66
        Sid: "CertbotRoute53Access",
67
        Effect: "Allow",
68
        Action: ["route53:ListHostedZones", "route53:GetChange"],
69
        Resource: ["*"],
70
      },
71
      {
72
        Sid: "CertbotRoute53ChangeAccess",
73
        Effect: "Allow",
74
        Action: ["route53:ChangeResourceRecordSets"],
75
        Resource: [zone.arn],
76
      },
77
    ],
78
  },
79
});
32 collapsed lines
80
const convexUserAccessKey = new aws.iam.AccessKey("ConvexUserAccessKey", {
81
  user: convexUser.name,
82
});
83

84
const dockerCompose = new sst.x.DevCommand("DockerCompose", {
85
  dev: {
86
    command: "docker compose up",
87
  },
88
  environment: {
89
    MYSQL_URL: $interpolate`mysql://root:root@mysql:3306`,
90
    S3_STORAGE_EXPORTS_BUCKET: exportsBucket.name,
91
    S3_STORAGE_SNAPSHOT_IMPORTS_BUCKET: snapshotImportsBucket.name,
92
    S3_STORAGE_MODULES_BUCKET: modulesBucket.name,
93
    S3_STORAGE_FILES_BUCKET: filesBucket.name,
94
    S3_STORAGE_SEARCH_BUCKET: searchBucket.name,
95
    S3_ENDPOINT_URL: "https://s3.us-east-1.amazonaws.com",
96
    AWS_ACCESS_KEY_ID: convexUserAccessKey.id,
97
    AWS_SECRET_ACCESS_KEY: convexUserAccessKey.secret,
98
    AWS_REGION: "us-east-1",
99
    CONVEX_CLOUD_ORIGIN: "http://127.0.0.1:3210",
100
    CONVEX_SITE_ORIGIN: "http://127.0.0.1:3211",
101
    DO_NOT_REQUIRE_SSL: "true",
102
    DISABLE_BEACON: "true",
103
    REDACT_LOGS_TO_CLIENT: "true",
104
  },
105
});
106

107
const convexDev = new sst.x.DevCommand(
108
  "ConvexDevServer",
109
  { dev: { command: "bun run convex dev" } },
110
  { dependsOn: [dockerCompose] }
111
);
112

113
const site = new sst.aws.StaticSite("Site", {
8 collapsed lines
114
  dev: {
115
    command: "bun run dev",
116
    url: "http://127.0.0.1:5173",
117
  },
118
  build: {
119
    command: "bun run build",
120
    output: "./dist",
121
  },
122
  environment: {
123
    VITE_CONVEX_URL: $dev ? "http://127.0.0.1:3210" : `http://api.${domain}`,
124
  },
125
});
126

127
if (!$dev) {
128
  const publicKeySecret = new sst.Secret("PublicKeySecret");
129

130
  const keyPair = new aws.ec2.KeyPair("KeyPair", {
131
    keyName: "ConvexSelfHostedKeyPair",
132
    publicKey: publicKeySecret.value,
133
  });
134
  const securityGroup = new aws.ec2.SecurityGroup("BackendSecurityGroup", {
135
    vpcId: vpc.id,
136
    ingress: [
137
      // Allow HTTP traffic
138
      {
139
        protocol: "tcp",
140
        fromPort: 80,
141
        toPort: 80,
142
        cidrBlocks: ["0.0.0.0/0"],
143
      },
144
      // Allow HTTPS traffic
145
      {
146
        protocol: "tcp",
147
        fromPort: 443,
148
        toPort: 443,
149
        cidrBlocks: ["0.0.0.0/0"],
150
      },
151
      // Allow SSH traffic
152
      {
153
        protocol: "tcp",
154
        fromPort: 22,
155
        toPort: 22,
156
        cidrBlocks: ["0.0.0.0/0"],
157
      },
158
      {
159
        protocol: "-1",
160
        fromPort: 0,
161
        toPort: 0,
162
        cidrBlocks: ["10.0.0.0/16"],
163
      },
164
    ],
165
    // Allow all outbound traffic
166
    egress: [
167
      {
168
        protocol: "-1",
169
        fromPort: 0,
170
        toPort: 0,
171
        cidrBlocks: ["0.0.0.0/0"],
172
      },
173
    ],
174
  });
175

176
  const userData = $interpolate`#!/bin/bash
177
mkdir -p /home/ubuntu/convex-backend
178

179
curl -o /home/ubuntu/convex-backend/docker-compose.yml https://raw.githubusercontent.com/get-convex/convex-backend/main/self-hosted/docker/docker-compose.yml
180

181
touch /home/ubuntu/convex-backend/.env
182
cat > /home/ubuntu/convex-backend/.env << EOL
183
AWS_ACCESS_KEY_ID=${convexUserAccessKey.id}
184
AWS_SECRET_ACCESS_KEY=${convexUserAccessKey.secret}
185
AWS_REGION=us-east-1
186

187
S3_STORAGE_EXPORTS_BUCKET=${exportsBucket.name}
188
S3_STORAGE_SNAPSHOT_IMPORTS_BUCKET=${snapshotImportsBucket.name}
189
S3_STORAGE_MODULES_BUCKET=${modulesBucket.name}
190
S3_STORAGE_FILES_BUCKET=${filesBucket.name}
191
S3_STORAGE_SEARCH_BUCKET=${searchBucket.name}
192
S3_ENDPOINT_URL=https://s3.us-east-1.amazonaws.com
193

194
MYSQL_URL=mysql://${database.username}:${database.password}@${database.host}:${database.port}
195

196
DO_NOT_REQUIRE_SSL=true
197
REDACT_LOGS_TO_CLIENT=true
198

199
CONVEX_CLOUD_ORIGIN=http://api.${domain}
200
CONVEX_SITE_ORIGIN=http://${domain}
201

202
EOL
203

204
chmod 600 /home/ubuntu/convex-backend/.env
205
chown -R ubuntu:ubuntu /home/ubuntu/convex-backend
206

207
# Remove old docker packages
208
for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do apt-get remove $pkg -y; done
209

210
# Install docker
211
apt-get update -y
212
apt-get install ca-certificates curl nginx -y
213
install -m 0755 -d /etc/apt/keyrings
214
curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
215

216
chmod a+r /etc/apt/keyrings/docker.asc
217

218
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "\${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
219

220
apt-get update
221
apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y
222

223
# Add the current user to the docker group
224
usermod -aG docker ubuntu
225

226
# Start and enable Docker
227
systemctl start docker
228
systemctl enable docker
229

230

231
touch /etc/nginx/sites-available/convex-backend
232

233
cat > /etc/nginx/sites-available/convex-backend << EOL
234
server {
235
    listen 80;
236
    server_name ${domain};
237

238
    location / {
239
        proxy_pass http://localhost:3210;
240
        proxy_http_version 1.1;
241
        proxy_set_header Upgrade $http_upgrade;
242
        proxy_set_header Connection 'upgrade';
243
        proxy_set_header Host $host;
244
        proxy_cache_bypass $http_upgrade;
245
    }
246
}
247

248
server {
249
    listen 80;
250
    server_name api.${domain};
251

252
    location / {
253
        proxy_pass http://localhost:3211;
254
        proxy_http_version 1.1;
255
        proxy_set_header Upgrade $http_upgrade;
256
        proxy_set_header Connection 'upgrade';
257
        proxy_set_header Host $host;
258
        proxy_cache_bypass $http_upgrade;
259
    }
260
}
261

262
server {
263
    listen 80;
264
    server_name dashboard.${domain};
265

266
    location / {
267
        proxy_pass http://localhost:6791;
268
        proxy_http_version 1.1;
269
        proxy_set_header Upgrade $http_upgrade;
270
        proxy_set_header Connection 'upgrade';
271
        proxy_set_header Host $host;
272
        proxy_cache_bypass $http_upgrade;
273
    }
274
}
275
EOL
276

277
ln -sf /etc/nginx/sites-available/convex-backend /etc/nginx/sites-enabled/
278

279
systemctl restart nginx
280
systemctl enable nginx
281

282
# Navigate to the directory with docker-compose.yml and .env
283
# and run docker compose up.
284
# The chown ensures that ubuntu user can manage files if needed later,
285
# but docker compose up itself will be run by root here.
286
cd /home/ubuntu/convex-backend
287
docker compose up -d
288

289
# Install certbot
290
snap install --classic certbot
291
ln -s /snap/bin/certbot /usr/bin/certbot
292

293
snap set certbot trust-plugin-with-root=ok
294
snap install certbot-dns-route53
295

296
# setup aws credentials
297
mkdir -p ~/.aws
298
cat > ~/.aws/credentials << EOL
299
[default]
300
aws_access_key_id = ${convexUserAccessKey.id}
301
aws_secret_access_key = ${convexUserAccessKey.secret}
302
EOL
303
`;
304

305
  // Place the EC2 instance in a public subnet of the VPC
306
  const ec2 = new aws.ec2.Instance("Backend", {
307
    instanceType: "t3.micro",
308
    ami: "ami-084568db4383264d4", // ubuntu 24.04 LTS
309
    userData: userData,
310
    // Use a public subnet to ensure internet connectivity
311
    subnetId: vpc.nodes.publicSubnets[0]?.id,
312
    vpcSecurityGroupIds: [securityGroup.id],
313
    // Ensure the instance gets a public IP
314
    associatePublicIpAddress: true,
315
    keyName: keyPair.keyName,
316
  });
317

318
  // create a route 53 record
319
  const baseRecord = new aws.route53.Record("BaseRecord", {
320
    zoneId: zone.zoneId,
321
    name: domain,
322
    type: "A",
323
    ttl: 300,
324
    records: [ec2.publicIp],
325
  });
326
  const apiRecord = new aws.route53.Record("ApiRecord", {
327
    zoneId: zone.zoneId,
328
    name: `api.${domain}`,
329
    type: "A",
330
    ttl: 300,
331
    records: [ec2.publicIp],
332
  });
333
  const dashboardRecord = new aws.route53.Record("DashboardRecord", {
334
    zoneId: zone.zoneId,
335
    name: `dashboard.${domain}`,
336
    type: "A",
337
    ttl: 300,
338
    records: [ec2.publicIp],
339
  });
340

341
  return {
342
    AccessKeyId: convexUserAccessKey.id,
343
    SecretAccessKey: convexUserAccessKey.secret,
344
    PublicIp: ec2.publicIp,
345
    PublicDns: ec2.publicDns,
346
    ExportsBucket: exportsBucket.name,
347
    SnapshotImportsBucket: snapshotImportsBucket.name,
348
    ModulesBucket: modulesBucket.name,
349
    FilesBucket: filesBucket.name,
350
    SearchBucket: searchBucket.name,
351
    Database: database.database,
352
    DatabaseHost: database.host,
353
    DatabasePort: database.port,
354
    DatabaseUsername: database.username,
355
    DatabasePassword: database.password,
356
  };
357
}

From the highlighted lines, we can see that:

We set a base domain which you can set however you want. I did this for this demo. For a better way please see how the SST handles their domains for their Console.
We add permissions to the IAM user based on what the certbot-dns-route53 plugin needs. This is spelled out in the Certbot DNS Route53 Plugin docs.
Adjust the VITE_CONVEX_URL environment variable on the StaticSite resource.
Then only in $dev we deploy the EC2 instance.
Create the Public Key, Key Pair, and Security Group for the EC2 instance.
I’ll go over the userData in more detail in the next section.
Hardcode the AMI ID for the Ubuntu 24.04 LTS image. These are region-specific, so you will need to find the correct one for your region, and the latest version.
We make the EC2 instance and put it into the public subnet of the VPC.
- set the security group to the new security group we created.
- set the key pair to the new key pair we created.
Create the route 53 records for the base domain, api, and dashboard.
Then return a load of information!

Deploy with SST

Now that we have the resources defined, we can deploy them with the SST CLI. You can use any stage name you want. It is recommended though that you use a different stage than the one you are using for your local development. Without the --stage flag, SST will default to your personal stage.

1
bun sst deploy --stage demo

User Data

Really the only must-have part of the userData script is where we setup the environment variables for the Convex backend, and the basic nginx configuration. This is because we have access to the SST resources when defining the userData script. The other parts, like installing docker, adding the current user to the docker group, and enabling docker services, could be done my SSHing into the EC2 instance and running the commands.

For this reason, I have included a bash script that includes all the commands to run on the EC2 instance. I will leave off the parts that used interpolation to set the environment variables.

Also, at the end of the script, I’m going to include the rest of the commands I did not include in the userData script. i.e Certbot, and the final nginx configuration with SSL. The sudo command is needed if not run as root during the userData script.

1
#!/bin/bash
2

3
mkdir -p /home/ubuntu/convex-backend
4

5
curl -o /home/ubuntu/convex-backend/docker-compose.yml https://raw.githubusercontent.com/get-convex/convex-backend/main/self-hosted/docker/docker-compose.yml
6

7
touch /home/ubuntu/convex-backend/.env
8
# remove due to interpolation in the sst.config.ts file
9
# cat > /home/ubuntu/convex-backend/.env << EOL
10
# AWS_ACCESS_KEY_ID=${convexUserAccessKey.id}
11
# AWS_SECRET_ACCESS_KEY=${convexUserAccessKey.secret}
12
# AWS_REGION=us-east-1
13

14
# S3_STORAGE_EXPORTS_BUCKET=${exportsBucket.name}
15
# S3_STORAGE_SNAPSHOT_IMPORTS_BUCKET=${snapshotImportsBucket.name}
16
# S3_STORAGE_MODULES_BUCKET=${modulesBucket.name}
17
# S3_STORAGE_FILES_BUCKET=${filesBucket.name}
18
# S3_STORAGE_SEARCH_BUCKET=${searchBucket.name}
19
# S3_ENDPOINT_URL=https://s3.us-east-1.amazonaws.com
20

21
# MYSQL_URL=mysql://${database.username}:${database.password}@${database.host}:${database.port}
22

23
# DO_NOT_REQUIRE_SSL=true
24
# REDACT_LOGS_TO_CLIENT=true
25

26
# CONVEX_CLOUD_ORIGIN=http://api.${domain}
27
# CONVEX_SITE_ORIGIN=http://${domain}
28

29
# EOL
30

31
# these only need to be run if the command to make the directory is not run by the `ubuntu` user
32
# chmod 600 /home/ubuntu/convex-backend/.env
33
# chown -R ubuntu:ubuntu /home/ubuntu/convex-backend
34

35
# Remove old docker packages
36
for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg -y; done
37

38
# Install docker & nginx
39
sudo apt-get update -y
40
sudo apt-get install ca-certificates curl nginx -y
41
sudo install -m 0755 -d /etc/apt/keyrings
42
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
43

44
sudo chmod a+r /etc/apt/keyrings/docker.asc
45

46
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
47

48
sudo apt-get update
49
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y
50

51
# Add the current user to the docker group
52
usermod -aG docker ubuntu
53
newgrp docker # this is needed to add the current user to the docker group
54

55
# Start and enable Docker
56
systemctl start docker
57
systemctl enable docker
58

59

60
touch /etc/nginx/sites-available/convex-backend
61
# remove due to interpolation in the sst.config.ts file
62
# cat > /etc/nginx/sites-available/convex-backend << EOL
63
# server {
64
#     listen 80;
65
#     server_name ${domain};
66

67
#     location / {
68
#         proxy_pass http://localhost:3210;
69
#         proxy_http_version 1.1;
70
#         proxy_set_header Upgrade $http_upgrade;
71
#         proxy_set_header Connection 'upgrade';
72
#         proxy_set_header Host $host;
73
#         proxy_cache_bypass $http_upgrade;
74
#     }
75
# }
76

77
# server {
78
#     listen 80;
79
#     server_name api.${domain};
80

81
#     location / {
82
#         proxy_pass http://localhost:3211;
83
#         proxy_http_version 1.1;
84
#         proxy_set_header Upgrade $http_upgrade;
85
#         proxy_set_header Connection 'upgrade';
86
#         proxy_set_header Host $host;
87
#         proxy_cache_bypass $http_upgrade;
88
#     }
89
# }
90

91
# server {
92
#     listen 80;
93
#     server_name dashboard.${domain};
94

95
#     location / {
96
#         proxy_pass http://localhost:6791;
97
#         proxy_http_version 1.1;
98
#         proxy_set_header Upgrade $http_upgrade;
99
#         proxy_set_header Connection 'upgrade';
100
#         proxy_set_header Host $host;
101
#         proxy_cache_bypass $http_upgrade;
102
#     }
103
# }
104
# EOL
105

106
ln -sf /etc/nginx/sites-available/convex-backend /etc/nginx/sites-enabled/
107

108
systemctl restart nginx
109
systemctl enable nginx
110

111
# Navigate to the directory with docker-compose.yml and .env
112
# and run docker compose up.
113
# The chown ensures that ubuntu user can manage files if needed later,
114
# but docker compose up itself will be run by root here.
115
cd /home/ubuntu/convex-backend
116
docker compose up -d
117

118
# Install certbot
119
snap install --classic certbot
120
ln -s /snap/bin/certbot /usr/bin/certbot
121

122
snap set certbot trust-plugin-with-root=ok
123
snap install certbot-dns-route53
124

125
## setup aws credentials on the root user
126
sudo su -
127
mkdir -p ~/.aws
128
touch ~/.aws/credentials
129

130
# cat > ~/.aws/credentials << EOL
131
# [default]
132
# aws_access_key_id = <your-access-key-id>
133
# aws_secret_access_key = <your-secret-access-key>
134
# EOL
135

136
#certbot certonly --dns-route53 -d <your-domain> -d api.<your-domain> -d dashboard.<your-domain>
137

138
# check on docker
139
# docker ps

SSL

Once Certbot has generated the SSL certificates, we need to add them to the nginx configuration. Certbot should also setup auto-renewal with a cron job. You can check their docs for more information.

Please note that the base-domain.com is just an example domain. You can use your own domain. Just make sure to set them to their correct values.

This is a very simple nginx configuration that listens on port 80 and redirects to port 443. Then based on the domain, it will proxy the request to the correct port on the EC2 instance.

For example, if you go to https://api.base-domain.com, it will proxy the request to http://localhost:3211.

1
server {
2
  listen 80;
3
  server_name api.base-domain.com dashboard.base-domain.com base-domain.com;
4
  return 301 https://$host$request_uri;
5
}
6

7
server {
8
  listen 443 ssl;
9
  server_name api.base-domain.com;
10

11
  ssl_certificate /etc/letsencrypt/live/base-domain.com/fullchain.pem;
12
  ssl_certificate_key /etc/letsencrypt/live/base-domain.com/privkey.pem;
13

14
  location / {
15
    proxy_pass http://localhost:3210;
16
    proxy_http_version 1.1;
17
    proxy_set_header Upgrade $http_upgrade;
18
    proxy_set_header Connection 'upgrade';
19
    proxy_set_header Host $host;
20
    proxy_cache_bypass $http_upgrade;
21
  }
22
}
23

24
server {
25
  listen 443 ssl;
26
  server_name base-domain.com;
27

28
  ssl_certificate /etc/letsencrypt/live/base-domain.com/fullchain.pem;
29
  ssl_certificate_key /etc/letsencrypt/live/base-domain.com/privkey.pem;
30

31
  location / {
32
    proxy_pass http://localhost:3211;
33
    proxy_http_version 1.1;
34
    proxy_set_header Upgrade $http_upgrade;
35
    proxy_set_header Connection 'upgrade';
36
    proxy_set_header Host $host;
37
    proxy_cache_bypass $http_upgrade;
38
  }
39
}
40

41
server {
42
  listen 443 ssl;
43
  server_name dashboard.base-domain.com;
44

45
  ssl_certificate /etc/letsencrypt/live/base-domain.com/fullchain.pem;
46
  ssl_certificate_key /etc/letsencrypt/live/base-domain.com/privkey.pem;
47

48
  location / {
49
    proxy_pass http://localhost:6791;
50
    proxy_http_version 1.1;
51
    proxy_set_header Upgrade $http_upgrade;
52
    proxy_set_header Connection 'upgrade';
53
    proxy_set_header Host $host;
54
    proxy_cache_bypass $http_upgrade;
55
  }
56
}

Update All Urls to SSL

Now that we have the SSL certificates, we need to update all the urls in the environment variables.

Make sure to update the CONVEX_CLOUD_ORIGIN and CONVEX_SITE_ORIGIN environment variables to the new values, and restart docker compose.

1
cd /home/ubuntu/convex-backend
2
docker compose up -d --force-recreate
3

4
# check on docker
5
docker ps

Conclusion

Well, once again we have Convex Self-Hosted on AWS! This time with an EC2 instance. Now the question is, which way is better? Well, as always, it depends.

For EC2

The EC2 instance setup will run much cheaper than the ECS setup as we can run the Convex backend and dashboard on the same instance, instead of the 2 ECS services, plus Load Balancer, plus API Gateway. Making it a good choice for smaller projects. Plus, at this time the self-hosted Convex backend cannot horizontally scale, so the ECS elastic scaling is lost to us.

I also like that the EC2 instance version does not require me to regenerate the Convex admin key, as the container uses the same volume each time.

This blog post is shorter than the ECS one, which might be a good thing? Maybe? It is hard to show the time it takes to setup the server if the userData script fails. I must have deployed 20 servers before I got it to where it is now. This was a small skill issue on my part. Although, after reading the AWS docs, I figured it out.

For ECS

More of the infrastructure is defined in the SST config file, and their is less manual work. This makes is easier to deploy and manage, IMHO. This holds true especially for the SSL setup. With SST I set the domain property on the given resource, and as long as their is a Hosted Zone in the AWS account, the DNS will all be automatically created. THANK YOU SST!

Not having to manage a server is always a nice thing. Taking a Docker image and telling ECS to run it is a lot easier than all the work that comes with managing a server.

I never like going through the manual process of setting up a server. I know it is really not that hard, and I have done it many times, but after finding SST, I have tried to avoid it as much as possible.

Resources

You can find the ECS version of this blog post here.

And the full code example of this EC2 version here.

Self-Hosted Convex on AWS with SST using EC2

Share this post