LetsEncrypt errors in Gaia on AWS with ELB

Apps Storage (Gaia)
#1

We’ve purchased a domain from AWS Route 53, got a valid SSL certificate for the domain and registered it in Certificate Manager. We also have an ALB deployed and a domain that resolves to the ELB endpoint.

We are trying to deploy a Blockstack Gaia instance with the image: blockstack-gaia_hub-ephemeral-2.5.3-hvm - ami-03e024fda2bc257d9 behind the Application Load Balancer so we can offload the SSL Certificate to the balancer. However, the instance never “resolves” our domain correctly, it always tries to resolve to either the private or public instance IP. We’ve tried several times, even just a single instance with an Elastic IP and we get the same result.

Is there some way to configure Gaia so that we can deploy behind a load balancer?

Here is the error message from journalctl -xe:

Feb 18 14:36:38 ip-172-31-35-111 bash[1578]: [ datahub.tcpdev.org ] Record (107.23.30.20
Feb 18 14:36:38 ip-172-31-35-111 bash[1578]: 3.216.65.218
Feb 18 14:36:38 ip-172-31-35-111 bash[1578]: 34.225.213.203
Feb 18 14:36:38 ip-172-31-35-111 bash[1578]: 34.231.187.112
Feb 18 14:36:38 ip-172-31-35-111 bash[1578]: 35.171.128.86
Feb 18 14:36:38 ip-172-31-35-111 bash[1578]: 54.156.194.52) doesnt match public IP (54.92.190.45) - sleeping for 70s....
Feb 18 14:36:45 ip-172-31-35-111 systemd[1]: Condition check resulted in LetsEncrypt Init Service being skipped.
#2

The problem here is that the startup scripts assume there is no loadbalancer in front of the instance. AWS proxies through nginx, and provides several IP’s. The host has no way of knowing what that LB ip might be (easily).

I haven’t worked on this in a while, but if you have a cert and a LB, it would be easiest to simply run gaia on the host directly.

the AMI was built from this: https://github.com/blockstackpbc/gaia-docker

and for local testing, i was using this which doesn’t do any letsencrypt setup: https://github.com/blockstackpbc/gaia-docker/blob/master/docker-compose.yaml

you could probably launch a generic AMI, install docker/docker-compose and run that compose script. essentially what you would want to do is to launch a generic linux VM in a private subnet (i.e. no public access), then setup your LB to forward the ports you need to that VM. add some security-group rules to allow all access to the gaia ports etc.

something like this should work, i think (the paths i have here may be old and no longer correct, but the commands should be what you need):

$ git clone \
      -b master \
      --single-branch \
      https://github.com/blockstack/gaia \
     master
$ cd gaia/docker
$ docker-compose \
      --project-directory ./docker \
      -f ./docker/docker-compose.yaml \
      up -d
$ curl -s localhost/hub_info | jq
<json output>
1 Like
#3

probably best to locally change file in the cloned repo gaia.env


the image tag should be set to v2.6.0 rather than v2.5.3

1 Like