This adds the initial configuration changes and testing for loading geotiff data from an S3 bucket. It currently uses MinIO as a temporary S3 server. It includes a lot of new configuration/environment variable options that maybe should have been added earlier for performance. Some are specific to curl/s3 loading and authentication.
TODO:
Test performance differences between local geotiff and local S3 server. This won't be a fair comparison as S3 will be remote, but it should reveal any major problems (I hope).
Allow more of the configuration values to be specified from environment variables.
Update the helm chart to load these environment variables from the values.yaml
Think of a transition plan for geosphere-test (and geosphere) for switching to S3. Either transition all geotiffs (manual copy/upload) or all future geotiffs or wait to enable it at a later date.
This is step one in getting this to work. The other steps include:
tile-gen needs to glob/search the S3 bucket for all geotiffs instead of expecting them locally
geo2grid needs to push geotiffs to S3. It probably shouldn't do local and S3 as it will slow processing...but maybe if tested we can check how long this really would be and do it on geosphere-test to verify operations.
