blog
Posts Tags Categories
blog

Google Cloud Storage via Cloudflare CDN for hosting Static Site

 included in GCP
   1012 words   5 minutes 

Features such as Content Delivery Network, DDoS Protection, Distributed Domain Name services plus their generous free tier makes make CloudFlare a no-brainer to host your static site.

Create a new Google Cloud Platform project

First, you have to create a new project. A project is the fundamental organizational component of Google Cloud, containing cloud resources, collaborators, permissions, and billing information.

You’ll need to provide a project name, which will be the human readable identifier for your project. Google will automatically convert this name to a globally unique identifier, which you will often use when interacting with Google Cloud APIs.

/posts/gcp/gcp_storage_via_cloudflare/001_create_project.png

After confirming, GCP will start the process of initializing your new project.

/posts/gcp/gcp_storage_via_cloudflare/002_create_project.png

Once finished, you’ll need to enable billing to get started building solutions on GCP. Project billing is managed through billing accounts, which associates a user or organization with payment information.

Create Service Account

Services and accounts in one project will be walled off from resources in other projects unless explicitly enabled. This provides a fundamental layer of security that can be built upon further via Google’s Identity & Access Management(IAM) system and the principle of least privilege. IAM is a platform-wide access control system that provides granular control over all GCP resources. As it’s commonly phrased, IAM specifies WHO can do WHAT to WHICH THING.

  • WHO: Actor. Actors are any entity that can take action against a Google Cloud resource. This includes both users and service accounts.

  • WHAT: Actions. In general, policies apply to specific actions an actor can take, creating the concept of permissions.

  • WHICH THING: Virtually everything can be thought of as a resource in GCP.

In general, it’s best practice to use service accounts whenever possible. Let’s get started with this by creating a service account and providing it with some resource-specific roles:

/posts/gcp/gcp_storage_via_cloudflare/003_create_service_account.png

The name should be meaningful, generally including how it will be used.

/posts/gcp/gcp_storage_via_cloudflare/004_create_service_account.png

For this service, we want to grant Cloud Storage permissions with a Storage Admin role.

/posts/gcp/gcp_storage_via_cloudflare/005_create_service_account.png

/posts/gcp/gcp_storage_via_cloudflare/006_create_service_account.png

Create Service Account Key.

/posts/gcp/gcp_storage_via_cloudflare/007_create_service_account.png

A JSON key for the account will be downloaded to your machine. This JSON file is a key pair for the service account you created.

/posts/gcp/gcp_storage_via_cloudflare/008_create_service_account.png

/posts/gcp/gcp_storage_via_cloudflare/009_create_service_account.png

Download the key JSON file. We will need this to fill in .env

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

{
  "type": "service_account",
  "project_id": "...",
  "private_key_id": "...",
  "private_key": "...",
  "client_email": "...",
  "client_id": "...",
  "auth_uri": "...",
  "token_uri": "...",
  "auth_provider_x509_cert_url": "...",
  "client_x509_cert_url": "..."
}

Create CNAME in Cloudflare

For this project, I will use the domain storage.raibis.com. Creating a subdomain in my Cloudflare account, domain raibis.lt. Adding CNAME record pointing to Google Storage c.storage.googleapis.com

/posts/gcp/gcp_storage_via_cloudflare/016_cloudflate.png

Verify Domain in Google Cloud platform

Before using storage.raibis.lt domain for Google Storage. We have to verify domain ownership by adding TXT record.

Visit Webmaster Central for domain verification

/posts/gcp/gcp_storage_via_cloudflare/018_verify_domain.png

/posts/gcp/gcp_storage_via_cloudflare/019_verify_domain.png

Add TXT domain record in Cloudflare.

/posts/gcp/gcp_storage_via_cloudflare/020_verify_domain.png

Then go back to Webmaster Central and press Verify button.

After successful verification, you should receive such a message.

/posts/gcp/gcp_storage_via_cloudflare/021_verify_domain.png

Now we are ready to proceed with Cloud Storage.

Create Cloud Storage Bucket

Cloud Storage is a general-purpose storage solution for unstructured data.

At the highest level, all data in Cloud Storage belongs to exactly one bucket. Buckets serve two primary purposes—organizing objects at a very high level and providing a control plane over those objects.

Buckets’ names can be regular and domain names.

Domain-named buckets allow bucket names to match valid site domains, such as storage.raibis.lt. Bucket creator must prove ownership of the domain using Google Webmaster Tools (GWT).

/posts/gcp/gcp_storage_via_cloudflare/010_create_bucket.png

/posts/gcp/gcp_storage_via_cloudflare/011_create_bucket.png

/posts/gcp/gcp_storage_via_cloudflare/012_create_bucket.png

Adding public bucket access.

/posts/gcp/gcp_storage_via_cloudflare/013_create_bucket.png

/posts/gcp/gcp_storage_via_cloudflare/014_create_bucket.png

Now we can proceed with some code writing.

Node.js code for file upload to the bucket

Simple Node Express server for file upload

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

require("dotenv").config();
const { format } = require("util");
const express = require("express");
const { Storage } = require("@google-cloud/storage");
const Multer = require("multer");

const storage = new Storage({
  projectId: process.env.GCP_PROJECT_ID,
  credentials: {
    type: process.env.GCP_TYPE,
    client_email: process.env.GCP_CLIENT_EMAIL,
    client_id: process.env.GCP_CLIENT_ID,
    private_key: process.env.GCP_PRIVATE_KEY?.replace(/\\n/g, "\n"),
  },
});

const userId = "user1234";

const app = express();

app.use(express.json());

const multer = Multer({
  storage: Multer.memoryStorage(),
  limits: {
    fileSize: 5 * 1024 * 1024,
  },
});

const bucket = storage.bucket(process.env.GCP_BUCKET);

app.post("/", multer.single("file"), (req, res, next) => {
  if (!req.file) {
    res.status(400).send("No file uploaded.");
    return;
  }
  const blob = bucket.file(`${userId}/${req.file.originalname}`);
  const blobStream = blob.createWriteStream({
    resumable: false,
  });

  blobStream.on("error", (err) => {
    next(err);
  });

  blobStream.on("finish", () => {
    const publicUrl = format(
      `https://storage.googleapis.com/${bucket.name}/${blob.name}`
    );
    res.status(200).send(publicUrl);
  });

  blobStream.end(req.file.buffer);
});

const port = process.env.PORT;
app.listen(port, () => {
  console.log(`Example app listening on port ${port}`);
});

Create .env file

1
2
3
4
5
6
7

PORT=8080
GCP_BUCKET="storage.raibis.lt"
GCP_PROJECT_ID="..."
GCP_CLIENT_ID="..."
GCP_TYPE="service_account"
GCP_CLIENT_EMAIL="..."
GCP_PRIVATE_KEY="..."

Upload file to Cloud Storage Bucket

/posts/gcp/gcp_storage_via_cloudflare/015_upload_photo.png

Direct Cloud Storage file access URL

Static website congiguration

File cashed in Cloudflare CDN

https://storage.raibis.lt/user1234/Google_Storage-Logo.wine.png

As you can see, the image file is cached in Cloudflare CDN and will serve from Cloudflare instead of Cloud Storage.

/posts/gcp/gcp_storage_via_cloudflare/022_cloudflare_image.png

You will get an error, if you visit a nonexisting object or do not directly point website index file location.

/posts/gcp/gcp_storage_via_cloudflare/023_configure_websettings.png

This is so because Cloud Storage is not configured to act as a website: not showing an index page and a 404 Not found page in case trying to access a nonexistent object.

Let’s add to the bucket two files: index.html and 404.html.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Index</title>
  </head>
  <body>
    <p>index.html</p>
    <img
      src="https://storage.raibis.lt/user1234/Google_Storage-Logo.wine.png"
      alt="Google Storage Logo"
    />
  </body>
</html>

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>404</title>
  </head>
  <body>
    404 Not Found
  </body>
</html>

Configure Cloud Storage

/posts/gcp/gcp_storage_via_cloudflare/024_configure_websettings.png

/posts/gcp/gcp_storage_via_cloudflare/025_configure_websettings.png

Congratulation, now you have Static Site served from Cloud Storage and cashed on Cloudflare CDN.

Updated on 2022-04-06
 GCP, Cloudflare, JS, Node
Back | Home