As a result, the search engine crawlers may not index your website's content correctly, leading to lower search rankings and reduced visibility. To address this issue, a reverse proxy with Cloudflare Workers and prerender.io can be used to ensure that search engine crawlers can access and index your JS-based site content effectively.
Here's a step-by-step guide to setting it up.
1. Sign up for Cloudflare and Prerender.io
First, create an account with both Cloudflare (https://www.cloudflare.com) and Prerender.io (https://prerender.io). If you already have an account with Cloudflare, ensure that your domain is added and configured correctly.
2. Set up Webflow with Cloudflare
To set up a Webflow site with a Cloudflare proxy, follow the following 6 steps:
- Add your domain to Cloudflare and configure it correctly.
- Ensure that the DNS record is set to proxy (orange cloud turned on).
- In Webflow, disable SSL and copy the blank DNS records (without SSL).
- Add the copied records to the DNS settings in Cloudflare.
- Re-enable SSL in the Webflow settings.
- Ignore any warnings in Webflow regarding AAAA-records.
3. Set up a Cloudflare Worker
4. Configure the reverse proxy
Here's a sample code snippet to get you started:
Copy this code from GitHub Gist.
Replace 'your_prerender_token’ with your actual Prerender.io API token. This code checks if the request comes from a crawler and, if so, sends it to Prerender.io for rendering. Otherwise, it forwards the request to your origin server.
5. Deploy the Cloudflare Worker
Save and deploy your Cloudflare Worker. The Worker will now run for all requests to your domain.
6. Test your setup