分享一个可处理相对/绝对路径的在线代理

在线代理,指的是访问http://a.com/b.com即可访问b.com的内容的一种特殊的代理。目前99%的这类代理都非常简单——接收到你的请求,程序向b.com发送请求,将返回的结果发给你。

很明显,这只适合代理api,因为很多页面的资源路径都是相对/绝对路径,而并非绝对URL。设想一下,如果你访问http://a.com/b.com,而b.com中有一个/main.css——那么浏览器会去访问http://a.com/main.css。显然在线代理会去请求https://main.css,理所应当的就会失败了。

为了解决这个问题,我写的这个在线代理用cookie存储你首次访问时的站点域名,随后你的任何请求都会被处理加上这个域名。这样就可以完全复制这个网站了(前提是没有被站点防火墙制裁)。

项目地址: RavelloH/edge-proxy-middleware: 基于edge-function的代理API,全网最强大的在线代理,可处理任何路径。

特性

  • 基于边缘网络,始终选择距你最近的可用区,延迟低;
  • edge-function无需冷启动,响应快;
  • 可路径完全相同的复制代理任意站点,可处理相对路径;
  • 可使用Vercel部署,免费;

在线demo: https://bridge.ravelloh.top/

简单来说,在要代理的域名前加上此API的地址即可。

https://bridge.ravelloh.top/github.com
https://bridge.ravelloh.top/https://github.com
https://bridge.ravelloh.top/api/edge?url=https://github.com

通过以上三种格式之一访问后,会自动识别到代理主机名github.com ,并写入到cookie:site中。

之后路径将与github.com完全对应,即:

https://bridge.ravelloh.top <===> https://github.com
https://bridge.ravelloh.top/RavelloH <===> https://github.com/RavelloH

要更改代理主机名,只需访问一个新主机名,例如https://bridge.ravelloh.top/ravelloh.top

注意事项

强烈建议 你自己部署此API。毕竟直接用这个API demo会耗我的Vercel额度,使用量多时不保证可用性。
(如果你有vercel账户,点击此处一键部署:)
Deploy with Vercel
Cookie一天后过期,届时请重新输入主机名。
尽管架构上完全可以用于代理任意网站,但本API的主要用途是代理API。
部分网站可能会将edge function的服务器IP段视为爬虫而造成访问异常。

21 个赞

其实用子域名去包含目的地址信息是更好的选择……类似 https__example_com 这样,因为虽然 DNS 支持用下划线,但是基本上没有人用下划线建站……

2 个赞

这确实,不过这样部署起来就麻烦的多了,目前我这种的可以实现一键部署,之后vercel随便绑定个域名就可以了

vercel有大小限制…… :face_savoring_food:

每月100GB单人应该够用了

我的意思是,单次请求体的大小

测了一下单次最大256MB(https://bridge.ravelloh.top/https://speed.cloudflare.com/__down?during=download&bytes=1073741824),不拿来下载文件只是当页面代理的话完全不会达到这个限制

大佬牛逼

可以CFpages部署吗?
我用pages部署了另一个在线代理(siteproxy),感觉不太好用,很慢 :sweat_smile:

可以,我刚才适配了下cf,现在复制cloudflare/worker.js里的内容到worker编辑器里就能直接部署了。我自己也部署了一个cf上的demo:
https://bridge.ravelloh.tech
不过实测,cf确实更慢



(上Vercel,下cloudflare)

多谢大佬,空了也弄个 :saluting_face:

https://proxy2.deno.dev

/**
 * edge-proxy-middleware for Deno Deploy
 * 基于边缘网络的代理API,可处理任何相对路径
 * 原项目: https://github.com/RavelloH/edge-proxy-middleware
 */

function parseCookies(cookieString) {
  const cookies = {};
  if (cookieString) {
    cookieString.split(";").forEach((cookie) => {
      const parts = cookie.split("=");
      const name = parts[0].trim();
      const value = parts[1] ? parts[1].trim() : "";
      cookies[name] = value;
    });
  }
  return cookies;
}

/**
 * 处理路径重写逻辑,支持多种URL格式:
 * 1. /domain.com -> /api/edge?url=http://domain.com
 * 2. /https://domain.com -> /api/edge?url=https://domain.com  
 * 3. /domain.com/ -> /api/edge?url=http://domain.com/
 */
function handleRewrites(url) {
  const pathname = url.pathname;
  
  // 处理根路径 - 显示默认页面
  if (pathname === "/") {
    return "/api/edge?url=/";
  }
  
  // 处理 https: 或 http: 开头的完整URL路径
  const protocolMatch = pathname.match(/^\/(https?:)\/(.+)/);
  if (protocolMatch) {
    const protocol = protocolMatch[1];
    const site = protocolMatch[2];
    return `/api/edge?url=${protocol}//${site}`.replace("///", "//");
  }
  
  // 处理以斜杠结尾的域名路径
  if (pathname.endsWith("/") && pathname !== "/") {
    const site = pathname.slice(1, -1);
    return `/api/edge?url=http://${site}/`.replace("///", "//");
  }
  
  // 处理普通域名路径(不以/api/开头的)
  if (pathname !== "/" && !pathname.startsWith("/api/")) {
    const site = pathname.slice(1);
    return `/api/edge?url=http://${site}`;
  }
  
  return null;
}

async function handleRequest(request) {
  const url = new URL(request.url);
  
  // 处理路径重写,支持多种访问格式
  const rewritePath = handleRewrites(url);
  if (rewritePath && !url.pathname.startsWith("/api/edge")) {
    // 重写URL到API端点
    const newUrl = new URL(rewritePath, url.origin);
    // 保留原始查询参数
    for (const [key, value] of url.searchParams.entries()) {
      newUrl.searchParams.append(key, value);
    }
    url.pathname = newUrl.pathname;
    url.search = newUrl.search;
  }
  
  // 只处理API路径,其他返回404
  if (!url.pathname.startsWith("/api/edge")) {
    return new Response(`
<!DOCTYPE html>
<html>
<head>
  <title>Edge Proxy Middleware - Deno Deploy</title>
  <meta charset="utf-8">
  <style>
    body { font-family: -apple-system, sans-serif; max-width: 800px; margin: 50px auto; padding: 0 20px; }
    h1 { color: #333; }
    code { background: #f5f5f5; padding: 2px 6px; border-radius: 3px; }
    .example { background: #f8f9fa; padding: 15px; border-radius: 5px; margin: 10px 0; }
  </style>
</head>
<body>
  <h1>🌐 Edge Proxy Middleware</h1>
  <p>基于 Deno Deploy 的边缘代理服务,支持任意网站代理。</p>
  
  <h2>使用方法</h2>
  <p>在要代理的域名前加上此服务的地址:</p>
  
  <div class="example">
    <strong>格式1:</strong> <code>${url.origin}/github.com</code><br>
    <strong>格式2:</strong> <code>${url.origin}/https://github.com</code><br>
    <strong>格式3:</strong> <code>${url.origin}/api/edge?url=https://github.com</code><br>
    <strong>格式4:</strong> <code>${url.origin}/api/edge?r=aHR0cHM6Ly9naXRodWIuY29t</code> (base64编码)
  </div>
  
  <h2>快速测试</h2>
  <div class="example">
    <p>点击以下链接测试代理功能:</p>
    <p><a href="/httpbin.org/get" target="_blank">🔗 测试 httpbin.org</a></p>
    <p><a href="/api/edge?url=https://httpbin.org/ip" target="_blank">🔗 获取IP信息</a></p>
    <p><a href="/github.com" target="_blank">🔗 代理 GitHub</a></p>
  </div>
  
  <p>访问后会自动识别代理主机名并写入 Cookie,之后路径将完全对应。</p>
  
  <p><small>💡 提示:Cookie 24小时后过期,届时请重新访问主机名。</small></p>
  
  <hr>
  <p><a href="https://github.com/RavelloH/edge-proxy-middleware">GitHub 项目地址</a></p>
</body>
</html>
    `, {
      status: 200,
      headers: { "Content-Type": "text/html; charset=utf-8" }
    });
  }
  
  // 从 Cookie 中获取已设置的代理站点
  const cookies = parseCookies(request.headers.get("cookie"));
  let targetDomain = cookies.site || "https://github.com"; // 默认为 GitHub
  console.log("当前代理目标:", targetDomain);

  const path = url.pathname;
  let requestedUrl = url.searchParams.get("url");
  let setCookie = false;
  let hostFromUrl = "";

  // 支持更多参数格式
  if (!requestedUrl) {
    // 检查是否有 r 参数(base64编码)
    const rParam = url.searchParams.get("r");
    if (rParam) {
      try {
        requestedUrl = atob(rParam); // base64解码
        console.log("从r参数解码URL:", requestedUrl);
      } catch (e) {
        console.error("base64解码失败:", e.message);
      }
    }
  }

  if (requestedUrl) {
    // 如果是根路径且没有设置站点,显示项目README
    if (requestedUrl === "/" && targetDomain === "https://github.com") {
      requestedUrl =
        "https://raw.githubusercontent.com/RavelloH/edge-proxy-middleware/main/README.md";
    }
    try {
      // 检测是否为完整的URL(包含域名)
      if (
        requestedUrl.match(
          /^https?:\/\/[a-zA-Z0-9-]+(\.[a-zA-Z0-9-]+)+(\:[0-9]+)?(\/|$)/
        ) &&
        !requestedUrl.match(
          /^https?:\/\/[^\/]+\.(jpg|jpeg|png|gif|svg|webp|ico|css|js)$/i
        )
      ) {
        // 解析URL并更新目标域名
        const urlObj = new URL(requestedUrl);
        hostFromUrl = urlObj.protocol + "//" + urlObj.host;

        // 如果检测到新的主机名,更新Cookie
        if (hostFromUrl !== targetDomain) {
          targetDomain = hostFromUrl;
          setCookie = true;
        }
      } else {
        // 处理相对路径,拼接到目标域名
        if (requestedUrl.startsWith("/")) {
          requestedUrl = requestedUrl.substring(1);
        }
        requestedUrl = `${targetDomain}/${requestedUrl
          .replace("https://", "")
          .replace("http://", "")}`;
      }
    } catch (error) {
      console.error("URL处理错误:", error);
      // 出错时作为相对路径处理
      if (requestedUrl.startsWith("/")) {
        requestedUrl = requestedUrl.substring(1);
      }
      requestedUrl = `${targetDomain}/${requestedUrl}`;
    }
  } else {
    // 使用当前路径构建请求URL
    requestedUrl = `${targetDomain}${path
      .replace("https://", "")
      .replace("http://", "")}`;
    console.log("使用当前路径:", path);
  }

  // 构建查询参数(排除url、site和r参数)
  const originalQueryParams = new URLSearchParams(url.search);
  const newQueryParams = new URLSearchParams();

  for (const [key, value] of originalQueryParams.entries()) {
    if (!["url", "site", "r"].includes(key)) {
      newQueryParams.append(key, value);
    }
  }

  // 添加查询参数到请求URL
  const queryString = newQueryParams.toString();
  if (queryString && !requestedUrl.includes("?")) {
    requestedUrl += "?" + queryString;
  } else if (queryString) {
    requestedUrl += "&" + queryString;
  }

  requestedUrl = decodeURIComponent(requestedUrl);
  console.log("最终请求URL:", requestedUrl);

  if (!requestedUrl) {
    return new Response("缺少 url 参数且未设置站点 Cookie", {
      status: 400,
    });
  }

  // 构建转发请求头(过滤掉不需要的头部)
  const forwardedHeaders = new Headers();
  const excludeHeaders = ["host", "connection", "content-length", "cf-ray", "cf-connecting-ip", "x-forwarded-for"];
  
  for (const [key, value] of request.headers.entries()) {
    const lowerKey = key.toLowerCase();
    if (!excludeHeaders.includes(lowerKey)) {
      forwardedHeaders.set(key, value);
    }
  }

  // 添加代理标识
  forwardedHeaders.set("User-Agent", forwardedHeaders.get("User-Agent") + " (EdgeProxy/1.0)");

  try {
    // 设置请求超时和更好的错误处理
    const controller = new AbortController();
    const timeoutId = setTimeout(() => controller.abort(), 30000); // 30秒超时

    // 发起代理请求
    const proxyResponse = await fetch(requestedUrl, {
      method: request.method,
      headers: forwardedHeaders,
      body: request.method !== "GET" && request.method !== "HEAD" ? request.body : undefined,
      signal: controller.signal,
    });

    clearTimeout(timeoutId);

    // 检查响应状态
    if (!proxyResponse.ok) {
      console.log(`目标服务器响应状态: ${proxyResponse.status} ${proxyResponse.statusText}`);
      
      // 如果是404或其他客户端错误,返回友好的错误页面
      if (proxyResponse.status === 404) {
        return new Response(`
<!DOCTYPE html>
<html>
<head>
  <title>页面未找到 - Edge Proxy</title>
  <meta charset="utf-8">
  <style>
    body { font-family: -apple-system, sans-serif; max-width: 600px; margin: 100px auto; padding: 0 20px; text-align: center; }
    .error { color: #dc3545; }
    .info { background: #f8f9fa; padding: 20px; border-radius: 8px; margin: 20px 0; }
    code { background: #e9ecef; padding: 2px 6px; border-radius: 3px; }
  </style>
</head>
<body>
  <h1 class="error">😔 页面未找到</h1>
  <div class="info">
    <p><strong>目标URL:</strong> <code>${requestedUrl}</code></p>
    <p><strong>状态码:</strong> ${proxyResponse.status}</p>
    <p>该页面可能不存在,或者目标服务器暂时不可用。</p>
  </div>
  <p><a href="/">← 返回首页</a></p>
</body>
</html>`, {
          status: 404,
          headers: { "Content-Type": "text/html; charset=utf-8" }
        });
      }
    }

    const contentType = proxyResponse.headers.get("content-type") || "text/plain";
    const responseHeaders = {
      "Content-Type": contentType,
      "Access-Control-Allow-Origin": "*",
      "Access-Control-Allow-Methods": "GET, POST, PUT, DELETE, OPTIONS",
      "Access-Control-Allow-Headers": "*",
    };

    // 设置站点Cookie(24小时过期)
    if (setCookie && hostFromUrl) {
      responseHeaders["Set-Cookie"] = `site=${hostFromUrl}; path=/; max-age=86400; SameSite=Lax; Secure`;
    }

    return new Response(proxyResponse.body, {
      status: proxyResponse.status,
      statusText: proxyResponse.statusText,
      headers: responseHeaders,
    });
  } catch (err) {
    console.error(`代理请求失败: ${err.message}`);
    
    // 区分不同类型的错误
    let errorMessage = "代理服务器错误";
    let statusCode = 502;
    
    if (err.name === 'AbortError') {
      errorMessage = "请求超时";
      statusCode = 504;
    } else if (err.message.includes('fetch')) {
      errorMessage = "无法连接到目标服务器";
      statusCode = 503;
    }
    
    return new Response(`
<!DOCTYPE html>
<html>
<head>
  <title>${errorMessage} - Edge Proxy</title>
  <meta charset="utf-8">
  <style>
    body { font-family: -apple-system, sans-serif; max-width: 600px; margin: 100px auto; padding: 0 20px; text-align: center; }
    .error { color: #dc3545; }
    .info { background: #f8f9fa; padding: 20px; border-radius: 8px; margin: 20px 0; text-align: left; }
    code { background: #e9ecef; padding: 2px 6px; border-radius: 3px; word-break: break-all; }
  </style>
</head>
<body>
  <h1 class="error">⚠️ ${errorMessage}</h1>
  <div class="info">
    <p><strong>目标URL:</strong><br><code>${requestedUrl}</code></p>
    <p><strong>错误信息:</strong> ${err.message}</p>
    <p><strong>时间:</strong> ${new Date().toLocaleString('zh-CN')}</p>
  </div>
  <p>请检查目标URL是否正确,或稍后重试。</p>
  <p><a href="/">← 返回首页</a></p>
</body>
</html>`, {
      status: statusCode,
      headers: {
        "Content-Type": "text/html; charset=utf-8",
        "Access-Control-Allow-Origin": "*",
      },
    });
  }
}

// Deno Deploy 服务启动
Deno.serve({
  port: 8000, // 可选:指定端口
}, (request) => {
  return handleRequest(request);
});```
2 个赞

没有设置跨域~

Fix. 修复跨域问题 · RavelloH/edge-proxy-middleware@6984090

已经修复了

好像不行,post跨域的话会发送个OPTIONS请求

是在网页里调用的吗,我用postman什么的倒是都可以

嗯,网页里调的

感觉还可以拦截一些favicon 关键字..这些都是没啥用的请求

佬,服务器上怎么部署

Vercel上的暂时没法直接部署,不过可以用Cloudflare版的,功能是一样的:

  1. 命令行运行npx wrangler generate node-proxy
  2. 修改生成的文件夹中的index.js,把 edge-proxy-middleware/cloudflare/worker.js at main · RavelloH/edge-proxy-middleware · GitHub 直接复制进去
  3. 之后运行wrangler dev --port 8787就行,8787是端口号

期间可能要登录一下Cloudflare的账号

1 个赞