mono - chore: moving keyv to better mono structure#1838
Conversation
![aikido-pr-checks[bot]](https://avatars.githubusercontent.com/in/898896?s=60&v=4){kind=small}
Summary of ChangesHello @jaredwray, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request undertakes a substantial refactoring of the project's monorepo architecture. The primary goal is to enhance the clarity and organization of the codebase by grouping related packages into logical categories such as core components, compression adapters, serialization utilities, and various storage adapters. This change impacts file paths across the repository, necessitating updates to configuration files, documentation, and build scripts to ensure consistency and proper functionality within the new structure. Highlights
Changelog
Ignored Files
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1838 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 32 32
Lines 2434 2434
Branches 446 446
=========================================
Hits 2434 2434 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3c5eee2488
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
jaredwray
deleted the
mono---chore-moving-keyv-to-better-mono-structure
branch
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request significantly refactors the monorepo structure, organizing packages into directories like core, storage, and compression. While this is a great organizational improvement, a critical Path Traversal vulnerability was identified in the new documentation generation script website/src/docs.ts, which could allow an attacker to read or write arbitrary files on the build server. Additionally, consider reverting the Node.js version bump in .nvmrc to an LTS release for stability, and address maintainability and readability suggestions in website/src/docs.ts.
| async function createDoc(adapterName: string, path: string, outputPath: string, parent:string) { | ||
| const originalFileName = "README.md"; | ||
| const newFileName = `${adapterName}.md`; | ||
| const packageJSONPath = `${path}/${adapterName}/package.json`; | ||
| const packageJSONContent = await fs.promises.readFile(packageJSONPath); | ||
| const packageJSON = JSON.parse(packageJSONContent.toString()); | ||
| const originalFileText = await fs.promises.readFile(`${path}/${adapterName}/${originalFileName}`, "utf8"); | ||
| let newFileText = "---\n"; | ||
| newFileText += `title: '${packageJSON.name}'\n`; | ||
| newFileText += `sidebarTitle: '${packageJSON.name}'\n`; | ||
| newFileText += `parent: '${parent}'\n`; | ||
| newFileText += "---\n"; | ||
| newFileText += "\n"; | ||
| newFileText += originalFileText; | ||
|
|
||
| newFileText = cleanDocumentFromImage(newFileText); | ||
|
|
||
| await fs.promises.mkdir(outputPath, {recursive: true}); | ||
| await fs.promises.writeFile(`${outputPath}/${newFileName}`, newFileText); | ||
| } |
There was a problem hiding this comment.
The createDoc function is vulnerable to path traversal. The adapterName parameter, which is derived from directory listings, is used to construct file paths for reading and writing without proper sanitization. An attacker who can control the names of files or directories in the source directories could use path traversal sequences (e.g., ../) to cause the script to read or write files from arbitrary locations on the file system where the script is executed. This could lead to information disclosure or arbitrary file writes on the build server. It is recommended to use the path module for constructing paths and to sanitize the input from readdir.
async function createDoc(adapterName: string, basePath: string, outputPath: string, parent:string) {
const path = await import('node:path');
const safeAdapterName = path.basename(adapterName);
const originalFileName = "README.md";
const newFileName = `${safeAdapterName}.md`;
const packageJSONPath = path.join(basePath, safeAdapterName, 'package.json');
const packageJSONContent = await fs.promises.readFile(packageJSONPath);
const packageJSON = JSON.parse(packageJSONContent.toString());
const originalFileText = await fs.promises.readFile(path.join(basePath, safeAdapterName, originalFileName), "utf8");
let newFileText = "---\
";
newFileText += `title: '${packageJSON.name}'\
`;
newFileText += `sidebarTitle: '${packageJSON.name}'\
`;
newFileText += `parent: '${parent}'\
`;
newFileText += "---\
";
newFileText += "\
";
newFileText += originalFileText;
newFileText = cleanDocumentFromImage(newFileText);
await fs.promises.mkdir(outputPath, {recursive: true});
await fs.promises.writeFile(path.join(outputPath, newFileName), newFileText);
}| @@ -1 +1 @@ | |||
| 20 No newline at end of file | |||
| 24 No newline at end of file | |||
There was a problem hiding this comment.
The Node.js version has been updated from 20 (LTS) to 24 (Current). For better stability and long-term support, it is recommended to use an LTS version. Consider using Node.js 22, which is the next LTS release, or sticking with Node.js 20.
22
| const basePath = await getBasePath(); | ||
|
|
||
| console.log("base path:" + basePath); | ||
| console.log("docs path:" + await getRelativeDocsPath()); |
There was a problem hiding this comment.
The function getRelativeDocsPath is called here, but its return value is only used for logging and is not used elsewhere. This line and the getRelativeDocsPath function (lines 100-108) appear to be dead code and can be removed to improve maintainability.
| document = document.replace(`<h1 align="center"><img width="250" src="https://jaredwray.com/images/keyv.svg" alt="keyv"></h1>`, ""); | ||
| document = document.replace(`[<img width="100" align="right" src="https://jaredwray.com/images/keyv.svg" alt="keyv">](https://github.com/jaredwra/keyv)`, ""); | ||
| document = document.replace(`[<img width="100" align="right" src="https://jaredwray.com/images/keyv-symbol.svg" alt="keyv">](https://github.com/jaredwra/keyv)`, ""); | ||
| return document; |
There was a problem hiding this comment.
This function can be written more concisely by chaining the replace calls. This improves readability and avoids reassigning the document variable multiple times.
| document = document.replace(`<h1 align="center"><img width="250" src="https://jaredwray.com/images/keyv.svg" alt="keyv"></h1>`, ""); | |
| document = document.replace(`[<img width="100" align="right" src="https://jaredwray.com/images/keyv.svg" alt="keyv">](https://github.com/jaredwra/keyv)`, ""); | |
| document = document.replace(`[<img width="100" align="right" src="https://jaredwray.com/images/keyv-symbol.svg" alt="keyv">](https://github.com/jaredwra/keyv)`, ""); | |
| return document; | |
| return document | |
| .replace(`<h1 align="center"><img width="250" src="https://jaredwray.com/images/keyv.svg" alt="keyv"></h1>`, "") | |
| .replace(`[<img width="100" align="right" src="https://jaredwray.com/images/keyv.svg" alt="keyv">](https://github.com/jaredwra/keyv)`, "") | |
| .replace(`[<img width="100" align="right" src="https://jaredwray.com/images/keyv-symbol.svg" alt="keyv">](https://github.com/jaredwra/keyv)`, ""); |
1 participant
Please check if the PR fulfills these requirements
What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
mono - chore: moving keyv to better mono structure