[Feat]: Evaluations for the trace spans by AmanAgarwal041 · Pull Request #614 · openlit/openlit

AmanAgarwal041 · 2025-02-28T19:38:35Z

Overview:

This PR enables a feature of evaluating your trace spans for Hallucinations, Bias & Toxicity.

You can enable this functionality by creating evaluation config in the settings screen
You can even enable the auto evaluations by proving cron recurring time which will create a cron job and evaluate your traces in the background.
For this it uses the vault feature where you can select your creds to make the evaluations happen
If you don't want the auto evaluations , you can even run evaluations for each trace spans
For evaluations it uses litellm in the background, so you can configure your own provider and models for the evaluations
You also see some dashboard panels in the dashboard page with Number of Hallucinations, Bias & Toxicity detected parameters.
Database config, user profile, evaluation config, api keys have been moved to settings page
This PR also decouples types from other places in one single directory
This PR also decouples server and client helpers to improve the dev experience

Fixes #470

Visuals (If applicable):

Checklist:

PR name follows conventional commit format: [Feat]: ... or [Fix]: ....
Added visuals for changes (If applicable)
Checked OpenLIT contribution guidelines

Summary by Sourcery

Implements trace span evaluations for Hallucinations, Bias, and Toxicity using LiteLLM, enabling users to configure evaluations, schedule automatic evaluations via cron jobs, and view evaluation results on a dashboard. Moves database config, user profile, evaluation config, and API keys to the settings page.

New Features:

Adds the ability to evaluate trace spans for Hallucinations, Bias, and Toxicity.
Introduces evaluation configurations in the settings screen.
Enables automated evaluations with cron recurring time.
Integrates with Vault for credential management.
Allows manual evaluation of trace spans.
Utilizes litellm for evaluations, allowing custom provider and model configuration.
Displays dashboard panels for Hallucinations, Bias, and Toxicity metrics.
Adds an 'Evaluation' tab to the request details page to display evaluation results.

sourcery-ai · 2025-02-28T19:38:39Z

🧙 Sourcery has finished reviewing your pull request!

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!
Generate a plan of action for an issue: Comment @sourcery-ai plan on
an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey @AmanAgarwal041 - I've reviewed your changes - here's some feedback:

Overall Comments:

Consider creating a separate component for the logic that fetches and processes evaluation data to improve readability and maintainability.
The addition of evaluation functionality introduces a dependency on Python and litellm; ensure these dependencies are well-documented for setup.

Here's what I looked at during the review

🟢 General issues: all looks good
🟢 Security: all looks good
🟢 Testing: all looks good
🟢 Complexity: all looks good
🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

sourcery-ai · 2025-02-28T19:40:24Z

@@ -0,0 +1,55 @@
+// Environment variables for auto evaluation
+const EVALUATION_CONFIG_ID = process.env.EVALUATION_CONFIG_ID;


suggestion (code-quality): Prefer object destructuring when accessing and using properties. (use-object-destructuring)

Suggested change

const EVALUATION_CONFIG_ID = process.env.EVALUATION_CONFIG_ID;

const {EVALUATION_CONFIG_ID} = process.env;

Explanation
Object destructuring can often remove an unnecessary temporary reference, as well as making your code more succinct.

From the Airbnb Javascript Style Guide

sourcery-ai · 2025-02-28T19:40:24Z

@@ -0,0 +1,55 @@
+// Environment variables for auto evaluation
+const EVALUATION_CONFIG_ID = process.env.EVALUATION_CONFIG_ID;
+const CRON_ID = process.env.CRON_ID;


suggestion (code-quality): Prefer object destructuring when accessing and using properties. (use-object-destructuring)

Suggested change

const CRON_ID = process.env.CRON_ID;

const {CRON_ID} = process.env;

Explanation
Object destructuring can often remove an unnecessary temporary reference, as well as making your code more succinct.

From the Airbnb Javascript Style Guide

sourcery-ai · 2025-02-28T19:40:24Z

+// Environment variables for auto evaluation
+const EVALUATION_CONFIG_ID = process.env.EVALUATION_CONFIG_ID;
+const CRON_ID = process.env.CRON_ID;
+const API_URL = process.env.API_URL;


suggestion (code-quality): Prefer object destructuring when accessing and using properties. (use-object-destructuring)

Suggested change

const API_URL = process.env.API_URL;

const {API_URL} = process.env;

Explanation
Object destructuring can often remove an unnecessary temporary reference, as well as making your code more succinct.

From the Airbnb Javascript Style Guide

sourcery-ai · 2025-02-28T19:40:24Z

+	const { fireRequest, isLoading: isLoadingModify } = useFetchWrapper();
+	const { fireRequest: getVaultKeys, isLoading: isLoadingVaultKeys } =
+		useFetchWrapper();


suggestion (code-quality): Combine destructure assignments. (combine-object-destructuring)

Suggested change

const { fireRequest, isLoading: isLoadingModify } = useFetchWrapper();

const { fireRequest: getVaultKeys, isLoading: isLoadingVaultKeys } =

useFetchWrapper();

const {fireRequest, isLoading: isLoadingModify, fireRequest: getVaultKeys, isLoading: isLoadingVaultKeys} = useFetchWrapper();

Explanation
Object destructuring can often remove an unnecessary temporary reference, as well as making your code more succinct.

From the Airbnb Javascript Style Guide

sourcery-ai · 2025-02-28T19:40:24Z

+	if (!validationParam.success)
+		return Response.json(validationParam.err, {
+			status: 400,
+		});


suggestion (code-quality): Use block braces for ifs, whiles, etc. (use-braces)

Suggested change

if (!validationParam.success)

return Response.json(validationParam.err, {

status: 400,

});

if (!validationParam.success) {

return Response.json(validationParam.err, {

status: 400,

});

}

Explanation
It is recommended to always use braces and create explicit statement blocks.

Using the allowed syntax to just write a single statement can lead to very confusing
situations, especially where subsequently a developer might add another statement
while forgetting to add the braces (meaning that this wouldn't be included in the condition).

sourcery-ai · 2025-02-28T19:40:25Z

+	if (validateVaultId) {
+		throwIfError(
+			!updatedSecretData?.id,
+			getMessage().EVALUATION_VAULT_SECRET_NOT_FOUND
+		);
+	} else {
+		if (!updatedSecretData.id) {
+			updatedConfig.vaultId = "";
+		}
+	}


suggestion (code-quality): Merge else clause's nested if statement into else if (merge-else-if)

Suggested change

if (validateVaultId) {

throwIfError(

!updatedSecretData?.id,

getMessage().EVALUATION_VAULT_SECRET_NOT_FOUND

);

} else {

if (!updatedSecretData.id) {

updatedConfig.vaultId = "";

}

}

if (validateVaultId) {

throwIfError(

!updatedSecretData?.id,

getMessage().EVALUATION_VAULT_SECRET_NOT_FOUND

);

}

else if (!updatedSecretData.id) {

updatedConfig.vaultId = "";

}

Explanation
Flattening if statements nested within else clauses generates code that is
easier to read and expand upon.

sourcery-ai · 2025-02-28T19:40:25Z

+	excludeVaultValue: boolean = true
+) {
+	const query = `SELECT * ${
+		!!excludeVaultValue ? "EXCEPT value" : ""


suggestion (code-quality): Invert ternary operator to remove negation (invert-ternary)

Suggested change

!!excludeVaultValue ? "EXCEPT value" : ""

!excludeVaultValue ? "" : "EXCEPT value"

Explanation
Negated conditions are more difficult to read than positive ones, so it is best
to avoid them where we can. By inverting the ternary condition and swapping the
expressions we can simplify the code.

sourcery-ai · 2025-02-28T19:40:25Z

-    }
+	return withAuth(
+		async function middleware(request: NextRequest, _next: NextFetchEvent) {
+			const pathname = request.nextUrl.pathname;


suggestion (code-quality): Prefer object destructuring when accessing and using properties. (use-object-destructuring)

Suggested change

const pathname = request.nextUrl.pathname;

const {pathname} = request.nextUrl;

Explanation
Object destructuring can often remove an unnecessary temporary reference, as well as making your code more succinct.

From the Airbnb Javascript Style Guide

sourcery-ai · 2025-02-28T19:40:26Z

+				if (isAuth || isAllowedRequestWithoutToken || isCronJobRoute) {
+					if (isCronJobRoute) {
+						const cronJobToken = request.headers.get("X-CRON-JOB");
+						if (cronJobToken) {
+							return NextResponse.next();
+						}
+					}
+					return NextResponse.next();
+				}
+			}



suggestion (code-quality): Merge nested if conditions (merge-nested-ifs)

Suggested change

if (isApiPage) {

if (isAuth || isAllowedRequestWithoutToken || isCronJobRoute) {

if (isCronJobRoute) {

const cronJobToken = request.headers.get("X-CRON-JOB");

if (cronJobToken) {

return NextResponse.next();

}

}

return NextResponse.next();

}

}

if (isApiPage && (isAuth || isAllowedRequestWithoutToken || isCronJobRoute)) {

if (isCronJobRoute) {

const cronJobToken = request.headers.get("X-CRON-JOB");

if (cronJobToken) {

return NextResponse.next();

}

}

return NextResponse.next();

}

Explanation
Reading deeply nested conditional code is confusing, since you have to keep track of which
conditions relate to which levels. We therefore strive to reduce nesting where
possible, and the situation where two if conditions can be combined using
and is an easy win.

sourcery-ai · 2025-02-28T19:40:26Z

+        parsed_content = json.loads(content)
+        return parsed_content


suggestion (code-quality): Inline variable that is immediately returned (inline-immediately-returned-variable)

Suggested change

parsed_content = json.loads(content)

return parsed_content

return json.loads(content)

AmanAgarwal041 added 2 commits February 28, 2025 19:25

feat: evaluations init

69c0860

feat: evaluations fixes

1a90244

AmanAgarwal041 requested a review from a team as a code owner February 28, 2025 19:38

AmanAgarwal041 requested a review from amanagarwal042 February 28, 2025 19:38

github-actions Bot added the client Issue related to OpenLIT Client label Feb 28, 2025

sourcery-ai Bot reviewed Feb 28, 2025

View reviewed changes

AmanAgarwal041 added 5 commits March 4, 2025 14:07

fix: ui issues

89f1d2c

feat: evaluations error display, cron logger table update

0e03fba

conflict resolve, main merge

67c2445

Merge branch 'main' of github.com:dokulabs/doku into feat/evaluations

edd1fc6

reverted sdk python changes, upgraded minor version

3d13f64

amanagarwal042 changed the title ~~[Feat]: evaluations for the trace spans~~ [Feat]: Evaluations for the trace spans Mar 16, 2025

Merge branch 'main' into feat/evaluations

9cbf5fd

amanagarwal042 merged commit da9769b into main Mar 17, 2025

amanagarwal042 deleted the feat/evaluations branch March 17, 2025 08:43

		@@ -0,0 +1,55 @@
		// Environment variables for auto evaluation
		const EVALUATION_CONFIG_ID = process.env.EVALUATION_CONFIG_ID;

	const EVALUATION_CONFIG_ID = process.env.EVALUATION_CONFIG_ID;
	const {EVALUATION_CONFIG_ID} = process.env;

	const CRON_ID = process.env.CRON_ID;
	const {CRON_ID} = process.env;

	const API_URL = process.env.API_URL;
	const {API_URL} = process.env;

	!!excludeVaultValue ? "EXCEPT value" : ""
	!excludeVaultValue ? "" : "EXCEPT value"

	const pathname = request.nextUrl.pathname;
	const {pathname} = request.nextUrl;

	parsed_content = json.loads(content)
	return parsed_content
	return json.loads(content)

Uh oh!

Uh oh!

Conversation

AmanAgarwal041 commented Feb 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview:

Visuals (If applicable):

Checklist:

Summary by Sourcery

Uh oh!

sourcery-ai Bot commented Feb 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

sourcery-ai Bot Feb 28, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AmanAgarwal041 commented Feb 28, 2025 •

edited

Loading

sourcery-ai Bot commented Feb 28, 2025 •

edited

Loading