Repository navigation
Rename TextChunker from SemanticTextPartitioner - #683
Merged
Adrian Bonar (adrianwyatt) merged 2 commits intoApr 26, 2023
Merged
Adrian Bonar (adrianwyatt) merged 2 commits into
Adrian Bonar (adrianwyatt) merged 2 commits into
Conversation
This commit refactors the SemanticTextPartitioner class, which splits text into chunks based on punctuation and new lines, and renames it to TextChunker. The new name better reflects the functionality of the class and avoids confusion with the SemanticText and SemanticPartitioner classes, which are different abstractions for text analysis. The TextChunker class is also moved from the SemanticFunctions.Partitioning namespace to the Text namespace, as it is a utility class rather than a semantic function. The commit updates the references to the class in the ConversationSummarySkill and the DocumentImportController classes, as well as the documentation.
Lee Miller (lemillermicrosoft)
requested review from
Abby Harrison (awharrison-28),
Devis Lucato (dluc) and
Shawn Callegari (shawncal)
April 26, 2023 23:09
Shawn Callegari (shawncal)
approved these changes
Apr 26, 2023
Adrian Bonar (adrianwyatt)
enabled auto-merge (squash)
April 26, 2023 23:37
Devis Lucato (dluc)
pushed a commit
that referenced
this pull request
Apr 29, 2023
### Motivation and Context To match changes in #450 ### Description This pull request renames the SemanticTextPartitioner class to TextChunker, and updates all references to this class in the code and documentation. The TextChunker class is responsible for splitting text into smaller chunks based on tokens and paragraphs, which is useful for processing large texts or code files. The new name TextChunker reflects the functionality of the class more clearly and avoids confusion with the SemanticPartitioner class, which is a different concept that splits text into semantic segments based on embeddings. Details: - Rename SemanticTextPartitioner.cs to TextChunker.cs and update the class name and namespace accordingly. The TextChunker class is now in the Text namespace, which is more consistent with its purpose and the other classes in the namespace. - Update all references to SemanticTextPartitioner in the code and documentation to use TextChunker instead. This includes the ConversationSummarySkill and the DocumentImportController classes, which use the TextChunker class to split text into sentences and paragraphs for summarization and import. - Update the README.md file for the GitHub Repo Q&A Bot sample to explain how the TextChunker class works with memory and embeddings. - Make some minor formatting changes to the TextChunker class, such as adding line breaks and spaces for readability, and removing an unused using directive.
Desmond Howard (dehoward)
pushed a commit
to lemillermicrosoft/semantic-kernel
that referenced
this pull request
Jun 1, 2023
### Motivation and Context To match changes in microsoft#450 ### Description This pull request renames the SemanticTextPartitioner class to TextChunker, and updates all references to this class in the code and documentation. The TextChunker class is responsible for splitting text into smaller chunks based on tokens and paragraphs, which is useful for processing large texts or code files. The new name TextChunker reflects the functionality of the class more clearly and avoids confusion with the SemanticPartitioner class, which is a different concept that splits text into semantic segments based on embeddings. Details: - Rename SemanticTextPartitioner.cs to TextChunker.cs and update the class name and namespace accordingly. The TextChunker class is now in the Text namespace, which is more consistent with its purpose and the other classes in the namespace. - Update all references to SemanticTextPartitioner in the code and documentation to use TextChunker instead. This includes the ConversationSummarySkill and the DocumentImportController classes, which use the TextChunker class to split text into sentences and paragraphs for summarization and import. - Update the README.md file for the GitHub Repo Q&A Bot sample to explain how the TextChunker class works with memory and embeddings. - Make some minor formatting changes to the TextChunker class, such as adding line breaks and spaces for readability, and removing an unused using directive.
golden-aries
pushed a commit
to golden-aries/semantic-kernel
that referenced
this pull request
Oct 10, 2023
### Motivation and Context To match changes in microsoft#450 ### Description This pull request renames the SemanticTextPartitioner class to TextChunker, and updates all references to this class in the code and documentation. The TextChunker class is responsible for splitting text into smaller chunks based on tokens and paragraphs, which is useful for processing large texts or code files. The new name TextChunker reflects the functionality of the class more clearly and avoids confusion with the SemanticPartitioner class, which is a different concept that splits text into semantic segments based on embeddings. Details: - Rename SemanticTextPartitioner.cs to TextChunker.cs and update the class name and namespace accordingly. The TextChunker class is now in the Text namespace, which is more consistent with its purpose and the other classes in the namespace. - Update all references to SemanticTextPartitioner in the code and documentation to use TextChunker instead. This includes the ConversationSummarySkill and the DocumentImportController classes, which use the TextChunker class to split text into sentences and paragraphs for summarization and import. - Update the README.md file for the GitHub Repo Q&A Bot sample to explain how the TextChunker class works with memory and embeddings. - Make some minor formatting changes to the TextChunker class, such as adding line breaks and spaces for readability, and removing an unused using directive.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
To match changes in #450
Description
This pull request renames the SemanticTextPartitioner class to TextChunker, and updates all references to this class in the code and documentation. The TextChunker class is responsible for splitting text into smaller chunks based on tokens and paragraphs, which is useful for processing large texts or code files. The new name TextChunker reflects the functionality of the class more clearly and avoids confusion with the SemanticPartitioner class, which is a different concept that splits text into semantic segments based on embeddings.
Details:
Contribution Checklist
dotnet format