You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thank you for considering contributing to this project! We appreciate your support and are excited to collaborate with you. To ensure a smooth and efficient collaboration process, please follow these guidelines.
4
+
5
+
## How to Contribute
6
+
7
+
### Reporting Bugs
8
+
9
+
If you find a bug, please report it by opening an issue in the [issue tracker](http://example.com/issues). Before reporting, please check if the issue has already been reported. When reporting a bug, include:
10
+
11
+
- A clear and descriptive title.
12
+
- A detailed description of the problem.
13
+
- Steps to reproduce the issue.
14
+
- Any relevant screenshots or logs.
15
+
16
+
### Suggesting Enhancements
17
+
18
+
If you have an idea for an enhancement, please suggest it by opening an issue in the [issue tracker](http://example.com/issues). Before suggesting, please check if the enhancement has already been proposed. When suggesting an enhancement, include:
19
+
20
+
- A clear and descriptive title.
21
+
- A detailed description of the enhancement.
22
+
- Any benefits or improvements it would bring.
23
+
24
+
### Submitting Pull Requests
25
+
26
+
If you'd like to contribute code, please follow these steps:
27
+
28
+
1.**Fork the Repository**: Click the "Fork" button at the top right of the repository page to create a copy of the repository on your GitHub account.
29
+
30
+
2.**Clone Your Fork**: Clone your forked repository to your local machine.
3.**Create a Branch**: Create a new branch for your changes.
37
+
38
+
```sh
39
+
git checkout -b feature-branch-name
40
+
```
41
+
42
+
4.**Make Changes**: Make your changes to the codebase.
43
+
44
+
5.**Commit Changes**: Commit your changes with a clear and concise commit message.
45
+
46
+
```sh
47
+
git add .
48
+
git commit -m "Description of your changes"
49
+
```
50
+
51
+
6.**Push to Fork**: Push your changes to your forked repository.
52
+
53
+
```sh
54
+
git push origin feature-branch-name
55
+
```
56
+
57
+
7.**Open a Pull Request**: Open a pull request from your forked repository to the main repository. Provide a clear and descriptive title and description of your changes.
58
+
59
+
### Code Style
60
+
61
+
To maintain a consistent codebase, please adhere to the following coding standards:
62
+
63
+
- Follow the existing code style and conventions.
64
+
- Write clear and concise comments where necessary.
65
+
- Ensure your code is well-documented.
66
+
67
+
### Review Process
68
+
69
+
All submissions will be reviewed by the project maintainers. Feedback will be provided, and changes may be requested. Please be responsive to feedback and make the necessary updates to your submission.
70
+
71
+
### License
72
+
73
+
By contributing to this project, you agree that your contributions will be licensed under the [Apache 2.0 License](LICENSE).
74
+
75
+
### Code of Conduct
76
+
77
+
Please note that this project is governed by a [Code of Conduct](CODE_OF_CONDUCT.md). By participating, you are expected to adhere to this code.
78
+
79
+
Thank you for contributing! We appreciate your efforts and look forward to working with you.
Enables the rapid orchestration and reuses of the latest large model capabilities, achieves low-latency, real-time multi-modal interactions with AI Agents.
@@ -23,17 +24,21 @@ Enables the rapid orchestration and reuses of the latest large model capabilitie
23
24
24
25
### Playground
25
26
26
-
We provide a [playground](https://astra-agents.agora.io/) for you to play with.
We provide a [live playground](https://astra-agents.agora.io/) where you can experiment and interact with the Astra powered voice agent.
27
32
28
33
### Local Agent
29
34
30
-
Currently, the agent we have built runs only on Linux. However, we have a Docker image ready for you to build and run the agent on Windows and macOS.
35
+
And of course, you are more than welcome to run our voice agent locally. We have a Docker image ready for you to build and run the agent on both macOS and Windows.
31
36
32
37
To start, make sure you have:
33
38
34
-
- Agora App ID and App Certificate([Read here on how](https://docs.agora.io/en/3.x/video-calling/reference/manage-agora-account?platform=android))
39
+
- Agora App ID and App Certificate([Read here on how](https://docs.agora.io/en/video-calling/get-started/manage-agora-account?platform=web))
35
40
- Azure's [speech-to-text](https://azure.microsoft.com/en-us/products/ai-services/speech-to-text) and [text-to-speech](https://azure.microsoft.com/en-us/products/ai-services/text-to-speech) API keys
36
-
-[OpenAI](https://openai.com/index/openai-api/) API keys.
41
+
-[OpenAI](https://openai.com/index/openai-api/) API keys
The Astra Service is built from various Astra extensions developed in different programming languages. The concept of a graph is used to describe the relationships between these extensions and illustrate the flow of data. Additionally, sharing and downloading extensions are made easy through the Astra cloud store and Astra package manager.
An extension is the fundamental unit of composition. Developers can create extensions in various languages and combine them in different ways to build diverse scenarios and applications. The Astra framework emphasizes cross-language collaboration, allowing extensions written in different programming languages to seamlessly work together within the same application or service.
86
-
87
-
For example, if an application requires real-time communication (RTC) features and advanced AI capabilities, a developer might choose to write RTC-related extensions in C++ for its performance advantages in processing audio and video data. At the same time, they could develop AI extensions in Python to leverage its extensive libraries and frameworks for data analysis and machine learning tasks.
88
-
89
-
#### Supported Languages
90
-
91
-
Up until June 2024, we support extensions written in following languages,
92
-
93
-
- C++
94
-
- Golang
95
-
- Python (Planned in July)
96
-
97
-
### Graph
98
-
99
-
A graph is used to describe the data flow between extensions, a graph in Astra orchestrates how different extensions interact. For example, the text output from a speech-to-text (STT) extension might be directed to a large language model (LLM) extension. Essentially, a graph defines which extensions are involved and the direction of data flow between them. Developers can customize this flow, directing outputs from one extension, such as an STT, into another, like an LLM.
100
-
101
-
In Astra, there are four main types of data flow between extensions:
102
-
103
-
- Command
104
-
- Data
105
-
- Image frame
106
-
- PCM frame
107
-
108
-
By specifying the direction of these data types in the graph, developers can enable mutual invocation and unidirectional data flow between plugins. This is especially useful for PCM and image data types, making audio and video processing simpler and more intuitive.
109
-
110
-
### Agent App
111
-
112
-
A runnable server-side participant application compiled to combine multiple **Extensions** following **Graph** rules to accomplish more sophisticated operations.
113
-
114
-
### Cloud Store
115
-
116
-
Cloud Store is a centralized platform for developers to share their extensions and access those created by others.
117
-
118
-
### Package Manager
119
-
120
-
Simplifies the process of uploading, sharing, downloading, and installing Astra extensions. Extensions can specify dependencies on other extensions and the environment, and the package manager automatically manages these dependencies, making the installation and release of extensions extremely convenient.
76
+
🎉 Congratulations! You now have our Astra.ai voice agent running locally.
121
77
122
78
</br>
123
79
124
-
# Fine-tune your agent
80
+
# Agent Customization
125
81
126
82
### Example
127
83
128
-
This project provides an example Agent App to help you get started.
129
-
It uses following Extensions:
84
+
Our voice agent is a great place for you to get started with, it uses following Extensions:
130
85
131
86
-_agora_rtc_ / [Agora](https://docs.agora.io/en) for RTC transport + VAD + Azure speech-to-text (STT)
132
87
-_azure_tts_ / [Azure](https://azure.microsoft.com/en-us/products/ai-services/text-to-speech) for text-to-speech (TTS)
133
88
-_openai_chatgpt_ / [OpenAI](https://openai.com/index/openai-api/) for LLM
134
89
-_chat_transcriber_ / A utility ext to forward chat logs into channel
135
90
-_interrupt_detector_ / A utility ext to help interrupt agent
This will generate an agent executable. We can change the source code in `agents/addon/extension/openai_chatgpt/openai_chatgpt.go` for instance to adjust your prompts and OpenAI parameters.
118
+
This code generates an agent executable. To customize your prompts and OpenAI parameters, modify the source code in agents/addon/extension/openai_chatgpt/openai_chatgpt.go.
162
119
163
-
Once done, we can use the following commands to start a server you then can test out with Astra Agent playground like we did in previous steps.
120
+
Once you have made the necessary changes, you can use the following commands to start a server. You can then test it out using the Astra voice agent playground as we did in previous steps.
🎉 Congratulations! You have created your first personalized voice agent. We appreciate your effort and look forward to seeing it in the extension store. We’d love it if you could share it within the community.
138
+
139
+
<br />
140
+
141
+
# Astra Service
142
+
143
+
Now let's discuss what's under the hood. The Astra Service is composed of various Astra extensions, developed in different programming languages. These extensions are interconnected using the concept of a graph, which describes their relationships and illustrates the flow of data. Furthermore, sharing and downloading extensions are simplified through the Astra cloud store and the Astra package manager.
An extension is the fundamental unit of composition within the Astra framework. Developers can create extensions in various programming languages and combine them to build diverse scenarios and applications. Astra emphasizes cross-language collaboration, allowing extensions written in different languages to work together seamlessly within the same application or service.
154
+
155
+
For example, if an application requires real-time communication (RTC) features and advanced AI capabilities, a developer might choose to write RTC-related extensions in C++ for its performance advantages in processing audio and video data. Meanwhile, they could develop AI extensions in Python to leverage its extensive libraries and frameworks for data analysis and machine learning tasks.
156
+
157
+
#### Supported Languages
158
+
159
+
As of June 2024, we support extensions written in the following languages:
160
+
161
+
- C++
162
+
- Golang
163
+
- Python (planned for July)
164
+
165
+
### Graph
166
+
167
+
A graph in Astra describes the data flow between extensions, orchestrating their interactions. For example, the text output from a speech-to-text (STT) extension might be directed to a large language model (LLM) extension. Essentially, a graph defines which extensions are involved and the direction of data flow between them. Developers can customize this flow, directing outputs from one extension, such as an STT, into another, like an LLM.
168
+
169
+
In Astra, there are four main types of data flow between extensions:
170
+
171
+
- Command
172
+
- Data
173
+
- Image frame
174
+
- PCM frame
175
+
176
+
By specifying the direction of these data types in the graph, developers can enable mutual invocation and unidirectional data flow between plugins. This is especially useful for PCM and image data types, simplifying audio and video processing.
177
+
178
+
### Astra Agent App
179
+
180
+
An Astra Agent App is a runnable server-side application that combines multiple Extensions following Graph rules to accomplish more sophisticated operations.
181
+
182
+
### Astra Store
183
+
184
+
The Astra Store is a centralized platform where developers can share their extensions and access those created by others.
185
+
186
+
### Astra Package Manager
187
+
188
+
The Astra Package Manager simplifies the process of uploading, sharing, downloading, and installing Astra extensions. Extensions can specify dependencies on other extensions and the environment, and the package manager automatically manages these dependencies, making the installation and release of extensions extremely convenient.
189
+
180
190
</br>
181
191
182
192
## TODO
@@ -189,6 +199,12 @@ make run-server
189
199
190
200
</br>
191
201
192
-
## Code Contributors
202
+
## Contributing
203
+
204
+
Contributions are welcome! Please read the [contribution guidelines](CONTRIBUTING.md) first.
205
+
206
+
</br>
207
+
208
+
## License
193
209
194
-
A heartfelt thanks to all contributors!
210
+
This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details.
0 commit comments