-
Notifications
You must be signed in to change notification settings - Fork 730
Closed
Milestone
Description
Hi,
I recently got PS with GStreamer working on Mac High Sierra. It was a bit convoluted and I'd like to share my results in case it helps other users. Let me know if you think there's an appropriate place to include the following info…
Install PocketSphinx
Attempts
- FAILED:
Very old version (0.8) which doesn't match docsbrew install cmu-pocketsphinx - Found tap to install HEAD on GH
GStreamer Integration
The PS docs say that leveraging GStream makes continuous mode easier, however, we couldn't get it working.
- Installed via
brew install gstreamer gst-plugins-base gst-plugins-good(last one was for testing) - Build gst-pocketsphinx plugin
FAILED: Reinstall PS to build the plugin; couldn't find gstreamer; posted question on both the homebrew and PS MLs- Cloned and built PS from scratch following the last script from this SO thread
- Manually copied the plugin (
*.sofile) to/usr/local/lib/gstreamer-1.0; NB: aliasing did not work, maybe a symlink would have
- Manually copied the plugin (
- Test
- Recognize sound file:
gst-launch-1.0 filesrc location=~/Documents/Reference/Foreign\ Language/Speak\ From\ Day\ One/audio_book/00_about.mp3 ! mpegaudioparse ! avdec_mp3 ! audioconvert ! audioresample ! pocketsphinx ! fdsink fd=1 - Recognize live sound
- Tried
but thought it wasn't working because no output; realized thatgst-launch-1.0 autoaudiosrc ! audioconvert ! audioresample ! pocketsphinx name=asr ! fakesinkfakesinkmeans throw away output! - Tried
gst-launch-1.0 autoaudiosrc ! audioconvert ! audioresample ! pocketsphinx name=asr ! fdsink fd=1, which returned (poorly) recognized text, but warned thatautoaudiosrc0-actual-src-osxaudi: Can't record audio fast enough. - Added
queueper the gst ML to get:gst-launch-1.0 autoaudiosrc ! queue ! audioconvert ! audioresample ! pocketsphinx name=asr ! fdsink fd=1. This thread also seemed to have some interesting ideas about how to address this.
- Tried
- Try python bindings
brew installWorkaround:export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python3.6/site-packages(per this GH issue)- Link Homebrew python module target to python3:
echo "/usr/local/lib/python3.6/site-packages" > /usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/Homebrew.pthper this SO thread
Usage Example - Live Audio
#!/usr/bin/env python
# Adapted from:
# - https://github.com/GStreamer/gst-python/blob/master/examples/helloworld.py
# - With parts from https://cmusphinx.github.io/wiki/gstreamer/
import sys
import signal
import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst
#For IPC
import socket
def on_element_message(bus, message, loop):
"""Receive element messages from the bus."""
msgtype = message.get_structure().get_name()
if msgtype != 'pocketsphinx':
return
if message.get_structure().get_value('final'):
final_result(message.get_structure().get_value('hypothesis'),
message.get_structure().get_value('confidence'))
def final_result(hyp, confidence):
#Debug - reference https://docs.python.org/3/tutorial/inputoutput.html#fancier-output-formatting
#print('Final: {}; confidence: {}'.format(hyp, confidence))
send_command(hyp)
def send_command(command):
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Connect the socket to the port where the server is listening
server_address = ('localhost', 9999)
#print >>sys.stderr, 'connecting to %s port %s' % server_address
sock.connect(server_address)
try:
# Send data
#print >>sys.stderr, 'sending "%s"' % command
sock.sendall(command.encode()) #.encode per https://stackoverflow.com/a/37376668/424245
# Look for the response
amount_received = 0
amount_expected = len(command)
while amount_received < amount_expected:
data = sock.recv(16)
amount_received += len(data)
#print >>sys.stderr, 'received "%s"' % data
finally:
#print >>sys.stderr, 'closing socket'
sock.close()
def main(args):
# Before using Python threads, or libraries using threads (GStreamer for example), you have to call GObject.threads_init(). Contrary to the naming, this function isn't provided by gobject but initializes thread support in PyGObject (it was called gobject.threads_init() in pygtk). Think of it as gi.threads_init().
# Since PyGObject 3.10.2, calling GObject.threads_init() this is no longer needed.
# Per https://wiki.gnome.org/Projects/PyGObject/Threading
GObject.threads_init()
Gst.init(None) # Cleanup via gst-deinit is normally not needed in an application as the resources will automatically be freed when the program terminates https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-Gst.html#gst-deinit
# Doc: https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstParse.html#gst-parse-launch
pipeline = Gst.parse_launch('autoaudiosrc ! queue ! audioconvert !' +
' audioresample ! pocketsphinx name=asr !' +
' fakesink') # to debug, use "fdsink fd=1" for last arg to write to stdout
if not pipeline:
sys.stderr.write("Failed to create pipeline\n")
sys.exit(1)
pocketsphinx = pipeline.get_by_name('asr')
#From https://raspberrypi.stackexchange.com/a/56818
pocketsphinx.set_property('kws', 'path/to/command.list')
# create and event loop and feed gstreamer bus mesages to it
loop = GObject.MainLoop.new(None, False)
bus = pipeline.get_bus()
bus.add_signal_watch()
bus.connect ('message::element', on_element_message, loop)
# start play back and listed to events
pipeline.set_state(Gst.State.PLAYING)
try:
loop.run()
except:
pass
# cleanup
print('Cleaning up...')
pipeline.set_state(Gst.State.NULL)
# Block until pipeline changes state (per http://gstreamer-devel.966125.n4.nabble.com/Gstreamer-critical-on-pipeline-set-state-Gst-State-NULL-tp4678899p4678912.html), but the docs say changes to null are never async and suggest an alternate check (https://lazka.github.io/pgi-docs/Gst-1.0/classes/Element.html#Gst.Element.set_state)
pipeline.get_state(Gst.CLOCK_TIME_NONE)
loop.quit()
print('Done!')
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv))Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels