Genie Community Forum

Use Almond in my Python Program

Developer Support
#1

Hi, I have a python program that has speech input capabilities and a music player. I want to turn it into a smart assistant able to respond to inputs. I have been playing around with NLP for a while, and I stumbled upon Almond and was amazed at how much it conforms to the vision of what I want my program to be like.

Is there a way to embed Almond into my Python Program? Functioning like a dependency, where I can call its method. Most notably, its NLP and IOT capabilities.

Thank you for your time.

I am Yours most Sincerely,
Adam Lim

#2

Hi @mostaste,
Welcome to our community!

The easiest way to consume Almond from a Python program is through the Almond HTTP API. The API documentation is at https://almond.stanford.edu/doc/my-api.md, and you can use the pyalmond client library as a wrapper.

The HTTP API is provided by the cloud instance of Almond, or by a locally running almond-server, which can you run directly as a subprocess of your Python program, or through docker.

Alternatively, if you use the GTK version of Almond, a very similar API is accessible through DBus.

1 Like
#3

Thank you for the information!

Also, how do I get a customized LUINet model? It says that I have to email and ask.

I Am Yours Most Sincerely,
Adam Lim

#4

We release some pretrained LUInet models (nowadays called GenieNLP) at https://wiki.almond.stanford.edu/releases
Otherwise, you can train your own. To train models that match our papers, follow the instructions in that wiki page, or use one of the starter folders from genie-toolkit. To train a model with real Thingpedia devices, follow the instructions in the thingpedia-common-devices repository.

Additionally, we can also arrange to collaborate, and we can help you build the model. Perhaps you can tell us what you’re interested on? Is this for a product or for research?

1 Like
#5

This is a personal project, which I hope that one day might turn into a product. I want to build a Conversational User Interface(CUI) . A home computer that can execute many tasks and aid people in their daily lives.

Some of the basic things include, Date, Time, Weather, Calculator, Note/Reminder, Music Player, Emails, and much much more. The scope of which I have not yet dreamt about.

The CUI basically functions as a Natural Language Processor, and you navigate the entire system with only conversation.

For more complex tasks, you would issue a command to drop into a protocol, say Music Protocol for instance. And from within Music Protocol, you can issue smaller commands like turning the volume up, play the next track, pause the song, etc.

Even the most basic things, Setting an Alarm, Creating Reminders, Playing Music is enough to make me happy.

And probably a Conversational Shell where the user does not expect to execute any tasks but just talking with the CUI.

#6

Is it also possible to train the model to execute function calls within a python program rather than for thingpedia?

#7

The model is trained to execute ThingTalk programs, which are small programs in our domain-specific language. Each program calls functions in Thingpedia. Currently, both ThingTalk and Thingpedia are implemented in JS, and need a JS runtime to execute.

In any case, what you describe sounds very similar to the Almond assistant, which also hopes to be a general purpose voice assistant supporting common commands. You can find our roadmap on our wiki - you’ll notice a reference to the top 10 skills.
So maybe Almond can serve as the “outer shell” for what you’re trying to build?

#8

Perhaps so. How do you advise I proceed from this?

I am yours most sincerely,
Adam Lim

#9

Well, the idea is to build your custom commands as Thingpedia skills, that you invoke from a plain install of Almond. The Thingpedia skills can then talk to your Python backend over IPC or network.