Genie Community Forum

Requirements

I am having some trouble trying to determine if almond can run 100% offline. I know for stuff like web look ups ect you have to have net access. But curious if any of the NLP things, the skills, ect have to have internet access. For example, lets say the ONLY thing I want it to do is interact with my local home automation system and i have the skill for that. If it’s disconnected from the internet or firewalled or whatever. Can it function? Will the NLP/Sentiment Analyisys/Intent discovery all work locally?

Side: I am looking in to alternatives to snips since sono’s decided to gut that project.

Hi @jason.brunk, welcome to the community!

Almond supports running with intermittent or unreliable connectivity (phone or laptop), but it is not designed to run 100% offline. In particular, it needs the network for the following:

Voice

Currently, speech to text inside Almond (using the full version of Almond Server, not the portable one) delegates to the Microsoft speech API. A version of the API that operates locally exists, but it still needs network because it is licensed and billed per API call, even though it runs locally. OTOH, if you use Ada (Home Assistant’s voice frontend) you might be able to plug in a different voice frontend, such as rhasspy or deepspeech.

NLP

NLP is a fairly heavy classical preprocessing pipeline (tokenization, part of speech tagging, named entity recognition), followed by a mid-size neural network. Both are currently run on our servers (at https://nlp.almond.stanford.edu). The requirements are about 2.5G disk, 1.5G RAM, 1 vCPU for preprocessing, and 6G disk, 4G RAM, 1.5 vCPU for the neural network (you don’t need a GPU for inference). This is big enough that we don’t feel like people should download and run this by themselves.

The software is fully open source, and the models will be available for download. Documentation for how to run this exists but it’s somewhat piecemeal; I have a plan to consolidate it eventually.

Thingpedia

Almond needs to contact the Thingpedia server to download the skill code and metadata. It does that once when you set up a skill, and periodically to check for updates. The skill code is cached in the Almond cache directory, which is inside the mounted volume if you use docker, and in ~/.cache otherwise.
If you set up the skills ahead of time, you might be able to cut off the connection here. This is not a well-tested configuration though, and you might get spurious errors. In particular, you will get spurious error if the NLP neural network interprets a command as belonging to a skill you have not cached. You might also see the skills become out of date compared to the NLP model, which will cause some commands to suddenly fail with “Sorry, I did not understand that.”

The services themselves

Almond talks to the web services that back the skills directly, as you noticed. If you’re only interested in the Home Assistant skill, then you might be able to run Home Assistant on the local network and cordon off the rest. Note though that certain skills which are available in Home Assistant, like weather, are actually native skills in Almond, so you’ll need to safelist them if you want to use them.

Hope this helps!

1 Like

Thank’s for this write up!

Is this all still true for “Almond Edge” or can this be run completely offline now?

Hello!

Yes, the architecture is the same for the latest version of Almond. To run completely offline, you’ll need to run the NLP model locally. Models are available for download from Datasets & Software Download | Open Virtual Assistant Lab (scroll to the bottom). You can point almond-server directly to the model in a local directory, or run the model in a different machine using genie server.

By the way, the model is quite a bit bigger than it used to be, due to the use of the bootleg named entity recognizer. For our currently trained model, we recommend at least 20G of disk, 30G of RAM, and a GPU to get acceptable performance.

1 Like