Skill development question: asking questions to friends

NatalieThurlby · May 17, 2021, 10:02am

Hi,

I’m a Research Software Developer working at the University of Bristol. I’m looking to help an academic here (an anthropology professor) in putting together a student project creating a skill for Thingapedia, about using virtual assistants (Almond) to co-operate on tasks. This isn’t my area, so I’m just looking for enough information to really be able to connect him with other IoT researchers here, and help him put together enough of a plan to move forwards with.

A simple example of the kind of co-operation that he is looking for would be the ability to ask Almond to “Ask my university friends if anyone has the notes for yesterday’s lecture”.

I’d like to just check my understanding of a couple of things first. So, my understanding is that Thingapedia contains Almond’s skills (and this kind of co-operation would be a skill) and these skills are written in JavaScript. Please let me know if I’ve misunderstood!

I’ve seen the skill tutorials and they look really useful, but I’m also wondering if there is any existing skills which are similar or anywhere that I could look in the repos or documentation that show an example of sharing data with friends, or managing existing circles of friends. My intuition is that this could build on top of lots of existing services, so there would be a lot of different ways to do this, so an example of something similar would be really useful I’m sure.

Thanks so much for your help!

gcampax · May 17, 2021, 9:55pm

Hi Natalie,

Thanks for reaching out!

What you suggest sounds quite useful. We did some work on “social” voice assistants in our paper Controlling Fine-Grain Sharing in Natural Language with a Virtual Assistant, which appeared in Ubicomp 2018. We introduced a system that would allow any requests of the assistant to be forwarded to another assistant over a chat platform.
Unfortunately, since the publication the Almond platform has evolved quite a bit, and that functionality is no longer available. Indeed, in the last year we’ve been going through a significant redesign of the Almond stack, as part of the upcoming Almond 2.0 release, and communication skills are no longer available. Existing code for the Slack, Twitter skills are in the thingpedia-common-devices repository but they are not functional at the moment. The skills also had very limited “friend-management” functionality, and could only send and receive messages.

This said, I think it should be relatively easy to build a specialized skill that sends message on a specific communication platform. As you noted, this will be a small JS package that bridges the SDK or API of the communication platforms with a set of high-level commands you will define. The commands and the JS package will then be uploaded to Thingpedia. You will also need to train the NLP model locally to take advantage of full natural language functionality (this requires access to a machine with a GPU); at the moment, we don’t have the resources to train models for third-parties.

Note that some communication platforms are more open to third-parties than other, so an early design decision will be the platform to use. Slack, Matrix, Telegram, Twitter are relatively open, Whatsapp, SMS can be used with some challenges, others are closed.

Hope this helps!