Genie Community Forum

Follow-up & Contextual Commands

Skill Suggestions
#1

Hi all!

I’m working on extending Almond with more complex interactions and transactions spawning multiple conversation turns.

I would like your feedback on how you would interact with the assistant after the assistant completed your first task. What commands you would expect to be supported next? How would you chain or transition from one command to the next?

I have created a survey here:

If you could provide your answers, that would be great!
You can answer multiple times with different ideas, if you like.

1 Like
#2

I’m interested in the outcome, will you be sharing the results?

1 Like
#3

Hi Dean! Welcome to the community!

I will probably not share the raw results, because I would like people answering the survey to do so freely without worrying about their answers being publicly available (even though the survey asks no personal information, and should have no privacy implication).
I will definitely share my findings though, in terms of which commands people like, which functionality is requested, which functionality is already covered by Almond and which will be supported by the new mechanism we’re working on.
Stay tuned!

2 Likes
#4

Very good point!

I’m personally interested in the specific topic of multiturn dialog management.
See also my rants about the lack of context management in GoogleAssistant and Amazon Alexa:

To manage dialogical context in an open-ended assistant, the keyword is maybe “coreference resolution” concept of pragmatics. See article: https://medium.com/deeppavlov/building-a-knowledge-graph-based-dialogue-system-at-the-2nd-convai-summer-school-ec2d0aa060e5

In broad terms, that assistant could keep memory (history) of intents & entities of (successful) turns. New user request could be “completed” with implicit intents/entities previously annotated (my horrible definition of “coreference”… you will forgive me because I’m not a linguist).

Here an example I submitted in the survey:

// turn 1
U: what’s the weather in Genova?
A: Current weather for Genova: blablabla
// context: { intent: weather, entities: { city: Genova, date: current_time} }

// turn 2
U: And tomorrow in Milano?
// candidate intent: weather
// candidate entities: city, time
A: Tomorrow weather for Milano: blablabla

An adjacent research concerns to integrate context with info user teached to his personal assistant. See the following example:

U: learn the my mom number is 0101234567
A: I get it!

some time after:

U: call my mom
A: calling phone number 0101234567

In this example there is more than a strict-context multiturn. Instead the assistant completes the context with facts learned in previous training dialogs… I love this feature :slight_smile:

Besides, in a more closed-ended / task-completion based dialog, maybe the old fashioned state-machine approach (for preplanned workflows) is a practical way. See my article:

#5

Now that Genie has satellite support, it would also be very useful to incorporate which satellite into the context.

Eg, if integrated with Home Assistant:

“Genie, turn off the lights”

Should turn off all of the lights in the area the satellite that heard the audio is in, not for the whole house (which could be covered by “Turn off ALL of the lights” though)

Similarly “Genie, play Gangam Style” should only play in the one room, not in every room throughout the whole house.