Genie Community Forum

What if Natural language utterances conflict?

Hi Almond Authors and Community! My name is Shokhan and I am a ML Researcher. First of all, I really appreciate this project and glad to be involved in! Thanks!
Currently I am testing one skill and I have a technical question:

  • What if my natural language utterance conflicts with similar one from another skill? So, when I type my natural language(exactly as is, because I did not train it yet) to test the skill, NLP model recognizes it as the command for another public skill and thus runs not my skill, but the public one. How to overcome this issue? What happens when I train the model?

Hi @shokhan,

Welcome to the community!

Because you have a developer account, you’re using the developer model. This model contains all skills that were submitted to Thingpedia, including those that are not approved yet, and for this model, conflicts are indeed to be expected.

There are three ways to solve the conflict:

  • if you type exactly the sentence you trained with, your skill should be preferred to you, because your skill came later and it’s visible to you; note: for get commands, you need to say exactly “get” followed by the phrase you entered in Thingpedia; for when commands, you should say exactly “notify me” followed by the phrase you entered. If this doesn’t work, it’s a bug, and I can take a look if you don’t mind telling me what sentence you’re trying.
  • after training, you can use your skill name as a way to disambiguate, that is, you can say “ask foo to …” where “foo” is the name of your skill; this is similar to how Alexa and Google Assistant work
  • upon review, and before a skill is made public, conflicts are resolved; it is expected that public skills are sufficiently distinct, and when they are not distinguishable, they should use a shared base class

Does this make sense?

1 Like

Hi @gcampax,
Thanks for the immediate reply!

Yes, your answer make sense. I was typing get commands with “get” followed by the phrase. I wanted to show you some examples, but suddenly NLP does not recognize any of my natural language commands(it was working a while ago), but tt programs run smoothly(typed with \t). Now I am still trying to make it work around with natural language…
If you have access to my skills and have some time, any help would be great) Let me know if it is possible.

So, I checked the Astana Times, and it appears to work? Also it should be relatively easy to distinguish provided you mention Astana or “astanatimes”.

As for the exchange rate, the first problem is that you don’t have a command with zero or one parameter, so you hit the complexity limit on the exact match.
The second problem is that your type declarations are too loose, and therefore the training set is not good enough to learn those sentences. You should try to make use of ThingTalk types like Currency and Entity. You can find the list of ThingTalk types in the reference, as well as the list of entity types. Most likely, you’ll use at least Entity(tt:currency_code).
Also note that numbers are preprocessed: if you want to recognize a number in the sentence, you must declare your parameter to have a numeric type (Number, Measure or Currency).

Couple more issues I noted: your canonical forms of parameters should be short phrases for the parameter name (one or two words) - slot-filling questions go into #_[prompt] instead.
Also a style issue: by convention, only primitive parameters use the p_ prefix, function parameters have no prefix.

Hope this helps!

1 Like

Thank you very much! These tips and notes are really clarifying things a lot!
In exchange rate, I added function with one(optional) parameter, used Entity(tt:currency_code) as type for currencies, but the problem remains, no reaction to natural language requests.
But previously(when I was using only one function) it was working and asking me to fill in required arguments(I retried doing it with one function again - but no success). Can’t figure out what went wrong, any further help is appreciated)

EDIT: It recognizes only natural language texts without parameters. As I understand, to make use of free-form parameters in texts, we have train it with different params, am I right? But what was that case when it was asking me to enter the required arguments?

For the exact matcher, the system can recognize phrases without placeholders, and phrases with at most one placeholder, provided the value is between quotes.
(Except, I’ve just discovered a bug: https://github.com/stanford-oval/almond-cloud/issues/614)

The full model (after a round of training, which I started), can recognize unquoted values, because it is trained with all possible values. It can also recognize sentences with more than one value, provided the relevant primitive templates are added.

Note that placeholders (primitive template parameters) are different than function parameters. Function parameters determine what information is passed to your code, while placeholders determine what information is extracted from the sentence. If you type in a sentence without parameters, but it maps to a function with required parameters, then Almond will ask for the value separately.

So to test, you should declare your functions to have all the parameters you need, and then write a whole bunch of primitive templates, some with many placeholders and some with only few or none. In fact, it is a good idea to write many primitive templates for different parameter combinations regardless, as it helps training the full model.

1 Like