Table 1
Examples of sentences containing emotemes (underlined) and/or emotive expressions (bold type font). English translations were prepared to reflect both types if possible.
| Example of a sentence (English translation) | emotemes | emotive expressions |
|---|---|---|
| (1) Kyo wa nante kimochi ii hi nanda! (Today is such a nice day!) | yes | yes |
| (2) Iyaa, sore wa sugoi desu ne! (Whoa, what do you know!) | yes | no |
| (3) Ryoushin wa minna jibun no kodomo wo aishiteiru. (All parents love their children.) | no | yes |
| (4) Kore wa hon desu. (This is a book.) | no | no |
Table 2
Examples from affect lexicon used in ML-Ask (N = noun, V = verb, Phr = phrase, Id = idiom, Adj = adjective, Adv = adverb).
 |
Figure 1
Conceptual flow of the ML-Ask software procedures.
Figure 2
Examples of valence shifting using Contextual Valence Shifters.
Figure 3
Mapping of Nakamura’s classification of emotions on Russell’s 2D space.
Figure 4
Output example for ML-Ask.
Figure 5
ML-Ask output.
Figure 6
Explanations of ML-Ask output.
Table 4
Results of each previous reported evaluation of ML-Ask. Evalauted functions are: Emo/N-emo = Emotive/non-emotive, EmoVal = Emotive value estimation, EmoType = Emotion type determination, Valence = Determining valence of emotion (positive/negative), Activ = Determining activation of emotion (active/passive), Engage = Estimation of emotional engagement in conversation. Evaluation metrics are: F1 = F-measure, U = Unanimity score, Acc = Accuracy, κ = Agreement with human (Kappa), ρ = Agreement with human (Pearson’s “rho” ρ).
| Paper No. | Year of publ. | Conference/Journal | Dataset used | Evalauted functions | Eval. metr. | Results reported |
|---|---|---|---|---|---|---|
| [20] | 2008 | Conference | Separate sentences | Emo/N-emo EmoVal EmoType | F1 U F1 | 0.830 0.630 0.450 |
| [12] | 2009 | Journal | Separate sentences | Emo/N-emo EmoVal EmoType | F1 U F1 | 0.830 0.630 0.450 |
| [23] | 2009 | Conference | Separate sent., Conversations | Emo/N-emo EmoType | F1 F1 | 0.840 0.450 |
| [25] | 2009 | Conference | Conversations, Blogs | Valence EmoType | Acc Acc | 0.900 0.850 |
| [21] | 2009 | Conference | Separate sentences | Emo/N-emo EmoType | Acc F1 | 0.900 0.367 |
| BBS | Emo/N-emo EmoType Annot. | Acc κ | up to .75 0.681 | |||
| [3] | 2010 | Journal | Conversations | Engage | ρ | up to .597 |
| [26] | 2010 | Journal | Separate sentences, Conversations | Emo/N-emo EmoType Valence | F1 F1 Acc | 0.830 0.470 0.800 |
| [29] | 2012 | Conference | Blogs | Emo/N-emo Valence/Activ EmoType | Acc Acc Acc | 0.988 0.886 0.734 |
| [34] | 2013 | Journal | Blogs | Emo/N-emo Valence/Activ EmoType | F1 F1 F1 | 0.994 0.939 0.847 |
| [33] | 2013 | Journal | Fairytales | Valence/Activ EmoType | Acc Acc | 0.606 0.576 |
| [35] | 2013 | Journal | Conversations, Blogs | Valence EmoType | Acc Acc | 0.600 0.680 |
Table 5
Example of benchmarking made for the current version of ML-Ask.
| Rate(iter/s) | mlask-4.2-simple (regex) | mlask-4.2 (regex) | mlask-4.3-simple (noregex) | mlask-4.3 (noregex) | |
|---|---|---|---|---|---|
| dataset A (1 sentence from Figure 5) | |||||
| mlask-4.2-simple | 206882875/s | – | –12% | –19% | –14% |
| mlask-4.2 | 234881024/s | 14% | – | –8% | –2% |
| mlask-4.3 | 239674513/s | 16% | 2% | –6% | – |
| mlask-4.3-simple | 254654171/s | 23% | 8% | – | 6% |
| dataset B (90 sentences used in [20, 21]) | |||||
| mlask-4.2-simple | 126370709/s | – | –9% | –17% | –24% |
| mlask-4.2 | 139257523/s | 10% | – | –9% | –17% |
| mlask-4.3-simple | 152876500/s | 21% | 10% | – | –8% |
| mlask-4.3 | 166995436/s | 32% | 20% | 9% | – |