patternMinor
Specific case of a ranking model
Viewed 0 times
caserankingmodelspecific
Problem
So the problem I'm solving is this: I have a list of conversations of 3 messages each (for eg. "hi", " how are you", "remind me to fix this bug" is one conversation, and my problem will have many of these) and need to pick out the message that is most likely to be a task ("remind me to fix this bug" would be the message in the conversation above; "don't forget to commit your code" would be another in some other conversation). What kind of machine learning/NLP model could I use to relatively rank the messages in a conversation and "learn" across conversations? And what would my training data look like?
My thoughts: each data point in the training data is one conversation (a set of 3 messages) along with the message with the highest likelihood of being a "task." I was thinking of using a linear weighted function of features such as TF-IDF score. But how would I learn the weights?
My thoughts: each data point in the training data is one conversation (a set of 3 messages) along with the message with the highest likelihood of being a "task." I was thinking of using a linear weighted function of features such as TF-IDF score. But how would I learn the weights?
Solution
Build a boolean classifier that classifies a message as either "task" or "not a task". Choose a classifier that can output a confidence score (many classifiers can, including ensembles like random forests; neural networks and deep learning; logistic regression; and more).
Train the classifier on many messages.
For a collection of 3 messages, apply the classifier to each message and get the confidence score. Then, use the confidence scores to rank the 3 messages: e.g., choose the message that has the highest confidence/probability of being "task" as the one that is most likely to be the task.
This is different from what you had in mind: here each instance is a message, not a conversation.
Which features to use is a separate question, but you'll probably want to use standard NLP features (bag of words, word2vec, and so on). That belongs in a separate question. As a warning, if each message is very short as indicated in your example, the performance of your classifier might not be great.
Also, you will need a large collection of messages where you know the label (classification) of each.
Train the classifier on many messages.
For a collection of 3 messages, apply the classifier to each message and get the confidence score. Then, use the confidence scores to rank the 3 messages: e.g., choose the message that has the highest confidence/probability of being "task" as the one that is most likely to be the task.
This is different from what you had in mind: here each instance is a message, not a conversation.
Which features to use is a separate question, but you'll probably want to use standard NLP features (bag of words, word2vec, and so on). That belongs in a separate question. As a warning, if each message is very short as indicated in your example, the performance of your classifier might not be great.
Also, you will need a large collection of messages where you know the label (classification) of each.
Context
StackExchange Computer Science Q#58191, answer score: 3
Revisions (0)
No revisions yet.