Due to limited computation resources, only the development set and a small portion of the training set are used for training the D-score framework. In addition, the current version of D-score framework only considers dialogue contexts. Hence, we omit the information regarding external facts here and will incorporate grounding on external knowledge in our future research.
train.convos.txt, dev.convos.txt contains a Reddit response and its preceding conversational context. Long conversational contexts are truncated by keeping the last 100 words. The file contains 5 columns:
The converational context may contain:
EOS: special symbol indicating a turn transition
START: special symbol indicating the start of the conversation

D-score dataset보다 훨씬 많은 양의 데이터 수집함
train.convos.txt
기존의 DSTC7의 학습 데이터 셋이 아닌 일부만 선정하여 학습용 데이터 수집
(2012-03, 2012-09, 2013-03, 2013-06, 2013-09, 2014-03, 2014-06)
208000여개의 Conversation 수집
dev.convos.txt
기존의 DSTC7의 데이터 그대로 수집
(2017-01, 2017-02, 2017-03)
85000여개의 Conversation 수집
In the evaluation dataset, there are 1000 dialogue contexts and 10 responses per context. These 10 responses contain hypotheses from 9 different generative models plus the origin human response. For each dialogue response, three crowdsourced annotators provided scores based on two criteria, relevance and informativeness. The scores for each criterion are based on the 5-point Likert scale. The overall score is obtained by combining the two judgments with equal weights.
subreddit name