On March 17, the patent of "Short Text Dialogue Method, Device, Equipment and Storage Medium" applied by Tencent Technology (Shenzhen) Co., Ltd. was authorized. The patent abstract shows that the method includes: obtaining the query text; calling the short text dialogue model based on the language model to predict the query text and obtaining the reply text; Layers include both self-attention and attention for supervising query text; output reply text. The application replaces the S2S framework with a short text dialogue model based on a language model, and at the same time uses the mixed attention layer in the model to supervise the hidden layer representation of the query text, so that the short text dialogue model can not only capture long-short distance dependencies, but also It can focus on the information of the query text when predicting, and can generate a reply text that is closely related to the query text in semantics and rich in form.

来源：数据猿