中文题名: | 基于注意力机制的风格化标题生成技术研究 |
姓名: | |
保密级别: | 公开 |
论文语种: | 中文 |
学科代码: | 080901 |
学科专业: | |
学生类型: | 学士 |
学位: | 理学学士 |
学位年度: | 2020 |
学校: | 北京师范大学 |
校区: | |
学院: | |
第一导师姓名: | |
第一导师单位: | |
第二导师姓名: | |
提交日期: | 2020-06-24 |
答辩日期: | 2020-05-14 |
外文题名: | Towards Stylized Image Captioning Based on Attention Mechanism |
中文关键词: | |
外文关键词: | Stylized Image Captioning ; Attention Mechanism ; Deep Learning ; Multimodel |
中文摘要: |
风格化标题生成的目的是生成更人性化的标题,这些标题可以准确地描述图像并具有某些风格特征。现有的一些方法将不包含风格特征的事实标题数据集和风格化标题数据集组合为一个大型数据集以训练模型,但是事实标题数据集的大小要大得多,因此会导致模型更偏向于生成事实标题。其他方法首先使用事实标题数据集对模型进行预训练,然后使用风格化标题数据集进行微调。但是,第二步可能会导致模型生成具有风格但与图像不相关的标题。为了平衡风格化和图像内容保留,本文提出了一个风格化标题生成的系统,该基于多任务学习框架,同时生成相同图像的事实和风格化标题来保留图像内容。此外,该系统在Transformer模型的基础上添加了融合注意力模块,可以将图像和先前的单词融合到上下文向量中,这有助于解码器生成与图像更相关的风格化标题。模型可以生成具有风格特征并且与图像保持相关性的风格化标题。在两个公共常用的数据集上的实验表明,本文提出的模型优于目前的基线系统。
﹀
|
外文摘要: |
Stylized Image Captioning aims to generate more human-like captions which can describe the image accurately and have some style characteristics. Some existing methods combine factual caption datasets which don’t contain style characteristics and stylized caption datasets as a whole large dataset to train the model, but the fact that the size of factual captions dataset is much larger leads to more factual generated captions. Others first pre-train the model using factual caption dataset, and then fine-tune using stylized caption datasets. However, the second step may generate captions with changed style but lack of relevance with images. To balance stylization and content preservation, in this paper, we propose a Multi-Task Learning framework that generates factual and stylized captions of the same image at the same time. Moreover, we design a Transformer-based model with Fused Attention to fuse the image and previous words to a context vector which helps decoder to generate more image-relevant captions. In this way, our model can generate stylized captions having style and also maintaining strong relevance with images. Experiments on two benchmark datasets show that our model outperforms the state-of-the-art systems.
﹀
|
参考文献总数: | 27 |
优秀论文: | |
插图总数: | 3 |
插表总数: | 6 |
馆藏号: | 本080901/20013 |
开放日期: | 2021-06-24 |