Pointer Generator dan Coverage Weighting untuk Memperbaiki Peringkasan Abstraktif

  • Agna Silpi Alpiani Telkom University
  • Suyanto Suyanto Telkom University
Abstract views: 135 , PDF downloads: 147
Test Output downloads: 0
Nilai ROUGE score model attention mechanism downloads: 0
Nilai ROUGE score model pointer generator + coverage weighting downloads: 0

Abstract

Model Long Short – Term Memory (LSTM) sequence-to-sequence telah banyak digunakan untuk menyelesaikan tantangan dalam peringkasan teks. Namun, model ini masih memiliki dua masalah utama yaitu, kemunculan kata diluar kosakata atau sering disebut out-of-vocabulary (OOV), dan perulangan kata. Pada makalah ini, pointer generator dan coverage weighting diterapkan untuk mengatasi dia masalah tersebut. Dimulai dengan model sequence-to-sequence dasar. Kemudian kami kembangkan dengan attention mechanism yang telah ditambahkan coverage weigting pada perhitungannya untuk mengurangi terjadinya perulangan kata, dan mengganti encoder menjadi bi-directional LSTM. Setelah itu kami mengimplementasikan pointer generator yang dapat  menunjuk kembali ke kata dalam teks sumber dan menghasilkan kata jika bertemu dengan kata OOV. Menggunakan dataset artikel berita bahasa Inggris CNN/Daily Mail dan metrik evaluasi ROUGE score, model kami menghasilkan hasil yang mirip dengan ringkasan teks sumber.

Downloads

Download data is not yet available.

References

Kågebäck, M., Mogren, O., Tahmasebi, N., & Dubhashi, D. (2014). Extractive summarization using continuous vector space models. In Proceedings of the 2nd Workshop on Continuous Vector Space Models and their Compositionality (CVSC) (pp. 31-39).

Pachantouris, G. (2005). GreekSum–A Greek Text Summarizer. Word Journal of the International Linguistic Association, 1-45.

Jing, H. (2000). Sentence reduction for automatic text summarization. In Sixth Applied Natural Language Processing Conference.

Cheung, J. C. K., & Penn, G. (2014). Unsupervised sentence enhancement for automatic summarization. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 775-786).

Rush, A. M., Chopra, S., & Weston, J. (2015). A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685.

Chopra, S., Auli, M., & Rush, A. M. (2016). Abstractive sentence summarization with attentive recurrent neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 93-98).

Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.

Vinyals, O., Fortunato, M., & Jaitly, N. (2015). Pointer networks. In Advances in Neural Information Processing Systems (pp. 2692-2700).

Gulcehre, C., Ahn, S., Nallapati, R., Zhou, B., & Bengio, Y. (2016). Pointing the unknown words. arXiv preprint arXiv:1603.08148.

Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2016). Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843.

Gu, J., Lu, Z., Li, H., & Li, V. O. (2016). Incorporating copying mechanism in sequence-to-sequence learning. arXiv preprint arXiv:1603.06393.

Miao, Y., & Blunsom, P. (2016). Language as a latent variable: Discrete generative models for sentence compression. arXiv preprint arXiv:1609.07317.

Nallapati, R., Zhou, B., Gulcehre, C., & Xiang, B. (2016). Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023.

See, A., Liu, P. J., & Manning, C. D. (2017). Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368.

Tu, Z., Lu, Z., Liu, Y., Liu, X., & Li, H. (2016). Modeling coverage for neural machine translation. arXiv preprint arXiv:1601.04811.

Chen, Q., Zhu, X. D., Ling, Z. H., Wei, S., & Jiang, H. (2016, July). Distraction-Based Neural Networks for Modeling Document. In IJCAI (pp. 2754-2760).

Hermann, K. M., Kocisky, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015). Teaching machines to read and comprehend. In Advances in neural information processing systems (pp. 1693-1701).

Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.

Lin, C. Y. (2004). Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out.