Extending Transformer to Predict Both the Order and Occurrence Times of Elements in a Sequence
- Authors
- Ryu, Hyewon; Yu, Sara; Yong Lee, Ki
- Issue Date
- Feb-2024
- Publisher
- Institute of Electrical and Electronics Engineers Inc.
- Keywords
- Positional encoding; Sequence prediction; Timestamped sequences; Transformer
- Citation
- Proceedings - 2024 IEEE International Conference on Big Data and Smart Computing, BigComp 2024, pp 371 - 372
- Pages
- 2
- Journal Title
- Proceedings - 2024 IEEE International Conference on Big Data and Smart Computing, BigComp 2024
- Start Page
- 371
- End Page
- 372
- URI
- https://scholarworks.sookmyung.ac.kr/handle/2020.sw.sookmyung/160065
- DOI
- 10.1109/BigComp60711.2024.00074
- ISSN
- 0000-0000
- Abstract
- Recently, sequence prediction techniques using Transformers have become essential in various fields. However, so far Transformers have only focused on predicting the next elements in a sequence and do not predict their occurrence times. Therefore, in this paper, we propose an extension of Transformer to predict not only the next elements but also their occurrence times. For this purpose, we extend Transformer in three ways: (1) We propose a new positional encoding method that can reflect both the order and occurrence time of each element in a sequence, (2) We extend the output layer of Transformer to simultaneously predict the next element and its occurrence time, and (3) We refine the loss function to measure the difference between sequences considering both the order and occurrence times of elements. Through experiments using real datasets, we confirmed that the proposed model more accurately predicts the order and occurrence time of each element than the existing Transformer. © 2024 IEEE.
- Files in This Item
-
Go to Link
- Appears in
Collections - 공과대학 > 소프트웨어학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.sookmyung.ac.kr/handle/2020.sw.sookmyung/160065)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.