Detailed Information

Cited 0 time in webofscience Cited 0 time in scopus
Metadata Downloads

Extending Transformer to Predict Both the Order and Occurrence Times of Elements in a Sequence

Authors
Ryu, HyewonYu, SaraYong Lee, Ki
Issue Date
Feb-2024
Publisher
Institute of Electrical and Electronics Engineers Inc.
Keywords
Positional encoding; Sequence prediction; Timestamped sequences; Transformer
Citation
Proceedings - 2024 IEEE International Conference on Big Data and Smart Computing, BigComp 2024, pp 371 - 372
Pages
2
Journal Title
Proceedings - 2024 IEEE International Conference on Big Data and Smart Computing, BigComp 2024
Start Page
371
End Page
372
URI
https://scholarworks.sookmyung.ac.kr/handle/2020.sw.sookmyung/160065
DOI
10.1109/BigComp60711.2024.00074
ISSN
0000-0000
Abstract
Recently, sequence prediction techniques using Transformers have become essential in various fields. However, so far Transformers have only focused on predicting the next elements in a sequence and do not predict their occurrence times. Therefore, in this paper, we propose an extension of Transformer to predict not only the next elements but also their occurrence times. For this purpose, we extend Transformer in three ways: (1) We propose a new positional encoding method that can reflect both the order and occurrence time of each element in a sequence, (2) We extend the output layer of Transformer to simultaneously predict the next element and its occurrence time, and (3) We refine the loss function to measure the difference between sequences considering both the order and occurrence times of elements. Through experiments using real datasets, we confirmed that the proposed model more accurately predicts the order and occurrence time of each element than the existing Transformer. © 2024 IEEE.
Files in This Item
Go to Link
Appears in
Collections
공과대학 > 소프트웨어학부 > 1. Journal Articles

qrcode

Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.

Related Researcher

Researcher Lee, Ki Yong photo

Lee, Ki Yong
공과대학 (소프트웨어학부(첨단))
Read more

Altmetrics

Total Views & Downloads

BROWSE