Recomendados
Recomendados
Más contenido relacionado
Similar a NS-CUK Joint Journal Club: Minwoo Choi, Review on "Spatio-Temporal Wind Speed Forecasting using Graph Networks and Novel Transformer Architectures", Appied Energy 333 2023
Similar a NS-CUK Joint Journal Club: Minwoo Choi, Review on "Spatio-Temporal Wind Speed Forecasting using Graph Networks and Novel Transformer Architectures", Appied Energy 333 2023 (20)
Más de ssuser4b1f48
Más de ssuser4b1f48 (20)
Último
Último (20)
NS-CUK Joint Journal Club: Minwoo Choi, Review on "Spatio-Temporal Wind Speed Forecasting using Graph Networks and Novel Transformer Architectures", Appied Energy 333 2023
- 1. Min-Woo Choi Network Science Lab Dept. of Artificial Intelligence The Catholic University of Korea E-mail: choimin1231@catholic.ac.kr 2023. 03. 06 Publish: 2023 Journal: Energy (IF: 11.45)
- 2. 1 Introduction • Background • Purpose Methodology • Data description • Experiment setup Results Conclusions Limitation and Future work
- 3. 2 1. Introduction Limitation • Previous studies have focused on either physical models based on detailed atmospheric laws or statistical approaches such as machine learning algorithms and time series analysis techniques • These methods are limited in their ability to capture the complex spatio-temporal correlations of wind resources which can lead to inaccurate forecasts. • The purpose of this paper is to present a generic framework for multi-step spatio-temporal forecasting, using Graph Neural Networks (GNNs) and optional update functions such as Transformer or LSTM networks. Purpose of study
- 4. 3 2. Methodology Data description • Location: Norwegian Meteorological Institute • Period : June 23, 2015, and February 28, 2022 • Number of data: 14 out of 25 available stations • Resolution: 10 min • Variable: 8 feature (air temperature, air pressure, dew point, relative humidity, wind direction, wind speed …) • Pre-processing: If measurements for a single time-step were missing for a particular station, linear interpolation was used to fill the missing entries. • Training/Validation/Test set: 60%/20%/20% feature matrix, 𝑓𝑡 ∈ RNx8
- 5. 4 2. Methodology Experiment setup • Look-back window of 32 time-steps was suitable for the 10-min and 1-h ahead forecasts • Increased to 64 for the 4-h forecasts Multilevel wavelet decomposition
- 6. 5 2. Methodology Fast Fourier Transformer • FFTtransformer based on signal decomposition and an adapted attention mechanism, named FFT-Attention. • FFTransformer is comprised of two streams one which analyses signals with clear periodicity and another that should learn trend components • The basic configuration is the same as that of a original transformer, but to better facilitate the analysis of periodic signals in the left-hand stream, we introduce the FFT- Attention mechanism. FFT is applied to the key, query and value inputs Real and imaginary FFT outputs are concatenated with the frequency values, to provide information on the corresponding frequency for the values in a particular position, similar to the motivation behind positional encoding, before being fed to an MHA block. • Outputs from the FFT-Attention module are then concatenated and projected, then inverse FFT transform values back to the time domain.
- 7. 6 2. Methodology Spatial-temporal framework All models were constructed in an encoder-only setting, i.e. without decoders, and used as update functions, 𝜙 (⋅) , in two-layer GNNs. Considering a node, 𝑖, its input features were 𝑣𝑖 ∈ R1×(𝑆+𝑃)×8 𝑆 = historical look-back window 𝑃 = Predict the next time-steps
- 8. 7 2. Methodology Benchmark models • ST-MLP • ST-LSTM • ST-Transformer • ST-LogSparse • ST-Informer • ST-Autoformer • ST-FFTransformer
- 9. 8 3. Results Results of 10 min – 4 h prediction
- 11. 10 4. Conclusions • The novel FFTransfomer was proposed which is based on signal decomposition using wavelet transform and an adapted attention mechanism in the frequency domain - making it significantly outperforming all other models for 4 hour ahead forecasts.
- 12. 11 Thank you!
Notas del editor
- 안녕하십니까, 저는 기상환경 관련 연구를 수행했던 최민우라고 하구요. 저의 전공지식을 토대로 이오준 교수님과 앞으로 연구를 진행해 나갈 예정입니다. 앞으로 영어 실력의 향상을 위해 영어로 발표하겠으나. 아직은 대본을 대부분 참조한다는 점 양해부탁드립니다. The topic of my presentation is Short-term wind speed forecasting based on spatial-temporal graph transformer networks.
- The order of contents is as follows:
- First, the limitations of previous studies in wind speed prediction are as follows. When we use a spatiotemporal prediction model to predict wind speed based on CNN, a square array or nodes with regular intervals are required. Like that grid. However, in actually it is not composed of regular data. Therefore, as a related study, Fu extracted spatial correlation using the STAN model. and Kyodayar extracted spatiotemporal information of wind speed using the GCDLA model. There are few related studies, but taking advantage of related studies, this study proposed STGTN to predict wind speed.
- The data used were wind farms located in Danish offshore, and data were collected at 10-minute intervals with 111 wind turbine nodes. The data set was divided into 3:1:1, and the wind speed was predicted using the historical data of each 12 points.
- Benchmark models were constructed to validate the proposed model. First, SVR model not include the spatial information, DL-STF and STAN models based on spatial-temporal information and the STGTN-T model using Transformer was used instead of MLP model.
- The ST-LogSparse and ST-Informer performed consistently better than the ST-Transformer model across all forecasting horizons in terms of both MSE and MAE, which showed the potential improvements brought by the ProbSparse and convolutional attention mechanisms for wind forecasting. In general, all Transformer-based models attained better results ST-Autoformer model performed very well for the 1- and 6-step forecasts, its performance was seen to degrade for the 24-step setting, where it was inferior to all other Transformerbased models. Both the ST-FFTransformer and ST-Autoformer achieved the best accuracy for three settings each, likely because of data decomposition. One reason for the superior performance of the FFTransformer architecture for the longer forecasting setting might be that it separately processes the frequency and trend components, using the two streams described in Section 4.2, with attention and feedforward modules applied to both. On the other hand, the Autoformer model is mainly focused on processing periodic components using the Auto-Correlation module and does not perform significant processing on extracted trend components after series decomposition, making it potentially struggle to learn longer-term trends that lack periodicity. Even though the ST-FFTransformer achieved good results across different horizons, it did not outperform the ST-Autoformer in the short-term. This was despite more advanced decomposition using MDWD, which was initially thought better at extracting trend and periodic components at different frequencies, compared to the simple moving average operation used in the Autoformer. In terms of decomposition, this could indicate that the MDWD is not necessarily superior to the ‘Series Decomposition’, or more likely, that the repeated use of decomposition in the Autoformer might work slightly better than the single decomposition performed on the inputs in the ST-FFTransformer.
- Even though this study did not focus on discussing better connectivity strategies for wind forecasting or using learnable adjacency matrices, we conduct a brief investigation into whether some of the connections could potentially be removed. The sharp decrease indicated that the models were able to successfully leverage spatial correlations to improve forecasts, proving the effectiveness of the proposed GNN architectures. Nevertheless, since MAEs converged to constant values for non-complete graphs, it indicated that a number of connections could potentially be removed without impairing predictive performance. Further work would therefore be desirable to investigate better methods for which to construct the graphs. or learn optimal connectivity for spatio-temporal wind forecasting. spatial correlations were thought less important than for the 1- and 4-h forecasts, due to the large physical distances between nodes resulting in wind fields not having time to propagate in the immediate short-term.
- The proposed model is stable and has excellent spatio-temporal predictability.
- Anyone have any questions?