1.
oleg.ovcharenko@kaust.edu.sa
O. Ovcharenko, V. Kazei, D. Peter, T. Alkhalifah
Transfer learning for low frequency extrapolation
from shot gathers for FWI applications
June 6th, 2019

2.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Team 2
Vladimir Kazei,
Post-doctoral Fellow
Oleg Ovcharenko,
PhD student
Tariq Alkhalifah,
Professor
Daniel Peter,
Assistant Professor
KAUST
Saudi Arabia
Temperature on June 6th
Max: 41°C
Min: 23°C
Average: 32°C
Today in London: ~18°C

3.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Team 3
Vladimir Kazei,
Post-doctoral Fellow
Oleg Ovcharenko,
PhD student
Tariq Alkhalifah,
Professor
Daniel Peter,
Assistant Professor
KAUST
Saudi Arabia
Temperature on June 6th
Max: 41°C
Min: 23°C
Average: 32°C
Today in London: ~18°C

4.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Previous work 4
2017
2018
Today
“Low-Frequency Data Extrapolation Using a Feed-
Forward ANN”,
80th EAGE Annual conference, 2018
“Neural network based low-frequency data
extrapolation”,
SEG FWI Workshop: Where are we getting?
“Transfer learning for low frequency extrapolation
from shot gathers for FWI applications”,
81st EAGE Annual conference, 2019

5.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Limitations of real-world acquisitions 5
Lack of low-frequency data
- Due to instrumental limitations
- Due to noise

6.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Low-frequency data in FWI 6
- Inverts large-scale velocity structures
- Less chance to get stuck in local minima
- Reveals deep model structures / below salt
fHigh
fLow
Multiple
local minima
Smooth
(Kazei et al., 2016)

7.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
FWI with different misfits 7
(Bozdag 2011, Choi & Alkhalifah 2013, Leeuwen & Herrmann, 2014 …)
Pros:
- Established workflow
- Direct image quality control
Cons:
- Computational costs
- Prone to event mismatching
- Sensitivity hard to control
Kalita et al., 2018

8.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
FWI with update conditioning 8
(Alkhalifah, 2015; Kazei, et al., 2016;
Yao et al., 2018; Ovcharenko, et al., 2018 ) etc…
Pros:
- Easily accessible sensitivity
- Direct image quality control
- Can be used together with any misfit
Cons:
- Computational costs
- Prone to event mismatching
Ovcharenko, et al., 2018

9.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
9Extrapolation of low-frequency data
Pros:
- Cheaper computations
Cons:
- Not well explored robustness
- Wavefield approximations
Hu et al., 2014; Li & Demanet, 2015, 2016, Ovcharenko et al., 2018)
etc…

10.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
10
This work: shot-to-shot extrapolation
Beat tone inversion
(Hu et al., 2014)
Bandwidth extension for atomic events
(Li & Demanet, 2015, 2016)
Deep learning freq domain for CSG – Ovcharenko et al., 2017,
Trace to trace deep learning – Sun & Demanet, 2018
Extrapolation of low-frequency data

11.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Why extrapolation should be possible? 11

12.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
12Why extrapolation should be possible?

13.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
One trace, one shot, one dataset 13
Accuracy
Computational complexity
Trace-to-Trace
Shot-to-Shot
Data-to-Data
(Ovcharenko et al, 2017, 2018)
(Sun & Demanet, 2018)

14.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
14Mono-frequency shot gather

15.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Supervised learning to link data in frequency domain 15
One network is trained to extrapolate for One frequency

16.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
ML to learn the phenomena 16
High Low
?
Non-linear function

17.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
17
Training dataset

18.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Generators which didn’t work well 18

19.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Generator on 1D profiles 19

20.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Realistic random models based on wavelets and style transfer 20
“Style transfer for generation of realistically textured subsurface models”,
Ovcharenko et al., SEG Technical Program Expanded Abstracts 2019
“Realistically Textured Random Velocity Models for Deep Learning Applications”,
Kazei et al., 81st EAGE Conference and Exhibition 2019

21.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Size of training dataset 21
Total training samples
=
random models
x
shots in the model

22.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
22
Deep learning model

23.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Selection of network architecture 23
Custom design Ready-to-use design

24.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Selection of network architecture 24
Custom design Ready-to-use design

25.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Learned filters applied to a different dataset 25

26.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Benchmark of network architectures 26
(Bianco et al., 2018,
"Benchmark Analysis of Representative
Deep Neural Network Architectures”)

27.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Benchmark of network architectures 27
(Bianco et al., 2018,
"Benchmark Analysis of Representative
Deep Neural Network Architectures”)

28.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
MobileNet (Howard et al., 2017) 28
NVIDIA Titan V
TensorFlow 1.12.0Python 3.6
Keras 2.2.4
Matlab R2016b
Training for one frequency ~ 5 min
Inference time < 1 sec
Total params ~ 3.9M

29.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
29
Examples

30.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
30Input and target data
(R x F x 2) (R x 1 x 2)

31.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
31Parametrization
(R x F x 3) (R x 1 x 2)

32.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
32
Example 1/2. Central part of BP 2004

33.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
FWI for the central part of BP 2004 benchmark model 33
64 sources and receivers
32 known frequency in range 3-5 Hz
https://github.com/vkazei/fastFWI
Successive mono-frequency inversions at
0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz
Acoustic

34.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Extrapolated low-frequency data for 0.25 Hz 34
True Pred Diff
Re
Im
Phase

35.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Extrapolated low-frequency data for 0.55 Hz 35
True Pred Diff
Re
Im
Phase

36.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Extrapolated low-frequency data for 0.93 Hz 36
True Pred Diff
Re
Im
Phase

37.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Initial model 37

38.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Conventional regularized FWI 38

39.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
39
FWI

40.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
40
What can go wrong with inversion?

41.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
41
What can go wrong with inversion?
FWI
Frequency
Extrapolation

42.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
42
What can go wrong with inversion?
FWI
Frequency
Extrapolation
Normalization
Training data
Training
Assumption
Implementation

43.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
43
What can go wrong with inversion?
FWI
Frequency
Extrapolation
Normalization
Training data
Training Misfit
Solver
Initial guess
Formulation
Implementation
Assumption
Implementation

44.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
44
What can go wrong with inversion?
FWI
Frequency
Extrapolation
Normalization
Training data
Training Misfit
Solver
Initial guess
Formulation
Implementation
Assumption
Implementation
True Pred Diff
Re
Im
Phase

45.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
45
What can go wrong with inversion?
FWI
Frequency
Extrapolation
Normalization
Training data
Training Misfit
Solver
Initial guess
Formulation
Implementation
Assumption
Implementation
True Pred Diff
Re
Im
0.25 Hz

46.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
46
What can go wrong with inversion?
FWI
Frequency
Extrapolation
Normalization
Training data
Training Misfit
Solver
Initial guess
Formulation
Implementation
Assumption
Implementation
True Pred Diff
Re
Im
0.25 Hz4.5 Hz

47.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
47
It is a rocky road
Training
Misfit
Solver

48.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
48
FWI
True vs Extrapolated

49.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
0.25 Hz 49
0.25Hz
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
0.25Hz
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
True Extrapolated
0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz

50.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
0.55 Hz 50
0.55Hz
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
0.55Hz
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
True Extrapolated
0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz

51.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
0.93 Hz 51
0.93Hz
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
True Extrapolated
0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz

52.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
4.5 Hz 52
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
0 5 10 15 20
km
0
2
4
6
km
2
3
4
km/s
True Extrapolated
0.25 0.55 0.93 2.04 2.66 3.46 4.50 Hz

53.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
53
Do we overfit the data?

54.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
54Left part of BP 2004
True Pred Diff
Re
Im
Phase

55.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
55Marmousi II
True Pred Diff
Re
Im
Phase
True Pred Diff
Re
Im
Phase

56.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
56
Example 2/2. One shot from field

57.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Streamer acquisition 57

58.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Single shot 58

59.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Time domain 59

60.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Frequency domain 60

61.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Low-frequency part of spectrum 61

62.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Used range 4 - 10 Hz 62

63.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Regular sampling of the range 63

64.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Raw input data for extrapolation 64

65.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
65Normalized input data for extrapolation

66.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Train one net for each frequency 66
13 frequencies to extrapolate for:
0.1000 0.1334 0.1778 0.2371 0.3162 0.4217 0.5623 0.7499 1.0000 1.3335 1.7783 2.3714 3.1623 Hz

67.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Extrapolated low frequency 67
Synthetic Field
A reference sample of low-
frequency data for a random
velocity model

68.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Extrapolated low frequency 68
Synthetic Field
Extrapolated low-frequency data
for a real-world shot

69.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Extrapolated low frequency 69
Synthetic Field
Extrapolated low-frequency data
for a real-world shot
Expected patterns
in low-frequencies

70.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
70
Trained on synthetic, tested on synthetic = OK

71.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
71
Trained on synthetic, tested on field = ???

72.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Conclusions 72
Wavenumber analysis justifies feasibility of bandwidth extrapolation
Pre-trained networks are applicable with mild retraining
Need a priori constraints on random velocity models
Lowest frequencies are better extrapolated
Regularized full-waveform inversion to tolerate inaccuracies

73.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Acknowledgements 73
Mahesh Kalita (KAUST/ION)
Xiangliang Zhang (KAUST)
Gerhard Pratt (UWO)
Tristan van Leeuven (Utrecht University)
Yunyue Elita Li (NUS)
ovcharenkoo.com ai.vkazei.com

74.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
References 74
Alkhalifah, T., 2015. Conditioning the full-waveform inversion gradient to welcome anisotropy. Geophysics, 80(3), pp.R111-R122.
Bozdağ, E., Trampert, J. and Tromp, J., 2011. Misfit functions for full waveform inversion based on instantaneous phase and envelope
measurements. Geophysical Journal International, 185(2), pp.845-870.
Choi, Y. and Alkhalifah, T., 2013. Frequency-domain waveform inversion using the phase derivative. Geophysical Journal International, 195(3), pp.
1904-1916.
Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M. and Adam, H., 2017. Mobilenets: Efficient convolutional
neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.
Hu*, W., 2014. FWI without low frequency data-beat tone inversion. In SEG Technical Program Expanded Abstracts 2014 (pp. 1116-1120). Society
of Exploration Geophysicists.
Kazei, V., Tessmer, E. and Alkhalifah, T., 2016. Scattering angle-based filtering via extension in velocity. In SEG Technical Program Expanded
Abstracts 2016 (pp. 1157-1162). Society of Exploration Geophysicists.
Kazei, V. and Alkhalifah, T., 2019. Scattering Radiation Pattern Atlas: What anisotropic elastic properties can body waves resolve?. Journal of
Geophysical Research: Solid Earth.
Li, Y.E. and Demanet, L., 2015. Phase and amplitude tracking for seismic event separation. Geophysics, 80(6), pp.WD59-WD72.
Li, Y.E. and Demanet, L., 2016. Full-waveform inversion with extrapolated low-frequency data. Geophysics, 81(6), pp.R339-R348.
Kalita, M., Kazei, V., Choi, Y. and Alkhalifah, T., 2018. Regularized full-waveform inversion for salt bodies. In SEG Technical Program Expanded
Abstracts 2018 (pp. 1043-1047). Society of Exploration Geophysicists.
Ovcharenko, O., Kazei, V., Peter, D. and Alkalifah, T., 2017. Neural network based low-frequency data extrapolation. In 3rd SEG FWI workshop:
What are we getting.
Ovcharenko, O., Kazei, V., Peter, D., Zhang, X. and Alkhalifah, T., 2018, June. Low-Frequency Data Extrapolation Using a Feed-Forward ANN.
In 80th EAGE Conference and Exhibition 2018.
Ovcharenko, O., Kazei, V., Peter, D. and Alkhalifah, T., 2018. Variance-based model interpolation for improved full-waveform inversion in the
presence of salt bodies. Geophysics, 83(5), pp.R541-R551.
Sun, H. and Demanet, L., 2018. Low frequency extrapolation with deep learning. In SEG Technical Program Expanded Abstracts 2018 (pp.
2011-2015). Society of Exploration Geophysicists.
van Leeuwen, T. and Herrmann, F.J., 2014. 3D frequency-domain seismic inversion with controlled sloppiness. SIAM Journal on Scientific
Computing, 36(5), pp.S192-S217.

75.
oleg.ovcharenko@kaust.edu.saLow frequency extrapolation from shot gathers
Conclusions 75
Wavenumber analysis justifies feasibility of bandwidth extrapolation
Pre-trained networks are applicable with mild retraining
Need a priori constraints on random velocity models
Lowest frequencies are better extrapolated
Regularized full-waveform inversion to tolerate inaccuracies