Getting Started

Getting Started#

Welcome to the NoisePy Colab Tutorial!

This tutorial will walk you through the basic steps of using NoisePy to compute ambient noise cross correlation functions.

First, we install the noisepy-seis package

# Uncomment and run this line if the environment doesn't have noisepy already installed:
# ! pip install noisepy-seis 

Warning: NoisePy uses obspy as a core Python module to manipulate seismic data. Restart the runtime now for proper installation of obspy on Colab.

Then we import the basic modules

from noisepy.seis import download, cross_correlate, stack_cross_correlations, __version__
from noisepy.seis.io import plotting_modules
from noisepy.seis.io.asdfstore import ASDFRawDataStore, ASDFCCStore, ASDFStackStore
from noisepy.seis.io.datatypes import CCMethod, ConfigParameters, FreqNorm, RmResp, TimeNorm
from datetime import datetime, timezone
from datetimerange import DateTimeRange
import os
import shutil
print(f"Using NoisePy version {__version__}")

path = os.path.join(".", "get_started_data")

os.makedirs(path,exist_ok=True)
raw_data_path = os.path.join(path, "RAW_DATA")
cc_data_path = os.path.join(path, "CCF")
stack_data_path = os.path.join(path, "STACK")

/opt/hostedtoolcache/Python/3.10.16/x64/lib/python3.10/site-packages/noisepy/seis/io/utils.py:13: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)
  from tqdm.autonotebook import tqdm

Using NoisePy version 0.1.dev1

Ambient Noise Project Configuration#

We store the metadata information about the ambient noise cross correlation workflow in a ConfigParameters() object. We first initialize it, then we tune the parameters for this cross correlation.

config = ConfigParameters() # default config parameters which can be customized
config.inc_hours = 12
config.sampling_rate = 20  # (int) Sampling rate in Hz of desired processing (it can be different than the data sampling rate)
config.cc_len = 3600  # (float) basic unit of data length for fft (sec)
    # criteria for data selection
config.ncomp = 3  # 1 or 3 component data (needed to decide whether do rotation)

config.acorr_only = False  # only perform auto-correlation or not
config.xcorr_only = True  # only perform cross-correlation or not

config.inc_hours = 12 # if the data is first 

config.lamin = 31       # min latitude
config.lamax = 42       # max latitude
config.lomin = -124     # min longitude
config.lomax = -115     # max longitude

# pre-processing parameters
config.step = 1800.0  # (float) overlapping between each cc_len (sec)
config.stationxml = False  # station.XML file used to remove instrument response for SAC/miniseed data
config.rm_resp = RmResp.INV  # select 'no' to not remove response and use 'inv' if you use the stationXML,'spectrum',
config.freqmin = 0.05
config.freqmax = 2.0
config.max_over_std  = 10  # threshold to remove window of bad signals: set it to 10*9 if prefer not to remove them

# TEMPORAL and SPECTRAL NORMALISATION
config.freq_norm = FreqNorm.RMA # choose between "rma" for a soft whitenning or "no" for no whitening. Pure whitening is not implemented correctly at this point.
config.smoothspect_N = 10  # moving window length to smooth spectrum amplitude (points)
    # here, choose smoothspect_N for the case of a strict whitening (e.g., phase_only)

config.time_norm = TimeNorm.NO  # 'no' for no normalization, or 'rma', 'one_bit' for normalization in time domain,
    # TODO: change time_norm option from "no" to "None"
config.smooth_N = 10  # moving window length for time domain normalization if selected (points)

config.cc_method = CCMethod.XCORR # 'xcorr' for pure cross correlation OR 'deconv' for deconvolution;
    # FOR "COHERENCY" PLEASE set freq_norm to "rma", time_norm to "no" and cc_method to "xcorr"

# OUTPUTS:
config.substack = True  # True = smaller stacks within the time chunk. False: it will stack over inc_hours
config.substack_windows = 1  # how long to stack over (for monitoring purpose)
    # if substack=True, substack_windows=2, then you pre-stack every 2 correlation windows.
    # for instance: substack=True, substack_windows=1 means that you keep ALL of the correlations

config.maxlag = 200  # lags of cross-correlation to save (sec)
config.substack = True

Step 0: download data#

This step will download data using obspy and save them into ASDF files locally. The data will be stored for each time chunk defined in hours by inc_hours.

The download will clean up the raw data by detrending, removing the mean, bandpassing (broadly), removing the instrumental response, merging gaps, ignoring too-gappy data.

Use the function download with the following arguments:

path:where to put the data
config: configuration settings, in particular:
- channel: list of the seismic channels to download, and example is shown below
- stations: list of the seismic stations, it can be “*” (not “all”)
- start_time
- end_time
client_url_key: the string for FDSN clients

config.networks = ["*"]
config.stations = ["A*"]
config.channels = ["BH?"]
config.start_date = datetime(2019, 2, 1, tzinfo=timezone.utc)
config.end_date = datetime(2019, 2, 2, tzinfo=timezone.utc)
timerange = DateTimeRange(config.start_date, config.end_date)

# Download data locally. Enters raw data path, channel types, stations, config, and fdsn server.
download(raw_data_path, config)

2025-01-14 01:06:34,062 139855575739264 INFO fdsn_download.download(): Download
        From: 2019-02-01T00:00:00.000000Z
        To: 2019-02-02T00:00:00.000000Z
        Networks: ['*']
        Stations: ['A*']
        Channels: ['BH?']
        

2025-01-14 01:06:36,288 139855575739264 INFO fdsn_download.download(): Fetched inventory

2025-01-14 01:06:36,289 139855575739264 INFO utils.log_raw(): TIMING:  2.227 secs for Getting inventory

2025-01-14 01:06:36,568 139854042105536 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHE
Request Submitted:
2025/01/14 01:06:36 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHE)

2025-01-14 01:06:36,796 139854042105536 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHE
Request Submitted:
2025/01/14 01:06:36 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHE)

2025-01-14 01:06:37,059 139854042105536 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHE
Request Submitted:
2025/01/14 01:06:37 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHE)

2025-01-14 01:06:37,688 139853893207744 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:37,723 139853710755520 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:38,401 139853729629888 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:38,689 139853930956480 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:38,747 139853912082112 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:38,755 139854023231168 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:38,834 139854042105536 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:38,852 139854060979904 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:39,817 139855575739264 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2025-01-14 01:06:40,085 139853893207744 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHZ
Request Submitted:
2025/01/14 01:06:40 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHZ)

2025-01-14 01:06:40,244 139855575739264 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2025-01-14 01:06:40,356 139853893207744 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHZ
Request Submitted:
2025/01/14 01:06:40 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHZ)

2025-01-14 01:06:40,439 139853710755520 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHE
Request Submitted:
2025/01/14 01:06:40 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHE)

2025-01-14 01:06:40,694 139853710755520 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHE
Request Submitted:
2025/01/14 01:06:40 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHE)

2025-01-14 01:06:40,825 139855575739264 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2025-01-14 01:06:40,965 139853710755520 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHE
Request Submitted:
2025/01/14 01:06:40 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHE)

2025-01-14 01:06:40,976 139855575739264 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2025-01-14 01:06:41,096 139853729629888 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHN
Request Submitted:
2025/01/14 01:06:41 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHN)

2025-01-14 01:06:41,097 139855575739264 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2025-01-14 01:06:41,217 139855575739264 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2025-01-14 01:06:41,328 139853729629888 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHN
Request Submitted:
2025/01/14 01:06:41 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHN)

2025-01-14 01:06:41,361 139855575739264 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2025-01-14 01:06:41,454 139853893207744 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:41,488 139855575739264 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2025-01-14 01:06:41,606 139853729629888 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHN
Request Submitted:
2025/01/14 01:06:41 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHN)

2025-01-14 01:06:41,862 139853729629888 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHN
Request Submitted:
2025/01/14 01:06:41 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHN)

2025-01-14 01:06:41,997 139853930956480 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:42,099 139853729629888 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHN
Request Submitted:
2025/01/14 01:06:42 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHN)

2025-01-14 01:06:42,104 139853729629888 WARNING fdsn_download.download_stream(): Could not download data for 2019-02-01T00:00:00.000000Z-2019-02-01T12:00:00.000000Z/AVM.BHN

2025-01-14 01:06:42,180 139853710755520 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:42,498 139855575739264 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2025-01-14 01:06:43,019 139855575739264 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2025-01-14 01:06:43,132 139855575739264 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2025-01-14 01:06:43,511 139853930956480 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHE
Request Submitted:
2025/01/14 01:06:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHE)

2025-01-14 01:06:43,535 139854023231168 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHE
Request Submitted:
2025/01/14 01:06:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHE)

2025-01-14 01:06:43,539 139853912082112 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ADO&location=%2A&channel=BHN
Request Submitted:
2025/01/14 01:06:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ADO.BHN)

2025-01-14 01:06:43,786 139854023231168 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHE
Request Submitted:
2025/01/14 01:06:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHE)

2025-01-14 01:06:43,794 139853912082112 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ADO&location=%2A&channel=BHN
Request Submitted:
2025/01/14 01:06:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ADO.BHN)

2025-01-14 01:06:44,039 139854023231168 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHE
Request Submitted:
2025/01/14 01:06:44 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHE)

2025-01-14 01:06:45,158 139854060979904 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHZ | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:45,339 139853710755520 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHN | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:45,517 139854042105536 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHE | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:45,540 139853893207744 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHZ | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:45,733 139853930956480 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHE | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:45,750 139853912082112 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHN | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:45,801 139853729629888 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHN | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:46,160 139854023231168 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHE | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:47,618 139855575739264 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2025-01-14 01:06:47,936 139855575739264 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2025-01-14 01:06:48,120 139855575739264 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2025-01-14 01:06:48,159 139853893207744 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHZ
Request Submitted:
2025/01/14 01:06:48 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHZ)

2025-01-14 01:06:48,242 139855575739264 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2025-01-14 01:06:48,358 139855575739264 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2025-01-14 01:06:48,418 139853893207744 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHZ
Request Submitted:
2025/01/14 01:06:48 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHZ)

2025-01-14 01:06:48,479 139855575739264 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2025-01-14 01:06:48,597 139855575739264 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2025-01-14 01:06:48,668 139853893207744 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHZ
Request Submitted:
2025/01/14 01:06:48 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHZ)

2025-01-14 01:06:48,867 139855575739264 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2025-01-14 01:06:49,115 139853710755520 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHZ | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:49,286 139854042105536 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHN | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:49,334 139854060979904 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHE | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:49,954 139853893207744 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHZ | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2025-01-14 01:06:50,391 139855575739264 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2025-01-14 01:06:50,540 139855575739264 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2025-01-14 01:06:50,678 139855575739264 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2025-01-14 01:06:50,944 139855575739264 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2025-01-14 01:06:51,061 139855575739264 INFO utils.log_raw(): TIMING: 16.999 secs for Total Download

List the files that were downloaded, just to make sure !

print(os.listdir(raw_data_path))

['2019_02_01_12_00_00T2019_02_02_00_00_00.h5', '2019_02_01_00_00_00T2019_02_01_12_00_00.h5', 'station.csv']

Plot the raw data, make sure it’s noise!

file = os.path.join(raw_data_path, "2019_02_01_00_00_00T2019_02_01_12_00_00.h5")
raw_store = ASDFRawDataStore(raw_data_path) # Store for reading raw data
timespans = raw_store.get_timespans()
plotting_modules.plot_waveform(raw_store, timespans[0], 'CI','ADO', 0.01, 0.4) # this function takes for input: filename, network, station, freqmin, freqmax for a bandpass filter

_images/523f565a7c018e0f2986ec04bada7cd1a9969cd0ab84886fc042bc273adb238f.png

Step 1: Cross-correlation#

This step will perform the cross correlation. For each time chunk, it will read the data, perform classic ambient noise pre-processing (time and frequency normalization), FFT, cross correlation, substacking, saving cross correlations in to a temp ASDF file (this is not fast and will be improved).

# For this tutorial make sure the previous run is empty
if os.path.exists(cc_data_path):
    shutil.rmtree(cc_data_path)

config.freq_norm = FreqNorm.RMA
cc_store = ASDFCCStore(cc_data_path) # Store for writing CC data

# print the configuration parameters. Some are chosen by default but we cab modify them
print(config)

client_url_key='SCEDC' start_date=datetime.datetime(2019, 2, 1, 0, 0, tzinfo=datetime.timezone.utc) end_date=datetime.datetime(2019, 2, 2, 0, 0, tzinfo=datetime.timezone.utc) sampling_rate=20 single_freq=True cc_len=3600 lamin=31 lamax=42 lomin=-124 lomax=-115 down_list=False networks=['*'] stations=['A*'] channels=['BH?'] step=1800.0 freqmin=0.05 freqmax=2.0 freq_norm=<FreqNorm.RMA: 'rma'> time_norm=<TimeNorm.NO: 'no'> cc_method=<CCMethod.XCORR: 'xcorr'> smooth_N=10 smoothspect_N=10 substack=True substack_windows=1 maxlag=200 inc_hours=12 max_over_std=10 ncomp=3 stationxml=False rm_resp=<RmResp.INV: 'inv'> rm_resp_out='VEL' respdir=None acorr_only=False xcorr_only=True stack_method=<StackMethod.LINEAR: 'linear'> keep_substack=False rotation=True correction=False correction_csv=None storage_options=defaultdict(<class 'dict'>, {})

Perform the cross correlation

cross_correlate(raw_store, config, cc_store)

2025-01-14 01:06:52,019 139855575739264 INFO correlate.cross_correlate(): Starting Cross-Correlation with 4 cores

2025-01-14 01:06:52,045 139855575739264 INFO utils.log_raw(): TIMING CC Main:  0.025 secs for get 11 channels

2025-01-14 01:06:52,046 139855575739264 INFO correlate.cc_timespan(): Checking for stations already done: 10 pairs

2025-01-14 01:06:52,048 139855575739264 INFO utils.log_raw(): TIMING CC Main:  0.002 secs for check for 4 stations already done (warm up cache)

2025-01-14 01:06:52,050 139855575739264 INFO utils.log_raw(): TIMING CC Main:  0.002 secs for check for stations already done

2025-01-14 01:06:52,050 139855575739264 INFO correlate.cc_timespan(): Still need to process: 4/4 stations, 11/11 channels, 10/10 pairs for 2019-02-01T00:00:00+0000 - 2019-02-01T12:00:00+0000

2025-01-14 01:06:52,322 139855575739264 INFO correlate._filter_channel_data(): Picked 20.0 as the closest sampling_rate to 20.0. 

2025-01-14 01:06:52,323 139855575739264 INFO correlate._filter_channel_data(): Filtered to 11/11 channels with sampling rate == 20.0

2025-01-14 01:06:52,324 139855575739264 INFO utils.log_raw(): TIMING CC Main:  0.274 secs for Read channel data: 11 channels

2025-01-14 01:06:52,543 139852905449152 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:52,689 139853087901376 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:52,698 139853050152640 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:52,729 139853031278272 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:52,730 139853106775744 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:52,733 139853012403904 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:52,764 139852924323520 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:52,776 139853069027008 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:55,348 139853050152640 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:55,459 139853087901376 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:55,474 139852905449152 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:56,724 139855575739264 INFO utils.log_raw(): TIMING CC Main:  4.401 secs for Preprocess: 11 channels

2025-01-14 01:06:56,725 139855575739264 INFO correlate.check_memory(): Require  0.06gb memory for cross correlations

2025-01-14 01:06:57,295 139855575739264 INFO utils.log_raw(): TIMING CC Main:  0.570 secs for Compute FFTs: 11 channels

2025-01-14 01:06:57,298 139855575739264 INFO correlate.cc_timespan(): Starting CC with 10 station pairs

2025-01-14 01:06:58,710 139855575739264 INFO utils.log_raw(): TIMING CC Main:  1.412 secs for Correlate and write to store

2025-01-14 01:06:58,806 139855575739264 INFO utils.log_raw(): TIMING CC Main:  6.786 secs for Process the chunk of 2019-02-01T00:00:00+0000 - 2019-02-01T12:00:00+0000

2025-01-14 01:06:58,835 139855575739264 INFO utils.log_raw(): TIMING CC Main:  0.025 secs for get 12 channels

2025-01-14 01:06:58,836 139855575739264 INFO correlate.cc_timespan(): Checking for stations already done: 10 pairs

2025-01-14 01:06:58,837 139855575739264 INFO utils.log_raw(): TIMING CC Main:  0.002 secs for check for 4 stations already done (warm up cache)

2025-01-14 01:06:58,839 139855575739264 INFO utils.log_raw(): TIMING CC Main:  0.002 secs for check for stations already done

2025-01-14 01:06:58,840 139855575739264 INFO correlate.cc_timespan(): Still need to process: 4/4 stations, 12/12 channels, 10/10 pairs for 2019-02-01T12:00:00+0000 - 2019-02-02T00:00:00+0000

2025-01-14 01:06:59,109 139855575739264 INFO correlate._filter_channel_data(): Picked 20.0 as the closest sampling_rate to 20.0. 

2025-01-14 01:06:59,110 139855575739264 INFO correlate._filter_channel_data(): Filtered to 12/12 channels with sampling rate == 20.0

2025-01-14 01:06:59,110 139855575739264 INFO utils.log_raw(): TIMING CC Main:  0.271 secs for Read channel data: 12 channels

2025-01-14 01:06:59,376 139853144524480 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHE | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:59,384 139853069027008 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHE | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:59,441 139853897402048 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHZ | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:59,447 139853106775744 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHN | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:59,458 139853031278272 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHZ | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:59,539 139853087901376 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHN | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:59,571 139853050152640 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHE | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:06:59,608 139853012403904 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHN | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:07:02,029 139853031278272 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHZ | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:07:02,270 139853069027008 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHE | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:07:02,361 139853897402048 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHZ | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:07:02,375 139853106775744 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHN | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2025-01-14 01:07:03,701 139855575739264 INFO utils.log_raw(): TIMING CC Main:  4.591 secs for Preprocess: 12 channels

2025-01-14 01:07:03,702 139855575739264 INFO correlate.check_memory(): Require  0.07gb memory for cross correlations

2025-01-14 01:07:04,330 139855575739264 INFO utils.log_raw(): TIMING CC Main:  0.627 secs for Compute FFTs: 12 channels

2025-01-14 01:07:04,331 139855575739264 INFO correlate.cc_timespan(): Starting CC with 10 station pairs

2025-01-14 01:07:05,617 139855575739264 INFO utils.log_raw(): TIMING CC Main:  1.286 secs for Correlate and write to store

2025-01-14 01:07:05,721 139855575739264 INFO utils.log_raw(): TIMING CC Main:  6.911 secs for Process the chunk of 2019-02-01T12:00:00+0000 - 2019-02-02T00:00:00+0000

2025-01-14 01:07:05,727 139855575739264 INFO utils.log_raw(): TIMING CC Main: 13.708 secs for Step 1 in total with 4 cores

Plot a single set of the cross correlation

pairs = cc_store.get_station_pairs()
timespans = cc_store.get_timespans(*pairs[0])
plotting_modules.plot_substack_cc(cc_store, timespans[0], 0.1, 1, 200, False)

_images/2a48d87a7602360c2d4115ec4282ce913cf3a35fcb2829b4898214f1f848e25f.png

_images/252a361ba377246741a2e5c7ede6f0c48b967ca17c874e0e0cce6756bfb4239b.png

_images/fecfdb1540f7e20687383f9f5568ed12b68ab1402da207f1656a4bbe965900b6.png

_images/93eb56571a9548a7ec83f362dc9f408272f8d3e492399495096e3ce9f85e4555.png

_images/8861a959ace9684173a99fa2eecf57cd2ca6f2b7714b9addf42ad3e808ca9055.png

_images/cd372efabf5829c7323e80260fac06a12abec33f90ffe5668f9404d8f4518f76.png

_images/5ee11f9325cedae3ebf1a72d80b5df6ed987c0402509f87601e5436d732cff9f.png

_images/01fa532b6343746abbc4ff2fd3af07c20d762ebe281f9407e1df0552747a208f.png

_images/4ebd46716c5f2c3880369f63a7e8d7da485a3608c74e2b5cd255086ec8b77626.png

_images/913ba72fc3a63df83af8f137e30dd415f7a4a602c8871eaa1543af5bdf1daaf3.png

_images/c9d98f40a939609633cef940dd54119075c630152a27c08f4677eed79c1008c3.png

_images/718c0521da1ce9a4f45ea168b1d2aefb223d2ab9fbe90317d4b9fd8ff283d4ba.png

_images/78981b3982ac2dca53b7adce03185ea70c492922284bb8e22c1480eeadfe0de1.png

_images/5adfb0213a8e12d6d800e52267dc8c8bc6cd38167c51a793ad6d51061fe79f32.png

_images/705c3bddd7d2491fe51f89bbb6bfb0df462e41e97cf92a398044b7fbd1c34ab7.png

_images/ec3b986a7d331d97bd0f858bc19a23f3d9ad5c65a3fff92a759e7b5566415831.png

_images/842076d424ac112cec6b294d6d3612c7e2213ac06e342d01e2fd33654f8b1870.png

_images/e2927630ce8a99e684f43e967b1192d538f1f6a2765d8070e44b3d625f4c35ec.png

_images/be421972564ba2be82232e5532d495d80c8d032634c5f08e856a4dba6c045d4a.png

_images/e7bca9ac4006a62a654903952016f53e8ff6406345ac828049d298595526dfda.png

_images/ec07caf79d1766abfa70b49bd924d26cbaf3bdab00c12615c128829b2c39b189.png

_images/0a6ebb30dc9dd908282953ddce191c830a606a4347f9dfadf9f2ff74b6642fa6.png

_images/ceb3c763775f249e93489d874690e2faf0df320d6e2f5e1ca211902218931afc.png

_images/7be603554648171aae0e7528c86d46fc21a380b9a55b91e91aff008faf823e29.png

_images/bd83985b2b716078e1b7907f040a06e69b1a3b71fbd85a1e041a9cd7225291dd.png

_images/6b762aa6922ca37046cadff91dbb922067fbe3177e777148a4cb486a1e710ae4.png

_images/c6258f8563f646769ea4082019ea2b1a9556aeb0befc476c819e9a2b99ec2ec2.png

_images/b32048394431f5fbe92b41385e24bf96057dc193811cebadb187cfe735a93972.png

_images/41505772ce7210cd0020276f59247a3c42ea8bc9facc9fd0ce839b9483d32f77.png

_images/f9a8bb76f7fa55b3ea6de72c3b1bd8fee2e846978b6f1819360325dcd3269890.png

_images/56c9360eea585b00740727e8eadb1374489de524e06d2dbbcd4e0ec9808732f0.png

_images/ebf74fb6a6533c900005d7fff149ea577d9831d24f2d03957241e6b152bd1a60.png

_images/c871c459f43797375157913a64052781241d38212d64777bfa2bff9fba6ed9eb.png

_images/2999b6ee8cc5aff7a7fad7d698a0f581a786dc9095a2df514abbaf8b45e4ea65.png

_images/2b3add2fd044f104b07c7651889354cf4cea8e8e7543dc3e706893c7b5af7465.png

_images/893c0460d50a7cdf2fdfb0ae6f13884108f2c1cfe1b17b5ad7e098a38d429968.png

_images/0761e3f66144d55318062d601bff38924ab62653d8bbc5f1dc2a15bf385f46b4.png

_images/2891f3dee3e31d3b5eba5f4d96919340fd6104d7751c4a7a15943492a19be971.png

_images/7f41af15fb7c4ebc071caff1403a4f58d5291780959a3c0514b9283d72d4b36e.png

_images/92f8dd529e398facb4731e7865f3501baec799de62e075edddd657151e3f8631.png

_images/cfaf4ffe13d378d86a39cd15b573c39c62e1be7678c175f466e60780ba33e6bb.png

_images/d5b78967f10084a5b53b0d362613aafd479ac3e65df21d9451ccddf36fd5325c.png

_images/8168c861fa8bc2ad557d74af95d41f290088285ac0cd3387546a225216fadd48.png

_images/9682c57e0bc0f4c390810bc6a9f44888f776d972016199c8094334eb9a880bca.png

_images/d8ed12b4020ce013c9059db369f7828064ad6f15fbfaa6f81ba8bb2eec2e4946.png

_images/b40ccb4d9e94e2e201d73bfdcc96f210693ba2fc8b111e99547eff603c0c1975.png

_images/66ef50e6c8ee5f970ce2ab60612a10e29901c76123e74f375a853e48844678d6.png

_images/5ccec1955c071eed5a0ab47a1b0915d6e993ee5c8653b1f1b2286a4968cd7e53.png

_images/2a69f52135294947e86173da72825439b433f7269fb3433f5bb50b4a0af749da.png

_images/42d55d5f1762b72d46b22c3c6483493f33f5a80295205a17d3eb63cde252b745.png

_images/26747c473f05fe2a45b3bf3dbfb8716fc0870a7222a88c5e76e6f564f580688f.png

_images/33ee066ca6a3e629088e0870f69a7d92dc2d38d40e1589956c5b89be2c3ff224.png

_images/ec8678da64d6cd6d0567c6d837e46d99354046853057e09352c470adfd8b83b2.png

_images/59699f684de014fa0773ec03a47ce3730c783ff8bf6d87a64464880574d8f3a9.png

_images/46b6d2653b98e168e9e58e052a73822d5180e8087876ec9a0ebf4e064340205e.png

_images/e1d0de1998cfe6fdf8a18fa385b58524e40d03d709180c014aa81c5b0fc1eb94.png

_images/aadf3f2a8daa4b3c432aba5e38615c2fbd0c5eade91c9ff9bcd0adae5c99d62e.png

_images/f5c424f5370913600a6d45b7fe9bafda6520f51f19476724da7842466019dde8.png

_images/86aaf569a68ba12175b2561231110acb20d5fb5103fd293d18639e8b7b36855c.png

_images/12e311eef2626e04511a99165324ea9af35ec71536853faaf0e4afe5538e26b5.png

_images/640fa0cfb330753cc097d2554cd0644b79082cb8a744e8782045d8d254171e12.png

_images/8b0735d90d06d5dae740d110c88e9f48d8d7422c777a5ab1c1c1fb14cbda2f8d.png

_images/073cd9ac21c551d6e7dc7d4b366637c2fa3ff1224d73b558be85530fa0f3db49.png

_images/fdf030f0750324804342e73452cf71068f3654ff1c7f32c4dc3613c66f64f2bd.png

_images/e1d21c8685a2822f414a821314b7c560472caf5cee89413ead15b3a6157abe36.png

_images/7adbae509faffdc31dfe256efef32d7186815efa828afaae27d6e033a07b71ef.png

Step 2: Stack the cross correlation#

This combines the time-chunked ASDF files to stack over each time chunk and at each station pair.

# open a new cc store in read-only mode since we will be doing parallel access for stacking
cc_store = ASDFCCStore(cc_data_path, mode="r")
print(cc_store.get_station_pairs())
stack_store = ASDFStackStore(stack_data_path)
config.stations = ["*"] # stacking doesn't support prefixes yet, so allow all stations
stack_cross_correlations(cc_store, stack_store, config)

2025-01-14 01:07:24,061 139855575739264 INFO stack.initializer(): Station pairs: 10

[(CI.ALP, CI.ARV), (CI.ALP, CI.AVM), (CI.ADO, CI.ADO), (CI.ADO, CI.ALP), (CI.ADO, CI.ARV), (CI.ADO, CI.AVM), (CI.AVM, CI.AVM), (CI.ALP, CI.ALP), (CI.ARV, CI.ARV), (CI.ARV, CI.AVM)]

2025-01-14 01:07:27,645 140098930662272 INFO stack.stack_store_pair(): Stacking CI.ADO_CI.ALP/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2025-01-14 01:07:27,655 140019165670272 INFO stack.stack_store_pair(): Stacking CI.ADO_CI.ADO/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2025-01-14 01:07:27,661 140223541717888 INFO stack.stack_store_pair(): Stacking CI.ADO_CI.ARV/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2025-01-14 01:07:27,678 139975268232064 INFO stack.stack_store_pair(): Stacking CI.ADO_CI.AVM/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2025-01-14 01:07:27,697 140019165670272 INFO utils.log_raw(): TIMING:  0.037 secs for loading CCF data
2025-01-14 01:07:27,699 140098930662272 INFO utils.log_raw(): TIMING:  0.049 secs for loading CCF data
2025-01-14 01:07:27,702 139975268232064 WARNING stack.validate_pairs(): continue! not enough cross components for cross-correlation (CI.ADO, CI.AVM) in 2019-02-01T00:00:00+0000 - 2019-02-01T12:00:00+0000. ncomp=3, n_pairs=6
2025-01-14 01:07:27,706 140019165670272 INFO utils.log_raw(): TIMING:  0.009 secs for stack/rotate all station pairs (CI.ADO, CI.ADO)
2025-01-14 01:07:27,708 140098930662272 INFO utils.log_raw(): TIMING:  0.009 secs for stack/rotate all station pairs (CI.ADO, CI.ALP)
2025-01-14 01:07:27,723 140223541717888 INFO utils.log_raw(): TIMING:  0.057 secs for loading CCF data
2025-01-14 01:07:27,727 140019165670272 INFO utils.log_raw(): TIMING:  0.020 secs for writing stack pair (CI.ADO, CI.ADO)
2025-01-14 01:07:27,730 140019165670272 INFO stack.stack_store_pair(): Stacking CI.ALP_CI.ALP/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2025-01-14 01:07:27,733 140223541717888 INFO utils.log_raw(): TIMING:  0.010 secs for stack/rotate all station pairs (CI.ADO, CI.ARV)
2025-01-14 01:07:27,745 139975268232064 INFO utils.log_raw(): TIMING:  0.062 secs for loading CCF data
2025-01-14 01:07:27,756 139975268232064 INFO utils.log_raw(): TIMING:  0.011 secs for stack/rotate all station pairs (CI.ADO, CI.AVM)
2025-01-14 01:07:27,767 140098930662272 INFO utils.log_raw(): TIMING:  0.058 secs for writing stack pair (CI.ADO, CI.ALP)
2025-01-14 01:07:27,769 140098930662272 INFO stack.stack_store_pair(): Stacking CI.ALP_CI.ARV/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2025-01-14 01:07:27,774 140019165670272 INFO utils.log_raw(): TIMING:  0.039 secs for loading CCF data
2025-01-14 01:07:27,781 140019165670272 INFO utils.log_raw(): TIMING:  0.007 secs for stack/rotate all station pairs (CI.ALP, CI.ALP)
2025-01-14 01:07:27,794 140223541717888 INFO utils.log_raw(): TIMING:  0.060 secs for writing stack pair (CI.ADO, CI.ARV)
2025-01-14 01:07:27,796 140223541717888 INFO stack.stack_store_pair(): Stacking CI.ALP_CI.AVM/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2025-01-14 01:07:27,800 140019165670272 INFO utils.log_raw(): TIMING:  0.018 secs for writing stack pair (CI.ALP, CI.ALP)
2025-01-14 01:07:27,800 140019165670272 INFO stack.stack_store_pair(): Stacking CI.ARV_CI.ARV/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2025-01-14 01:07:27,814 139975268232064 INFO utils.log_raw(): TIMING:  0.058 secs for writing stack pair (CI.ADO, CI.AVM)
2025-01-14 01:07:27,817 139975268232064 INFO stack.stack_store_pair(): Stacking CI.ARV_CI.AVM/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2025-01-14 01:07:27,819 140223541717888 WARNING stack.validate_pairs(): continue! not enough cross components for cross-correlation (CI.ALP, CI.AVM) in 2019-02-01T00:00:00+0000 - 2019-02-01T12:00:00+0000. ncomp=3, n_pairs=6
2025-01-14 01:07:27,824 140098930662272 INFO utils.log_raw(): TIMING:  0.052 secs for loading CCF data
2025-01-14 01:07:27,833 140098930662272 INFO utils.log_raw(): TIMING:  0.009 secs for stack/rotate all station pairs (CI.ALP, CI.ARV)
2025-01-14 01:07:27,838 139975268232064 WARNING stack.validate_pairs(): continue! not enough cross components for cross-correlation (CI.ARV, CI.AVM) in 2019-02-01T00:00:00+0000 - 2019-02-01T12:00:00+0000. ncomp=3, n_pairs=6
2025-01-14 01:07:27,842 140223541717888 INFO utils.log_raw(): TIMING:  0.043 secs for loading CCF data
2025-01-14 01:07:27,846 140019165670272 INFO utils.log_raw(): TIMING:  0.041 secs for loading CCF data

2025-01-14 01:07:27,851 140223541717888 INFO utils.log_raw(): TIMING:  0.009 secs for stack/rotate all station pairs (CI.ALP, CI.AVM)
2025-01-14 01:07:27,854 140019165670272 INFO utils.log_raw(): TIMING:  0.008 secs for stack/rotate all station pairs (CI.ARV, CI.ARV)
2025-01-14 01:07:27,863 139975268232064 INFO utils.log_raw(): TIMING:  0.043 secs for loading CCF data
2025-01-14 01:07:27,871 139975268232064 INFO utils.log_raw(): TIMING:  0.008 secs for stack/rotate all station pairs (CI.ARV, CI.AVM)
2025-01-14 01:07:27,874 140019165670272 INFO utils.log_raw(): TIMING:  0.020 secs for writing stack pair (CI.ARV, CI.ARV)
2025-01-14 01:07:27,875 140019165670272 INFO stack.stack_store_pair(): Stacking CI.AVM_CI.AVM/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2025-01-14 01:07:27,888 140019165670272 WARNING stack.validate_pairs(): continue! not enough cross components for auto-correlation (CI.AVM, CI.AVM) in 2019-02-01T00:00:00+0000 - 2019-02-01T12:00:00+0000. ncomp=3, n_pairs=3
2025-01-14 01:07:27,891 140098930662272 INFO utils.log_raw(): TIMING:  0.057 secs for writing stack pair (CI.ALP, CI.ARV)
2025-01-14 01:07:27,900 140019165670272 INFO utils.log_raw(): TIMING:  0.023 secs for loading CCF data
2025-01-14 01:07:27,903 140019165670272 INFO utils.log_raw(): TIMING:  0.003 secs for stack/rotate all station pairs (CI.AVM, CI.AVM)
2025-01-14 01:07:27,913 140223541717888 INFO utils.log_raw(): TIMING:  0.061 secs for writing stack pair (CI.ALP, CI.AVM)
2025-01-14 01:07:27,915 140019165670272 INFO utils.log_raw(): TIMING:  0.011 secs for writing stack pair (CI.AVM, CI.AVM)
2025-01-14 01:07:27,923 139975268232064 INFO utils.log_raw(): TIMING:  0.052 secs for writing stack pair (CI.ARV, CI.AVM)

2025-01-14 01:07:28,478 139855575739264 INFO utils.log_raw(): TIMING:  4.419 secs for step 2 in total

pairs = stack_store.get_station_pairs()
print(f"Found {len(pairs)} station pairs")
sta_stacks = stack_store.read_bulk(timerange, pairs) # no timestamp used in ASDFStackStore

Found 10 station pairs

2025-01-14 01:07:28,695 139855575739264 INFO utils.log_raw(): TIMING:  0.210 secs for loading 10 stacks

Plot the stacks

print(os.listdir(cc_data_path))
print(os.listdir(stack_data_path))

['2019_02_01_12_00_00T2019_02_02_00_00_00.h5', '2019_02_01_00_00_00T2019_02_01_12_00_00.h5']
['CI.ARV', 'CI.ADO', 'CI.AVM', 'CI.ALP']

plotting_modules.plot_all_moveout(sta_stacks, 'Allstack_linear', 0.1, 0.2, 'ZZ', 1)

2025-01-14 01:07:28,706 139855575739264 INFO plotting_modules.plot_all_moveout(): Plottting: Allstack_linear, 10 station pairs

200 8001

_images/0830eb0797ba54ff1a4ddb75cabeae843c6dae0d7851e0535d889f142f445817.png