Getting Started

Getting Started#

Welcome to the NoisePy Colab Tutorial!

This tutorial will walk you through the basic steps of using NoisePy to compute ambient noise cross correlation functions.

First, we install the noisepy-seis package

# Uncomment and run this line if the environment doesn't have noisepy already installed:
# ! pip install noisepy-seis 

Warning: NoisePy uses obspy as a core Python module to manipulate seismic data. Restart the runtime now for proper installation of obspy on Colab.

Then we import the basic modules

from noisepy.seis import download, cross_correlate, stack_cross_correlations, __version__
from noisepy.seis.io import plotting_modules
from noisepy.seis.io.asdfstore import ASDFRawDataStore, ASDFCCStore, ASDFStackStore
from noisepy.seis.io.datatypes import CCMethod, ConfigParameters, FreqNorm, RmResp, TimeNorm
from dateutil.parser import isoparse
from datetimerange import DateTimeRange
import os
import shutil
print(f"Using NoisePy version {__version__}")

path = os.path.join(".", "get_started_data")

os.makedirs(path,exist_ok=True)
raw_data_path = os.path.join(path, "RAW_DATA")
cc_data_path = os.path.join(path, "CCF")
stack_data_path = os.path.join(path, "STACK")

/opt/hostedtoolcache/Python/3.10.14/x64/lib/python3.10/site-packages/noisepy/seis/io/utils.py:13: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)
  from tqdm.autonotebook import tqdm

Using NoisePy version 0.1.dev1

Ambient Noise Project Configuration#

We store the metadata information about the ambient noise cross correlation workflow in a ConfigParameters() object. We first initialize it, then we tune the parameters for this cross correlation.

config = ConfigParameters() # default config parameters which can be customized
config.inc_hours = 12
config.samp_freq= 20  # (int) Sampling rate in Hz of desired processing (it can be different than the data sampling rate)
config.cc_len= 3600  # (float) basic unit of data length for fft (sec)
    # criteria for data selection
config.ncomp = 3  # 1 or 3 component data (needed to decide whether do rotation)


config.acorr_only = False  # only perform auto-correlation or not
config.xcorr_only = True  # only perform cross-correlation or not

config.inc_hours = 12 # if the data is first 

config.lamin = 31       # min latitude
config.lamax = 42       # max latitude
config.lomin = -124     # min longitude
config.lomax = -115     # max longitude
config.net_list = ["*"] # look for all network codes



 # pre-processing parameters
config.step= 1800.0  # (float) overlapping between each cc_len (sec)
config.stationxml= False  # station.XML file used to remove instrument response for SAC/miniseed data
config.rm_resp= RmResp.INV  # select 'no' to not remove response and use 'inv' if you use the stationXML,'spectrum',
config.freqmin = 0.05
config.freqmax = 2.0
config.max_over_std  = 10  # threshold to remove window of bad signals: set it to 10*9 if prefer not to remove them

# TEMPORAL and SPECTRAL NORMALISATION
config.freq_norm= FreqNorm.RMA # choose between "rma" for a soft whitenning or "no" for no whitening. Pure whitening is not implemented correctly at this point.
config.smoothspect_N = 10  # moving window length to smooth spectrum amplitude (points)
    # here, choose smoothspect_N for the case of a strict whitening (e.g., phase_only)

config.time_norm = TimeNorm.NO  # 'no' for no normalization, or 'rma', 'one_bit' for normalization in time domain,
    # TODO: change time_norm option from "no" to "None"
config.smooth_N= 10  # moving window length for time domain normalization if selected (points)

config.cc_method= CCMethod.XCORR # 'xcorr' for pure cross correlation OR 'deconv' for deconvolution;
    # FOR "COHERENCY" PLEASE set freq_norm to "rma", time_norm to "no" and cc_method to "xcorr"

# OUTPUTS:
config.substack = True  # True = smaller stacks within the time chunk. False: it will stack over inc_hours
config.substack_len = config.cc_len  # how long to stack over (for monitoring purpose): need to be multiples of cc_len
    # if substack=True, substack_len=2*cc_len, then you pre-stack every 2 correlation windows.
    # for instance: substack=True, substack_len=cc_len means that you keep ALL of the correlations

config.maxlag= 200  # lags of cross-correlation to save (sec)
config.substack = True

Step 0: download data#

This step will download data using obspy and save them into ASDF files locally. The data will be stored for each time chunk defined in hours by inc_hours.

The download will clean up the raw data by detrending, removing the mean, bandpassing (broadly), removing the instrumental response, merging gaps, ignoring too-gappy data.

Use the function download with the following arguments:

path:where to put the data
config: configuration settings, in particular:
- channel: list of the seismic channels to download, and example is shown below
- stations: list of the seismic stations, it can be “*” (not “all”)
- start_time
- end_time
client_url_key: the string for FDSN clients

config.stations = ["A*"]
config.channels =  ["BHE","BHN","BHZ"]
config.start_date =  isoparse("2019-02-01T00:00:00Z")
config.end_date = isoparse("2019-02-02T00:00:00Z")
timerange = DateTimeRange(config.start_date, config.end_date)

# Download data locally. Enters raw data path, channel types, stations, config, and fdsn server.
download(raw_data_path, config)

2024-06-24 21:33:31,525 140264513891200 INFO fdsn_download.download(): Download
        From: 2019-02-01T00:00:00.000000Z
        To: 2019-02-02T00:00:00.000000Z
        Stations: ['A*']
        Channels: ['BHE', 'BHN', 'BHZ']
        

2024-06-24 21:33:36,376 140264513891200 INFO fdsn_download.download(): Fetched inventory

2024-06-24 21:33:36,377 140264513891200 INFO utils.log_raw(): TIMING: 4.8516 secs. for Getting inventory

2024-06-24 21:33:36,626 140262999787072 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ADO&location=%2A&channel=BHZ
Request Submitted:
2024/06/24 21:33:36 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ADO.BHZ)

2024-06-24 21:33:36,630 140262860904000 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ADO&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:36 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ADO.BHN)

2024-06-24 21:33:36,630 140262626031168 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHZ
Request Submitted:
2024/06/24 21:33:36 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHZ)

2024-06-24 21:33:36,630 140262894466624 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:36 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHN)

2024-06-24 21:33:36,631 140262592468544 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:36 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHN)

2024-06-24 21:33:36,833 140262999787072 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ADO&location=%2A&channel=BHZ
Request Submitted:
2024/06/24 21:33:36 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ADO.BHZ)

2024-06-24 21:33:36,834 140262894466624 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:36 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHN)

2024-06-24 21:33:36,834 140262626031168 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHZ
Request Submitted:
2024/06/24 21:33:36 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHZ)

2024-06-24 21:33:36,846 140262592468544 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:36 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHN)

2024-06-24 21:33:37,043 140262626031168 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHZ
Request Submitted:
2024/06/24 21:33:37 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHZ)

2024-06-24 21:33:37,056 140262592468544 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:37 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHN)

2024-06-24 21:33:37,248 140262626031168 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHZ
Request Submitted:
2024/06/24 21:33:37 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHZ)

2024-06-24 21:33:37,255 140262592468544 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:37 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHN)

2024-06-24 21:33:37,586 140262609249856 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:37,805 140262983005760 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:38,008 140262877685312 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:38,299 140262860904000 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:38,466 140262894466624 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:38,502 140262999787072 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:38,642 140262626031168 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:38,817 140262592468544 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:40,305 140264513891200 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2024-06-24 21:33:40,720 140264513891200 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2024-06-24 21:33:40,969 140264513891200 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2024-06-24 21:33:41,070 140262999787072 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T00%3A00%3A00.000000&endtime=2019-02-01T12%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHZ
Request Submitted:
2024/06/24 21:33:41 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHZ)

2024-06-24 21:33:41,217 140264513891200 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2024-06-24 21:33:41,295 140262609249856 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:41,372 140264513891200 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2024-06-24 21:33:41,620 140264513891200 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2024-06-24 21:33:41,768 140262983005760 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:41,822 140264513891200 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2024-06-24 21:33:41,828 140262877685312 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:42,047 140264513891200 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2024-06-24 21:33:42,187 140262999787072 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T11:59:59.950000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:42,672 140264513891200 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2024-06-24 21:33:43,208 140264513891200 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2024-06-24 21:33:43,341 140264513891200 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2024-06-24 21:33:43,455 140264513891200 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2024-06-24 21:33:43,800 140262626031168 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ADO&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ADO.BHN)

2024-06-24 21:33:43,802 140262877685312 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHN)

2024-06-24 21:33:43,806 140262860904000 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ADO&location=%2A&channel=BHE
Request Submitted:
2024/06/24 21:33:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ADO.BHE)

2024-06-24 21:33:43,807 140262983005760 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHZ
Request Submitted:
2024/06/24 21:33:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHZ)

2024-06-24 21:33:43,808 140262999787072 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHE
Request Submitted:
2024/06/24 21:33:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHE)

2024-06-24 21:33:44,002 140262626031168 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ADO&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ADO.BHN)

2024-06-24 21:33:44,007 140262877685312 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHN)

2024-06-24 21:33:44,008 140262860904000 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ADO&location=%2A&channel=BHE
Request Submitted:
2024/06/24 21:33:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ADO.BHE)

2024-06-24 21:33:44,009 140262983005760 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ALP&location=%2A&channel=BHZ
Request Submitted:
2024/06/24 21:33:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ALP.BHZ)

2024-06-24 21:33:44,013 140262999787072 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHE
Request Submitted:
2024/06/24 21:33:43 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHE)

2024-06-24 21:33:44,208 140262860904000 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ADO&location=%2A&channel=BHE
Request Submitted:
2024/06/24 21:33:44 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ADO.BHE)

2024-06-24 21:33:44,213 140262877685312 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:44 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHN)

2024-06-24 21:33:44,418 140262860904000 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ADO&location=%2A&channel=BHE
Request Submitted:
2024/06/24 21:33:44 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ADO.BHE)

2024-06-24 21:33:44,421 140262877685312 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=ARV&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:44 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(ARV.BHN)

2024-06-24 21:33:45,563 140262609249856 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHN | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:45,581 140262894466624 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHZ | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:45,690 140262626031168 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHN | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:45,709 140262592468544 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHE | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:45,901 140262999787072 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHE | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:45,955 140262983005760 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHZ | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:46,205 140262860904000 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHE | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:46,391 140262877685312 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHN | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:48,159 140264513891200 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2024-06-24 21:33:48,423 140264513891200 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2024-06-24 21:33:48,485 140262894466624 WARNING fdsn_download.download_stream(): <class 'obspy.clients.fdsn.header.FDSNInternalServerException'>/Service responds: Internal server error
HTTP Status code: 500
Detailed response of server:

500
Error 500: STP clients exceeded. Please try again later OR download data from the SCEDC public data set hosted on the cloud (AWS). For details, please see https://scedc.caltech.edu/cloud/ 
More Details:
handler exited, code: 1  reason: Internal Server Error
Request:
http://service.scedc.caltech.edu/fdsnws/dataselect/1/query?starttime=2019-02-01T12%3A00%3A00.000000&endtime=2019-02-02T00%3A00%3A00.000000&network=CI&station=AVM&location=%2A&channel=BHN
Request Submitted:
2024/06/24 21:33:48 UTC
Service version:
Service: fdsnws-dataselect  version: 1.1.0 for get_waveforms(AVM.BHN)

2024-06-24 21:33:48,597 140264513891200 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2024-06-24 21:33:48,725 140264513891200 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2024-06-24 21:33:48,848 140264513891200 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2024-06-24 21:33:49,089 140264513891200 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2024-06-24 21:33:49,375 140264513891200 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2024-06-24 21:33:49,507 140262626031168 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHE | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:49,567 140264513891200 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2024-06-24 21:33:49,717 140262592468544 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHZ | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:49,840 140262609249856 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHZ | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:49,843 140262894466624 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHN | 2019-02-01T11:59:59.950000Z - 2019-02-01T23:59:59.900000Z | 20.0 Hz, 864000 samples using inv

2024-06-24 21:33:50,946 140264513891200 INFO fdsn_download.download(): Downloaded BHE/bhe_00

2024-06-24 21:33:51,093 140264513891200 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2024-06-24 21:33:51,212 140264513891200 INFO fdsn_download.download(): Downloaded BHZ/bhz_00

2024-06-24 21:33:51,328 140264513891200 INFO fdsn_download.download(): Downloaded BHN/bhn_00

2024-06-24 21:33:51,443 140264513891200 INFO utils.log_raw(): TIMING: 19.9184 secs. for Total Download

List the files that were downloaded, just to make sure !

print(os.listdir(raw_data_path))

['2019_02_01_00_00_00T2019_02_01_12_00_00.h5', 'station.csv', '2019_02_01_12_00_00T2019_02_02_00_00_00.h5']

Plot the raw data, make sure it’s noise!

file = os.path.join(raw_data_path, "2019_02_01_00_00_00T2019_02_01_12_00_00.h5")
raw_store = ASDFRawDataStore(raw_data_path) # Store for reading raw data
timespans = raw_store.get_timespans()
plotting_modules.plot_waveform(raw_store, timespans[0], 'CI','ADO',0.01,0.4) # this function takes for input: filename, network, station, freqmin, freqmax for a bandpass filter

_images/4c390ff435082a0cde0e5dd6f7ff67a42ea9d8ae18fed5275b0017d2c6a7e418.png

Step 1: Cross-correlation#

This step will perform the cross correlation. For each time chunk, it will read the data, perform classic ambient noise pre-processing (time and frequency normalization), FFT, cross correlation, substacking, saving cross correlations in to a temp ASDF file (this is not fast and will be improved).

# For this tutorial make sure the previous run is empty
if os.path.exists(cc_data_path):
    shutil.rmtree(cc_data_path)

config.freq_norm = FreqNorm.RMA
cc_store = ASDFCCStore(cc_data_path) # Store for writing CC data

# print the configuration parameters. Some are chosen by default but we cab modify them
print(config)

client_url_key='SCEDC' start_date=datetime.datetime(2019, 2, 1, 0, 0, tzinfo=tzutc()) end_date=datetime.datetime(2019, 2, 2, 0, 0, tzinfo=tzutc()) samp_freq=20 single_freq=True cc_len=3600 lamin=31 lamax=42 lomin=-124 lomax=-115 down_list=False net_list=['*'] stations=['A*'] channels=['BHE', 'BHN', 'BHZ'] step=1800.0 freqmin=0.05 freqmax=2.0 freq_norm=<FreqNorm.RMA: 'rma'> time_norm=<TimeNorm.NO: 'no'> cc_method=<CCMethod.XCORR: 'xcorr'> smooth_N=10 smoothspect_N=10 substack=True substack_len=3600 maxlag=200 inc_hours=12 max_over_std=10 ncomp=3 stationxml=False rm_resp=<RmResp.INV: 'inv'> rm_resp_out='VEL' respdir=None acorr_only=False xcorr_only=True stack_method=<StackMethod.LINEAR: 'linear'> keep_substack=False rotation=True correction=False correction_csv=None storage_options=defaultdict(<class 'dict'>, {}) stations_file=None

Perform the cross correlation

cross_correlate(raw_store, config, cc_store)

2024-06-24 21:33:52,300 140264513891200 INFO correlate.cross_correlate(): Starting Cross-Correlation with 4 cores

2024-06-24 21:33:52,326 140264513891200 INFO utils.log_raw(): TIMING CC Main: 0.0249 secs. for get 12 channels

2024-06-24 21:33:52,327 140264513891200 INFO correlate.cc_timespan(): Checking for stations already done: 10 pairs

2024-06-24 21:33:52,329 140264513891200 INFO utils.log_raw(): TIMING CC Main: 0.0019 secs. for check for 4 stations already done (warm up cache)

2024-06-24 21:33:52,330 140264513891200 INFO utils.log_raw(): TIMING CC Main: 0.0018 secs. for check for stations already done

2024-06-24 21:33:52,331 140264513891200 INFO correlate.cc_timespan(): Still need to process: 4/4 stations, 12/12 channels, 10/10 pairs for 2019-02-01T00:00:00+0000 - 2019-02-01T12:00:00+0000

2024-06-24 21:33:52,638 140264513891200 INFO correlate._filter_channel_data(): Picked 20.0 as the closest sampling frequence to 20.0. 

2024-06-24 21:33:52,639 140264513891200 INFO correlate._filter_channel_data(): Filtered to 12/12 channels with sampling rate == 20.0

2024-06-24 21:33:52,640 140264513891200 INFO utils.log_raw(): TIMING CC Main: 0.3099 secs. for Read channel data: 12 channels

2024-06-24 21:33:52,982 140261887833664 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:33:52,988 140261854271040 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:33:53,005 140261871052352 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:33:53,005 140261988488768 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:33:53,021 140262005270080 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:33:53,021 140262022051392 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:33:53,023 140261753615936 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:33:53,023 140262122706496 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

 WARNING (norm_resp): computed and reported sensitivities differ by more than 5 percent. 
	 Execution continuing.

2024-06-24 21:33:55,555 140261854271040 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:33:55,876 140262122706496 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHE | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:33:56,073 140261887833664 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHN | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:33:56,130 140261988488768 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHZ | 2019-02-01T00:00:00.000000Z - 2019-02-01T12:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:33:57,479 140264513891200 INFO utils.log_raw(): TIMING CC Main: 4.8390 secs. for Preprocess: 12 channels

2024-06-24 21:33:57,481 140264513891200 INFO correlate.check_memory(): Require  0.07gb memory for cross correlations

2024-06-24 21:33:58,059 140264513891200 INFO utils.log_raw(): TIMING CC Main: 0.5777 secs. for Compute FFTs: 12 channels

2024-06-24 21:33:58,060 140264513891200 INFO correlate.cc_timespan(): Starting CC with 10 station pairs

2024-06-24 21:33:59,492 140264513891200 INFO utils.log_raw(): TIMING CC Main: 1.4324 secs. for Correlate and write to store

2024-06-24 21:33:59,591 140264513891200 INFO utils.log_raw(): TIMING CC Main: 7.2901 secs. for Process the chunk of 2019-02-01T00:00:00+0000 - 2019-02-01T12:00:00+0000

2024-06-24 21:33:59,626 140264513891200 INFO utils.log_raw(): TIMING CC Main: 0.0243 secs. for get 12 channels

2024-06-24 21:33:59,627 140264513891200 INFO correlate.cc_timespan(): Checking for stations already done: 10 pairs

2024-06-24 21:33:59,628 140264513891200 INFO utils.log_raw(): TIMING CC Main: 0.0015 secs. for check for 4 stations already done (warm up cache)

2024-06-24 21:33:59,630 140264513891200 INFO utils.log_raw(): TIMING CC Main: 0.0020 secs. for check for stations already done

2024-06-24 21:33:59,631 140264513891200 INFO correlate.cc_timespan(): Still need to process: 4/4 stations, 12/12 channels, 10/10 pairs for 2019-02-01T12:00:00+0000 - 2019-02-02T00:00:00+0000

2024-06-24 21:33:59,915 140264513891200 INFO correlate._filter_channel_data(): Picked 20.0 as the closest sampling frequence to 20.0. 

2024-06-24 21:33:59,916 140264513891200 INFO correlate._filter_channel_data(): Filtered to 12/12 channels with sampling rate == 20.0

2024-06-24 21:33:59,917 140264513891200 INFO utils.log_raw(): TIMING CC Main: 0.2868 secs. for Read channel data: 12 channels

2024-06-24 21:34:00,181 140261988488768 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHN | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:34:00,202 140262592468544 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHE | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:34:00,212 140261753615936 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHN | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:34:00,228 140262122706496 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHE | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:34:00,229 140262626031168 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHN | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:34:00,293 140262894466624 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHE | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:34:00,333 140262038816320 INFO noise_module.preprocess_raw(): removing response for CI.ADO..BHZ | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:34:00,347 140262609249856 INFO noise_module.preprocess_raw(): removing response for CI.ALP..BHZ | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:34:03,108 140261988488768 INFO noise_module.preprocess_raw(): removing response for CI.ARV..BHZ | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:34:03,235 140262122706496 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHE | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:34:03,274 140262626031168 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHZ | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:34:03,282 140261753615936 INFO noise_module.preprocess_raw(): removing response for CI.AVM..BHN | 2019-02-01T12:00:00.000000Z - 2019-02-02T00:00:00.000000Z | 20.0 Hz, 864001 samples using inv

2024-06-24 21:34:04,713 140264513891200 INFO utils.log_raw(): TIMING CC Main: 4.7961 secs. for Preprocess: 12 channels

2024-06-24 21:34:04,714 140264513891200 INFO correlate.check_memory(): Require  0.07gb memory for cross correlations

2024-06-24 21:34:05,332 140264513891200 INFO utils.log_raw(): TIMING CC Main: 0.6174 secs. for Compute FFTs: 12 channels

2024-06-24 21:34:05,335 140264513891200 INFO correlate.cc_timespan(): Starting CC with 10 station pairs

2024-06-24 21:34:06,509 140264513891200 INFO utils.log_raw(): TIMING CC Main: 1.1742 secs. for Correlate and write to store

2024-06-24 21:34:06,600 140264513891200 INFO utils.log_raw(): TIMING CC Main: 6.9983 secs. for Process the chunk of 2019-02-01T12:00:00+0000 - 2019-02-02T00:00:00+0000

2024-06-24 21:34:06,606 140264513891200 INFO utils.log_raw(): TIMING CC Main: 14.3066 secs. for Step 1 in total with 4 cores

Plot a single set of the cross correlation

pairs = cc_store.get_station_pairs()
timespans = cc_store.get_timespans(*pairs[0])
plotting_modules.plot_substack_cc(cc_store, timespans[0], 0.1, 1, 200, False)

_images/2398da3c6219067ef56810e4f55912552dcffcd7c40dd779b367ca0b0ba5a054.png

_images/ff70a2c73c6401a0bf790b511bceab8e96749b1f068ebc6d9a857bb229e91910.png

_images/3e4bd72d0afb9c45b453bf613247053464e5aa5dfc7ba854993f5d12ae5bc740.png

_images/776cfe81d22a2e3628e25e9778c66fb930e3d6b8e9f89d3d1aa9d15853d8f042.png

_images/878e831e119d726df48f95b220a5f2ed65017f4cb2bc31c784537f253797c1dd.png

_images/821d31351fd9c953a8c19c9b38ca6db95b381cc21031d8923670a52d70468c76.png

_images/1a670539d4c3a905fa190ef3b1f8bb8918aa1a62795fdb0d35ffb64313589e53.png

_images/20b8526d7eafc0550666ffeb125ce26e36e4abb5d07395c58937d8eef1149ef1.png

_images/d94299406cdb3a3863a2ab1138c254a53b47b8e002130cfc5d8fb88e7f59cfa1.png

_images/83458d15a5afe4e964eaf1caa5684a74518f71df6a0304b1d982512936c0fecc.png

_images/8ceac39af8844f836ec43afcee09dbf89b52f32cf2f4c6d4fa6b19c430c31110.png

_images/c6039c3fd82da113fa03e26a8a0642b0c4fa1a8e2be1cc6e6567a1c3f184d782.png

_images/43008393a838b3c615c85808237567fe54f1b9bb5494842bd9a312f6e0b06f0b.png

_images/04850ccb42e57439347e8d2572028bf6c0d38c9c408bb6719c4fd8193f3db3b7.png

_images/fcc9d0986f3efbfb47f4cab72f441a0e0056a7a09d575ba12eeadc531f275e4b.png

_images/ef33019220af2c40f1a2d283a6abf88d7c4c282aab27f949337634919364f37d.png

_images/ecdb83f6c250da200b49b6a9fe4a1b1eaf40342fa78423212ddccec2b3d82e54.png

_images/aecbd016ba4158c209c72f8ee84805311b9affca5c07d50ba2e5b65ddc6ea5e3.png

_images/291ae477932140ffc5656960c7af55184951160ad852de0b559a326da02d68bc.png

_images/54f4a5e9167a43b1dbe04402866cb5fa2236718fcc10f3de73b4b422a6262bd0.png

_images/fbd42bcd299e895084a04a4b2dffc858629b20583cfc944bc419f0097857d218.png

_images/8625450e5b0896460b5ad53720aafa37a429b30fc5d14a874b47f79fa6c17e3d.png

_images/641589c3050ae45542a5db9bfa462bd387ca75930d35326a92a2264d835e39b0.png

_images/c9a5854edc7ba348a0f109cf44f0abc792c1037b832c091b83f92f0e556c134c.png

_images/37ac879f5f0e5c9862a9cbc28f404d771e1c95cd1a1467b6324bbdba21c6f049.png

_images/66929d8baca2a255d4e474619e9b37a6d28aade2a996fb225a31210b4a0d5897.png

_images/301a33482723c8ece14c3819416dc9f012408b2be9e66c72bdeada22c3613ade.png

_images/6e0c8963c4fa552d94fdc7588fd82d1555142810c96771b0f1eb2d4c83d567e5.png

_images/05ca5aea0d9c469bfce8c03d0e740bf74a1d99f0e141d363754a81d2842e539e.png

_images/2d6e8ca18b9494a12c7ff6bc200782b56eb1a231ad7b93d68d37900e97a1f07d.png

_images/ad2bb4df644086795f8dcaaa94725dbd5c5619d8a2459e98acb345bff3d9ba1b.png

_images/af5553a97609b00d4671849d28202f159a37bfc147c0fc74e42d791a3ec9ae6c.png

_images/91444b4a3c4c9c3dd7018ced90da00bf1b6dc8c634da13adce6abd46eac496ff.png

_images/93987508eef754b8e997d07b33edf23d94aaa61cc1d68cb8fd4b469f4a21b5c5.png

_images/d1b6d1fa9b12313ef30718c9e76a7ad912ab6514c04c296735095865b4c5db91.png

_images/9953d2f36e25b3df2f3923717861e09802a5939135202d9e7b709e31060c826e.png

_images/7ae72980a54340cb4d238aaf7f8fb20cd48684af111326be2611947c6c0bb286.png

_images/1e56cbfd57b30a58325eeadf98e134790977a0f2088841c874ca282c1fbc7898.png

_images/986afe7a6cde8f4b3864449d693788bc170499eef30bec4ca8840b6c2f23a08a.png

_images/096cb5ba96b79def39a333c17081e4c894c4d30975ade8ea7edea64c1d0baec5.png

_images/acd6a65c53b949ef45f0ce834ef5674b58d228cca34daf254001f752b2a49eb1.png

_images/a4ab17ee31417be333ef3357efb83f9e80b6230d220bbf5394df66aff0e2e91b.png

_images/3a72f49dd0694761bf1fd7cd227ee031a03c5e4e12c4a0f46a18da1621d11c2c.png

_images/51a0b438e5d671f5b9dcc5cfd43098d338d1546edd192e4760f6e645bc5bfdd9.png

_images/324e260628a8ac7c639b89f738ab31b6c3e7c4ad33eaafb52b2a4f1ddad6c414.png

_images/2acf2381a543b37a14361363e317066aacae84091a5a95c85fbdfadd5aa417e0.png

_images/d0dc783db3fb78cd69adf662225d407b3ba03c2a8d4f40a52a3b7454d27933ae.png

_images/5313eb99c079f772d1c759859e81fc0d4e2829c7055d38ba05c702a749f6bb12.png

_images/e221aa6a20a6f7de46671b831d2c1f3e93189e52a96e7077da03fd2e7b578d2a.png

_images/739950a36c25ca83b58d89165e72db7b1969de9bc604c58bd53586bdb62dffb8.png

_images/637466d9c321738793884db898179f2c280072d9f3d3eb5f868d3251af60b976.png

_images/bb5c0f0ea336e8cfbc8e091ff2a3f7c807995df684ed273117acb08c5662cbf1.png

_images/4265f56b1c7a051cbbcb3e082b86c118ff3ce27e269a424dc6f7479733dc983f.png

_images/32e48f630f73ea2bf7b9291b0435f8accefe1858c9919c11c1cf41a148ded4f8.png

_images/6b7c77d0df973a8eb508c69a767cfcd1afb8c7cfac89a4fbb396c4b04e494e49.png

_images/a2f91cb632b6636d11bdb9ea155428fe04649e11fd998048df907b436c1d808c.png

_images/d05064f5b7d69ac34d21dc78ed81182fa78c5559763d67543a03ce2c3c61b080.png

_images/e66873ac90d9237e2709aa40d930a755bc3c1f06b0a81c1224d4cf22d7db2f54.png

_images/b209c6683a14f7bc97940e24ab6bcdd03b2391052147345369c80b97e2d13fae.png

_images/9df963cf6a51e15f63a268f6f6e648db8af05c009b4bf4626001901002e16d12.png

_images/a47e11456ebc063d9e164a815326497389ef8c7f94993cfd898ea12b01582880.png

_images/ad9c9252ce8171612ea2766e2f9f025fd7373077a4c558defa43ff9c42b6272f.png

_images/65e1e502ffe51f2399f81530c9f40d34a2c0293558b3bda763d52a6f87c2d06e.png

_images/35c1c34a6923e834416b1e46f3c145cc3b435ddeab064d031043d1c75d2f6210.png

_images/51a91e8e943a0160ac3d808208ed0326cc4ca88c4e953798442804f8e8100741.png

_images/8e4a047ed9cad10890d9274e31b7c7275d0ef9b8096495b0a15d775a45ab1d50.png

_images/3d52f77718c9dbbc2f94368232560a0f62916f11beafda98ddcdc869c9b6859c.png

_images/127825c9f571c1dd31282fc82304614ae49e19995cc5b70b9cccc57fb56267d5.png

_images/06a904d8e8d5bdce6a9c39718ee646ef1100233b15da306d170f1916f92a1482.png

_images/46808298545bd1c1fb6f0fde2dca4c789a929eb69646f936a98c7255b3842694.png

_images/cbb5c79cfc00da787d4ce4ea905fcca75ff2bd7aa8bc7f1af68046ca7f05d3c1.png

_images/0d3e3484c3a25d4cc21f33c5cc33dc614ec74e56cfbcdfd1c93de81dd557720c.png

_images/1649c835df4aa0d73b191ab80d3f9695408b7a67ee671057cddbfcbfa6a760fc.png

_images/c91cbae908034a49bf94ea327b4d30363b743309f679af0af08f97aabe8d7b13.png

_images/913a6a715e46b4f5244a9fd2a31f8cdffb59ca2e5a3de3d2a68fd3af04b570e4.png

_images/6682587f59d710ca40392ee419a47fc6b5022d2ee9a3792684883924dd76572a.png

_images/e836a2983c0435fcbbb5b6bb25750735c50149052ccbc4514098b286825f344d.png

_images/f9bdab5119c3e9178ae37e64e078d47f96ebceff764f633b18bc77c07ba592a3.png

Step 2: Stack the cross correlation#

This combines the time-chunked ASDF files to stack over each time chunk and at each station pair.

# open a new cc store in read-only mode since we will be doing parallel access for stacking
cc_store = ASDFCCStore(cc_data_path, mode="r")
print(cc_store.get_station_pairs())
stack_store = ASDFStackStore(stack_data_path)
config.stations = ["*"] # stacking doesn't support prefixes yet, so allow all stations
stack_cross_correlations(cc_store, stack_store, config)

2024-06-24 21:34:28,240 140264513891200 INFO stack.initializer(): Station pairs: 10

[(CI.ADO, CI.ARV), (CI.ADO, CI.AVM), (CI.ADO, CI.ALP), (CI.ALP, CI.ARV), (CI.AVM, CI.AVM), (CI.ALP, CI.ALP), (CI.ALP, CI.AVM), (CI.ADO, CI.ADO), (CI.ARV, CI.ARV), (CI.ARV, CI.AVM)]

2024-06-24 21:34:31,713 140023920368512 INFO stack.stack_store_pair(): Stacking CI.ADO_CI.ADO/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2024-06-24 21:34:31,755 140023920368512 INFO utils.log_raw(): TIMING: 0.0368 secs. for loading CCF data
2024-06-24 21:34:31,760 139865300167552 INFO stack.stack_store_pair(): Stacking CI.ADO_CI.ARV/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2024-06-24 21:34:31,763 140023920368512 INFO utils.log_raw(): TIMING: 0.0076 secs. for stack/rotate all station pairs (CI.ADO, CI.ADO)
2024-06-24 21:34:31,781 140023920368512 INFO utils.log_raw(): TIMING: 0.0178 secs. for writing stack pair (CI.ADO, CI.ADO)
2024-06-24 21:34:31,784 140023920368512 INFO stack.stack_store_pair(): Stacking CI.ALP_CI.ALP/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2024-06-24 21:34:31,798 140360789511040 INFO stack.stack_store_pair(): Stacking CI.ADO_CI.AVM/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2024-06-24 21:34:31,814 140188919909248 INFO stack.stack_store_pair(): Stacking CI.ADO_CI.ALP/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2024-06-24 21:34:31,818 139865300167552 INFO utils.log_raw(): TIMING: 0.0534 secs. for loading CCF data
2024-06-24 21:34:31,823 140023920368512 INFO utils.log_raw(): TIMING: 0.0358 secs. for loading CCF data
2024-06-24 21:34:31,828 139865300167552 INFO utils.log_raw(): TIMING: 0.0099 secs. for stack/rotate all station pairs (CI.ADO, CI.ARV)
2024-06-24 21:34:31,830 140023920368512 INFO utils.log_raw(): TIMING: 0.0079 secs. for stack/rotate all station pairs (CI.ALP, CI.ALP)
2024-06-24 21:34:31,848 140023920368512 INFO utils.log_raw(): TIMING: 0.0178 secs. for writing stack pair (CI.ALP, CI.ALP)
2024-06-24 21:34:31,851 140023920368512 INFO stack.stack_store_pair(): Stacking CI.ALP_CI.ARV/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2024-06-24 21:34:31,869 140360789511040 INFO utils.log_raw(): TIMING: 0.0650 secs. for loading CCF data
2024-06-24 21:34:31,878 139865300167552 INFO utils.log_raw(): TIMING: 0.0496 secs. for writing stack pair (CI.ADO, CI.ARV)
2024-06-24 21:34:31,880 140360789511040 INFO utils.log_raw(): TIMING: 0.0109 secs. for stack/rotate all station pairs (CI.ADO, CI.AVM)
2024-06-24 21:34:31,880 139865300167552 INFO stack.stack_store_pair(): Stacking CI.ALP_CI.AVM/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2024-06-24 21:34:31,891 140188919909248 INFO utils.log_raw(): TIMING: 0.0716 secs. for loading CCF data
2024-06-24 21:34:31,901 140188919909248 INFO utils.log_raw(): TIMING: 0.0097 secs. for stack/rotate all station pairs (CI.ADO, CI.ALP)
2024-06-24 21:34:31,901 140023920368512 INFO utils.log_raw(): TIMING: 0.0467 secs. for loading CCF data
2024-06-24 21:34:31,911 140023920368512 INFO utils.log_raw(): TIMING: 0.0098 secs. for stack/rotate all station pairs (CI.ALP, CI.ARV)

2024-06-24 21:34:31,931 139865300167552 INFO utils.log_raw(): TIMING: 0.0481 secs. for loading CCF data
2024-06-24 21:34:31,941 139865300167552 INFO utils.log_raw(): TIMING: 0.0103 secs. for stack/rotate all station pairs (CI.ALP, CI.AVM)
2024-06-24 21:34:31,944 140360789511040 INFO utils.log_raw(): TIMING: 0.0642 secs. for writing stack pair (CI.ADO, CI.AVM)
2024-06-24 21:34:31,947 140360789511040 INFO stack.stack_store_pair(): Stacking CI.ARV_CI.ARV/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2024-06-24 21:34:31,956 140188919909248 INFO utils.log_raw(): TIMING: 0.0552 secs. for writing stack pair (CI.ADO, CI.ALP)
2024-06-24 21:34:31,959 140188919909248 INFO stack.stack_store_pair(): Stacking CI.ARV_CI.AVM/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2024-06-24 21:34:31,966 140023920368512 INFO utils.log_raw(): TIMING: 0.0554 secs. for writing stack pair (CI.ALP, CI.ARV)
2024-06-24 21:34:31,969 140023920368512 INFO stack.stack_store_pair(): Stacking CI.AVM_CI.AVM/2019-02-01T00:00:00+0000 - 2019-02-02T00:00:00+0000
2024-06-24 21:34:31,993 140360789511040 INFO utils.log_raw(): TIMING: 0.0395 secs. for loading CCF data
2024-06-24 21:34:31,996 139865300167552 INFO utils.log_raw(): TIMING: 0.0544 secs. for writing stack pair (CI.ALP, CI.AVM)
2024-06-24 21:34:32,002 140360789511040 INFO utils.log_raw(): TIMING: 0.0091 secs. for stack/rotate all station pairs (CI.ARV, CI.ARV)
2024-06-24 21:34:32,003 140023920368512 INFO utils.log_raw(): TIMING: 0.0317 secs. for loading CCF data
2024-06-24 21:34:32,007 140023920368512 INFO utils.log_raw(): TIMING: 0.0046 secs. for stack/rotate all station pairs (CI.AVM, CI.AVM)
2024-06-24 21:34:32,016 140188919909248 INFO utils.log_raw(): TIMING: 0.0547 secs. for loading CCF data
2024-06-24 21:34:32,019 140023920368512 INFO utils.log_raw(): TIMING: 0.0111 secs. for writing stack pair (CI.AVM, CI.AVM)
2024-06-24 21:34:32,020 140360789511040 INFO utils.log_raw(): TIMING: 0.0179 secs. for writing stack pair (CI.ARV, CI.ARV)
2024-06-24 21:34:32,026 140188919909248 INFO utils.log_raw(): TIMING: 0.0096 secs. for stack/rotate all station pairs (CI.ARV, CI.AVM)
2024-06-24 21:34:32,056 140188919909248 INFO utils.log_raw(): TIMING: 0.0304 secs. for writing stack pair (CI.ARV, CI.AVM)

2024-06-24 21:34:32,606 140264513891200 INFO utils.log_raw(): TIMING: 4.3682 secs. for step 2 in total

pairs = stack_store.get_station_pairs()
print(f"Found {len(pairs)} station pairs")
sta_stacks = stack_store.read_bulk(timerange, pairs) # no timestamp used in ASDFStackStore

Found 10 station pairs

2024-06-24 21:34:32,817 140264513891200 INFO utils.log_raw(): TIMING: 0.2025 secs. for loading 10 stacks

Plot the stacks

print(os.listdir(cc_data_path))
print(os.listdir(stack_data_path))

['2019_02_01_00_00_00T2019_02_01_12_00_00.h5', '2019_02_01_12_00_00T2019_02_02_00_00_00.h5']
['CI.ADO', 'CI.AVM', 'CI.ALP', 'CI.ARV']

plotting_modules.plot_all_moveout(sta_stacks, 'Allstack_linear', 0.1, 0.2, 'ZZ', 1)

2024-06-24 21:34:32,829 140264513891200 INFO plotting_modules.plot_all_moveout(): Plottting: Allstack_linear, 10 station pairs

200 8001

_images/1c00fab119f5f4ccf0fe44d722e9d664e9e463c116be3a309a62ffe98123e5d4.png