Preprocessing for AIND ephys pipeline

aind-ephys-preprocessing

Description

This capsule is designed to preprocess data for the AIND pipeline.

This capsule preprocessed the ephys stream with the following steps:

• Phase shift correction (see phase_shift)
• High-pass filtering
• Bad channel removal (using the method developed by IBL)
• Denoising: using one of the following methods
  • Common Median Referenc (CMR)
  • High-pass spatial filtering (see highpass_spatial_filter())
• (optional) Drift correction (estimation and interpolation)

Inputs

The data/ folder must include the session data and at least one JSON file generated by the aind-ephys-job-dispatch capsule is required (1 is recommended).

Parameters

Parameters are read from a params.json file that must be in the root folder of the capsule (next to code/, i.e. ../params.json from code/), with a logging and a preprocessing section (keys such as n_jobs (default -1), t_start, t_stop).

A full list of parameters can be found in the aind-ephys-pipeline repository, and are defined in this schema.

Output

The output of this capsule is the following:

• results/preprocessed_{recording_name} folder, containing the preprocessed data saved to binary
• results/motion_{recording_name} file, containing the motion estimation data (if motion is estimate or apply)
• results/preprocessed_{recording_name}.json file, containing the JSON file to reload the processing chain
• results/preprocessedviz_{recording_name}.json file, all the information required to make a visualization of the preprocessing downstream
• results/data_process_preprocessing_{recording_name}.json file, a JSON file containing a DataProcess object from the aind-data-schema package.