pyadps Documentation
Python package for ADCP data processing
pyadps provides tools for reading, quality controlling, and processing
Acoustic Doppler Current Profiler (ADCP) data — designed for Teledyne RDI
ADCPs recording in the PD0 binary format. PD0 files from other RDI models,
such as Ocean Surveyor and DVS, can also be read; take extra care when
processing that data, since the pipeline’s defaults were tuned against
Workhorse deployments.
Background
pyadps was built to process PD0 files from moored ADCP deployments that
lack navigation (GPS) data. It was developed primarily for the COSINE and
ECO-IOD mooring programs in the north Indian Ocean, which together span
over 600 ADCP deployments — a scale that made it worth standardizing and
documenting the processing steps rather than repeating ad hoc scripts for
each dataset. The processing pipeline is designed to encourage a close
look at the quality-control results before the velocity output is treated
as final. The package currently processes only data recorded in Earth
coordinates; Beam-coordinate support and the associated coordinate
transformation are planned for a future release.
Features
Read RDI binary files (PD0 format) with xarray integration
Robust reading of corrupted or truncated binary files, with per-ensemble checksum verification and partial-data recovery
Six-step quality control pipeline: time axis correction, sensor health, signal quality, profile operations, and velocity checks
Interactive web interface (Streamlit) — no Python knowledge required
Batch processing and multi-file combining
CF Convention compliant output
Reproducible processing via
config.iniexportExtensively tested: 4,200+ automated tests (
pytest), 98% coverage
Quick Example
import pyadps
# Read an RDI file
ds = pyadps.read("deployment.000")
# Check system configuration
config = ds.fixed_leader.system_configuration()
print(f"Frequency: {config['Frequency']}")
# Access velocity data
velocity = ds['velocity']
print(f"Shape: {velocity.shape}")
# Export to NetCDF
ds.to_netcdf('output.nc')
Processing Pipeline
The diagram below adds the decision logic behind the steps that involve a judgment call — time diagnostics, sensor health, signal quality, and profile operations — on top of the overall pipeline order. Each of these appears as its own box: work down the diamonds inside it, answering the question at each one, before moving on to the next box in the main column. Velocity processing has no decision points of its own — its four checks (magnetic declination, velocity thresholds, despike, flatline) are each independently optional and can be applied in any combination.
![digraph pipeline {
rankdir=TB;
bgcolor="transparent";
splines=ortho;
nodesep=0.5;
ranksep=0.8;
fontname="Helvetica,Arial,sans-serif";
node [shape=box, style="rounded,filled", fontname="Helvetica,Arial,sans-serif",
fontsize=14, margin="0.28,0.16", penwidth=1.4];
// Diamonds get their own tighter, taller margin so they read as
// narrow decision points rather than wide ovals-with-corners.
// (applied per-node below via margin="0.12,0.28")
edge [color="#6b7280", arrowsize=0.75, penwidth=1.1, fontname="Helvetica,Arial,sans-serif",
fontsize=11.5];
// ---------------------------------------------------------------
// Palette
// Terminal (start/end): deep navy fill, white text
// Process / action: soft sage fill, forest-green border
// Decision (diamond): warm ivory fill, amber border
// Cluster panels: two muted blue-gray tones, alternating
// ---------------------------------------------------------------
read [label="Read PD0 File", fillcolor="#1f3a5f", color="#14293f",
fontcolor="white", fontsize=14.5, style="rounded,filled", penwidth=1.6];
check_view [label="Check and View Data", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
// -- Time Diagnostics --
subgraph cluster_time {
label=<<B><FONT FACE="Helvetica" POINT-SIZE="16" COLOR="#1f3a5f">Time Diagnostics</FONT></B>>;
style="rounded,filled";
color="#5b7ea3";
fillcolor="#f3f7fb";
margin=22;
penwidth=1.2;
time_check [label="Is the time axis\nirregular?", shape=diamond,
fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
time_minor [label="Is it a minor issue\n(within tolerance)?", shape=diamond,
fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
time_fix [label="Snap / Fill\nTime Gaps", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
{ rank=same; time_check; time_minor; time_fix; }
time_check -> time_minor [xlabel="Yes", fontcolor="#2e7d5b"];
time_minor -> time_fix [xlabel="Yes", fontcolor="#2e7d5b"];
}
// -- Sensor Health --
subgraph cluster_sensor {
label=<<B><FONT FACE="Helvetica" POINT-SIZE="16" COLOR="#1f3a5f">Sensor Health</FONT></B>>;
style="rounded,filled";
color="#5b7ea3";
fillcolor="#e3ecf5";
margin=22;
penwidth=1.2;
sensor_check [label="Are sensors\nworking (per\ndeployment config)?", shape=diamond,
fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
sensor_alt [label="Is an alternate/\nexternal sensor\navailable?", shape=diamond,
fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
sensor_replace [label="Replace with alternate\nsensor data", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
sensor_fixed [label="Use a fixed value\n(deployment config)", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
{ rank=same; sensor_check; sensor_alt; sensor_replace; sensor_fixed; }
sensor_check -> sensor_alt [xlabel="No", fontcolor="#b3261e"];
sensor_alt -> sensor_replace [xlabel="Yes", fontcolor="#2e7d5b", weight=5];
sensor_alt -> sensor_fixed [xlabel="No", fontcolor="#b3261e"];
sensor_replace -> sensor_fixed [style=invis];
}
// -- Signal Quality --
subgraph cluster_signal {
label=<<B><FONT FACE="Helvetica" POINT-SIZE="16" COLOR="#1f3a5f">Signal Quality</FONT></B>>;
style="rounded,filled";
color="#5b7ea3";
fillcolor="#f3f7fb";
margin=22;
penwidth=1.2;
sig_q1 [label="Apply Echo Intensity\nThreshold?", shape=diamond,
fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
sig_q1b [label="Echo intensity data\ncollected in air?", shape=diamond,
fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
sig_noise [label="Noise Floor\nIdentification", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
sig_userdef [label="User-Defined\nThreshold", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
{ rank=same; sig_q1; sig_q1b; sig_noise; sig_userdef; }
sig_q2 [label="Apply Percent Good\nThreshold?", shape=diamond,
fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
sig_pgadvisor [label="PG Threshold\nAdvisor", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
{ rank=same; sig_q2; sig_pgadvisor; }
sig_q3 [label="Change default\nCorrelation / Error\nVelocity / False\nTarget thresholds?", shape=diamond,
fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
sig_qcconfig [label="QC Test\nConfiguration", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
{ rank=same; sig_q3; sig_qcconfig; }
sig_q4 [label="Has the orientation\nsensor gone bad?", shape=diamond,
fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
sig_orient [label="Fix Orientation", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
{ rank=same; sig_q4; sig_orient; }
sig_q1 -> sig_q1b [xlabel="Yes", fontcolor="#2e7d5b"];
sig_q1b -> sig_noise [xlabel="Yes", fontcolor="#2e7d5b", weight=5];
sig_q1b -> sig_userdef [xlabel="No", fontcolor="#b3261e"];
sig_noise -> sig_userdef [style=invis];
sig_q1 -> sig_q2 [xlabel="No", fontcolor="#b3261e", weight=10];
sig_noise -> sig_q2;
sig_userdef -> sig_q2;
sig_q2 -> sig_pgadvisor [xlabel="Yes", fontcolor="#2e7d5b"];
sig_q2 -> sig_q3 [xlabel="No", fontcolor="#b3261e", weight=10];
sig_pgadvisor -> sig_q3;
sig_q3 -> sig_qcconfig [xlabel="Yes", fontcolor="#2e7d5b"];
sig_q3 -> sig_q4 [xlabel="No", fontcolor="#b3261e", weight=10];
sig_qcconfig -> sig_q4;
sig_q4 -> sig_orient [xlabel="Yes", fontcolor="#2e7d5b"];
}
// -- Profile Operations --
subgraph cluster_profile {
label=<<B><FONT FACE="Helvetica" POINT-SIZE="16" COLOR="#1f3a5f">Profile Operations</FONT></B>>;
style="rounded,filled";
color="#5b7ea3";
fillcolor="#e3ecf5";
margin=22;
penwidth=1.2;
profile_q1 [label="Data collected\nin air —\npre-deployment\nor post-recovery?", shape=diamond,
fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
profile_trim [label="Trim Ends", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
{ rank=same; profile_q1; profile_trim; }
profile_q2 [label="Cells contaminated by\nside-lobe echo?", shape=diamond,
fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
profile_cut [label="Side Lobe Cut", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
{ rank=same; profile_q2; profile_cut; }
profile_q3 [label="Any other cells\nor ensembles\ncontaminated?", shape=diamond,
fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
profile_manual [label="Manual Cut", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
{ rank=same; profile_q3; profile_manual; }
profile_regrid [label="Regrid", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
profile_q1 -> profile_trim [xlabel="Yes", fontcolor="#2e7d5b"];
profile_q1 -> profile_q2 [xlabel="No", fontcolor="#b3261e", weight=10];
profile_trim -> profile_q2;
profile_q2 -> profile_cut [xlabel="Yes", fontcolor="#2e7d5b"];
profile_q2 -> profile_q3 [xlabel="No", fontcolor="#b3261e", weight=10];
profile_cut -> profile_q3;
profile_q3 -> profile_manual [xlabel="Yes", fontcolor="#2e7d5b"];
profile_q3 -> profile_regrid [xlabel="No", fontcolor="#b3261e", weight=10];
profile_manual -> profile_regrid;
}
// -- Velocity Processing --
subgraph cluster_velocity {
label=<<B><FONT FACE="Helvetica" POINT-SIZE="16" COLOR="#1f3a5f">Velocity Processing</FONT></B>>;
style="rounded,filled";
color="#5b7ea3";
fillcolor="#f3f7fb";
margin=22;
penwidth=1.2;
vel_magdec [label="Magnetic Declination\nCorrection (Optional)", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
vel_thresh [label="Apply Velocity\nThresholds (Optional)", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
vel_despike [label="Despike Data\n(Optional)", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
vel_flatline [label="Flatline Detection\n(Optional)", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
vel_magdec -> vel_thresh -> vel_despike -> vel_flatline;
}
finalize [label="Finalize & Export\n(NetCDF / CSV)", fillcolor="#1f3a5f", color="#14293f",
fontcolor="white", fontsize=14.5, style="rounded,filled", penwidth=1.6];
// Main trunk — heavier weight keeps this column straight and reads as
// the primary pipeline at a glance.
read -> check_view [weight=10, minlen=2];
check_view -> time_check [weight=10, minlen=2];
time_check -> sensor_check [xlabel="No", fontcolor="#b3261e", weight=10, minlen=4];
sensor_check -> sig_q1 [xlabel="Yes", fontcolor="#2e7d5b", weight=10, minlen=4];
sig_q4 -> profile_q1 [xlabel="No", fontcolor="#b3261e", weight=10, minlen=4];
profile_regrid -> vel_magdec [weight=10, minlen=4];
vel_flatline -> finalize [weight=10, minlen=4];
// Time Diagnostics merges back into the trunk.
time_minor -> sensor_check [xlabel="No — continue,\nuse external tools\nafter processing", fontcolor="#b3261e"];
time_fix -> sensor_check;
// Sensor Health merges back into the trunk.
sensor_replace -> sig_q1;
sensor_fixed -> sig_q1;
// Signal Quality merges back into the trunk.
sig_orient -> profile_q1;
}](_images/graphviz-7cd7ed7cdcdd3c4d3d0966c4055830a677f7d714.png)
Package Structure
pyadps is organized into three main components:
Module Reference
Additional Resources
Getting Started
Install
pyadps: See Installation for instructionsRead the Quick Start: Quick Start provides a 5-minute introduction
Try the Web App: Web Application covers the interactive interface
Go deeper: I/O Module and Processing Module cover the module details
Module Quick Reference
I/O Functions
Function |
Description |
|---|---|
|
Load complete ADCP dataset |
|
Read file header metadata |
|
Read configuration data |
|
Read sensor data |
|
Read velocity arrays |
Processing Classes
Class |
Description |
|---|---|
|
Check sensor diagnostics |
|
Assess signal quality |
|
Validate velocities |
|
Profile operations |
|
Processed data container |
Accessor Methods
Accessor |
Namespace |
Key Methods |
|---|---|---|
Header |
|
|
Fixed Leader |
|
|
Variable Leader |
|
|
Support
Documentation: You’re reading it!
Issue Tracker: GitHub Issues
Source Code: GitHub Repository
License
pyadps is released under the MIT License.
Note
This version of the package was developed with extensive use of Claude (Anthropic) as a coding assistant.