pyadps Documentation

Python package for ADCP data processing

pyadps provides tools for reading, quality controlling, and processing Acoustic Doppler Current Profiler (ADCP) data — designed for Teledyne RDI ADCPs recording in the PD0 binary format. PD0 files from other RDI models, such as Ocean Surveyor and DVS, can also be read; take extra care when processing that data, since the pipeline’s defaults were tuned against Workhorse deployments.

Background

pyadps was built to process PD0 files from moored ADCP deployments that lack navigation (GPS) data. It was developed primarily for the COSINE and ECO-IOD mooring programs in the north Indian Ocean, which together span over 600 ADCP deployments — a scale that made it worth standardizing and documenting the processing steps rather than repeating ad hoc scripts for each dataset. The processing pipeline is designed to encourage a close look at the quality-control results before the velocity output is treated as final. The package currently processes only data recorded in Earth coordinates; Beam-coordinate support and the associated coordinate transformation are planned for a future release.

Features

  • Read RDI binary files (PD0 format) with xarray integration

  • Robust reading of corrupted or truncated binary files, with per-ensemble checksum verification and partial-data recovery

  • Six-step quality control pipeline: time axis correction, sensor health, signal quality, profile operations, and velocity checks

  • Interactive web interface (Streamlit) — no Python knowledge required

  • Batch processing and multi-file combining

  • CF Convention compliant output

  • Reproducible processing via config.ini export

  • Extensively tested: 4,200+ automated tests (pytest), 98% coverage

Quick Example

import pyadps

# Read an RDI file
ds = pyadps.read("deployment.000")

# Check system configuration
config = ds.fixed_leader.system_configuration()
print(f"Frequency: {config['Frequency']}")

# Access velocity data
velocity = ds['velocity']
print(f"Shape: {velocity.shape}")

# Export to NetCDF
ds.to_netcdf('output.nc')

Processing Pipeline

The diagram below adds the decision logic behind the steps that involve a judgment call — time diagnostics, sensor health, signal quality, and profile operations — on top of the overall pipeline order. Each of these appears as its own box: work down the diamonds inside it, answering the question at each one, before moving on to the next box in the main column. Velocity processing has no decision points of its own — its four checks (magnetic declination, velocity thresholds, despike, flatline) are each independently optional and can be applied in any combination.

digraph pipeline {
    rankdir=TB;
    bgcolor="transparent";
    splines=ortho;
    nodesep=0.5;
    ranksep=0.8;
    fontname="Helvetica,Arial,sans-serif";

    node [shape=box, style="rounded,filled", fontname="Helvetica,Arial,sans-serif",
          fontsize=14, margin="0.28,0.16", penwidth=1.4];
    // Diamonds get their own tighter, taller margin so they read as
    // narrow decision points rather than wide ovals-with-corners.
    // (applied per-node below via margin="0.12,0.28")
    edge [color="#6b7280", arrowsize=0.75, penwidth=1.1, fontname="Helvetica,Arial,sans-serif",
          fontsize=11.5];

    // ---------------------------------------------------------------
    // Palette
    //   Terminal (start/end): deep navy fill, white text
    //   Process / action:      soft sage fill, forest-green border
    //   Decision (diamond):    warm ivory fill, amber border
    //   Cluster panels:        two muted blue-gray tones, alternating
    // ---------------------------------------------------------------

    read     [label="Read PD0 File", fillcolor="#1f3a5f", color="#14293f",
              fontcolor="white", fontsize=14.5, style="rounded,filled", penwidth=1.6];

    check_view [label="Check and View Data", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];

    // -- Time Diagnostics --
    subgraph cluster_time {
        label=<<B><FONT FACE="Helvetica" POINT-SIZE="16" COLOR="#1f3a5f">Time Diagnostics</FONT></B>>;
        style="rounded,filled";
        color="#5b7ea3";
        fillcolor="#f3f7fb";
        margin=22;
        penwidth=1.2;

        time_check [label="Is the time axis\nirregular?", shape=diamond,
                    fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
        time_minor [label="Is it a minor issue\n(within tolerance)?", shape=diamond,
                    fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
        time_fix   [label="Snap / Fill\nTime Gaps", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];

        { rank=same; time_check; time_minor; time_fix; }

        time_check -> time_minor [xlabel="Yes", fontcolor="#2e7d5b"];
        time_minor -> time_fix [xlabel="Yes", fontcolor="#2e7d5b"];
    }

    // -- Sensor Health --
    subgraph cluster_sensor {
        label=<<B><FONT FACE="Helvetica" POINT-SIZE="16" COLOR="#1f3a5f">Sensor Health</FONT></B>>;
        style="rounded,filled";
        color="#5b7ea3";
        fillcolor="#e3ecf5";
        margin=22;
        penwidth=1.2;

        sensor_check [label="Are sensors\nworking (per\ndeployment config)?", shape=diamond,
                      fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
        sensor_alt   [label="Is an alternate/\nexternal sensor\navailable?", shape=diamond,
                      fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
        sensor_replace [label="Replace with alternate\nsensor data", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
        sensor_fixed   [label="Use a fixed value\n(deployment config)", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];

        { rank=same; sensor_check; sensor_alt; sensor_replace; sensor_fixed; }

        sensor_check -> sensor_alt [xlabel="No", fontcolor="#b3261e"];
        sensor_alt -> sensor_replace [xlabel="Yes", fontcolor="#2e7d5b", weight=5];
        sensor_alt -> sensor_fixed [xlabel="No", fontcolor="#b3261e"];
        sensor_replace -> sensor_fixed [style=invis];
    }

    // -- Signal Quality --
    subgraph cluster_signal {
        label=<<B><FONT FACE="Helvetica" POINT-SIZE="16" COLOR="#1f3a5f">Signal Quality</FONT></B>>;
        style="rounded,filled";
        color="#5b7ea3";
        fillcolor="#f3f7fb";
        margin=22;
        penwidth=1.2;

        sig_q1  [label="Apply Echo Intensity\nThreshold?", shape=diamond,
                 fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
        sig_q1b [label="Echo intensity data\ncollected in air?", shape=diamond,
                 fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
        sig_noise   [label="Noise Floor\nIdentification", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
        sig_userdef [label="User-Defined\nThreshold", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
        { rank=same; sig_q1; sig_q1b; sig_noise; sig_userdef; }

        sig_q2 [label="Apply Percent Good\nThreshold?", shape=diamond,
                fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
        sig_pgadvisor [label="PG Threshold\nAdvisor", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
        { rank=same; sig_q2; sig_pgadvisor; }

        sig_q3 [label="Change default\nCorrelation / Error\nVelocity / False\nTarget thresholds?", shape=diamond,
                fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
        sig_qcconfig [label="QC Test\nConfiguration", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
        { rank=same; sig_q3; sig_qcconfig; }

        sig_q4 [label="Has the orientation\nsensor gone bad?", shape=diamond,
                fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
        sig_orient [label="Fix Orientation", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
        { rank=same; sig_q4; sig_orient; }

        sig_q1 -> sig_q1b [xlabel="Yes", fontcolor="#2e7d5b"];
        sig_q1b -> sig_noise [xlabel="Yes", fontcolor="#2e7d5b", weight=5];
        sig_q1b -> sig_userdef [xlabel="No", fontcolor="#b3261e"];
        sig_noise -> sig_userdef [style=invis];
        sig_q1 -> sig_q2 [xlabel="No", fontcolor="#b3261e", weight=10];
        sig_noise -> sig_q2;
        sig_userdef -> sig_q2;

        sig_q2 -> sig_pgadvisor [xlabel="Yes", fontcolor="#2e7d5b"];
        sig_q2 -> sig_q3 [xlabel="No", fontcolor="#b3261e", weight=10];
        sig_pgadvisor -> sig_q3;

        sig_q3 -> sig_qcconfig [xlabel="Yes", fontcolor="#2e7d5b"];
        sig_q3 -> sig_q4 [xlabel="No", fontcolor="#b3261e", weight=10];
        sig_qcconfig -> sig_q4;

        sig_q4 -> sig_orient [xlabel="Yes", fontcolor="#2e7d5b"];
    }

    // -- Profile Operations --
    subgraph cluster_profile {
        label=<<B><FONT FACE="Helvetica" POINT-SIZE="16" COLOR="#1f3a5f">Profile Operations</FONT></B>>;
        style="rounded,filled";
        color="#5b7ea3";
        fillcolor="#e3ecf5";
        margin=22;
        penwidth=1.2;

        profile_q1 [label="Data collected\nin air —\npre-deployment\nor post-recovery?", shape=diamond,
                    fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
        profile_trim [label="Trim Ends", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
        { rank=same; profile_q1; profile_trim; }

        profile_q2 [label="Cells contaminated by\nside-lobe echo?", shape=diamond,
                    fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
        profile_cut [label="Side Lobe Cut", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
        { rank=same; profile_q2; profile_cut; }

        profile_q3 [label="Any other cells\nor ensembles\ncontaminated?", shape=diamond,
                    fillcolor="#fdf6e3", color="#c98a1f", fontcolor="#4a3a10", fontsize=12, margin="0.12,0.3"];
        profile_manual [label="Manual Cut", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
        { rank=same; profile_q3; profile_manual; }

        profile_regrid [label="Regrid", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];

        profile_q1 -> profile_trim [xlabel="Yes", fontcolor="#2e7d5b"];
        profile_q1 -> profile_q2 [xlabel="No", fontcolor="#b3261e", weight=10];
        profile_trim -> profile_q2;

        profile_q2 -> profile_cut [xlabel="Yes", fontcolor="#2e7d5b"];
        profile_q2 -> profile_q3 [xlabel="No", fontcolor="#b3261e", weight=10];
        profile_cut -> profile_q3;

        profile_q3 -> profile_manual [xlabel="Yes", fontcolor="#2e7d5b"];
        profile_q3 -> profile_regrid [xlabel="No", fontcolor="#b3261e", weight=10];
        profile_manual -> profile_regrid;
    }

    // -- Velocity Processing --
    subgraph cluster_velocity {
        label=<<B><FONT FACE="Helvetica" POINT-SIZE="16" COLOR="#1f3a5f">Velocity Processing</FONT></B>>;
        style="rounded,filled";
        color="#5b7ea3";
        fillcolor="#f3f7fb";
        margin=22;
        penwidth=1.2;

        vel_magdec   [label="Magnetic Declination\nCorrection (Optional)", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
        vel_thresh   [label="Apply Velocity\nThresholds (Optional)", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
        vel_despike  [label="Despike Data\n(Optional)", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];
        vel_flatline [label="Flatline Detection\n(Optional)", fillcolor="#e7f2ec", color="#2e7d5b", fontcolor="#1c3d2e"];

        vel_magdec -> vel_thresh -> vel_despike -> vel_flatline;
    }

    finalize [label="Finalize & Export\n(NetCDF / CSV)", fillcolor="#1f3a5f", color="#14293f",
              fontcolor="white", fontsize=14.5, style="rounded,filled", penwidth=1.6];

    // Main trunk — heavier weight keeps this column straight and reads as
    // the primary pipeline at a glance.
    read -> check_view [weight=10, minlen=2];
    check_view -> time_check [weight=10, minlen=2];
    time_check -> sensor_check [xlabel="No", fontcolor="#b3261e", weight=10, minlen=4];
    sensor_check -> sig_q1 [xlabel="Yes", fontcolor="#2e7d5b", weight=10, minlen=4];
    sig_q4 -> profile_q1 [xlabel="No", fontcolor="#b3261e", weight=10, minlen=4];
    profile_regrid -> vel_magdec [weight=10, minlen=4];
    vel_flatline -> finalize [weight=10, minlen=4];

    // Time Diagnostics merges back into the trunk.
    time_minor -> sensor_check [xlabel="No — continue,\nuse external tools\nafter processing", fontcolor="#b3261e"];
    time_fix -> sensor_check;

    // Sensor Health merges back into the trunk.
    sensor_replace -> sig_q1;
    sensor_fixed -> sig_q1;

    // Signal Quality merges back into the trunk.
    sig_orient -> profile_q1;
}

Package Structure

pyadps is organized into three main components:

Module Reference

Additional Resources

Getting Started

  1. Install pyadps: See Installation for instructions

  2. Read the Quick Start: Quick Start provides a 5-minute introduction

  3. Try the Web App: Web Application covers the interactive interface

  4. Go deeper: I/O Module and Processing Module cover the module details

Module Quick Reference

I/O Functions

Function

Description

pyadps.read()

Load complete ADCP dataset

pyadps.read_header()

Read file header metadata

pyadps.read_fixed_leader()

Read configuration data

pyadps.read_variable_leader()

Read sensor data

pyadps.read_velocity()

Read velocity arrays

Processing Classes

Class

Description

SensorHealthRunner

Check sensor diagnostics

SignalQualityRunner

Assess signal quality

VelocityCheckRunner

Validate velocities

ProfileOperationRunner

Profile operations

ProcessedDataset

Processed data container

Accessor Methods

Accessor

Namespace

Key Methods

Header

ds.header.*

check_file(), summary()

Fixed Leader

ds.fixed_leader.*

system_configuration(), validate()

Variable Leader

ds.variable_leader.*

bit_result_summary(), validate_timestamps()

Support

License

pyadps is released under the MIT License.

Note

This version of the package was developed with extensive use of Claude (Anthropic) as a coding assistant.