Wind speed from a photograph
of the ocean.
We are training a model that estimates wind speed from images of the ocean surface. The training data is roughly 207,000 BuoyCAM panoramas from the NDBC network, each paired with a colocated anemometer reading. The current model holds out thirteen buoys it has never seen and predicts wind on those buoys with a mean absolute error of 1.72 knots.
Anemometers are mechanical, expensive, and need to be physically mounted at every location where wind data matters. A camera facing water cannot replace one for high-stakes meteorology, but for the much larger set of cases where someone has a camera and wants a wind estimate — security cameras at marinas, GoPros on offshore platforms, fixed webcams at surf breaks — there is currently no answer.
The hypothesis is straightforward. The texture of the water surface is determined by the wind that produced it. Capillary ripples, gravity waves, whitecap density, foam streaks, spray — these are all visible in a photograph and they all scale with wind speed. A vision model trained on enough labeled examples should be able to read them.
v15 reaches 1.72 knots uncalibrated mean absolute error on a buoy-ID holdout split. This is the first WindLabs result below 2.0 knots. Every prior architecture from v6 through v14 — frozen DINOv2, fine-tuned DINOv2, EfficientNet, a custom CNN trained from scratch, ViT-Small, CORAL ordinal regression — converged to roughly 2.3 to 2.7 knots. v15 broke that ceiling by combining a larger high-wind-tail dataset with multi-slice cross-attention fusion across the six panorama slices.
We also ran v15 on the MU-SSiD dataset, 800 DSLR images of the Malaysian coast labelled by Beaufort class. The model produced a monotonic ranking — predicted wind increased with each Beaufort class — and roughly 80 percent of predictions fell within one Beaufort class of the label. The model has never seen a Malaysian DSLR image, a coastal viewing angle, or a non-NDBC camera. Generalization to foreign hardware appears to work in the ordinal sense.
Per-site calibration is the central commercial mechanism. Camera-specific bias dominates raw error on a webcam the model has never seen. A short linear regression — fit on twenty to fifty paired image-and-anemometer observations from one specific camera — removes that bias. The calibration coefficients are three numbers per site. They are stable in time and add no inference cost.
The backbone is DINOv2 ViT-L/14, with the last four blocks unfrozen. Each panorama is split into six slices in the dataloader and each slice goes through the backbone independently. Per-patch spatial attention weights let the model emphasize wave patches over sky patches. A cross-attention layer then fuses the six slice representations into a single observation-level prediction. We use Huber loss and train on Harvard's Cannon HPC cluster on A100 GPUs.
The training set combines two NDBC archives. The Final dataset contains roughly 1,860 observations above 25 knots from storm-period BuoyCAM frames that were missing from the more recent Matched dataset. Adding the high-wind tail back to the training set was as important as the architecture change. An ablation that disentangles the two effects is planned but has not yet been run.
This summer we are deploying a GoPro and an Inspeed Vortex cup anemometer on a buoy in Quisset Harbor, Falmouth, Massachusetts. The anemometer is colocated with the camera. A Raspberry Pi logs anemometer pulses with timestamps that match the GoPro frames. This is the first WindLabs validation against ground truth from a colocated instrument rather than a numerical weather model. The Quisset deployment runs through August.
The intended writeup is a methods paper for the Journal of Atmospheric and Oceanic Technology. The first round of pilot conversations with offshore wind operators and maritime data companies will follow Quisset. There is no working API endpoint yet and there are no paying customers.
If you have an ocean-facing camera near a known weather station and want to know what the model predicts, send a frame to [email protected]. Same address for general questions.