Building a Physics-Based Synthetic Drilling Telemetry Simulator

Synthetic drilling telemetry is easy to make look plausible.

It is much harder to make every channel describe the same physical operation.

A credible electronic drilling recorder, or EDR, dataset needs more than realistic-looking rate of penetration, hookload, standpipe pressure, torque, and pump traces. Those signals must agree on what the rig is doing, what is supporting the drillstring, whether the bit is on bottom, whether circulation exists, how much pipe is in the hole, when a connection is required, how the bottomhole assembly reached its current depth, and whether the reported load and pressure states can physically coexist.

This project began as a drilling-data generator conditioned on short public field intervals. The early system could reproduce on-bottom drilling behavior reasonably well, particularly where public Utah FORGE data provided a real interval against which effective parameters could be tuned.

The problem was everything outside the measured interval.

The simulator could produce convincing drilling telemetry after initialization, but it did not yet represent a complete drilling operation. It could begin at depth, drill a short section, and generate plausible channels without first answering a more basic question:

How did the rig physically get the bit there?

Solving that question changed the project from a drilling-trace generator into a drilling rig simulator.

The final architecture follows one central rule:

Simulate the rig first. Telemetry is a byproduct.

The completed system is now publicly available as the Drilling Telemetry Simulator. The first tagged public prerelease is v0.1.0a1.

Release At A Glance

Item	Released state
Public version	`v0.1.0a1`
Repository	SyntheticFunk/drilling-telemetry-simulator
Python distribution and CLI	`rig-telemetry-sim`
Python import package	`rig_telemetry_sim`
Production physics cadence	1 second
Default public EDR cadence	5 seconds
Bundled examples	18 total, 7 complete operations and 11 explicit developer fixtures
Directional demonstration	Complete PDM build-and-hold operation with rotate and slide drilling
Test gates	296 default-tier tests and 21 release-tier tests on Python 3.10 and 3.12
Public run artifacts	Seven core files, plus optional static WITSML 1.4.1.1 interchange
Code license	Apache 2.0
Bundled FORGE-derived compact data	CC BY 4.0, with explicit attribution and adaptation notices

The release is intentionally labelled 0.1.0a1. It is a public engineering prerelease, not a claim that the simulator is a finished commercial drilling product.

The Original Model

The initial simulator focused on the part of drilling for which public data was available.

Given a short interval of drilling telemetry, the development process could tune effective parameters governing:

pump and hydraulic behavior
rotary response
weight on bit
hookload
torque
motor behavior
bit-rock interaction
rate of penetration

That approach worked well inside the measured interval. The simulator could reproduce the broad character of a real drilling trace because its parameters were anchored to observed field behavior.

But the surrounding operation was still abstracted.

The first production-oriented version effectively generated the drilling state first and then assigned a rig activity to it. Depth, telemetry, and activity labels were coordinated, but the rig itself was not yet the first-order source of truth.

That created subtle but important problems.

A simulator could say that the rig was tripping while changing bit depth too quickly. It could move from drilling directly into a survey without working the pipe. Pumps could appear to stop instantaneously. A connection could exist as a named state without a complete sequence of slips, support transfer, top-drive handling, and block movement.

Most importantly, the system could initialize directly at a downhole reference interval.

That was useful for isolated calibration and regression testing, but it was not a complete synthetic drilling operation.

Reversing The Architecture

The architecture was restructured around a causal rig procedure engine.

Instead of commanding depth directly, the simulator now models operations such as:

picking up the bottomhole assembly
making up and handling stands
setting and releasing slips
transferring the string between hook and slips
connecting and disconnecting the top drive
moving and braking the travelling block
tripping into and out of the hole
starting and stopping pumps
starting and stopping surface rotation
tagging bottom
establishing weight on bit
drilling until available block travel requires a connection
working pipe before a survey
acquiring and transmitting directional surveys
laying down the BHA at surface

The normal operating lifecycle is:

surface
  -> pick up BHA
  -> trip in
  -> arrive off bottom
  -> establish circulation and rotation
  -> tag bottom
  -> establish WOB
  -> drill
  -> connect, work pipe, and survey as required
  -> finish drilling
  -> trip out
  -> lay down BHA
  -> validated terminal surface state

There is no downhole teleportation in a normal generated run.

An existing hole may already be hundreds or thousands of metres deep, but the bit still begins at zero measured depth. The rig must deploy the BHA and drillpipe from surface and physically reach the downhole interval.

Prepositioned states still exist, but only as explicit developer fixtures for focused calibration closure, unit tests, subsystem diagnostics, and benchmarking. They are clearly separated from normal generated operations.

One Unified Physical State

The rig procedure engine does not generate independent telemetry. It issues commands to one unified physical plant.

At each one-second physics step, the simulator updates the current state of:

pumps
flow
standpipe pressure
differential pressure reference state
travelling-block position, velocity, and acceleration
top-drive speed and acceleration
drillstring support topology
tubular inventory
bit depth
hole depth
axial load
hookload
weight on bit
rotary torque
torque-and-drag
positive-displacement motor behavior
bit-rock interaction
actual trajectory
survey and telemetry transport state

Procedure labels describe what the rig is trying to do.

They do not directly create physical outputs.

For example, stop_feed does not automatically remove motor load. It stops the penetration command. If the string remains compressed and WOB remains present, the motor remains loaded until the bit is physically picked up.

Likewise, a slide request does not immediately apply full oriented steering response. The top drive must first decelerate below the configured stopped-rotation tolerance before slide penetration is allowed.

These distinctions matter because many of the hardest bugs were not problems with individual equations. They were ownership problems. A procedure label, control command, actuator state, hydraulic state, and physical reaction could otherwise describe different moments in time.

One-Second Physics, Sampled EDR

The simulator separates physical integration from public observation.

The qualified production configuration is:

physics timestep: 1 second
default public EDR cadence: 5 seconds

The rig evolves physically every second. Public EDR telemetry samples that evolving state at the configured cadence.

Changing the public telemetry cadence does not change the physical trajectory.

This separation was necessary because five- or ten-second physics steps were too coarse for machinery dynamics. A pump could traverse an entire start or stop transient inside one step, making a finite acceleration model appear instantaneous in the exported data.

At one-second resolution, transitions become physically visible:

pump command changes
  -> actual pump accelerates
  -> actual flow changes
  -> standpipe pressure responds
  -> hydraulic completion predicate is reached
  -> procedure advances

The same principle applies to top-drive rotation and travelling-block movement.

Public EDR remains compact, while privileged one-second truth is available for engineering review, validation, and supervised-learning labels.

The public normal-run API intentionally supports a one-second physics cadence. Broader physics-step claims were removed after testing showed that coarse steps changed procedure timing materially.

Physical Completion Gating

Rig activities do not advance solely because a timer expires.

Each action has a physical completion condition.

Examples include:

pumps must physically reach the stopped tolerance before circulation is broken
surface rotation must stop before breakout
the travelling block must reach its target and settle before the next handling action
finite block velocity must be physically braked rather than deleted
slips must latch before support transfers
support mode, hookload, and axial boundary conditions must agree in the same solved state
survey activity must begin only after the string is stationary and supported in slips
mud-pulse transport must retain a valid circulating path throughout transmission
slide penetration must wait for actual rotation to stop
a connection must occur when physically available drilling travel is exhausted
terminal success must pass the same state invariants used during the operation

Timers may represent minimum dwell periods or safety timeouts, but they do not replace physical completion.

This approach produces more than realistic-looking transitions. It prevents the simulator from entering combinations of states that a real rig could not occupy.

An Authoritative Physical Pipe Tally

One of the most important changes was restoring a design principle from the original prototype: one authoritative physical pipe tally.

The simulator resolves the BHA first, then generates or accepts one physical drillpipe tally. For generated operations, joint lengths are sampled once from a bounded distribution using the deterministic scenario seed. Those exact joints are grouped into stands according to the selected rig profile.

For a triple, a stand might look like:

STAND-0001
  J-0001: 10.70 m
  J-0002:  9.61 m
  J-0003:  9.98 m

actual stand length: 30.29 m

That actual 30.29 m stand is what the rig trips, connects, and drills with.

The simulator no longer maintains one sampled planning tally and a second nominal physical inventory. The same physical tally controls:

tubular inventory
connected length
deployed length
stand grouping
trip geometry
connection spacing
available drilling footage
final partial footage
inventory closure

Planning associations and survey requests attach to those physical objects rather than being generated in a separate tubular universe.

Survey acquisition identity remains independent from pipe, stand, and drilling-interval identity. Every actual survey receives a unique monotonic acquisition ID.

This removed a class of failures where final surveys reused the last available joint number, duplicate survey IDs appeared, or event metadata described a planning joint rather than the footage that was physically drilled.

A 100 Metre Example

In the released 600 to 700 m rotary example, the rig does not simply drill three nominal 30 m stands and a final 10 m section.

The physical intervals are determined by the resolved BHA, sampled pipe tally, block geometry, available stand footage, and final target depth. A representative deterministic run produced intervals near:

600.000 -> 623.255 m
623.255 -> 651.811 m
651.811 -> 681.993 m
681.993 -> 700.000 m

The final interval is explicitly partial. The currently connected stand may still have additional available footage, but the bit stops exactly at total depth.

No extra joint is invented, and no previous survey or interval identity is reused.

Every complete export includes:

pipe_tally.csv

This allows a downstream user to reconcile irregular connection spacing directly against the exact synthetic inventory that generated it.

The package can also accept a valid user-supplied physical tally, but normal bundled operations generate a deterministic synthetic tally. The software does not imply that generated pipe lengths are measured field inventory.

Working Pipe And Surveying

The early rig-first implementation transitioned too quickly from drilling into survey acquisition.

The released generic static-survey sequence is:

finish drilling
  -> stop feed
  -> maintain circulation where required
  -> work pipe upward
  -> work pipe downward to approximately 1 m off bottom
  -> stop block
  -> set slips
  -> transfer string support to slips
  -> survey settle period
  -> tool window
  -> survey acquisition

The work-pipe cycle uses the same travelling-block and string mechanics as every other axial movement. It is not a direct depth mutation.

Hookload, torque-and-drag, surface torque, bit depth, and standpipe pressure evolve from the physical state.

The semantic boundary is also explicit. Static survey/* activity does not begin until the string is stationary, the slips are latched, and the slips physically own support.

Circulation-dependent mud-pulse surveys may remain in slips while retaining flow. Hydraulic state and string support are independent physical requirements.

Mud-pulse transmission is evaluated throughout its transmission window. A frame cannot continue after circulation is lost, the BHA is tripped out, or the operation reaches terminal state.

The exported surveys.csv has a stable 17-column schema even when no survey is acquired. A zero-survey operation writes the full header and zero data rows rather than an unparseable blank file.

Hydraulics And Differential Pressure

Standpipe pressure is derived from the modeled hydraulic system.

The pressure budget includes reduced-order contributions from:

drillstring losses
BHA losses
bit/nozzle losses
annular losses
PDM differential pressure where applicable

Public differential pressure is modeled as an operator-zeroed surface measurement:

DIFP = current SPP - captured settled off-bottom SPP zero

The public zero remains frozen until a physically meaningful hydraulic lifecycle event requires a new reference.

This preserves small natural variations caused by actual pump-speed and flow changes.

In rotary-only drilling, DIFP may remain near zero because there is no motor load-dependent pressure term.

In motor-assisted drilling, physical PDM loading produces motor torque and differential pressure. That pressure enters the standpipe-pressure budget, and public DIFP rises naturally.

The distinction is important:

rotary-only drilling:
  DIFP near zero, with small process variation

PDM-assisted drilling:
  DIFP includes the loaded motor pressure contribution

The motor does not turn off because the pump command changes. It responds to actual connected flow as the pump accelerates or decelerates.

The transient hydraulic and motor power states were reconciled so pressure, flow, PDM differential pressure, hydraulic input, mechanical output, and modeled losses represent one energy-consistent physical state rather than two independently convenient diagnostics.

Axial, Rotary, And Torque-And-Drag Load Paths

The simulator separates three useful load concepts:

string hookload
rig-side suspended assembly load
public surface load-cell hookload

Public hookload is:

surface hookload
  = resolved string hookload
  + rig-side suspended assembly load

This means public hookload may remain nonzero at surface even with no deployed drillstring. The modeled load-cell boundary may still carry the travelling block, hook, links, top-drive-side assembly, and other configured suspended rig-side mass.

That baseline is intentional.

The capacity model separately distinguishes gross hook load from string payload so the rig cannot ignore the suspended assembly when checking its own equipment limits.

For directional and high-angle operations, the reduced-order soft-string torque-and-drag model contributes directly to the actual hookload path during:

pickup
slack-off
off-bottom rotation
work-pipe movement
on-bottom drilling

The model is reduced order, but it is applied consistently rather than calculated and discarded.

Rotary demand is resolved through one shared drive budget.

Surface torque must cover:

rotary inertia
wellbore and contact drag
delivered bit torque

The bit cannot consume the complete drive envelope before contact drag is added afterward. The constructed state is validated before it is exposed.

Planned Directional Trajectories, Slides, And MWD

The released simulator supports user-authorable planned directional trajectories. It does not claim to be a commercial well-planning system or to support a general vendor well-plan interchange format.

The public directional machinery includes:

minimum-curvature planned and actual trajectory geometry
rotate and slide steering modes
PDM-assisted directional drilling
actual-rotation stop gating before slide penetration
surveys
idealized EM transport
mud-pulse QPSK transport
high-angle torque-and-drag

The minimum-curvature implementation was checked for:

monotonic measured depth
coordinate sign conventions
inclination and azimuth continuity
dogleg-severity units
northing, easting, and TVD behavior
trajectory station consistency

Slide steering is physically gated by actual rotation. The controller may request a slide immediately, but the simulator does not permit oriented slide footage until surface and string rotation fall below the stopped threshold.

PDM bit rotation can continue from actual hydraulic flow while surface rotation remains stopped.

The Public Build-And-Hold Example

The final release added a complete public directional demonstration:

build_and_hold_motor_full_operation.json

The bundled example:

starts at surface
picks up the BHA
trips physically into the existing hole
drills with a PDM
alternates rotate and slide intervals
builds from 0° toward a 20° target
reaches a final inclination of approximately 19.787°
enters a tangent/hold section
acquires six unique EM surveys
trips back to surface
lays down the BHA
reaches validated terminal state

The release validation recorded 13 genuine slide intervals, approximately 82.45 m of slide footage, and no slide penetration while surface RPM was nonzero. The full operation completes in roughly 10 seconds on the development system.

This example makes the public scope explicit: the simulator is not limited to vertical drilling, but its directional capability remains a reduced-order planned-trajectory and steering model rather than a commercial planning suite.

Field Anchors And Calibration Scope

Public Utah FORGE intervals were used during development to condition selected reference scenarios.

The measured strip anchors the synthetic world.

It is not the spawn point.

The generated operation still begins at surface, trips into the well physically, reaches the anchored interval causally, drills, and returns to surface.

The surrounding trip, connections, drilling outside the measured strip, formation extension, and surface operation are synthetic. They are not presented as historical reconstructions of the real FORGE operations.

The current public v0.1.0a1 API does not provide a general-purpose automatic calibration workflow for arbitrary user drilling data.

Instead, the release provides:

bundled field-anchored reference scenarios
compact public target artifacts
machine-readable calibration ledgers
lower-level development utilities
explicit provenance showing which values were conditioned, fitted, assumed, or treated as calibrated proxies

No private customer drilling data or proprietary field telemetry is part of the public release.

Representative Reference Closure

Representative in-sample closure from the released reference cases includes:

FORGE 56 median ROP:
  simulated: approximately 7.521 m/h
  reference: approximately 7.550 m/h

FORGE 78B public DIFP:
  simulated: approximately 1,195 kPa
  reference: approximately 1,185 kPa

FORGE 78B motor torque:
  simulated: approximately 440 N·m
  reference proxy: approximately 443 N·m

These results demonstrate closure against the selected intervals.

They are not claims of universal field validation, literal formation-property identification, or out-of-sample predictive accuracy.

Some effective parameters are structurally non-identifiable from a short interval. The calibration ledgers therefore distinguish:

conditioned operating controls
fitted parameters
effective calibrated proxies
assumptions
inactive baseline values
independent comparison channels where available

Hostile Engineering Audit

Once the simulator appeared to pass ordinary human inspection, it was subjected to a multi-agent hostile release-candidate audit.

Independent reviewers examined:

drilling operations and rig procedure logic
MWD and directional behavior
hydraulics, PDM pressure, and units
hoisting, axial mechanics, rotary mechanics, and T&D
numerical methods and stochastic processes
architecture and API ownership
performance and scaling
tests and repository cleanup
packaging, provenance, and licensing
impossible-state integration behavior

A separate synthesis pass reproduced, reconciled, deduplicated, and prioritized the findings.

The audit accepted 16 canonical issues for correction.

Identity And Lifecycle Findings

survey identity and physical pipe-tally reconciliation
mud-pulse lifecycle ownership
slide steering during residual rotation
invalid terminal-state semantics

Load, Motion, And Energy Findings

omitted high-angle off-bottom T&D hookload
PDM shutdown based on command rather than actual flow
transient hydraulic and motor energy inconsistency
unbounded drawworks stopping
gross hook-capacity enforcement
double consumption of the rotary torque envelope

API, Geometry, And Release Findings

unsupported public physics timesteps
mutable result snapshots and ambiguous partial exports
conflicting BHA-length authorities
incomplete calibration ledgers
misleading reconstruction terminology
third-party data licensing metadata

The remediation pass closed all 16 findings.

Subsequent independent scans examined more than 100,000 one-second physics states across representative operations without finding a new release-level impossible-state contradiction.

The final release also added focused corrections and tests for:

static survey activity beginning only after slips support
authoritative pipe_tally.csv export
T&D cache invalidation
immutable run results
stable empty-survey CSV schema
Python 3.10 compatibility
public directional build-and-hold behavior

Performance Optimization Without Changing Physics

The hostile performance audit found that deep directional runs spent most of their time rebuilding and interpolating the same historical T&D geometry.

The correction was not to skip physics or reduce path resolution.

Instead, the released implementation caches immutable or revision-owned geometry while continuing to recompute dynamic forces every required physics step.

The cache is invalidated by changes such as:

actual trajectory growth
bit endpoint movement into new geometry
tubular topology changes
connections
work-pipe movement
rotate and slide transitions

It is not invalidated merely because WOB, RPM, or load direction changes when geometry itself remains unchanged.

Seven semantic oracles were used to compare pre-optimization and post-optimization behavior. The checked physical outputs matched exactly.

Measured improvements included:

approximately 2.22× on a deep FORGE 16A 5,000-step prefix
approximately 1.73× on the 100 m surface-to-surface operation
approximately 1.26× on a compact high-angle case

The optimization changed repeated geometric work, not the physical model.

The test suite was also divided into a fast default tier and a slower release tier. Expensive reference cases remain part of the release gate without forcing every development iteration to repeat all field-reference and artifact-validation work.

Packaging, Portability, And Release Engineering

The released repository is a clean public export rather than a publication of the development archive.

The sanitized source tree contained approximately 118 files and 1.6 MB of source, tests, compact package data, and normative documentation.

It intentionally excludes:

private development history
the original prototype notebook
proprietary or non-redistributable manuals
local reference PDFs
raw customer or protected telemetry
virtual environments
human-review packets
generated run outputs
profiler data
stale builds and caches

Third-party technical material is cited through public sources rather than redistributed.

The release process validated:

direct wheel build
source distribution build
wheel rebuilt from the source distribution
fail-closed artifact membership
clean installs outside the checkout
loading all 18 bundled examples
deterministic byte-identical exports
documentation links
licensing and provenance
credential and private-path sanitation
installation from a fresh public GitHub clone

The first GitHub push exposed a Python 3.10 compatibility problem before tagging. The fix added compatible importlib.resources and TOML-loading paths plus a conditional tomli dependency. GitHub Actions then passed on Python 3.10 and 3.12.

At the tagged release, each Python matrix path passed:

296 default-tier tests
21 release-tier tests
wheel build and install validation
artifact validation

The tag v0.1.0a1 identifies the exact public source state that passed those checks.

Outputs

A complete normal run exports:

surface_telemetry.csv
surface_telemetry.parquet
pipe_tally.csv
surveys.csv
events.json
ground_truth.json
run_metadata.json

The files serve different roles:

File	Role
`surface_telemetry.csv`	Stable field-facing EDR-style surface projection.
`surface_telemetry.parquet`	The same logical public surface schema in Parquet.
`pipe_tally.csv`	The exact physical joints and stand grouping used by the run.
`surveys.csv`	Successfully acquired and received directional surveys.
`events.json`	Privileged rig-procedure and interval event history.
`ground_truth.json`	Privileged physical, trajectory, survey, and terminal truth.
`run_metadata.json`	Version, scenario hash, seed, model IDs, provenance, licences, file roles, and tally hash.

An optional static WITSML 1.4.1.1 adapter can also export representative:

well and wellbore objects
surface log data
tubular and BHA-run context
received-survey trajectory
a manifest

The adapter is intentionally static. It is not an ETP server or a complete WITSML operations platform.

Public surface telemetry remains compact and field-facing.

Privileged outputs expose additional simulator truth for engineering review, including machinery commands, physical actuator states, hydraulic components, support topology, string forces, T&D components, PDM state, trajectory truth, and calibration diagnostics.

Trying The Release

git clone https://github.com/SyntheticFunk/drilling-telemetry-simulator.git
cd drilling-telemetry-simulator

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install .

List the bundled examples:

rig-telemetry-sim examples list

Run a complete rotary operation:

rig-telemetry-sim examples run \
  substantial_trip_100m_drilling_example.json \
  --output ./output/rotary-100m

Run the public directional build-and-hold operation:

rig-telemetry-sim examples run \
  build_and_hold_motor_full_operation.json \
  --output ./output/build-and-hold

The output directory must be new.

The repository contains detailed public documentation for installation, scenario authoring, directional configuration, model semantics, references, licensing, testing, and contribution.

Intended Uses

The simulator is intended for:

synthetic drilling telemetry generation
drilling-data pipeline testing
EDR integration testing
static WITSML interchange testing
analytics development
machine-learning dataset generation
drilling-state classification research
simulation and controls experimentation
educational inspection of coupled drilling systems
testing algorithms when real operational data is unavailable, restricted, incomplete, or unsuitable for redistribution

What The Project Is Not

The project is not:

a rig digital twin
a historical replay of FORGE operations
a commercial well-planning suite
a general vendor well-plan interchange system
an anti-collision system
a well-control simulator
field-planning software
a safety-critical engineering package
a replacement for OEM or operator models
a claim of universal field validation

The physics are deliberately reduced order.

Known Limitations

Documented limitations include:

no detailed torsional-wave propagation
no full static or dynamic stick-slip model
reduced-order soft-string T&D
reduced-order PDM behavior
no RSS steering
no anti-collision model
no detailed wire-rope or drawworks structural dynamics
no full surge/swab or hole-cleaning model
no detailed iron-roughneck or connection-torque transient
simplified BHA handling
bit-colocated ideal surveys
no detailed MWD sag, magnetic interference, vibration, or sensor correction
synthetic geology outside measured reference intervals
no general automatic calibration workflow for arbitrary user data
one-second normal production physics cadence

Those limits are documented rather than hidden behind plausible-looking telemetry.

The Main Engineering Lesson

The central lesson from this project is that realistic telemetry is not primarily an output-format problem.

It is a state-ownership problem.

A convincing EDR record emerges when:

the rig has one physical configuration
the string has one support path
the pipe tally is authoritative
machinery has finite dynamics
procedures wait for real completion
drilling grows the hole causally
directional response waits for physical steering conditions
loads, torques, pressures, and power share consistent budgets
surveys and MWD transport obey actual rig conditions
telemetry observes the system rather than controlling it

The first versions produced good traces where reality was available to tune against.

The released system makes the rest of the operation earn those traces.

That transition, from generating drilling-shaped signals to simulating a drilling operation, is what turned the project into an engineering system.

Project And Technical Documentation

Public Technical References

The simulator was developed using public technical and field sources, including:

Third-party manuals and publications are cited, not redistributed, unless their licence explicitly permits inclusion.