John Scarrott

Platform Lead · Rust & Systems Engineer

john@scarrotts.uk (+44) 7733298950 www.jscarrott.com github.com/jscarrott Barnstaple, Devon, UK

About

I'm a software engineer and platform lead at RazorSecure, where we build intrusion detection for the rail industry — on-train agents reporting back to a wayside and cloud platform. I set the technical direction across our platforms and am currently leading the migration of our on-train detection agent from Python to Rust.

I've been building software professionally for over a decade. I started out in safety-critical embedded systems — helicopter health monitoring written to DO-178 — before moving into high-performance networking, security and cloud-native infrastructure. These days I work mostly in Rust and Python, and I'm at my best on low-level, performance-sensitive problems and the tooling and systems around them.

What I tend to focus on:

Outside work I build things for fun — at the moment a physics simulator for testing an autonomous sailboat — and contribute to open source.

Experience

🚀 Senior Rust Software Engineer & Platform Lead — RazorSecure

December 2019 - Present · Remote

  • Platform technical lead: Set the technical direction for the product and author cross-platform roadmaps coordinating the on-train Agent, Frontend and Microservices/Backend platforms, so an architectural decision on one doesn't break another. Work is tracked as GitHub epics through an idea → buy-in → ticketed → planned → done lifecycle, and includes a push-based configuration-management design.
  • Rust rewrite of the on-train detection agent (Python → Rust): Driving an incremental "strangler-fig" migration that ships one stage per sprint with no big-bang cutover. Built the agent as a multi-call single binary (clap) on a Tokio runtime, with structured async task supervision, `panic=unwind` so a faulty monitor restarts rather than aborting the process, a lifecycle state machine, and self-monitoring with health heartbeats. Reimplemented the full monitor suite to strict parity with the Python agent (network DoS / port-scan / ARP, USB, syslog via journald, Suricata DPI, SNMP, GPS via gpsd, file/inotify, nftables and more) and drove the codebase toward pure-Rust dependencies, dropping C-FFI libraries such as libsnmp and paho-mqtt.
  • High-performance detection core: Designed and built a zero-copy Rust deep packet inspection library with eBPF (XDP) acceleration, sustaining sub-microsecond latency and 1 million+ packets per second as the basis of the intrusion-detection product, plus an L2-to-L7 firewall for an embedded security gateway protecting critical rail network infrastructure.
  • Cloud portability and on-prem deployment: Migrated storage and business logic off Google Cloud onto S3-compatible storage and made the entire platform deployable on-prem and on Azure — in-cluster Elasticsearch, Terraform fixes, and removal of cloud-provider-only assumptions across every Helm chart — including air-gapped Kubernetes clusters running alongside the cloud-hosted offering.
  • Deployment and release engineering: Consolidated dozens of per-microservice Helm deployments into a single platform Helm chart (bringing Redis, QuestDB and RabbitMQ into the chart), moved CI off developer machines into GitHub Actions, and introduced a Harbor OCI registry with Replicated-based on-prem distribution. Built release automation around git-cliff changelogs, Conventional-Commit linting, release actions and grouped Dependabot updates.
  • Data-store performance and platform uplift: Led deep PostgreSQL alert-store performance work — functional indexes on JSON fields, tsvector search predicates, partition pruning, bulk multi-row inserts and Celery tuning to eliminate out-of-memory failures during large batch jobs — alongside a MySQL 8.4 migration, a Pydantic v2 / FastAPI uplift across services, RabbitMQ 4.2 clustering, QuestDB ingest and memory-leak fixes, and an Elasticsearch migration to a Helm-hosted, tiered deployment with reindex-from-remote.
  • Rail-specific detection and domain features: Implemented rail geospatial logic (lat/long → ELR plus miles-and-chains) and a range of new monitors and protocol support — Suricata DPI, an SSH honeypot, the TRDP, MVB and EDSA rail protocols, mirrored-traffic and VLAN-strip handling, rate-limiting and time-based event prioritisation — plus a new customer-configuration microservice.
  • Observability: Built a Grafana proxy architecture with authenticated endpoints and config-map-preloaded dashboards, and instrumented the platform throughout with Datadog APM tracing and statsd metrics.
  • Test and documentation discipline: Built a multi-tier BDD test harness spanning the agent, processing and integration layers up to end-to-end tests driven through a real MQTT broker, with a JUnit aggregator that produces a Typst PDF test report and parallel Docker-based BDD in CI; established documentation conventions with topical docs and Mermaid / C4 / D2 architecture diagrams, and contributed to open-source Rust libraries for layer-2 network monitoring and flame-graph performance analysis.

⚙️ Software Engineer — Helitune/Beran Instruments

June 2015 - December 2019 · Torrington, North Devon

  • Next-generation protection and condition monitoring: Development and systems engineering of a next-generation system for large-plant monitoring. Championed the use of Rust for several of the system modules, running a dockerised signal-processing and logging system on an embedded target — being deployed at the NASA Ames Research Center.
  • Rotor Track and Balance systems: Developed and maintained a best-in-class RTB system in both carry-on and permanent-fit configurations, written to DO-178 Level C and D.
  • Modular C++ signal-processing system: Designed an all-new, module-based C++ signal-processing system that let existing and new products move onto a single platform. Being easily testable and replaceable, it shortened development time, made the system more robust, and fit well with agile working.
  • C#/WPF applications: Developed and maintained several large (100,000+ line) C#/WPF projects across the whole software lifecycle, and built a configuration-driven automated test application that saved hundreds of hours and caught production faults before release.
  • Version control and CI: Led the company's move from TFS to Git — teaching teams distributed version control and introducing code review — and built the CI workflow around it with packer/vagrant, vSphere and TeamCity.

Skills

Systems Programming
Rust (Tokio async, FFI elimination, multi-call binaries), C++, C
Backend & APIs
Python (FastAPI, Pydantic v2, Celery)
Databases
PostgreSQL (advanced indexing & partition tuning), MySQL, QuestDB, Elasticsearch, Redis, RabbitMQ
Containerisation & Orchestration
Docker, Podman, Kubernetes, Helm, GKE, AKS
Infrastructure & Release Engineering
Terraform/Terragrunt, Skaffold, Pants, Replicated, Harbor, GitHub Actions, ArgoCD, FluxCD
Observability
Grafana, Datadog APM, statsd
Domain & Protocols
Rail (TRDP, MVB, EDSA, ELR/miles-and-chains), DPI/Suricata, SNMP, MQTT, ARP, GPS/NMEA/gpsd
Engineering Practice
Strangler-fig migration, TDD & BDD (Pytest, cargo test), Conventional Commits & release automation, ADRs, C4/D2 architecture modelling, secure-by-design
Leadership
Cross-platform roadmapping, technical-direction setting, migration & rollback planning, effort estimation
Project Organisation
GitHub Issues & Epics, JIRA, Confluence, Agile/SCRUM
Previously (not current)
C#/WPF, F#, Java, PowerShell, SQL Server, NUnit, GoogleTest/GoogleMock, TeamCity, vSphere, packer/vagrant

Projects

⛵ Author — STDA Sailboat Simulator

2023- · GitHub

  • Building a physics-based simulator to design, develop and test an autonomous sailboat before deploying to real hardware.
  • Ported the 6-degree-of-freedom (6-DOF) dynamics core to Rust from a published autonomous-sailing model (Sailing Team Darmstadt, IRSC 2018), and extended it with route-following, real nautical-chart navigation and a speed-polar calibration instrument.
  • Rust and Python, working toward calibrating the model against real-world data for a hardware retrofit and autonomous-control trials.

🏠 Author — Homebox Home Assistant Add-on

2025- · GitHub

  • Home Assistant add-on that packages Homebox (a self-hosted home inventory tool) for one-click installation through the Home Assistant Supervisor.
  • My most-starred public project, used by the wider Home Assistant community.
  • Maintained add-on repository with a containerised build and release flow.

📖 Author — Md-book combiner

2023- · Github

  • Simple high-value tool for combining mdbooks from multiple repositories into one
  • Leveraged automated releases to make deployment simple
  • High complexity to value ratio for personal usecase

🔌 Author — jjui-pm

2026- · GitHub

  • CLI plugin manager, written in Rust, for jjui — a TUI for the Jujutsu (jj) version-control system.
  • Manages Lua plugins by reading and modifying the jjui config file; installable via cargo.
  • Reflects deep day-to-day use of Jujutsu and the Rust terminal-UI ecosystem.

🔧 Contributor — Rust nRF52 Hal

2018- · Github

  • Experienced developing at the cutting edge, implementing Rust only hardware abstraction on a micro-controller.
  • Being the first to run Rust on the nRF52840 chip proved a real problem solving challenge.

Education

🎓 BSc. in Computing and Psychology — The Open University

2011 - 2015 · England

  • Software development and the psychology behind user interface design

⚡ C for Real-Time Developers — Feabhas

2016 · Royal Wootton Bassett

  • Intensive five day course on writing low-level C with and without a real time operating system.

🔧 Advanced C++ Development — Feabhas

2016 · Royal Wootton Bassett

  • Intensive five day course on writing C++ on micro-controllers.
  • Learned many of the pitfalls with the C++ language.
  • Learned how to leverage the more technical features of the language.