Human factors in security operations

A testbed for how analysts think under pressure.

foyl Research is an open, telemetry-rich simulation testbed for studying attention, cognitive workload, and decision-making in security operations. It reframes realistic SOC tasks as controlled experiments: it manipulates signal noise, time pressure, and interface framing while logging millisecond-level interaction data, so socio-technical questions about analyst performance can be studied empirically.

2
instrumented SOC tasks
3
manipulated conditions
d′
signal-detection measures
0
personally identifying data
Abstract

Security analysts work in high-noise, high-tempo environments where alert fatigue, ambiguous evidence, and time pressure degrade judgement. Most security training and tooling is evaluated on whether it functions, not on how it shapes human performance. foyl Research treats a working simulation platform as a research instrument: an ecologically grounded environment in which realistic analyst tasks become controlled experimental probes.

Participants complete instrumented triage and phishing-detection tasks under randomly assigned conditions that vary signal-to-noise ratio, temporal pressure, and motivational framing. The platform records a fine-grained interaction stream and derives sensitivity (d′), decision criterion, time-to-first-diagnostic, and friction measures, alongside validated workload and self-efficacy instruments. The result is a reusable workbench for studying how interface and environmental factors move analyst attention, workload, and accuracy.

Instrument architecture

Three layers turn a simulation into a measurement device.

Each participant action flows through a logging pipeline; an experimental condition engine parameterises the task the participant sees.

Interactive simulation engine Realistic SOC tasks: alert triage · phishing detection fictional foyl SecIntel canon Telemetry & logging pipeline Timestamped action stream · time-to-first-diagnostic · response time Friction events: hesitation · backtracking · context-switching append-only · idempotent · anonymous participant code Experimental condition engine Signal noise (low/high) · time pressure (off/on) · framing (control/mastery) randomised between-subjects · URL-overridable for directed runs
Method

Independent variables and measures.

Manipulated conditions

FactorLevelsMotivation
Signal noiselow · highalert fatigue, discriminability
Time pressureoff · onspeed-accuracy tradeoff
Framingcontrol · masteryself-efficacy, motivation

Dependent measures

MeasureWhat it captures
d′ / criterionsensitivity and decision bias (signal-detection theory)
Accuracycorrect triage and quarantine/release decisions
Time-to-first-diagnosticonset-to-first-decision latency
Frictionhesitation, backtracking, context-switching
TLX

Workload

A NASA-TLX rating after each task captures perceived mental demand, temporal load, effort, and frustration.

SE

Self-efficacy

A short security self-efficacy scale, measured before and after, pairs with the framing manipulation.

C

Confidence

A post-task confidence rating supports criterion and calibration analysis against actual accuracy.

Take part

The study takes about 12-15 minutes and is completely anonymous.

No account, no email, no personal data. Every company, person, and incident is fictional.

Begin the study

A technical report describing the instrument and pilot results is in preparation. Researchers can review the aggregate dashboard and study information.