# Computer science with Propab

Training runs, ablations, benchmarks and algorithm analysis, set up properly, run to completion and reported with the curves and tables behind them.

## Real runs

- **Machine learning** — [replay](https://propabai.com/replay/share_A1gL8yr1_0wcY9GpqhQGr4ZaRoe4BgZk): “On a small transformer trained on a character-level language modelling task, ablate positional encodings, layer norm placement and dropout one at a time, and measure the effect on validation loss. Deliver the ablation table, curves and a short report.” Made: Ablation table, Loss curves, Report.

## Workflows ready today

### Algorithms and theory

- **Empirical scaling of a solver on an instance family** — Where a problem family becomes intractable on this machine, with fitted scaling across solvers and the theory that explains it.
- **Graph-algorithm study on generated families** — Runtime and solution quality of graph algorithms across generated graph families and sizes.
- **SMT solver benchmarking with validated answers** — Solver comparison on SMT-LIB with every model and unsat core checked.
- **Symbolic computation behind a proof** — Closed forms, recurrences and inequalities checked symbolically to support an argument.
- **Anytime evaluation of a heuristic optimiser** — Quality-vs-time curves of heuristics on TSPLIB-style instances against known optima.

### Compilers and languages

- **Is this peephole rewrite sound?** — An SMT proof or counterexample that an optimisation preserves semantics for all inputs.
- **Parser, grammar and AST work** — Grammar written and tested, or ASTs of real code analysed.
- **LLVM IR and optimisation-pipeline study** — IR built and run through optimisation passes, with before/after IR and JIT results.

### Computer architecture

- **Roofline model of a kernel** — Arithmetic intensity vs attainable performance for a kernel on a named CPU/GPU from its datasheet numbers.

### Computer vision

- **Score detections against ground truth** — COCO-style mAP/AR for a set of detections the researcher has.
- **Classical image processing and matching** — Feature detection, matching, homography and geometric correction on images.
- **Fine-tune a pretrained classifier on a small dataset** — Transfer-learning accuracy with calibration on a small public dataset (CIFAR-scale), CPU-sized.
- **Architecture claim at ImageNet scale** — Train and compare architectures on ImageNet-1k under a matched recipe. (needs a GPU you connect)
- **Robustness under corruptions** — Accuracy of pretrained models on CIFAR-10-C by corruption type and severity.
- **Train a detector or segmenter** — Detection/segmentation model trained and evaluated on COCO-style data. (needs a GPU you connect)
- **Label-noise and test-set overfitting audit** — Find mislabeled test items and compare accuracy on a re-collected test set (CIFAR-10.1).
- **Novel-view synthesis benchmark** — NeRF/3D Gaussian splatting quality (PSNR/SSIM/LPIPS) on a standard scene set. (needs a GPU you connect)

### Formal methods

- **Model-check a TLA+ specification** — TLC run over a bounded instance: invariants hold or a counterexample trace.
- **Find models or counterexamples with Alloy** — Alloy analysis of a relational model within a stated scope.
- **Symbolic execution of Python against its contracts** — Counterexamples to a Python function's pre/postconditions.
- **Property-based testing with shrinking** — Minimal failing input for a stated property of real code.
- **Machine-checked proof in Coq** — A theorem stated and proved in Coq using the standard library.
- **Is this program transformation sound?** — SMT encoding of a transformation with a proof or counterexample.

### Human-computer interaction

- **Analyse my controlled user study** — Mixed-effects analysis of a within-subjects experiment with effect sizes and CIs.
- **Power analysis and preregistration** — Required sample size and a preregistration document with the analysis script dry-run on simulated data.
- **Fit Fitts' law to pointing data** — Model fit, throughput and per-condition comparison from pointing trials.
- **Code interview transcripts** — Codebook, coded excerpts and inter-rater agreement for qualitative data the researcher provides.
- **Analyse deployment logs** — Usage patterns and retention from in-the-wild interaction logs.
- **Survey instrument analysis** — Reliability (alpha/omega) and factor structure of a questionnaire.
- **Reproduce a paper from its OSF artifact** — Re-run a published analysis from its OSF materials and compare numbers.

### Information retrieval

- **BM25 ad-hoc evaluation on a test collection** — nDCG/MAP/recall of BM25 on a public collection with qrels.
- **Dense retrieval across the full BEIR suite** — Zero-shot dense-retriever results on all BEIR datasets. (needs a GPU you connect)
- **Statistically sound system comparison** — Paired tests with multiple-comparison control over per-query metrics from run files.
- **Retrieval-augmented generation evaluation** — Retrieval and answer quality of a RAG pipeline on a QA benchmark. (needs a GPU you connect)
- **Corpus and query-log analytics** — SQL analytics over a large document or interaction corpus.

### Machine learning

- **Is an improvement real or seed noise?** — Seed-level significance test and effect size for a claimed gain.
- **Fit a scaling law** — Scaling exponents with bootstrap intervals from a run table.
- **Ablation study of a small model** — Causal ablation of components on a CPU-sized task with seeds and compute matched.
- **Budgeted hyperparameter search on tabular benchmarks** — Tuned baselines on an OpenML suite with the search budget disclosed.
- **Compare architectures at GPU scale** — Matched-recipe training runs of models too large for CPU. (needs a GPU you connect)
- **Language-model benchmark evaluation** — lm-evaluation-harness scores for open models with the harness version pinned. (needs a GPU you connect)
- **Benchmark contamination audit** — N-gram/near-duplicate overlap of a benchmark with a pretraining-corpus sample.
- **Reproduce a published result at CPU scale** — Re-run a paper's code from GitHub and compare its numbers.

### Natural language and LLMs

- **Tokenisation and perplexity experiments** — Tokenizer behaviour and perplexity of small models on a corpus.
- **Score translations or summaries** — BLEU, chrF and ROUGE with signatures for system outputs.
- **Fine-tune a small language model** — Fine-tuned small model (≤ ~125M params) with evaluation against the base.
- **LoRA fine-tune of a billion-parameter model** — Parameter-efficient fine-tuning of a 1B+ model with a disclosed search budget. (needs a GPU you connect)
- **Social-bias probing of open models** — BBQ/CrowS-Pairs-style results disaggregated by group. (needs a GPU you connect)
- **Semantic search over a corpus** — Embedding index and retrieval quality over a document set.
- **Text-classification baselines** — TF-IDF/linear and small-transformer baselines on a public dataset with error analysis.
- **Serve a model for high-throughput inference** — Throughput/latency of a served model under load (vLLM). (needs a GPU you connect)

### Robotics and control

- **Simulate a robot and close a control loop** — MuJoCo/PyBullet simulation with a feedback controller and tracking metrics.
- **Classical control design** — LQR, pole placement, Bode and root-locus analysis of a plant.
- **State estimation with Kalman-family filters** — EKF/UKF/particle-filter estimates and consistency checks on a dataset.
- **Motion-planning benchmark** — OMPL planners compared on success rate and path quality.
- **Embodied navigation in photorealistic scenes** — Habitat navigation agents on standard scene datasets. (needs a GPU you connect)
- **Pose-graph SLAM back end** — Least-squares pose-graph optimisation with loop closures on a dataset.

### Security and cryptography

- **Find an input that reaches a code location** — Symbolic execution of a binary to a target block (e.g. a crackme's success path).
- **Analyse a binary's structure** — Sections, symbols, disassembly and emulation of an ELF binary.
- **Break a textbook cryptographic construction** — Working attack (padding oracle, CBC bit-flip, small-exponent RSA) with explanation.
- **Vulnerability trends from NVD** — CVE counts, CWE mix and CVSS trends for a product or class.
- **Scan a sample corpus with YARA** — Rule matches and false-positive checks over files the researcher supplies.
- **Membership-inference evaluation** — Attack AUC against a trained model and the effect of a defence.

### Systems and databases

- **Analytical SQL on large Parquet data** — Query results, plans and engine comparison over multi-GB columnar data.
- **Query-optimiser study on TPC-H** — Plan quality and regret across queries and scale factors.
- **Deterministic simulation testing of a protocol** — A distributed protocol run under a seeded scheduler until a correctness bug appears, with the replay.
- **Find a tail-latency regression** — Percentile analysis with CIs and change-point detection on latency measurements.
- **Re-run a reproducibility package** — A paper's artifact re-executed from Zenodo/GitHub with results compared.