Selected work
Small language models specialised for authorized security testing and defensive research, trained as lightweight adapters on modest hardware and published with their datasets.
- Role
- Sole researcher
- Type
- Applied research
- Published
- 3 models, 4 datasets
- Hardware
- Kaggle T4 (14B), MLX (7B)
Signal path
- Input
- Hidden 01
- Hidden 02
- Output
The question
In my experience, general purpose models are uneven on security work, even for people who are authorized to test a system or who are defending one. I wanted to find out how far a small model that runs locally could be specialised for that work without a large training budget.
What I built
Three adapters and four datasets, all on Hugging Face. The 14B adapter was trained with QLoRA in 200 steps on a single Kaggle T4. The two 7B adapters are LoRA adapters in MLX format, so they run on Apple silicon.
I publish them for authorized security testing and defensive research, and each model card says what the model is for and where it falls short.
Where it stands
This is research in progress. The 14B adapter has not had a broad evaluation yet. The 7B distilled adapter has an evaluation of 24 prompts and a written list of known weak spots. A proper benchmark is the next piece of work.
Pipeline
- Store
Datasets
Four datasets, from 644 to 57,769 rows.
Leads to: 7B adapters, 14B adapter.
- Process
7B adapters
Two LoRA adapters in MLX format, so they run on Apple silicon.
Leads to: Evaluation.
- Decision
Evaluation
24 prompts on the 7B distilled adapter, and a written list of known weak spots.
Leads to: Model cards.
- Process
14B adapter
QLoRA, 200 steps on a single Kaggle T4. No broad evaluation yet.
Leads to: Model cards.
- Process
Model cards
What each model is for, what was evaluated and where it falls short.
Leads to: Hugging Face.
- Output
Hugging Face
Three adapters and four datasets, for authorized security testing and defensive research.
Decisions
Adapters, not full fine tunes
The 14B adapter trains about 69 million parameters, which is 0.46 percent of the model, at rank 16. The published adapter files are small next to the base models: about 23 MB each for the 7B adapters and about 275 MB for the 14B.
Datasets published alongside
The four datasets range from 644 rows to 57,769 rows. The largest is deduplicated by SHA-256 and built from two Apache 2.0 sources, with train, validation and test splits.
Size was not the lever
One dataset card records that a larger bulk dataset did not improve a strong base model; the set that card describes has 644 rows, and the distillation set has 794 examples.
Limits written down
The cards say what was evaluated and what was not: the 7B distilled card reports its 24 prompt evaluation, and the 14B card says none has been run yet. A model that people may rely on for security work should not be described more generously than the evidence allows.
Stack
- QLoRA
- LoRA
- MLX
- Hugging Face
- Kaggle
- Python