NeurIPS 2025 Workshop · 2025
Context-Masked Meta-Prompting for Privacy-Preserving LLM Adaptation in Finance
Sayash Raaj Hiraou
How does a financial institution adapt a large language model to its own tasks when the data that would teach the model can never leave the firm? The paper introduces context-masked meta-prompting, which iteratively optimizes hard prompts without exposing confidential context to the model, keeping the process low-cost and auditable.
Contribution Sole author: the method, the experiments and the paper. Research conducted at Fidelity Investments.
01Problem
Adapting a general-purpose LLM to a financial task usually means showing it examples drawn from the task, and those examples are often exactly the data that cannot leave the firm. The constraint is that prompt quality must improve without the model ever seeing the confidential context, while the process stays auditable.
02Approach
- An iterative meta-prompting procedure that treats the hard prompt as the object being optimized.
- A regeneration process built from feeder and propagation methods, designed so that proprietary context is never exposed to the LLM.
- Systematic prompt templates and evaluation harnesses that benchmark candidate prompts on each task, so improvements are measured rather than judged by eye.
03Evaluation
| Public dataset | Financial task it stands in for |
|---|---|
| SQuAD | Extractive question answering over financial documents |
| CNN/DailyMail | News summarization |
| SAMSum | Summarizing client interactions |
103.87%
improvement in ROUGE-L F1
question answering, GPT-3.5 Turbo
04Why it is relevant here
The problem has the same structure as much of my portfolio work: improve an objective under a hard constraint that cannot be relaxed (here, confidentiality), and prove the improvement with a measurable evaluation rather than anecdotes.