Specify Privacy Yourself: Assessing Inference-Time Personalized Privacy Preservation Ability of Large Vision-Language Model

Paper PDF | ACM DL

The official implementation of ACM Multimedia 2025 BNI Oral paper "Specify Privacy Yourself: Assessing Inference-Time Personalized Privacy Preservation Ability of Large Vision-Language Model".

Illustration with cases of our proposed Inference-Time Personalized Privacy Protection task.

News

[2025-10-21] 🎉 We have released the code and data for both evaluation and training! A preprint copy of our paper is also available here.

[2025-08-08] 🚀 This repository has been created.

[2025-08-01] 🎉 Our paper has been accepted by ACM Multimedia 2025 as a BNI Oral presentation!

Abstract

Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities but raise significant privacy concerns due to their abilities to infer sensitive personal information from images with high precision. While current LVLMs are relatively well aligned to protect universal privacy, e.g., credit card data, we argue that privacy is inherently personalized and context-dependent. This work pivots towards a novel task: can LVLMs achieve Inference-Time Personalized Privacy Protection (ITP$^3$), allowing users to dynamically specify privacy boundaries through language specifications? To this end, we present SPY-Bench, the first systematic assessment of ITP$^3$ ability, which comprises (1) 32,700 unique samples with image-question pairs and personalized privacy instructions across 67 categories and 24 real-world scenarios, and (2) novel metrics grounded in user specifications and context awareness. Benchmarking the ITP$^3$ ability of 21 SOTA LVLMs, we reveal that: (i) most models, even the top-performing o4-mini, perform poorly, with only ~24% compliance accuracy; (ii) they show quite limited contextual privacy understanding capability. Therefore, we implemented initial ITP$^3$ alignment methods, including a novel Noise Contrastive Alignment variant which achieves 96.88% accuracy while maintaining reasonable general performance. These results mark an initial step towards the ethical deployment of more controllable LVLMs.

Environment Setup

First, clone this repository to your local machine and navigate to the project directory:

git clone https://github.com/achernarwang/specify-privacy-yourself
cd specify-privacy-yourself

Then prepare the python environment with the following commands:

conda create -n spy python=3.12 -y
conda activate spy
pip install uv
uv pip install vllm qwen-vl-utils accelerate deepspeed tensorboard trl==0.15.0 liger-kernel==0.5.3
uv pip install flash-attn --no-build-isolation

Evaluating with SPY-Bench

Download Images

Download the test image dataset of VISPR from this link and extract it to benchmark/data/images/. After extraction, the benchmark directory structure should look like this:

benchmark/
├── data/
│   ├── images/
│   │   └── test2017
│   │       ├── 2017_10000580.jpg
│   │       └── ...
│   ├── label2text.json
│   └── ...
└── ...

Prepare Models

A complete list with download links of the evaluated LVLMs and fine-tuned checkpoints in our paper is provided in Additional Information, though our implementation can theoretically benchmark any LVLM that is supported by vLLM or has a serving endpoint compatible with OpenAI API.

If you want to evaluate an API model, please set the API key (and Endpoint URL if needed) in the benchmark/.env:

API_KEY = "<YOUR_API_KEY>"
BASE_URL = "<ENDPOINT_URL>" # optional if using models served by OpenAI

Evaluation Steps

1. Generate the responses of the evaluated models:

   cd benchmark
   # For open-sourced models (with distractors)
   python generate.py --model path/to/your/model --gpu_id 0 --batch_size 64 --add_distractors --result_dir results/with_distractors 
   # For API models (without distractors)
   python generate.py --model <your_model_name_or_id> --batch_size 64 --result_dir results/without_distractors

   The argument --add_distractors indicates whether to include distractor privacy instructions in the evaluation. If specified, the model needs to identify the correct privacy instruction among multiple distractors. Other arguments are self-explanatory and can be checked by python generate.py --help.

2. [Optional] If you want to evaluate the self-moderation method described in section 4.1 of our paper, please run the following command after step 1:

   python generate_self_mod.py --file results/with_distractors/resp/<generated_file>.jsonl --model path/to/your/model --gpu_id 0 --batch_size 64 --result_dir results/with_distractors_self_mod

   The evaluated model in this step should be the same as that in step 1.

3. Evaluate the generated results with a judgment model (specified by --eval_model). The judgment model does not require multimodal capability, so you can use pure LLMs in this step. If you decide to use an API model (in our paper we use GPT-4o), please also set EVAL_API_KEY and EVAL_BASE_URL in the benchmark/.env file.

   # using open-sourced models (require vLLM support) as judgment model
   python evaluate.py --eval_model /path/to/your/model --gpu_id 0 --batch_size 64 --result_dir results/with_distractors  --result_file resp/<generated_file>.jsonl
   # using API models as judgment model
   python evaluate.py --eval_model <your_model_name_or_id> --batch_size 64 --result_dir results/with_distractors  --result_file resp/<generated_file>.jsonl

   If you are evaluating the results generated with self-moderation method, please also specify the argument --resp_k as resp_3 in the command above.

4. Calculate the metrics for the evaluation results:

   python metrics.py -f eval/<evaluated_file>.jsonl --result_dir results/with_distractors

   If you are evaluating the results generated with self-moderation method, please specify the argument --eval_k as eval_3 in the command above. To calculate the overall score across SPY-Bench and general benchmarks (MMMU, OCRBench, MME), you can use the VLMEvalKit to obtain the results of these general benchmark and then specify the -g argument with the path to the general benchmark results file when running metrics.py.

Training

Prepare the training data (SPY-Tune)

First, download the training image dataset of VISPR from this link and extract it to train/data/images/. Then download the training annotations from this link and move it to train/data/.

The train directory should now look like this:

train/
├── configs/
├── data/
│   ├── images/
│   │   └── train2017
│   │       ├── 2017_10001018.jpg
│   │       └── ...
│   ├── train_data.json
│   ├── eval_data.json
│   └── ...
└── ...

Training Scripts

We provide training scripts for all the methods adopted in our paper, including SFT (train/train_sft.py), DPO / NCA (train/train_dpo.py), and NCA-P (train/train_our.py). Below is an example command for training with NCA-P:

export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
accelerate launch --config_file configs/deepspeed_zero2.yaml --num_processes 8 \ # Number of using GPUs
   train_our.py \
   --model_name_or_path /path/to/Qwen2-VL-7B-Instruct \
   --train_data_path data/train_data.json \
   --eval_data_path data/eval_data.json \
   --label_path data/label2text.json \
   --image_folder data \
   --shuffle True \
   --add_distractors True \
   --min_pixels 200704 \
   --max_pixels 1003520 \
   --num_train_epochs 3.0 \
   --save_strategy "epoch" \
   --logging_steps 10 \
   --eval_steps 100 \
   --per_device_train_batch_size 4 \
   --per_device_eval_batch_size 4 \
   --gradient_accumulation_steps 1 \
   --gradient_checkpointing \
   --learning_rate 3e-6 \
   --loss_type "nca_priv" \
   --beta 0.01 \
   --weight_decay 0.05 \
   --warmup_ratio 0.1 \
   --lr_scheduler_type "cosine" \
   --bf16 \
   --tf32 True \
   --torch_dtype bfloat16 \
   --use_liger \
   --attn_implementation flash_attention_2 \
   --output_dir runs/q2_ncap_b32_l3e-6_b001_e3_wd005_wr01 \
   --save_only_model True \
   --report_to tensorboard

You could refer to the train/scripts/ for more example commands.

Tip

If you are encountering CUDA out-of-memory issues during training, other than reducing the training batch size, you could also try to adjust the deepspeed settings in your Accelerate configuration file (under train/configs/) following the instructions here.

Acknowledgements

We gratefully acknowledge the developers and contributors of VISPR, 🤗Huggingface Libraries, vLLM Project, and VLMEvalKit, upon which our work is built. We also extend our gratitude to the authors of all evaluated models (see the list below) for sharing the model weights or service endpoints with the research community.

Citation

If you feel this repo is helpful to your research, please consider citing our work:

@inproceedings{wang2025specify,
  title={Specify Privacy Yourself: Assessing Inference-Time Personalized Privacy Preservation Ability of Large Vision-Language Models},
  author={Wang, Xingqi and Yi, Xiaoyuan and Xie, Xing and Jia, Jia},
  booktitle={Proceedings of the 33rd ACM International Conference on Multimedia},
  pages={12304--12313},
  year={2025}
}

Additional Information

In our paper, we evaluated the following LVLMs with SPY-Bench:

Model Name	Model Type	Source
LLaVA 1.5 13B	Open-source	🤗 HuggingFace
LLaVA NeXT Vicuna 13B	Open-source	🤗 HuggingFace
LLaVA OneVision Qwen2 7B	Open-source	🤗 HuggingFace
Llama 3.2 11B Vision Instruct	Open-source	🤗 HuggingFace
Pixtral 12B	Open-source	🤗 HuggingFace
GLM 4V 9B	Open-source	🤗 HuggingFace
Deepseek VL2	Open-source	🤗 HuggingFace
InternVL 2.5 4B/8B/38B/78B	Open-source	🤗 HuggingFace
Qwen2 VL 7B Instruct	Open-source	🤗 HuggingFace
Qwen2.5 VL 3B/7B/32B/72B Instruct	Open-source	🤗 HuggingFace
Phi 4 Multimodal Instruct	Open-source	🤗 HuggingFace
Mistral Small 3.1 24B Instruct 2503	Open-source	🤗 HuggingFace
GPT 4o 2024-11-20	Proprietary	OpenAI
Gemini 2.0 Flash	Proprietary	Google AI
o4-mini 2025-04-16	Proprietary	OpenAI

The information and download link of the fine-tuned checkpoints used in our paper are provided below:

Base Model	Finetuning Method	Checkpoint Link
Qwen2-VL-7B-Instruct	SFT	🤗 HuggingFace
Qwen2-VL-7B-Instruct	DPO	🤗 HuggingFace
Qwen2-VL-7B-Instruct	NCA	🤗 HuggingFace
Qwen2-VL-7B-Instruct	NCA-P	🤗 HuggingFace