实验室概况


上海交通大学人工智能安全实验室（AI Security Lab）现有教师6位、在读博士生10余位、在读硕士生20余位。 实验室聚焦人工智能技术自身的安全问题以及人工智能技术在安全领域的应用，主要研究内容安全、大模型安全、联邦学习、深度模型水印、深度伪造检测、生物特征身份认证等。 累计发表学术论文400余篇，其中多篇论文发表在IEEE TIFS / TPAMI / TFS / TCSVT / TCYB / TNNLS、ACL、ICCV、AAAI、IJCAI、EMNLP等顶级期刊和会议上。

团队成员

Avatar

刘功申

教授

人工智能安全, 自然语言处理, 大模型及其安全, 深度学习框架及其安全, 多模态大模型

Avatar

王士林

教授

多媒体信息分析与处理, 多媒体信息内容安全, 人工智能安全

Avatar

孟魁

高级实验师

内容安全, 系统安全与仿真

Avatar

张倬胜

助理教授

自然语言处理, 预训练语言模型, 自主智能体及其安全

Avatar

段苏峰

助理研究员

内容安全, 自然语言处理, 机器翻译

Avatar

黄林庆

助理研究员

低质数据挖掘与学习, 遥感影像智能解译, 人工智能安全及应用

近期事件

5月 23, 2025 — 5月 25, 2025
实验室参加ICASSP 2025苏州卫星会议
实验室参加ICASSP 2025苏州卫星会议

由上海交通大学2024年春季《人工智能安全技术》课程（NIS3353）讲义拓展而来（教师：张倬胜），旨在提供大模型相关的入门编程参考。通过简单实践，帮助同学快速入门大模型，更好地开展课程设计或学术研究。涵盖微调与部署、提示学习与思维链、知识编辑、模型水印、越狱攻击、多模态模型、大模型智能体与安全等内容。
3月 1, 2024 — 6月 30, 2024
《动手学大模型》系列编程实践教程
《动手学大模型》系列编程实践教程

由上海交通大学2024年春季《人工智能安全技术》课程（NIS3353）讲义拓展而来（教师：张倬胜），旨在提供大模型相关的入门编程参考。通过简单实践，帮助同学快速入门大模型，更好地开展课程设计或学术研究。涵盖微调与部署、提示学习与思维链、知识编辑、模型水印、越狱攻击、多模态模型、大模型智能体与安全等内容。
PDF 代码

最新成果

Efficient and Effective Model Extraction
Model extraction aims to steal a functionally similar copy from a machine learning as a service (MLaaS) API with minimal overhead, …
Efficient and Effective Model Extraction
Enhancing Visual Forced Alignment with Local Context-Aware Feature Extraction and Multi-Task Learning
This paper introduces a novel approach to Visual Forced Alignment (VFA), aiming to accurately synchronize utterances with corresponding …
Enhancing Visual Forced Alignment with Local Context-Aware Feature Extraction and Multi-Task Learning
Membership Encoding for Black-Box Neural Network Watermarking
Deep neural network watermarking is an emerging technique for protecting the copyright of models. Most existing black-box watermarking …
Membership Encoding for Black-Box Neural Network Watermarking
MIFAE-Forensics: Masked lmage-Frequency AutoEncoder for DeepFake Detection
With continuously evolving generative models and increasingly diverse face forgery products, there is a growing demand for DeepFake …
MIFAE-Forensics: Masked lmage-Frequency AutoEncoder for DeepFake Detection
Rethinking the Fragility and Robustness of Fingerprints of Deep Neural Networks
Fingerprints characterize deep neural networks that are deployed as black-boxes. To achieve copyright tracing and integrity …
Rethinking the Fragility and Robustness of Fingerprints of Deep Neural Networks
Acquiring Clean Language Models from Backdoor Poisoned Datasets by Downscaling Frequency Space
Despite the notable success of language models (LMs) in various natural language processing (NLP) tasks, the reliability of LMs is …
PDF Cite DOI
Acquiring Clean Language Models from Backdoor Poisoned Datasets by Downscaling Frequency Space
Personatalk: Preserving Personalized Dynamic Speech Style in Talking Face Generation
Recent visual speaker authentication methods claimed their effectiveness against deepfake attacks. However, the success is attributed …
Personatalk: Preserving Personalized Dynamic Speech Style in Talking Face Generation
Lip Feature Disentanglement for Visual Speaker Authentication in Natural Scenes
Recent studies have shown that lip shape and movement can be used as an effective biometric feature for speaker authentication. By …
PDF Cite 代码 DOI
Lip Feature Disentanglement for Visual Speaker Authentication in Natural Scenes
How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Study
Previous work has showcased the intriguing capability of large language models (LLMs) in retrieving facts and processing context …
PDF Cite DOI
How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Study
自然语言处理中的探针可解释方法综述
随着大规模预训练模型的广泛应用，自然语言处理的各领域（如文本分类和机器翻译）均取得了长足的发展。然而，受限于预训练模型的“黑盒”特性，其内部的决策模式以及编码的知识信息被认为是不透明的。以OpenAI发布的ChatGPT和GPT-4为代表的先进预训练模型为例，它们在各领 …
PDF Cite DOI
自然语言处理中的探针可解释方法综述
查看全部