Glossary
Red teaming
Deliberately attacking your own AI system to find failures before users or attackers do.
Red teaming probes for prompt injection, jailbreaks, data leakage, harmful output and tool misuse, adversarially, from the perspective of someone motivated to break it.
The durable output is not the report but the regression suite it leaves behind.
Scope it against your own system rather than the model. The interesting questions are what your tools can do, what data your retrieval can reach, and what a determined user could make the combination do, none of which the model provider's own safety testing has any visibility into.
Run it before launch and again after any change to tools or permissions. The attack surface of an AI system is defined by what it can reach, so widening its access reopens questions the previous test already answered.
Related terms, in context
The concepts you almost always meet alongside red teaming.
- Prompt injection
- An attack where instructions hidden in content the model reads override the behaviour you intended.
- Evaluation
- A labelled test set that turns 'it seems good' into a number you can regress against.
- Guardrails
- Constraints on what an AI system may do, enforced in code rather than requested in a prompt.
Where this shows up in our work
Red teaming is not an abstraction for us. It is a decision we make on live projects. It shows up most directly in ai evaluation & red teaming, where getting it wrong has a cost someone can measure.
If you are evaluating a vendor on this, the useful question is not whether they can define the term. It is what they measure, what they would refuse to do, and what happens in their system when the assumption behind red teaming stops holding.
Questions
What is Red teaming?
Deliberately attacking your own AI system to find failures before users or attackers do.
Does Orqent Labs build this?
Yes, AI Evaluation & Red Teaming. We work across India, covering all 19,238 PIN codes remotely.
Building something that involves red teaming?
We will tell you honestly whether it is the right approach for your problem.
Or email bd@dtrasglobal.com · call +91 74118 77878