Generative AI Monitoring
Generative AI Monitoring provides a unified way to track, evaluate, and improve the quality of your AI model outputs.
This service enables you to analyze your large language model (LLM) interactions across multiple quality and safety metrics, including hallucination, similarity, sentiment, PII detection, and more.
You can use these endpoints in two modes:
- Real-time — get results instantly for each request.
- Task-based — queue and retrieve results asynchronously for bulk or long-running evaluations.
These metrics can also be visualized in the Watchlog Dashboard.
Authentication
To use the API, obtain your API key from your dashboard.
You can find it in the GenAI Control Room.
Metrics
Below is a list of metrics you can use to monitor your application’s responses.
Each section provides both real-time and task-based usage examples.
Note:
For demonstration, we use the same prompt and response across most metrics:
Prompt: Hello, how are you?
Response: I’m doing great, thank you! How can I help you today?
Some metrics, such as PII detection, use a different example to better illustrate their functionality.
Hallucination
Description:
The Hallucination metric evaluates the consistency between a target response and additional sampled responses.
If the model truly understands the topic, it should produce similar and consistent outputs when asked the same question multiple times.
The metric returns a score between 0 and 1, where:
0indicates high consistency (low hallucination)- Values closer to
1indicate high inconsistency (potential hallucination)
The score combines semantic similarity and LLM-based consistency scores.
Real-time Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/hallucination' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?",
"response": "I’m doing great, thank you! How can I help you today?"
}'
You will receive a response similar to the example below, depending on your configuration.
{
"id": "87c3e64e-ed73-4fe8-9dcc-343760db2e16",
"prompt": "Hello, how are you?",
"response": "I’m doing great, thank you! How can I help you today?",
"metrics": {
"hallucination": {
"samples": [
"Hello! I'm just a program, so I don’t have feelings, but I’m here and ready to help you. How can I assist you today?"
],
"response": "I’m doing great, thank you! How can I help you today?",
"llm_score": 1,
"final_score": 0.6840837076306343,
"total_tokens": 180,
"semantic_score": 0.3681674152612686,
"created_at": "2025-10-23T06:49:57.828391Z"
}
}
}
Task-based Usage: : you can send your prompt and response like this
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/hallucination/task' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?",
"response": "I’m doing great, thank you! How can I help you today?"
}'
You will receive a response similar to the example shown below.
```json
{
"status": "queued",
"message": "Your hallucination metric calculation is being processed.",
"task_id": "a19c78d5-af68-4442-97ce-7f466b533cd9"
}
You can check the current status of your task using this endpoint.
Text Similarity
Description:
The Text Similarity metric calculates how closely a model’s response matches the prompt’s intent.
It uses cosine similarity between embeddings of the prompt and response.
A higher score indicates greater relevance.
Real-time Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/text-similarity' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?",
"response": "I’m doing great, thank you! How can I help you today?"
}'
The API response will look like the following example.
{
"id": "87c3e64e-ed73-4fe8-9dcc-343760db2e16",
"prompt": "Hello, how are you?",
"response": "I’m doing great, thank you! How can I help you today?",
"metrics": {
"text_similarity": {
"response.relevance_to_prompt": 0.4632212519645691,
"created_at": "2025-10-23T07:16:24.497812Z"
}
}
}
Task-based Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/text-similarity/task' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?",
"response": "I’m doing great, thank you! How can I help you today?"
}'
The API response will look like the following example.
{
"status": "queued",
"message": "Your text similarity metric calculation is being processed.",
"task_id": "e750f8de-4638-45cc-b4a4-df7d51f7cf7d"
}
You can check the current status of your task using this endpoint.
Detect PII (Personally Identifiable Information)
Description:
The PII Detection metric identifies sensitive entities such as credit card numbers, phone numbers, SSNs, and more, using Microsoft’s Presidio engine.
Results include:
pii_presidio.result: a JSON string containing detected entities with their positions, types, and confidence scores.pii_presidio.entities_count: the number of detected entities.
Real-time Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/detected-pii' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Please contact me at 123 Main St, Springfield, IL for further details. You can reach me at (123) 456-7890 or via email at [email protected]. For payment, use my credit card 4111-1111-1111-1111. My SSN is 123-45-6789 for verification."
}'
The API response will look like the following example.
{
"id": "483a8a1a-6545-4a6d-bc84-592bfee4507b",
"prompt": "Please contact me at 123 Main St, Springfield, IL for further details. You can reach me at (123) 456-7890 or via email at [email protected]. For payment, use my credit card 4111-1111-1111-1111. My SSN is 123-45-6789 for verification.",
"response": null,
"metrics": {
"detected_pii": {
"prompt.pii_presidio.result": "[{\"type\": \"CREDIT_CARD\", \"start\": \"176\", \"end\": \"195\", \"score\": \"1.0\"}, {\"type\": \"URL\", \"start\": \"122\", \"end\": \"129\", \"score\": \"0.5\"}, {\"type\": \"URL\", \"start\": \"131\", \"end\": \"142\", \"score\": \"0.5\"}, {\"type\": \"PHONE_NUMBER\", \"start\": \"91\", \"end\": \"105\", \"score\": \"0.4\"}]",
"prompt.pii_presidio.entities_count": 4,
"created_at": "2025-10-23T07:27:23.121571Z"
}
}
}
as you can see, based on my prompt this metric find 4 entities with positions and confidence score of that.
Task-based Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/detected-pii/task' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Please contact me at 123 Main St, Springfield, IL for further details. You can reach me at (123) 456-7890 or via email at [email protected]. For payment, use my credit card 4111-1111-1111-1111. My SSN is 123-45-6789 for verification."
}'
The API response will look like the following example.
{
"status": "queued",
"message": "Your detected PII metric calculation is being processed.",
"task_id": "da39c330-3239-477e-b1c6-3b20e930423e"
}
You can check the current status of your task using this endpoint.
Prompt Injection
Description:
This detector identifies potential prompt injection attacks.
It works by sending the test prompt along with a control instruction that asks the LLM to repeat a random string.
If the model ignores the instruction (and the random string is missing in its response), a prompt injection is suspected.
Real-time Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/detected-prompt-injection' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?"
}'
The API response will look like the following example.
{
"id": "67eba8e8-f203-4422-ba40-22176e3f63f5",
"prompt": "Hello, how are you?",
"response": null,
"metrics": {
"detected_prompt_injection": {
"score": 0,
"response": "07QZFK3T 07QZFK3T",
"created_at": "2025-10-23T07:47:29.225695Z"
}
}
}
Task-based Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/detected-prompt-injection/task' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?"
}'
The API response will look like the following example.
{
"status": "queued",
"message": "Your detected prompt injection metric calculation is being processed.",
"task_id": "1f341f4e-4fe3-4279-bcdd-e3baa1cac27a"
}
You can check the current status of your task using this endpoint.
Sensitive Pattern
Description:
The Sensitive Pattern metric scans text using groups of regex patterns to identify sensitive data such as SSNs, credit card numbers, phone numbers, mailing addresses, and emails.
If a match is found, the corresponding pattern group name is returned.
Real-time Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/detected-sensitive-pattern' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Please contact me at 123 Main St, Springfield, IL for further details. You can reach me at (123) 456-7890 or via email at [email protected]. For payment, use my credit card 4111-1111-1111-1111. My SSN is 123-45-6789 for verification."
}'
The API response will look like the following example.
{
"id": "483a8a1a-6545-4a6d-bc84-592bfee4507b",
"prompt": "Please contact me at 123 Main St, Springfield, IL for further details. You can reach me at (123) 456-7890 or via email at [email protected]. For payment, use my credit card 4111-1111-1111-1111. My SSN is 123-45-6789 for verification.",
"response": null,
"metrics": {
"detected_sensitive_pattern": {
"prompt.has_patterns": "SSN",
"created_at": "2025-10-23T07:57:28.763136Z"
}
}
}
Task-based Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/detected-sensitive-pattern/task' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Please contact me at 123 Main St, Springfield, IL for further details. You can reach me at (123) 456-7890 or via email at [email protected]. For payment, use my credit card 4111-1111-1111-1111. My SSN is 123-45-6789 for verification."
}'
The API response will look like the following example.
{
"status": "queued",
"message": "Your detected sensitive pattern metric calculation is being processed.",
"task_id": "42804b8a-98ba-4fc3-885f-cd326a8cc6ca"
}
You can check the current status of your task using this endpoint.
Text Sentiment
Description:
The Text Sentiment metric analyzes the emotional tone of the text using NLTK’s Vader sentiment analyzer.
It returns a score between -1 (most negative) and 1 (most positive).
Real-time Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/text-sentiment' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?"
}'
The API response will look like the following example.
{
"id": "67eba8e8-f203-4422-ba40-22176e3f63f5",
"prompt": "Hello, how are you?",
"response": null,
"metrics": {
"text_sentiment": {
"prompt.sentiment_nltk": 0,
"created_at": "2025-10-23T08:00:37.521746Z"
}
}
}
Task-based Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/text-sentiment/task' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?"
}'
The API response will look like the following example.
{
"status": "queued",
"message": "Your text sentiment metric calculation is being processed.",
"task_id": "c69b3856-70b5-4814-94d8-3aba0908aad7"
}
You can check the current status of your task using this endpoint.
Text Metrics
Description:
The Text Metrics endpoint calculates various readability and complexity scores, providing a detailed view of linguistic quality.
It includes submetrics such as:
flesch_kincaid_grade,smog_index,gunning_fog, etc.aggregate_reading_level: a combined readability score.difficult_words,syllable_count,sentence_count, and others.
These measures help assess the readability and complexity of generated text across multiple languages.
Real-time Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/text-metrics' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?"
}'
The API response will look like the following example.
{
"id": "67eba8e8-f203-4422-ba40-22176e3f63f5",
"prompt": "Hello, how are you?",
"response": null,
"metrics": {
"text_metrics": {
"prompt.has_patterns": null,
"prompt.letter_count": 14,
"prompt.lexicon_count": 4,
"prompt.sentence_count": 1,
"prompt.sentiment_nltk": 0,
"prompt.syllable_count": 5,
"prompt.character_count": 16,
"prompt.difficult_words": 0,
"prompt.monosyllable_count": 3,
"prompt.polysyllable_count": 0,
"prompt.flesch_reading_ease": 92.8,
"prompt.pii_presidio.result": "[]",
"prompt.aggregate_reading_level": 1,
"prompt.automated_readability_index": -0.7,
"prompt.pii_presidio.entities_count": 0,
"prompt.injection.proactive_detection": 0,
"created_at": "2025-10-23T08:09:52.725269Z"
}
}
}
Task-based Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/text-metrics/task' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?"
}'
The API response will look like the following example.
{
"status": "queued",
"message": "Your text metrics calculation is being processed.",
"task_id": "703786b1-70e8-403f-9c91-9adccfdf232b"
}
You can check the current status of your task using this endpoint.
Theme Similarity
Description:
The Theme Similarity metric evaluates whether a prompt or response resembles known categories, such as jailbreaks or refusals.
It uses cosine similarity against predefined thematic examples to produce a similarity score for each supported theme.
Real-time Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/theme-similarity' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?",
"response": "I’m doing great, thank you! How can I help you today?"
}'
The API response will look like the following example.
{
"id": "87c3e64e-ed73-4fe8-9dcc-343760db2e16",
"prompt": "Hello, how are you?",
"response": "I’m doing great, thank you! How can I help you today?",
"metrics": {
"theme_similarity": {
"prompt.has_patterns": null,
"prompt.letter_count": 14,
"prompt.lexicon_count": 4,
"prompt.sentence_count": 1,
"prompt.sentiment_nltk": 0,
"prompt.syllable_count": 5,
"response.has_patterns": null,
"response.letter_count": 39,
"prompt.character_count": 16,
"prompt.difficult_words": 0,
"response.hallucination": 0.6840837076306343,
"response.lexicon_count": 11,
"response.sentence_count": 2,
"response.sentiment_nltk": 0.8622,
"response.syllable_count": 13,
"response.character_count": 43,
"response.difficult_words": 1,
"prompt.monosyllable_count": 3,
"prompt.polysyllable_count": 0,
"prompt.flesch_reading_ease": 92.8,
"prompt.pii_presidio.result": "[]",
"prompt.jailbreak_similarity": 0.273514986038208,
"response.monosyllable_count": 9,
"response.polysyllable_count": 0,
"response.refusal_similarity": 0.3237978518009186,
"response.flesch_reading_ease": 99.73,
"response.pii_presidio.result": "[]",
"response.relevance_to_prompt": 0.4632212519645691,
"prompt.aggregate_reading_level": 1,
"response.aggregate_reading_level": 0,
"prompt.automated_readability_index": -0.7,
"prompt.pii_presidio.entities_count": 0,
"prompt.injection.proactive_detection": 0,
"response.automated_readability_index": -0.4,
"response.pii_presidio.entities_count": 0,
"created_at": "2025-10-23T08:23:53.681211Z"
}
}
}
Task-based Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/theme-similarity/task' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?",
"response": "I’m doing great, thank you! How can I help you today?"
}'
The API response will look like the following example.
{
"status": "queued",
"message": "Your theme similarity metric calculation is being processed.",
"task_id": "97d9353b-ad6f-4e39-bf5f-ede3507fb283"
}
You can check the current status of your task using this endpoint.
Text Topic
Description:
The Text Topic classifier assigns the input text to one of several predefined categories (e.g., law, finance, medical, education, politics, support) based on semantic similarity.
Real-time Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/text-topic' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?"
}'
The API response will look like the following example.
{
"id": "67eba8e8-f203-4422-ba40-22176e3f63f5",
"prompt": "Hello, how are you?",
"response": null,
"metrics": {
"text_topic": {
"prompt.closest_topic": "support",
"created_at": "2025-10-23T08:28:03.148594Z"
}
}
}
Task-based Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/text-topic/task' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?"
}'
The API response will look like the following example.
{
"status": "queued",
"message": "Your text topic metric calculation is being processed.",
"task_id": "2b7c1ba3-f454-41fd-bdbb-f132264c405b"
}
You can check the current status of your task using this endpoint.
Text Toxicity
Description:
The Text Toxicity metric measures how toxic or offensive a piece of text is.
It produces a score between 0 (non-toxic) and 1 (highly toxic).
Real-time Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/text-toxicity' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?"
}'
The API response will look like the following example.
{
"id": "67eba8e8-f203-4422-ba40-22176e3f63f5",
"prompt": "Hello, how are you?",
"response": null,
"metrics": {
"text_toxicity": {
"prompt.toxicity": 0.0013582110404968262,
"created_at": "2025-10-23T08:31:10.578737Z"
}
}
}
Task-based Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/text-toxicity/task' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?"
}'
The API response will look like the following example.
{
"status": "queued",
"message": "Your text toxicity metric calculation is being processed.",
"task_id": "9b3bb8cc-455c-4b71-b45f-16b263ae5b1b"
}
You can check the current status of your task using this endpoint.
Total Metrics
Description:
The Total Metrics endpoint computes all available metrics in a single call, returning a combined response for holistic model evaluation.
Real-time Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?",
"response": "I’m doing great, thank you! How can I help you today?"
}'
The API response will look like the following example.
{
"id": "87c3e64e-ed73-4fe8-9dcc-343760db2e16",
"prompt": "Hello, how are you?",
"response": "I’m doing great, thank you! How can I help you today?",
"metrics": {
"hallucination": {
"samples": [
"Hello! I’m just a program, but I’m here and ready to help you. How can I assist you today?"
],
"response": "I’m doing great, thank you! How can I help you today?",
"llm_score": 0.5,
"final_score": 0.4175884947180748,
"total_tokens": 168,
"semantic_score": 0.3351769894361496,
"created_at": "2025-10-23T08:35:31.752708Z"
},
"text_similarity": {
"response.relevance_to_prompt": 0.4632212519645691,
"created_at": "2025-10-23T08:35:31.753663Z"
},
"detected_pii": {
"prompt.toxicity": 0.0013582110404968262,
"prompt.has_patterns": null,
"prompt.letter_count": 14,
"prompt.closest_topic": "support",
"prompt.lexicon_count": 4,
"prompt.sentence_count": 1,
"prompt.sentiment_nltk": 0,
"prompt.syllable_count": 5,
"prompt.character_count": 16,
"prompt.difficult_words": 0,
"prompt.monosyllable_count": 3,
"prompt.polysyllable_count": 0,
"prompt.flesch_reading_ease": 92.8,
"prompt.pii_presidio.result": "[]",
"prompt.jailbreak_similarity": 0.273514986038208,
"prompt.aggregate_reading_level": 1,
"prompt.automated_readability_index": -0.7,
"prompt.pii_presidio.entities_count": 0,
"prompt.injection.proactive_detection": 0,
"created_at": "2025-10-23T08:35:31.754172Z"
},
"detected_prompt_injection": {
"score": 0,
"response": "4TM7K8TP",
"created_at": "2025-10-23T08:35:31.754869Z"
},
"detected_sensitive_pattern": {
"prompt.has_patterns": null,
"created_at": "2025-10-23T08:35:31.755362Z"
},
"text_sentiment": {
"prompt.sentiment_nltk": 0,
"created_at": "2025-10-23T08:35:31.755903Z"
},
"text_metrics": {
"prompt.toxicity": 0.0013582110404968262,
"prompt.has_patterns": null,
"prompt.letter_count": 14,
"prompt.closest_topic": "support",
"prompt.lexicon_count": 4,
"prompt.sentence_count": 1,
"prompt.sentiment_nltk": 0,
"prompt.syllable_count": 5,
"prompt.character_count": 16,
"prompt.difficult_words": 0,
"prompt.monosyllable_count": 3,
"prompt.polysyllable_count": 0,
"prompt.flesch_reading_ease": 92.8,
"prompt.pii_presidio.result": "[]",
"prompt.jailbreak_similarity": 0.273514986038208,
"prompt.aggregate_reading_level": 1,
"prompt.automated_readability_index": -0.7,
"prompt.pii_presidio.entities_count": 0,
"prompt.injection.proactive_detection": 0,
"created_at": "2025-10-23T08:35:31.756481Z"
},
"theme_similarity": {
"prompt.toxicity": 0.0013582110404968262,
"response.toxicity": 0.001183629035949707,
"prompt.has_patterns": null,
"prompt.letter_count": 14,
"prompt.closest_topic": "support",
"prompt.lexicon_count": 4,
"prompt.sentence_count": 1,
"prompt.sentiment_nltk": 0,
"prompt.syllable_count": 5,
"response.has_patterns": null,
"response.letter_count": 39,
"prompt.character_count": 16,
"prompt.difficult_words": 0,
"response.closest_topic": "support",
"response.hallucination": 0.4336949288845062,
"response.lexicon_count": 11,
"response.sentence_count": 2,
"response.sentiment_nltk": 0.8622,
"response.syllable_count": 13,
"response.character_count": 43,
"response.difficult_words": 1,
"prompt.monosyllable_count": 3,
"prompt.polysyllable_count": 0,
"prompt.flesch_reading_ease": 92.8,
"prompt.pii_presidio.result": "[]",
"prompt.jailbreak_similarity": 0.273514986038208,
"response.monosyllable_count": 9,
"response.polysyllable_count": 0,
"response.refusal_similarity": 0.3237978518009186,
"response.flesch_reading_ease": 99.73,
"response.pii_presidio.result": "[]",
"response.relevance_to_prompt": 0.4632212519645691,
"prompt.aggregate_reading_level": 1,
"response.aggregate_reading_level": 0,
"prompt.automated_readability_index": -0.7,
"prompt.pii_presidio.entities_count": 0,
"prompt.injection.proactive_detection": 0,
"response.automated_readability_index": -0.4,
"response.pii_presidio.entities_count": 0,
"created_at": "2025-10-23T08:35:31.757006Z"
},
"text_topic": {
"prompt.closest_topic": "support",
"created_at": "2025-10-23T08:35:31.757458Z"
},
"text_toxicity": {
"prompt.toxicity": 0.0013582110404968262,
"created_at": "2025-10-23T08:35:31.757893Z"
}
}
}
Task-based Usage:
curl -X 'POST' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/task' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>' \
-H 'Content-Type: application/json' \
-d '{
"prompt": "Hello, how are you?",
"response": "I’m doing great, thank you! How can I help you today?"
}'
The API response will look like the following example.
{
"status": "queued",
"message": "Your request to calculate metrics is being processed.",
"task_id": "29ef496c-ac10-4abc-937f-1603e1c8d132"
}
You can check the current status of your task using this endpoint.
Task Status for All Metrics
Description:
Use this endpoint to check the processing status or result of a submitted task.
Tasks can be in one of four states: queued, processing, processed, or failed.
usage:
curl -X 'GET' \
'https://gen-ai.watchlog.io/api/v1/monitoring/metrics/task/{task-id}/status' \
-H 'accept: application/json' \
-H 'Authorization: Bearer <YOUR-API-KEY>'
and get the result like this.
{
"task_id": "string",
"status": "string",
"message": "string",
"result": {
"additionalProp1": {}
},
"error": "string"
}
The task status can be one of the following: "queued", "processing", "processed", or "failed". If the task status is "processed", the result will appear in the result field. If the status is "failed", the error field will contain the corresponding error message.
Didn’t find the action or metric you were looking for? Let us know — we’re constantly improving and can add it for you.