Size in Bytes
The metric provides the value in bytes that a partition of desired stream uses on each Kafka broker at any given time.
Basic usage
Please refer to the example partition size in bytes metric provided in the API docs
This request is asking for the size in bytes of the
partition 2 of stream payment-events-stream on environment dev for last 30 minutes with the step-size of 10 minutes.
Basic Request
{
"metric": "io.axual.partition/size_bytes",
"stepSize": "PT10M",
"timeWindow": "PT30M",
"filter": {
"type": "AND",
"filters": [
{
"type": "FIELD",
"field": "environment",
"operation": "EQUALS",
"value": "dev"
},
{
"type": "FIELD",
"field": "stream",
"operation": "EQUALS",
"value": "payment-events-stream"
},
{
"type": "FIELD",
"field": "partition",
"operation": "EQUALS",
"value": "2"
}
]
}
}
The below part of sample response, represents the size of partition 2 on each kafka broker (pod) in bytes.
Basic Response
{
"type": "UNGROUPED",
"dataPoints": [
{
"timestamp": "2022-10-18T12:30:00",
"value": 0,
"labels": {
"__name__": "kafka_log_log_size",
"axual_cluster": "jupiter",
"partition": "2",
"pod": "jupiter-kafka-2",
"topic": "axual-qa-dev-payment-events-stream"
},
"unit": "Bytes"
},
{
"timestamp": "2022-10-18T12:40:00",
"value": 144098,
"labels": {
"__name__": "kafka_log_log_size",
"axual_cluster": "jupiter",
"partition": "2",
"pod": "jupiter-kafka-2",
"topic": "axual-qa-dev-payment-events-stream"
},
"unit": "Bytes"
},
{
"timestamp": "2022-10-18T12:50:00",
"value": 400366,
"labels": {
"__name__": "kafka_log_log_size",
"axual_cluster": "jupiter",
"partition": "2",
"pod": "jupiter-kafka-2",
"topic": "axual-qa-dev-payment-events-stream"
},
"unit": "Bytes"
},
{
"timestamp": "2022-10-18T12:30:00",
"value": 0,
"labels": {
"__name__": "kafka_log_log_size",
"axual_cluster": "jupiter",
"partition": "2",
"pod": "jupiter-kafka-1",
"topic": "axual-qa-dev-payment-events-stream"
},
"unit": "Bytes"
},
{
"timestamp": "2022-10-18T12:40:00",
"value": 149754,
"labels": {
"__name__": "kafka_log_log_size",
"axual_cluster": "jupiter",
"partition": "2",
"pod": "jupiter-kafka-1",
"topic": "axual-qa-dev-payment-events-stream"
},
"unit": "Bytes"
},
{
"timestamp": "2022-10-18T12:50:00",
"value": 400366,
"labels": {
"__name__": "kafka_log_log_size",
"axual_cluster": "jupiter",
"partition": "2",
"pod": "jupiter-kafka-1",
"topic": "axual-qa-dev-payment-events-stream"
},
"unit": "Bytes"
}
]
}
This metric could be used to know how much data is stored in each partition, and how it’s distributed among partitions and how much data growths per time unit.
Advanced usage
Using groupBy
If you want to get response grouped by some label - you can use groupBy
Request using groupBy
{
"metric": "io.axual.partition/size_bytes",
"groupBy": [
"pod"
],
"stepSize": "PT10M",
"timeWindow": "PT30M",
"filter": {
"type": "AND",
"filters": [
{
"type": "FIELD",
"field": "environment",
"operation": "EQUALS",
"value": "dev"
},
{
"type": "FIELD",
"field": "stream",
"operation": "EQUALS",
"value": "payment-events-stream"
},
{
"type": "FIELD",
"field": "partition",
"operation": "EQUALS",
"value": "2"
}
]
}
}
The below response represents the size in bytes of the partition on each Kafka broker (pod), with data grouped by pod.
Response using groupBy
{
"type": "GROUPED",
"groups": [
{
"labels": {
"pod": "jupiter-kafka-1"
},
"dataPoints": [
{
"timestamp": "2022-10-18T12:30:00",
"value": 0,
"labels": {
"__name__": "kafka_log_log_size",
"axual_cluster": "jupiter",
"partition": "2",
"pod": "jupiter-kafka-1",
"topic": "axual-qa-dev-payment-events-stream"
},
"unit": "Bytes"
},
{
"timestamp": "2022-10-18T12:40:00",
"value": 149754,
"labels": {
"__name__": "kafka_log_log_size",
"axual_cluster": "jupiter",
"partition": "2",
"pod": "jupiter-kafka-1",
"topic": "axual-qa-dev-payment-events-stream"
},
"unit": "Bytes"
},
{
"timestamp": "2022-10-18T12:50:00",
"value": 400366,
"labels": {
"__name__": "kafka_log_log_size",
"axual_cluster": "jupiter",
"partition": "2",
"pod": "jupiter-kafka-1",
"topic": "axual-qa-dev-payment-events-stream"
},
"unit": "Bytes"
}
]
},
{
"labels": {
"pod": "jupiter-kafka-2"
},
"dataPoints": [
{
"timestamp": "2022-10-18T12:30:00",
"value": 0,
"labels": {
"__name__": "kafka_log_log_size",
"axual_cluster": "jupiter",
"partition": "2",
"pod": "jupiter-kafka-2",
"topic": "axual-qa-dev-payment-events-stream"
},
"unit": "Bytes"
},
{
"timestamp": "2022-10-18T12:40:00",
"value": 144098,
"labels": {
"__name__": "kafka_log_log_size",
"axual_cluster": "jupiter",
"partition": "2",
"pod": "jupiter-kafka-2",
"topic": "axual-qa-dev-payment-events-stream"
},
"unit": "Bytes"
},
{
"timestamp": "2022-10-18T12:50:00",
"value": 400366,
"labels": {
"__name__": "kafka_log_log_size",
"axual_cluster": "jupiter",
"partition": "2",
"pod": "jupiter-kafka-2",
"topic": "axual-qa-dev-payment-events-stream"
},
"unit": "Bytes"
}
]
}
]
}