Understanding Key Properties in Storm UI
Storm UI is very helpful for troubleshooting issues encountered during Storm usage, but some properties have unclear meanings. Although they are all simple concepts, not knowing them can be quite frustrating.
One thing to note: when you hover over the title bar in the UI, you can see the specific details of that property. Several highly-ranked Google articles are essentially just organizing this information.
Most properties are straightforward — you know what they mean just by seeing the name. Here I’ll only list some properties that might cause confusion, to help everyone troubleshoot issues more easily.
emitted and transferred: “emitted” refers to the number of data items emitted, which is the number of times the emit method of OutputCollector is called. “transferred” is the actual number of tuples sent to the next task. At first glance they seem the same, right? In most cases they are indeed the same. However, for example, if a bolt emits data but no downstream bolt consumes it, the bolt’s transfer count will be 0. Another example: if bolt A uses the all group strategy (every bolt must receive it) to emit tuples to bolt B, then the transferred count will be a multiple of the emitted count.
execute latency and process latency: “execute latency” is straightforward — it’s the execution time of the execute() method in code. “process latency” is the time from when execute starts until the ack method is called. It can be considered as the time required for business code execution. Under normal circumstances, execute latency is greater than process latency, but if you never call ack, process latency will be much greater than execute latency.
Spout’s complete latency: You can refer to Storm’s ack mechanism. This is the time from when a tuple is emitted until that tuple is acked. More precisely, it’s the time difference between when the spout calls the emit method and when the ack method is called — in other words, it’s the time required for the tuple tree generated by that tuple to be fully processed.
