10 Writing accounting probes

In Chapter 7 we saw that Haizea collects data while running through the use of probes. While Haizea includes several probes, it is also possible for you to write your own probes by implementing a class that extends from the AccountingProbe class. A barebones probe would look like this:

from haizea.core.accounting import AccountingProbe

class MyProbe(AccountingProbe):
    
    def __init__(self, accounting):
        AccountingProbe.__init__(self, accounting)
        # Create counters, per-lease stats, per-run stats
    
    def finalize_accounting(self):
	# Collect information

    def at_timestep(self, lease_scheduler):
	# Collect information

    def at_lease_request(self, lease):
	# Collect information 

    def at_lease_done(self, lease):
	# Collect information

All the methods shown above are also present in AccountingProbe, but don't do anything. You have to override some or all of the methods to make sure that data gets collected. More specifically:

at_timestep

Override this method to perform any actions every time the Haizea scheduler wakes up. The lease_scheduler parameter contains Haizea's lease scheduler (an instance of the LeaseScheduler class), which you can use to gather scheduling data.

at_lease_request

Override this method to collect data after a lease has been requested.

at_lease_done

Override this method to collect data after a lease is done (this includes successful completion and rejected/cancelled/failed leases).

finalize_accounting

Override this method to perform any actions when data collection stops. This is usually where per-run data is computed.

Probes can collect three types of data:

Per-lease data:

Data attributable to individual leases or derived from how each lease was scheduled.

Per-run data:

Data from an entire run of Haizea

Counters:

A counter is a time-ordered list showing how some metric varied throughout a single run of Haizea.

The probe's constructor should create the counters and specify what per-lease data and per-run data will be collected by your probe (the methods to do this are described next). Notice how a probe's constructor receives a accounting parameter. When creating your probe, Haizea will pass an AccountingDataCollection object that you will be able to use in your probe's other methods to store data.

Once a probe has been implemented, it can be used in Haizea by specifying its full name in the probes option of the accounting section of the configuration file. For example, suppose you created a class called MyProbe in the foobar.probes module. To use the probe, the probes option would look like this:

probes: foobar.probes.MyProbe

When running Haizea, you have to make sure that foobar.probes.MyProbe is in your PYTHONPATH. After running Haizea with your probe, you can access the data it collects using the haizea-convert-data command described in Section